Data Engineer
J&C Associates
London, UK
19 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
2 years minimum
Working hours
Regular working hours
Job source
Tech stack
Java (Programming Language)
A/B Testing
Application Programming Interfaces (APIs)
Agile Methodology
Airflow
Amazon Web Services
Microsoft Azure
Databases
System Configuration
Continuous Integration
Data Security
Data Warehousing
+17 more
Relational Databases
DevOps
Python (Programming Language)
PostgreSQL
Cloud Services
Web Analytics
Scripting
Git
Pandas
Data Lakes
Pyspark
Data Analytics
Splunk
Text Files
Docker
Jenkins
Databricks
Job description
- As a senior data platform engineer, you will build and maintain scalable, rock solid self-serve solutions for data analysts and support them by understanding the content and context of data as well as collaborating with them to figure out the best way to Extract, Transform, Load and access it.
- The ideal candidate will have a passion for simplifying and automating workflows, for making things better, for being efficient and helping the team do more with less.
What you will do:
- Work with data analysts to determine the best schemas to translate available data sources into business domains.
- Build connectors to integrate with Postgres-based RDBMSs, cloud solutions like AWS, Azure.
- Write scripts to retrieve data from RDBMSs, logging systems (e.g., Splunk), APIs, text files.
- Work with external partners to facilitate data access.
- Make data workflows more efficient and automate them when possible.
- Lead sophisticated, large-scale projects in an Agile environment using Git, Jenkins, Docker.
Requirements
- 5-10 years of Industry Experience.
- Passion for continuous betterment (i.e., better workflows, automation, efficiency).
- Experience with Data Warehouse or Data Lake, choosing database architectures, optimizing SQL queries.
- Experience in a DevOps role, configuring and launching Docker containers, Jenkins for CI/CD, airflow for scheduling.
- 2-5 years of proven experience coding in Python/Java or any scripting language.
- Excellent in teamwork and communication skills.
Preferred skills:
- Experience and knowledge of Digital Analytics or Digital Marketing products.
- Experience with storage, compute services (e.g., databricks) on a major cloud vendor like AWS or Microsoft Azure is a big plus.
- Knowledge of statistics or experience with pandas, pyspark or R, to better connect with analysts.
- Knowledge of A/B testing or data-driven experimentation frameworks.
- Experience in a B2B environment is a plus.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www1.jobdiva.co.ukGood distractions
Talks and stories from around this role — technically off-topic, practically not.
Moments
Explore playlistsVideos
See allRelated articles
See all
EM
Eli McGarvie
about 3 years ago
EM
Eli McGarvie
Data Analyst Salary in the UK
about 3 years ago
BB
Benedikt Bischof
Making Data Warehouses Fast: A Developer’s Story
about 4 years ago
LM
Luis Minvielle
The Most Popular IT Jobs on the Market
over 2 years ago
DS
Dhannush Subramani
Top Big Data Technologies That You Need to Know
about 4 years ago
EM
Eli McGarvie
Highest Paying Tech Companies for Developers
over 3 years ago