Senior Data Engineer

FDJ UNITED

Greater London, UK

1 day ago

Apply on www.collegerecruiter.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Working hours

Regular working hours

Job source

www.collegerecruiter.com

Tech stack

Airflow Amazon S3 Application Frameworks Cloud Computing Cloud Database Continuous Integration Directed Acyclic Graph (Directed Graphs) Data Architecture Information Engineering Data Governance Extract Transform Load (ETL) Dimensional Modeling

+8 more

Python (Programming Language) SQL Databases Data Streaming Sql Optimization Apache Spark Data Lakes Pyspark Data Pipelines

Job description

Pipeline Development: Build ETL/ELT pipelines (Python, SQL), optimize Spark/PySpark jobs, manage Airflow DAGs, and implement dbt transformations.
Lakehouse Architecture: Design medallion-style architectures using S3 and modern table formats (Iceberg, Delta Lake) with optimized layouts.
Platform Infrastructure: Create reusable frameworks, enforce data contracts, build CI/CD pipelines, and establish observability practices.
Data Quality: Implement validation frameworks, data contracts, and SLIs/SLOs for critical pipelines.

Requirements

A hands-on engineering role building and optimising scalable ETL/ELT pipelines and Lakehouse architectures across a modern multi-cloud platform using Python, Spark, Airflow, and dbt.

Requires 5+ years of data engineering experience with strong expertise in SQL, CI/CD, data quality frameworks, and cloud technologies (S3, Athena, Redshift), with a plus for streaming and AI tooling experience., * 5+ years in data engineering

Strong Python, advanced SQL, Airflow, Spark/PySpark, dbt, and CI/CD proficiency
Experience with Lakehouse architectures, dimensional modeling, and cloud data warehouses (S3, Athena, Redshift)
Data quality frameworks, monitoring, and schema evolution expertise