Senior Data Engineer

Careerwise

London, UK

about 1 month ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Compensation

£70,000.0 - £80,000.0

Working hours

Regular working hours

Job source

computerjobs.com

Tech stack

Artificial Intelligence Microsoft Azure Continuous Integration Data Infrastructure Extract Transform Load (ETL) Data Warehousing Cursor (Graphical User Interface Elements) Python (Programming Language) Standard Sql Apache Spark Git Microsoft Fabric

+6 more

Data Lakes Pyspark Star Schema Data Management Data Pipelines Databricks

Job description

Senior Data Engineer

Location: Hybrid (2 Days office)

Type: Permanent

Salary: £70,000-£80,000 + Benefits

About the Role

We’re looking for a Senior Data Engineer to join a growing AI & Data team, building modern data platforms using Databricks and Microsoft Fabric. You’ll design and develop scalable data pipelines, data models and lakehouse solutions that support reporting, analytics and AI initiatives.

Key Responsibilities

Lead data pipeline architecture development/orchestration/maintenance.
Develop robust data models and ETL/ELT processes.
Optimise data performance and troubleshoot pipeline issues.
Build and maintain data warehouse/lakehouse solutions.
Collaborate with cross-functional teams and document technical solutions.
Support best practices around data quality, governance and CI/CD.

Required Skills

Strong experience with Databricks.
Excellent Python and SQL skills.
Good knowledge of Spark, PySpark, Delta Lake, Delta Live Tables, Unity Catalog and Medallion Architecture.
Experience with Azure DevOps, Git and CI/CD.
Strong understanding of data warehousing, star schema, fact & dimension modelling, SCD and CDC.
Experience building data platforms for BI, reporting and analytics.
Exposure to AI development tools such as Cursor is desirable.

Nice to Have

Azure certifications (AZ-900, DP-900, DP-203, DP-500).
Databricks Lakehouse Data Engineer certification.
MSc in Data Science or a related field.

Requirements

Strong experience with Databricks.
Excellent Python and SQL skills.
Good knowledge of Spark, PySpark, Delta Lake, Delta Live Tables, Unity Catalog and Medallion Architecture.
Experience with Azure DevOps, Git and CI/CD.
Strong understanding of data warehousing, star schema, fact & dimension modelling, SCD and CDC.
Experience building data platforms for BI, reporting and analytics.
Exposure to AI development tools such as Cursor is desirable.

Nice to Have