Senior Data Engineer

Careerwise
Perivale, United Kingdom
2 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior
Compensation
£ 80K

Job location

Perivale, United Kingdom

Tech stack

Artificial Intelligence
Azure
Continuous Integration
Data Infrastructure
ETL
Data Warehousing
Cursor (Graphical User Interface Elements)
Python
Standard Sql
Spark
GIT
Microsoft Fabric
Data Lake
PySpark
Star Schema
Data Management
Data Pipelines
Databricks

Job description

We're looking for a Senior Data Engineer to join a growing AI & Data team, building modern data platforms using Databricks and Microsoft Fabric. You'll design and develop scalable data pipelines, data models and lakehouse solutions that support reporting, analytics and AI initiatives., * Lead data pipeline architecture development/orchestration/maintenance.

  • Develop robust data models and ETL/ELT processes.
  • Optimise data performance and troubleshoot pipeline issues.
  • Build and maintain data warehouse/lakehouse solutions.
  • Collaborate with cross-functional teams and document technical solutions.
  • Support best practices around data quality, governance and CI/CD.

Requirements

  • Strong experience with Databricks.
  • Excellent Python and SQL skills.
  • Good knowledge of Spark, PySpark, Delta Lake, Delta Live Tables, Unity Catalog and Medallion Architecture.
  • Experience with Azure DevOps, Git and CI/CD.
  • Strong understanding of data warehousing, star schema, fact & dimension modelling, SCD and CDC.
  • Experience building data platforms for BI, reporting and analytics.
  • Exposure to AI development tools such as Cursor is desirable.

Nice to Have

  • Azure certifications (AZ-900, DP-900, DP-203, DP-500).
  • Databricks Lakehouse Data Engineer certification.
  • MSc in Data Science or a related field.

Apply for this position