Senior Data Engineer

Exciting opportunity for a Senior Data Engineer to own the maintenance, expansion, and performance of Databricks-based data platform. This is a hands-on role, you’ll be writing pipeline code, building jobs, and modelling as the main part of your day to day.

Role Responsibilities

Responsibilities will include:

• Designing, building, and operating Databricks Pipelines and Jobs
• Designing and building Delta Lake pipelines
• Owning Databricks deployment bundles, packaging, versioning, and promoting pipeline code and job definitions
• Building and maintaining environment-variable-driven Azure Data Factory pipelines
• Building the data layer that feeds Microsoft Fabric/OneLake

Essential skills will include:

• Hands-on experience building and running Databricks Pipeline/Job structure in a production
• Strong data modelling ability, able to look at raw source data and design sensible, well-structured silver/gold tables
• Strong understanding of medallion (bronze/silver/gold) design patterns, and data architecture right at table level
• Solid Delta Lake knowledge, deletion vectors, time travel, optimization, compatibility considerations
• Experience with Databricks Asset Bundles or equivalent deployment/CI practices for data pipelines
• Practical experience building environment-variable/parameter-driven Azure Data Factory pipelines
• Proven experience of managing large data architectures
• Strong SQL, and comfort in at least one of Python/Scala for pipeline logic

You will be the technical expert for the data solution, to work alongside the engineering lead to manage the data aspects of the digital solution.