Senior Databricks Engineer - £70,000 - Hybrid
Role details
Job location
Tech stack
Job description
We're looking for a hands-on Senior Databricks Engineer to lead the delivery of scalable data solutions within an Agile environment. Working closely with the Data Product Manager and Data Architect, you will shape and develop our data platform, delivering high-quality pipelines and insights that support strategic decision-making. You will also manage and coach a small team of Data Engineers, driving best practice, consistency, and governance.
Key Responsibilities
Translate business strategy into data solutions and ensure alignment with product goals.
Provide technical leadership, breaking initiatives into Features, Epics, and Stories and setting engineering standards.
Collaborate with the Data Architect to design and implement data architecture and build plans.
Build and maintain scalable data pipelines, ETL/ELT processes, and large-scale data workflows.
Optimise data systems for performance, reliability, and scalability.
Implement data quality processes and maintain data models, schemas, and documentation.
Operate CI/CD practices in Azure DevOps and contribute to Agile sprint cycles.
Troubleshoot and resolve pipeline issues promptly.
Stay current with industry trends and recommend improvements.
Ensure adherence to governance standards.
Line-manage and mentor a small team of Data Engineers.
Requirements
Extensive Databricks experience, including Unity Catalog.
Strong skills in Python, Spark, SQL and experience with SQL databases.
Terraform experience for cloud infrastructure as code.
Experience with Azure and workflow tools (Airflow, ADF).
Excellent problem-solving ability, communication skills, and attention to detail.
Experience across Waterfall and Agile methodologies.
Curious, inclusive, and committed to continuous learning.
