> Markdown version of [/jobs/ext/2821680-lead-data-engineer-databricks](https://www.wearedevelopers.com/jobs/ext/2821680-lead-data-engineer-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer (Databricks)

- **Company:** United Utilities
- **Location:** Warrington, UK
- **Experience:** Expert
- **Salary:** £21,840.0
- **Contract:** Permanent contract
- **Skills:** Unity 3d, Application Programming Interfaces (APIs), Airflow, Amazon Web Services, Architectural Patterns, Automation of Tests, Microsoft Azure, Continuous Integration, Information Engineering, Data Governance, Extract Transform Load (ETL), Python (Programming Language), Standard Sql, Data Streaming, Workflow Management Systems, Apache Spark, Data Lakes, Pyspark, Git Flow, Information Technology, Software Version Control, Data Pipelines, Databricks
- **Published:** September 10, 2026
- **Apply:** https://www.collegerecruiter.com/job/2860993882-lead-data-engineer-databricks

## About the Role

* Deep expertise in Databricks (Spark / PySpark, Delta Lake, Unity Catalog, Workflows)
 * Strong experience building ETL/ELT pipelines, both batch and streaming
 * Advanced Python and SQL skills
 * Hands-on experience with cloud platforms (Azure, AWS or GCP)
 * Solid understanding of data modelling and modern architectures
 * Strong knowledge of data governance, security and compliance
 * Experience with orchestration tools such as Airflow
 * Familiarity with CI/CD pipelines, Git workflows and automated testing
 * Exposure to real-time, IoT or environmental telemetry data

Experience & Qualifications

 * Bachelor's degree in Computer Science, Data Engineering or a related discipline
 * 5+ years' experience in data engineering roles, including 3+ years with Databricks

Proven ability to lead technically and mentor engineers

## Description

As the Lead Data Engineer on our River Water Quality Programme, you'll design, build, and optimise Databricks-based data pipelines that enable timely, reliable and well-governed data across the organisation.

You'll:

 * Design and develop scalable batch and streaming data pipelines using Databricks, Spark and Delta Lake
 * Optimise Spark jobs and workflows to meet near real-time monitoring SLAs
 * Build and maintain streaming pipelines using Structured Streaming and Delta Live Tables
 * Ensure strong data governance, including schema enforcement, validation rules and ACID compliance
 * Orchestrate and automate pipelines using Databricks Jobs, APIs or cloud-native schedulers
 * Collaborate with data scientists, analysts and environmental specialists to deliver high-quality, well-documented datasets
 * Monitor, troubleshoot and continuously improve production pipelines for reliability, performance and cost efficiency
 * Champion best practices across CI/CD, testing, version control and documentation

Act as a technical leader and mentor, raising standards across the data engineering community


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Reality and Beyond: Coding a Drone Using {Unity 3D .NET} and ChatGPT AI!](https://www.wearedevelopers.com/videos/702-reality-and-beyond-coding-a-drone-using-unity-3d-net-and-chatgpt-ai)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Software Engineer Salary in The UK](https://www.wearedevelopers.com/magazine/231-software-engineer-salary-in-the-uk)
- [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)
- [Fullstack Developer Salary UK](https://www.wearedevelopers.com/magazine/251-fullstack-developer-salary-uk)