> Markdown version of [/jobs/ext/3764997-data-engineering-technical-lead](https://www.wearedevelopers.com/jobs/ext/3764997-data-engineering-technical-lead). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineering Technical Lead

- **Company:** Vanguard
- **Location:** Manchester, UK
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Amazon Web Services, Automation of Tests, Code Review, Continuous Integration, Data Architecture, Information Engineering, Data Systems, DevOps, Machine Learning, Performance Tuning, SQL Databases, Data Streaming, Cloud Platform System, Apache Spark, Generative AI, Git, Build Management, Data Lakes, Pyspark, AWS Glue, Data Analytics, AWS Data Analytics, Spark Streaming, Data Management, Machine Learning Operations, Functional Programming, Terraform, Stream Processing, Data Pipelines, Databricks
- **Published:** October 16, 2026
- **Apply:** http://www.vanguardjobs.com/job/23748719/data-engineering-technical-lead-manchester-gb/?utm_medium=%22mcloud%2Djobads%22&utm_campaign=Default%20Campaign&utm_content=Data%20Engineering%20Technical%20Lead&utm_term=181560

## About the Role

· Expert-level experience with Databricks, PySpark, SQL, and Apache Spark

· Strong experience designing and delivering cloud-based data platforms and lakehouse architectures

· Hands-on experience with Delta Lake, Unity Catalog, Databricks Workflows, and performance tuning

· Experience building batch and streaming data pipelines

· Strong knowledge of data modelling and data product design

· Experience with Terraform, Git, CI/CD, and automated testing

· Proven leadership and stakeholder management skills

.

Nice-to-Have Skills & Experience

· AWS data platform experience

· Experience with Delta Live Tables (DLT) and MLflow

· Knowledge of AI/ML, MLOps, and Generative AI solutions on Databricks

· Financial Services industry experience

Special Factors

## Description

The Data Engineering Technical Lead will play a critical role in shaping and delivering Vanguard Europe's data and analytics capabilities. Working within the Chief Data & Analytics Office (CDAO), this role will lead the technical design, development, and evolution of data products that support business decision-making across Financial Advice Services, Marketing, Operations, Risk, and other key business functions.

Lead the design, development, and optimization of scalable data platforms and data products on AWS tech stack and Databricks. Drive engineering best practices, mentor data engineers, and deliver high-quality data solutions that support analytics, reporting, and AI initiatives.

In this role you will

· Design and build scalable data pipelines using Databricks, PySpark, SQL, AWS Glue and Lambda.

· Develop and manage Data Lake solutions using Bronze, Silver, and Gold data architectures.

· Implement batch and real-time data processing using Spark Structured Streaming and Auto Loader.

· Optimize Databricks workloads for performance, scalability, reliability, and cost efficiency.

· Establish data quality, governance, and security controls using Unity Catalog and enterprise best practices.

· Lead technical delivery, code reviews, and engineering standards across the data engineering team.

· Collaborate with Product Managers, Architects, Analysts, and business stakeholders to deliver data products.

· Implement CI/CD and Infrastructure as Code using Terraform and modern DevOps practices.

· Mentor engineers and promote best practices in Databricks and data engineering.