> Markdown version of [/jobs/ext/2216315-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/2216315-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** Understanding Recruitment
- **Location:** Greater London, UK
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Query Performance, Artificial Intelligence, Airflow, Data Analysis, Big Data, Cloud Computing, Data Architecture, Information Engineering, Data Infrastructure, Distributed Data Store, Elasticsearch, Python (Programming Language), NoSQL, Software Product Management, Azure Machine Learning, Search Technologies, Apache Spark, Backend, Event Driven Architecture, Storage Technologies, Real Time Data, Apache Kafka, Machine Learning Operations, Data Pipelines, Automation Anywhere
- **Published:** August 25, 2026
- **Apply:** https://www.understandingrecruitment.com/job/lead-data-engineer-9202/

## About the Role

* Strong experience in data engineering or backend engineering
 * Experience designing scalable pipelines and distributed data systems
 * Strong Python skills
 * Experience with relational and NoSQL databases
 * Hands-on experience with vector databases
 * Good understanding of data modelling, performance and storage architecture
 * Experience with technologies such as Spark, Airflow, Kafka or Elasticsearch/OpenSearch

Experience with AI/ML platforms, cloud infrastructure or event-driven architectures would be useful, but you don't need everything on the list.

## Description

Lead Data Engineer

Want to own the data architecture behind an AI product while the foundations are still being built?

This is an early-stage lead role where you'll shape the systems powering real-time data, AI workflows, vector search and analytics - with the freedom to make the key technical decisions.

What's in it for you?

 * Own the data architecture and technical direction
 * Build the data backbone of an AI-first product from the ground up
 * Work closely with AI, backend and product engineers
 * Solve problems across streaming, vector search and ML data infrastructure
 * Help define tooling, standards and the future data team
 * Competitive salary + equity
 * Join early enough to have genuine influence over how things are built

What you'll be doing

 * Architecting scalable batch and real-time data pipelines
 * Designing ingestion, transformation and storage systems
 * Building infrastructure for vector search, retrieval and ML workflows
 * Improving data quality, observability and reliability
 * Optimising large-scale datasets and query performance
 * Supporting AI training, inference and product features
 * Making long-term architecture and tooling decisions
 * Helping grow and mentor the data engineering function

What you'll bring

 * Strong experience in data engineering or backend engineering
 * Experience designing scalable pipelines and distributed data systems
 * Strong Python skills
 * Experience with relational and NoSQL databases
 * Hands-on experience with vector databases
 * Good understanding of data modelling, performance and storage architecture
 * Experience with technologies such as Spark, Airflow, Kafka or Elasticsearch/OpenSearch

Experience with AI/ML platforms, cloud infrastructure or event-driven architectures would be useful, but you don't need everything on the list.


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Developing the Backend with Stefan Lingler, CTO at Shpock](https://www.wearedevelopers.com/videos/100360-developing-the-backend-with-stefan-lingler-cto-at-shpock)
- [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [Nest.js - TypeScript in the backend can also be clean](https://www.wearedevelopers.com/videos/1033-nest-js-typescript-in-the-backend-can-also-be-clean)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [7 Most Popular Web Developer Jobs in Europe](https://www.wearedevelopers.com/magazine/163-7-most-popular-web-developer-jobs-in-europe)