> Markdown version of [/jobs/ext/1245549-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/1245549-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** Hexwired Recruitment Limited
- **Location:** London, UK
- **Experience:** Expert
- **Salary:** £80,000.0 - £115,000.0
- **Contract:** Permanent contract
- **Skills:** JavaScript (Programming Language), Artificial Intelligence, Airflow, Amazon Web Services, Automated Storage and Retrieval Systems, Microsoft Azure, Big Data, Cloud Computing, Information Engineering, Data Integrity, Relational Databases, Distributed Data Store, Elasticsearch, Monitoring of Systems, Python (Programming Language), PostgreSQL, Machine Learning, Node.Js, NoSQL, Performance Tuning, Search Technologies, Data Ingestion, Apache Spark, Database Performance, Backend, Pandas, Event Driven Architecture, Build Management, Storage Technologies, Apache Kafka, Stream Processing, Data Pipelines
- **Published:** July 12, 2026
- **Apply:** https://find.jobs/jobs-near-me/apply/ats-redirect/?id=2869613465-2

## About the Role

* 7+ years' experience in Data Engineering or Backend Engineering.
 * Strong Python development experience.
 * Experience building distributed data platforms and scalable pipelines.
 * Excellent knowledge of PostgreSQL (or similar relational databases) and NoSQL databases.
 * Experience with vector databases such as Qdrant, Milvus or pgvector.
 * Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars.
 * Strong understanding of data modelling, storage architecture and performance optimisation.

Desirable

 * Experience working on AI or Machine Learning platforms.
 * Knowledge of cloud platforms (AWS, Azure or GCP).
 * JavaScript/Node.js experience.
 * Event-driven architecture and stream processing experience.
 * Previous experience within a high-growth start-up environment.

## Description

* Design and build scalable batch and real-time data pipelines.
 * Develop data ingestion, transformation and storage solutions for operational and AI workloads.
 * Build and optimise vector search, retrieval systems and ML data pipelines.
 * Ensure data reliability, governance, monitoring and observability across the platform.
 * Work closely with AI, backend and product teams to support model training, inference and product development.
 * Optimise large-scale datasets and database performance.
 * Help define engineering standards and mentor future data engineers.


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis)
- [Advanced Typing in TypeScript](https://www.wearedevelopers.com/videos/496-advanced-typing-in-typescript)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment)
- [Data Science on Software Data](https://www.wearedevelopers.com/videos/162-data-science-on-software-data)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)
- [7 Most Popular Web Developer Jobs in Europe](https://www.wearedevelopers.com/magazine/163-7-most-popular-web-developer-jobs-in-europe)
- [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)