> Markdown version of [/jobs/ext/2206297-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/2206297-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** FBI &TMT
- **Location:** Kingston upon Thames, UK (Remote available)
- **Experience:** Expert
- **Salary:** £65,000.0
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Amazon Web Services, Microsoft Azure, Data Architecture, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), Database Queries, Distributed Computing Environment, Python (Programming Language), Operational Databases, Azure Data Lake, Data Streaming, Azure Service Bus, Cloud Platform System, Azure Data Factory, Apache Spark, Cloudformation, Data Lakes, Information Technology, Bicep, Apache Kafka, Machine Learning Operations, Video Streaming, Terraform, Databricks
- **Published:** August 24, 2026
- **Apply:** https://www.totaljobs.com/job/lead-data-engineer/fbi-tmt-job107889911

## About the Role

* Strong experience as a senior or lead data engineer building and scaling production data platforms
 * Deep hands on expertise in Databricks including Spark, Delta Lake and Databricks Workflows
 * Python and SQL proficiency
 * Infrastructure as code experience using Terraform, Bicep or CloudFormation
 * Cloud data platform experience, ideally Azure including Azure Data Factory, ADLS Gen2 and Azure DevOps
 * Strong understanding of data modelling, ETL and ELT patterns, distributed processing
 * Experience with security, governance and data platform operations
 * Experience leading or mentoring engineers, with strong stakeholder management
 * Rigorous approach to data quality and testing
 * Databricks certification

Desirable

 * Unity Catalog, MLflow and Databricks Asset Bundles
 * dbt or similar transformation frameworks
 * Retail or CPG data such as POS, inventory, supply chain
 * Streaming technologies such as Kafka or Event Hubs

Education

 * Degree in computer science, data engineering or related field, or equivalent practical experience

## Description

* Lead the design, development and optimisation of a Databricks based data platform, ensuring reliability, performance and scalability
 * Design enterprise scale data architectures on Databricks and cloud platforms such as Azure, AWS or GCP
 * Define lakehouse standards, data modelling approaches and governance frameworks
 * Lead design of batch and streaming platforms using Delta Lake, Spark and related technologies
 * Establish security, access control, data quality and compliance standards
 * Optimise performance, scalability and cloud cost management
 * Lead and mentor a small team of data engineers, setting technical standards and reviewing code
 * Partner with data science, analytics and product teams to ensure the platform supports their work
 * Own data quality, governance and observability across pipelines
 * Support AI and ML enablement including feature stores and model deployment patterns
 * Translate business requirements into technical solutions
 * Contribute to platform roadmap, tooling selection and continuous improvement


## Related Videos

- [Back(end) to the Future: Embracing the continuous Evolution of Infrastructure and Code](https://www.wearedevelopers.com/videos/440-back-end-to-the-future-embracing-the-continuous-evolution-of-infrastructure-and-code)
- [Infrastructure as Code: The Developer's Secret Weapon](https://www.wearedevelopers.com/videos/1221-infrastructure-as-code-the-developer-s-secret-weapon)
- [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [OLTP in the Lakehouse: Redefining Data for AI Workloads](https://www.wearedevelopers.com/videos/2038-oltp-in-the-lakehouse-redefining-data-for-ai-workloads)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)