> Markdown version of [/jobs/ext/1870949-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/1870949-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer

- **Company:** 17918
- **Location:** London, UK
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Agile Methodology, Cloud Computing, System Configuration, Continuous Integration, Data Infrastructure, Data Structures, Linux, Document-Oriented Databases, Python (Programming Language), NoSQL, Performance Tuning, Query Optimization, Rapid Application Development, Ansible, Data Processing, Scripting, Test-Driven Development (TDD), Infrastructure Automation Frameworks, Integration Frameworks, Real Time Data, Terraform, Data Pipelines, Serverless Computing, Jenkins
- **Published:** August 1, 2026
- **Apply:** https://www.apply4u.co.uk/jobs/x/42521452/

## About the Role

improvement Skills and Experience Essential Experience building data pipelines using modern data processing frameworks Familiarity with Agile or rapid application development methodologies Strong understanding of data modelling and different data structures, including when to apply them Hands-on experience with public cloud platforms, ideally including services such as compute, storage, serverless and data processing tools Experience configuring and tuning relational and NoSQL databases, including query optimisation Proficiency in programming or scripting languages such as Python Experience working with Test Driven Development practices and supporting tools Desirable Familiarity with CI tools such as Jenkins or Drone Knowledge of configuration management tools such as Ansible or Packer Experience using Infrastructure as Code tools, for example Terraform Working knowledge of Linux and Windows environments Ways of Working The organisation promotes a collaborative and delivery-focused
culture. Team members are encouraged to take ownership, experiment, solve complex problems and support each other in achieving shared goals. Open, timely communication and accountability are key values. Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community we have built, and we welcome applications from people of all backgrounds and perspectives. Our success is driven by our people, united by the spirit of partnership to deliver the best resourcing solutions for our clients. If you need any help or adjustments during the recruitment process for any reason , please let us know when you apply or talk to the recruiters directly so we can support you. TPBN1_UKTJ

## Description

Lead Data Engineer The Role As a Lead Data Engineer, you will play a key role in building and enhancing a secure, scalable data platform for a high-profile government programme. You will work with both batch and real-time data, leveraging modern cloud technologies and frameworks. This role involves ingesting data from multiple sources, transforming it into the correct formats, ensuring alignment with defined data models and enabling reliable access for downstream users. Key Responsibilities Design, develop, test and document data pipelines aligned to business requirements Perform logical and physical database design Enhance data platform capabilities to address evolving data and analytics challenges Analyse data workflows and identify opportunities for optimisation and improvement Ensure consistency between data models and overall architecture, including entity definitions and relationships Conduct root cause analysis on data issues to support business decision-making and continuous


## Related Videos

- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [The Road to MLOps: How Verivox Transitioned to AWS](https://www.wearedevelopers.com/videos/1050-the-road-to-mlops-how-verivox-transitioned-to-aws)
- [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis)
- [Docker network without Docker](https://www.wearedevelopers.com/videos/1418-docker-network-without-docker)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Our GitOps approach for deploying an Identity Provider and an API Gateway in a SaaS company](https://www.wearedevelopers.com/videos/776-our-gitops-approach-for-deploying-an-identity-provider-and-an-api-gateway-in-a-saas-company)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)
- [Best Companies to work for in London: Top 25 Companies in 2023](https://www.wearedevelopers.com/magazine/187-best-companies-to-work-for-in-london-top-25-companies-in-2023)
- [Fullstack Developer Salary UK](https://www.wearedevelopers.com/magazine/251-fullstack-developer-salary-uk)
- [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)