> Markdown version of [/jobs/ext/2806609-director-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/2806609-director-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Director: Lead Data Engineer

- **Company:** Projectsparticipate
- **Location:** London, UK
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Algorithmic Trading, Amazon Web Services, Microsoft Azure, Computer Programming, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Security, Data Visualization, Data Warehousing, Python (Programming Language), Open Source Technology, Power BI, Data Ingestion, Azure Data Factory, Apache Spark, Information Technology, Data Analytics, Apache Kafka, Video Streaming, Stream Processing, Data Pipelines, Databricks
- **Published:** September 9, 2026
- **Apply:** https://www.apply4u.co.uk/jobs/director-lead-data-engineer/46480820

## About the Role

Section / Function:Insert Section/FunctionTravel Required:NoSMCR Status:N/AEffective from:Dual Hat Department:Non-employing entity Dual Hat DepartmentDual Hat Reports to:Non-employing entity Manager's Job Rank/Corporate title and functionaltitleDual Hat SMCR:Client Dealing Function, Material Risk Takers, Manager of Certified Employees, Significant Management Function, CASS Oversight function, Algorithmic trading or N/AProfile SummaryWe are seeking a highly skilled and experienced Lead Data Engineer to join our talented team. The ideal candidate will have a strong background in data engineering, with experience in AI and a proven track record of designing and implementing robust data pipelines.ResponsibilitiesLead the design, development, and maintenance of scalable and efficient data pipelines to support our data-driven applicationsCollaborate with cross-functional teams, including data scientists, software engineers, and product managers, to understand data requirements and deliver
high-quality solutionsArchitect and implement data ingestion, transformation, and storage processes using tools such as Apache Spark, Azure Data Factory, and other similar technologiesOptimize data pipeline performance, ensuring data accuracy, reliability, and timely deliveryMentor and guide junior data engineers, fostering a culture of continuous learning and improvementStay up-to-date with the latest trends and best practices in data engineering and AI, and apply them to our projectsParticipate in the evaluation and selection of new technologies and tools to enhance our data engineering capabilitiesContribute to the development of AI-powered features and products, leveraging your knowledge of machine learning and data science conceptsRequired QualificationsBachelor's or Master's degree in Computer Science, Engineering, or a related field7+ years of experience in data engineering, with at least 3 years in a leadership roleStrong proficiency in Apache Spark, Azure Data Factory, and
other data pipeline toolsExperience with cloud platforms such as Azure, AWS, or GCPSolid understanding of data modeling, data warehousing, and ETL/ELT processesFamiliarity with AI and machine learning concepts, with hands\u2011on experience in implementing AI-driven solutionsExcellent problem-solving skills and the ability to design and optimize data architecturesStrong programming skills in languages such as Python, Scala, or JavaExperience with data quality, data governance, and data security best practicesExcellent communication and collaboration skills, with the ability to work effectively in a team environmentAbility to thrive in a fast-paced, dynamic environment and manage multiple prioritiesPreferred QualificationsAdvanced degree in Computer Science, Data Science, or a related fieldCertifications in relevant technologies, such as Azure Data Engineer or Databricks Certified DeveloperExperience with real-time data processing and streaming technologies like Apache Kafka or Azure
Event HubsKnowledge of data visualization tools, such as Power BI or TableauContributions to open-source projects or active participation in the data engineering community #J-18808-Ljbffr



## Related Videos

- [Beyond Dashboards: Fixing Text-to-SQL with Semantic RAG](https://www.wearedevelopers.com/videos/2036-beyond-dashboards-fixing-text-to-sql-with-semantic-rag)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source)
- [Enabling intelligent logistics automation: home-grown Industrial IoT platform at Austrian Post](https://www.wearedevelopers.com/videos/2018-enabling-intelligent-logistics-automation-home-grown-industrial-iot-platform-at-austrian-post)
- [OLTP in the Lakehouse: Redefining Data for AI Workloads](https://www.wearedevelopers.com/videos/2038-oltp-in-the-lakehouse-redefining-data-for-ai-workloads)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)
- [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)