> Markdown version of [/jobs/ext/3309284-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/3309284-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** VanRath - **Location:** UK (Remote available) - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Artificial Intelligence, Amazon Web Services, Data Analysis, Big Data, Cloud Computing, Data Architecture, Information Engineering, Data Governance, Data Infrastructure, Extract Transform Load (ETL), Data Warehousing, Python (Programming Language), Apache Spark, Data Lakes, Pyspark, AWS Glue, Data Analytics, Data Pipelines, Databricks - **Published:** September 11, 2026 - **Apply:** https://www.vanrath.com/job/lead-data-engineer-bbbh15929-1788947652/apply ## About the Role An established and growing organisation is investing heavily in its Data & Analytics capability and is now seeking an experienced Data Engineering Team Lead to join its team. This is a fully remote opportunity for an experienced data professional with a proven track record of leading Data Engineering teams, providing technical direction and delivering scalable data solutions across a complex organisation., You will be an experienced Data Engineering Team Lead, Data Engineering Manager or Senior Data Engineering professional with existing team leadership experience. You must be based in Northern Ireland and have unrestricted eligibility to work in the UK without sponsorship. This is a mandatory requirement for the position. You will ideally have: * Proven experience leading and developing a Data Engineering team * Strong commercial experience within Data Engineering, BI or Data & Analytics * Strong technical expertise across modern data engineering environments * Experience with Databricks and Python/PySpark * Strong knowledge of data warehousing, data modelling and big data concepts * Experience developing scalable ETL/ELT processes and data pipelines * Strong understanding of Delta Lake and modern data platform architecture * Experience working with cloud technologies, ideally AWS * Experience providing technical direction and setting engineering standards * Strong stakeholder management and communication skills * Excellent problem-solving and analytical abilities * The ability to balance strategic thinking, team leadership and hands-on technical delivery Experience with AWS Glue, APIs, advanced analytics, AI/ML, data governance or Agile/SCRUM environments would be advantageous., This role would suit an experienced Data Engineering Team Lead who is comfortable leading people, influencing stakeholders and remaining technically engaged, with the opportunity to make a significant impact on the organisation's wider Data & Analytics strategy. ## Description You will work closely with colleagues across Data, Analytics, IT and the wider business, providing technical leadership and ensuring the Data Engineering team delivers robust, scalable and high-quality solutions., As Data Engineering Team Lead, you will be responsible for providing both people and technical leadership across the Data Engineering function. Your responsibilities will include: * Leading, supporting and developing a team of Data Engineers * Setting technical direction and establishing best practice across the team * Managing priorities and ensuring effective delivery of data engineering projects * Designing, building and maintaining scalable data pipelines and ETL/ELT processes * Working hands-on with Databricks, Delta Lake, Python/PySpark and cloud technologies * Contributing to the development and evolution of the organisation's data architecture * Developing robust data models to support BI, reporting and analytics * Monitoring and optimising data pipelines, Spark jobs and workflows * Ensuring appropriate standards around data quality, governance, security and compliance * Working closely with BI, Analytics, IT and business stakeholders * Translating business requirements into effective and scalable data solutions * Supporting advanced analytics, AI and machine learning initiatives * Identifying opportunities to improve data engineering capability, processes and performance * Acting as a technical point of escalation and providing guidance on complex engineering challenges * Building a culture of collaboration, knowledge sharing and continuous improvement within the team ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [How we built an AI-powered code reviewer in 80 hours](https://www.wearedevelopers.com/videos/1511-how-we-built-an-ai-powered-code-reviewer-in-80-hours) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs) - [Fullstack Developer Salary UK](https://www.wearedevelopers.com/magazine/251-fullstack-developer-salary-uk) - [Software Engineer Salary in The UK](https://www.wearedevelopers.com/magazine/231-software-engineer-salary-in-the-uk) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)