> Markdown version of [/jobs/ext/69788-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/69788-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** Planning Inspectorate - **Location:** Bristol, UK (Remote available) - **Experience:** Expert - **Salary:** £20,756.0 - £59,644.0 - **Contract:** Permanent contract - **Skills:** Clean Code Principles, Microsoft Azure, Continuous Integration, Information Engineering, Data Governance, Data Security, Data Systems, Data Warehousing, DevOps, Monitoring of Systems, Python (Programming Language), SQL Azure, SQL Databases, Data Ingestion, Git, Data Lakes, Pyspark, Azure Synapse Analytics, Software Version Control, Data Pipelines, Databricks - **Published:** May 31, 2026 - **Apply:** https://uk.indeed.com/viewjob?jk=1b9751ebb6a4376f ## About the Role Do you have experience in Scalability?, * Experience working with various Azure services and technologies relevant to data engineering, such as: Python, PySpark, Data Factory, Databricks, Synapse Analytics, SQL Database and Data Lake Storage. Understanding of how to leverage these services to build scalable and robust data solutions. * Knowledge of data warehousing principles and experience in designing and implementing data warehouse solutions using technologies like Azure Synapse Analytics (formerly Azure SQL Data Warehouse). * Proficiency in Python and SQL. Ability to write efficient and maintainable code for data ingestion, transformation, and manipulation. * Understanding of data governance principles, data security, and privacy requirements. Knowledge of Azure security features, data encryption, access controls, and compliance standards like GDPR (General Data Protection Regulation) and Data Protection Act. * Experience with DevOps practices, including continuous integration and continuous deployment (CI/CD), version control systems (e.g., Git), and infrastructure-as-code concepts. Knowledge of Azure DevOps or similar tools for managing and automating data engineering pipelines. * Experience identifying and resolving data engineering issues, optimize data pipelines, and ensure high performance and scalability of data solutions. Familiarity with monitoring tools and techniques to identify and address bottlenecks or inefficiencies., * Leadership * Making Effective Decisions * Working Together Technical skills We'll assess you against these technical skills during the selection process: * Data engineering solution, This vacancy is using Success Profiles , and will assess your Behaviours, Experience and Technical skills., Successful candidates must undergo a criminal record check. People working with government assets must complete baseline personnel security standard (opens in new window) checks. ## Description You’ll join the Planning Inspectorate’s Data Service as a Lead Data Engineer, working within a co-leadership model alongside the Service Manager and another Lead Data Engineer. The Data Service underpins how data is shared, trusted and used across the organisation. In this role, you’ll lead the design and delivery of high quality data engineering solutions, including building and maintaining data pipelines and platforms that support analytics, decision making and operational insight. You’ll also contribute to the development of a strong, sustainable data engineering capability across the organisation. You’ll play a hands-on technical leadership role - working with others to shape solutions, guide delivery, and provide technical oversight, while also supporting and developing data engineers through coaching and day-to-day collaboration. Working closely with your peers, you’ll help ensure the service operates effectively, manages technical risk, and continues to improve., We’re looking for a Lead Data Engineer who brings technical experience and enjoys supporting delivery through collaboration. You’ll bring solid technical experience, sound engineering judgement, and the ability to support and develop others. You should be comfortable contributing to complex data engineering work, working collaboratively within a shared leadership structure, and contributing to technical decision-making as part of a shared leadership model. You’ll take a constructive, coaching-oriented approach to leadership, be confident working with a range of stakeholders, and be motivated by building reliable, well governed data solutions in a public sector context. You’ll lead and support the design and delivery of data engineering solutions, working with the Data Service team to build, operate and improve enterprise scale data pipelines and platforms. A significant part of your role will be supporting and developing other data engineers through technical leadership, code and design assurance, coaching and day-to-day guidance. You’ll contribute to setting and maintaining engineering standards and contribute to continuous improvement of ways of working. You’ll work closely with the Service Manager and the other Lead Data Engineer to plan and prioritise work, manage technical risks, and engage with stakeholders to ensure solutions meet user needs and align with the wider Data Service roadmap., * Technical Example: Provide an example of a data engineering solution you have designed, built, or significantly contributed to using Python and PySpark. We are interested in how you approach data engineering in practice, including design decisions, use of tooling, and how you ensure quality and reliability. There is a 250 word limit. All shortlisted candidates will be advised of the outcome shortly afterwards. Interview If you are shortlisted, you will: * Be asked to attend an interview to have a more in-depth discussion about your experience and suitability for the role * Interviews will be a blend of behaviour, experience and technical questions Candidates who pass the shortlist stage will receive full assessment details with their interview invitations., * UK nationals * nationals of the Republic of Ireland * nationals of Commonwealth countries who have the right to work in the UK * nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) * nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS) * individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020 * Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Software Engineer Salary in The UK](https://www.wearedevelopers.com/magazine/231-software-engineer-salary-in-the-uk) - [Fullstack Developer Salary UK](https://www.wearedevelopers.com/magazine/251-fullstack-developer-salary-uk) - [Frontend Developer Salary in The UK](https://www.wearedevelopers.com/magazine/233-frontend-developer-salary-in-the-uk)