Lead Data Engineer

You’ll join the Planning Inspectorate’s Data Service as a Lead Data Engineer, working within a co-leadership model alongside the Service Manager and another Lead Data Engineer.

The Data Service underpins how data is shared, trusted and used across the organisation. In this role, you’ll lead the design and delivery of high quality data engineering solutions, including building and maintaining data pipelines and platforms that support analytics, decision making and operational insight. You’ll also contribute to the development of a strong, sustainable data engineering capability across the organisation.

You’ll play a hands-on technical leadership role - working with others to shape solutions, guide delivery, and provide technical oversight, while also supporting and developing data engineers through coaching and day-to-day collaboration. Working closely with your peers, you’ll help ensure the service operates effectively, manages technical risk, and continues to improve., We’re looking for a Lead Data Engineer who brings technical experience and enjoys supporting delivery through collaboration. You’ll bring solid technical experience, sound engineering judgement, and the ability to support and develop others.

You should be comfortable contributing to complex data engineering work, working collaboratively within a shared leadership structure, and contributing to technical decision-making as part of a shared leadership model.

You’ll take a constructive, coaching-oriented approach to leadership, be confident working with a range of stakeholders, and be motivated by building reliable, well governed data solutions in a public sector context.

You’ll lead and support the design and delivery of data engineering solutions, working with the Data Service team to build, operate and improve enterprise scale data pipelines and platforms.

A significant part of your role will be supporting and developing other data engineers through technical leadership, code and design assurance, coaching and day-to-day guidance. You’ll contribute to setting and maintaining engineering standards and contribute to continuous improvement of ways of working.

You’ll work closely with the Service Manager and the other Lead Data Engineer to plan and prioritise work, manage technical risks, and engage with stakeholders to ensure solutions meet user needs and align with the wider Data Service roadmap., * Technical Example: Provide an example of a data engineering solution you have designed, built, or significantly contributed to using Python and PySpark. We are interested in how you approach data engineering in practice, including design decisions, use of tooling, and how you ensure quality and reliability. There is a 250 word limit.

All shortlisted candidates will be advised of the outcome shortly afterwards.

Interview

If you are shortlisted, you will:

• Be asked to attend an interview to have a more in-depth discussion about your experience and suitability for the role
• Interviews will be a blend of behaviour, experience and technical questions

Candidates who pass the shortlist stage will receive full assessment details with their interview invitations., * UK nationals

• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Do you have experience in Scalability?, * Experience working with various Azure services and technologies relevant to data engineering, such as: Python, PySpark, Data Factory, Databricks, Synapse Analytics, SQL Database and Data Lake Storage. Understanding of how to leverage these services to build scalable and robust data solutions.

• Knowledge of data warehousing principles and experience in designing and implementing data warehouse solutions using technologies like Azure Synapse Analytics (formerly Azure SQL Data Warehouse).
• Proficiency in Python and SQL. Ability to write efficient and maintainable code for data ingestion, transformation, and manipulation.
• Understanding of data governance principles, data security, and privacy requirements. Knowledge of Azure security features, data encryption, access controls, and compliance standards like GDPR (General Data Protection Regulation) and Data Protection Act.
• Experience with DevOps practices, including continuous integration and continuous deployment (CI/CD), version control systems (e.g., Git), and infrastructure-as-code concepts. Knowledge of Azure DevOps or similar tools for managing and automating data engineering pipelines.
• Experience identifying and resolving data engineering issues, optimize data pipelines, and ensure high performance and scalability of data solutions. Familiarity with monitoring tools and techniques to identify and address bottlenecks or inefficiencies., * Leadership
• Making Effective Decisions
• Working Together

Technical skills

We'll assess you against these technical skills during the selection process:

• Data engineering solution, This vacancy is using Success Profiles , and will assess your Behaviours, Experience and Technical skills., Successful candidates must undergo a criminal record check. People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Pulled from the full job description

• Annual leave
• Company pension
• Flexible schedule, New entrants to the Civil Service are expected to start on the minimum of the pay band. Internal rules apply to existing Civil Servants, i.e. level transfers move on current salary or the pay range minimum, transfers on promotion move to new pay range minimum or receive 10% increase. Either case is determined by whichever is the highest.

DDAT allowance of up to £20,756 based on Capability and base salary

Working hours:

Standard working week is 37 hours, flexible working hours policy available

Part time at 35 hours or above, flexible working hours policy available

The Planning Inspectorate has a long and proud history in ensuring a fair planning system for England. The work we do has a significant impact on people’s lives, the communities where they live and the economy.

We want our colleagues to be able to work more flexibly and more collaboratively, exploring new and innovative ways to improve the way we provide services., Alongside your salary of £59,644, Planning Inspectorate contributes £17,278 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.

• Flexible working options, including flexi-time, part-time, compressed hours and home working if suitable for the role.
• A working culture which encourages inclusion and diversity.
• 25 days annual leave increasing by 1 day per year up to 30 days after 5 years, plus 1 privilege day to mark the King’s birthday. In addition to the normal bank holidays.
• Generous family friendly policies.
• Fast and modern IT kit which utilises Windows 11 and Microsoft 365, enabling seamless working from home and collaboration across teams., Interviews will be held at The Planning Inspectorate, Bristol, further details will be sent with the invite to interview.

The Planning Inspectorate does not fund travel and subsistence for candidates to attend interviews

Reserve list

A reserve list may be held for a period of 12 months from which further appointments can be made.

Future Matches

Candidates who demonstrate strong potential and future readiness but score just below the required level at interview may be considered for alternative positions. These roles may be at a lower grade but align with their skills and offer opportunities for growth.

We encourage all candidates to make themselves familiar with the role and what we will be assessing you against which is detailed in the Person Specification.

ID and Documents

If the job needs specific qualifications or memberships and you get the job, you'll need to send scanned or photographed images of them before you start.

We must check that any successful candidates are allowed to work for the Civil Service before we employ them. If successful, you will be asked to do a right to work check using identity document validation technology (IDVT) through the services of an Identity Service Provider (IDSP), Civil Service Jobs. If we are unable to confirm your right to work and through the IDVT we will need to do a manual right to work check using original documents with you in person.

Visa Sponsorship

The Planning Inspectorate cannot offer Visa sponsorship to candidates, we do not hold a Visa sponsorship licence.