Data Engineer

DWPs Digital Group is looking for experienced Data Engineers to fill post in Management Information (MI) team.

The MI teamprovides strategic capabilities for MI reporting and data visualisations across DWP. Working with modern cloud environments alongside legacy on-premises technologies, youll play a key role in developing the DWP Data Engineering Strategy for MI products and services.

Successful candidates will join a fast-paced environment with a wide portfolio of products for analytical use and a challenging delivery roadmap. Youll focus on a variety of tasks, supporting platform live service and delivering high quality, data transformation products., * Building and maintain scalable data engineering solutions that power analytical reporting, working in line with DWPs engineering standards.

• Collaborating with stakeholders across DWP to gather and clarify requirements where needed and translate them into scalable back-end data processes that support MI reporting needs.
• Maintaining comprehensive technical documentation and collaborative workspaces in Confluence to improve transparency, knowledge sharing, and onboarding.
• Working closely with other junior data engineers to provide guidance, share technical expertise, and actively support ongoing knowledge sharing.
• Reviewing code developed by fellow engineers to ensure quality, maintainability, and adherence to DWP coding standards., Civil Servants applying on promotion will usually be appointed to the salary minimum or within 10% of existing salary.

Any move to DWP from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may, however, be eligible for other government schemes, including Tax Free Childcare. Determine your eligibility at the Childcare Choices Website.

For further information on reasonable adjustments, terms and conditions and how we recruit visit the How we recruit page.

Feedback will only be provided if you attend an interview or assessment.

Security

Successful candidates must undergo a basic (or equivalent) criminal record check.

Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window) .

See our vetting charter (opens in a new window) .

People working with government assets must complete baseline personnel security standard (opens in new window) checks.

Nationality requirements

This job is broadly open to the following groups:

• UK nationals
• nationals of the Republic of Ireland
• nationals of Commonwealth countries who have the right to work in the UK
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window)
• nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)
• individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020
• Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements (opens in a new window)

Working for the Civil Service

The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission’s recruitment principles (opens in a new window) .

The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.

The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.

Diversity and Inclusion The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window) .

Apply and further information

This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative.

The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window) .

Are you an experienced Data Engineer with strong Analytical skills?, * Practitioner: Ability toorchestrate PySpark jobs on EMR, managing dependencies, workflow execution, and operational efficiency.

• Working: Knowledge of designingevent-driven architectures, leveragingAWS Lambda functionsto triggerPySpark workflowsbased on real-time data ingestion and processing needs.
• Practitioner: Experienced in provisioning cloud infrastructure using Amazon EC2, and automating deployment, configuration, and data workflows with Bash scripting across scalable environments.
• Working: Ability to applyAgile best practices in developing CI/CD pipelines, using continuous integration, testing, and deploymentwithin a collaborative development framework.
• Practitioner: Understanding ofoptimising queries, storage formats, and data structuresto enhance efficiency, reduce processing time, and ensure high data quality standards in complex cloud architectures.

1. Technical statement (up to 250 words). The following statement is aligned to the required technical skill of Technical Breadth. This statement should be used to provide example(s) of how you meet the criterion below: * Practitioner: Good understanding ofdesigning and automating ETL pipelinesusingPySpark and Spark SQL, ensuring seamless extraction, transformation, and loading of structured and unstructured data across cloud environments.

Ensure that all examples provided in your statements are taken directly from your own experience and that you describe the examples in your own words. If you choose to use gen-AI to support your statements, you must follow the guidelines outlined in the Artificial intelligence and recruitment guide., You must meet the security requirements before you can be appointed. The level of security needed is security check.

You must have resided continuously in the UK for at least 3 of the last 5 years, 2 of which must have been the immediately preceding years from the point of applying for this job.

Alongside your salary of 44,447, Department for Work and Pensions contributes 12,876 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides (opens in a new window).

We also have a broad benefits package built around your work-life balance which includes:

• Working patterns to support work/life balance such as job sharing, term-time working, flexi-time and compressed hours.
• Generous annual leave at least 25 days on entry, increasing up to 30 days over time (prorata for part time employees), plus 9 days public and privilege leave.
• Support for financial wellbeing, including interest-free season ticket loans for travel, a cycle to work scheme and an employee discount scheme.
• Health and wellbeing support including our Employee Assistance Programme for specialist advice and counselling and the opportunity to join HASSRA a first-class programme of competitions, activities and benefits for its members (subscription payable monthly).
• Family friendly policies including enhanced maternity and shared parental leave pay after 1 years continuous service.
• Funded learning and development to support progress in your role and career. This includes industry recognised qualifications and accreditations, coaching, mentoring and talent development programmes.
• An inclusive and diverse environment with opportunities to join professional and interpersonal networks including Womens Network, National Race Network, National Disability Network (THRIVE) and many more.

Hybrid Working

This role may be suitable for hybrid working, which is where an employee works part of the week in their DWP office and part of the week from home. This is a voluntary, non-contractual arrangement and your office will be your contractual place of work.

If a hybrid working arrangement is suitable for the role and for you, you will normally be required to spend a minimum of 60% of your contracted working hours from your DWP office., Pay for this role in National locations is from 44,447 to 54,813.

The maximum salary for the grade is 46,547, however a Digital Allowance of up to 8,266 per annum is available for exceptional candidates. Digital Allowance is non-salary, and non-pensionable, and is classed as a temporary allowance. It is reviewed annually where it could be retained, amended, or removed.

Pay for this role in London is from 50,044 to 51,528.

Our offer to successful candidates will be based on an assessment of your skills and experience asdemonstratedat interview.

Existing Civil Servants who secure a new role on lateral transfer willmaintaintheir current substantive salary.

Civil Servants who gain promotion will move to the bottom of the grade pay scale or receive a 10% increase applied to their current substantive salary, whichever is greater.

Any temporary allowances that you are currently in receipt of will not form part of the calculation todetermineyour pay. Any allowances that are in payment will cease when you move into your new role.

Things you need to know

Artificial intelligence Artificial intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance (opens in a new window) for more information on appropriate and inappropriate use., If you are placed on a reserve list but we cannot immediately offer you a post, please note:

• If you are later offered the role you applied for, in a location you have expressed a preference for, and you decline the offer or are unable to take up the post within a reasonable timeframe you will be withdrawn from the campaign and removed from the reserve list, other than in exceptional circumstances (e.g. a contractual Part Time Working Pattern cannot be accommodated in the initial role offered or in cases of serious ill health).
• If DWP makes an offer of an alternative role or location to that which you originally applied for, and you decline that offer, you will be able to remain on the reserve list.

Disability Confident Scheme

If high application volumes are received, the benchmark for candidates to proceed to the next stage may be raised. In line with our commitment to the Disability Confident Scheme (DCS), we aim to advance all candidates applying under the DCS who meet the minimum standard. However, we may only progress those candidates who best meet the required standards.

Reasonable Adjustments

We consider visible and non-visible disabilities, neurodiversity or learning differences, chronic medical conditions, or mental ill health. Examples include dyslexia, epilepsy, autism, chronic fatigue, or schizophrenia.

If you need a change to be made so that you can make your application, you should contact the Government Recruitment Service via DWPRecruitment.grs@cabinetoffice.gov.uk as soon as possible before the closing date to discuss your needs.