> Markdown version of [/jobs/ext/2095534-data-engineer](https://www.wearedevelopers.com/jobs/ext/2095534-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Data Engineer - **Company:** Department for Work and Pensions (DWP) - **Location:** Birmingham, UK - **Salary:** £49,040.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Amazon Elastic Compute Cloud, Confluence, Bash Shell, Cloud Computing, Continuous Integration, Information Engineering, Extract Transform Load (ETL), Data Transformation, Apache Hive, Unstructured Data, Pyspark, Real Time Data, Data Pipelines - **Published:** August 17, 2026 - **Apply:** https://www.adzuna.co.uk/jobs/details/5845144439 ## About the Role Are you an experienced Data Engineer with strong Analytical skills?, * Practitioner: Ability toorchestrate PySpark jobs on EMR, managing dependencies, workflow execution, and operational efficiency. * Working: Knowledge of designingevent-driven architectures, leveragingAWS Lambda functionsto triggerPySpark workflowsbased on real-time data ingestion and processing needs. * Practitioner: Experienced in provisioning cloud infrastructure using Amazon EC2, and automating deployment, configuration, and data workflows with Bash scripting across scalable environments. * Working: Ability to applyAgile best practices in developing CI/CD pipelines, using continuous integration, testing, and deploymentwithin a collaborative development framework. * Practitioner: Understanding ofoptimising queries, storage formats, and data structuresto enhance efficiency, reduce processing time, and ensure high data quality standards in complex cloud architectures. 4. Technical statement (up to 250 words). The following statement is aligned to the required technical skill of Technical Breadth. This statement should be used to provide example(s) of how you meet the criterion below: * Practitioner: Good understanding ofdesigning and automating ETL pipelinesusingPySpark and Spark SQL, ensuring seamless extraction, transformation, and loading of structured and unstructured data across cloud environments. Ensure that all examples provided in your statements are taken directly from your own experience and that you describe the examples in your own words. If you choose to use gen-AI to support your statements, you must follow the guidelines outlined in the Artificial intelligence and recruitment guide., You must meet the security requirements before you can be appointed. The level of security needed is security check. You must have resided continuously in the UK for at least 3 of the last 5 years, 2 of which must have been the immediately preceding years from the point of applying for this job. ## Description DWPs Digital Group is looking for experienced Data Engineers to fill post in Management Information (MI) team. The MI teamprovides strategic capabilities for MI reporting and data visualisations across DWP. Working with modern cloud environments alongside legacy on-premises technologies, youll play a key role in developing the DWP Data Engineering Strategy for MI products and services. Successful candidates will join a fast-paced environment with a wide portfolio of products for analytical use and a challenging delivery roadmap. Youll focus on a variety of tasks, supporting platform live service and delivering high quality, data transformation products., * Building and maintain scalable data engineering solutions that power analytical reporting, working in line with DWPs engineering standards. * Collaborating with stakeholders across DWP to gather and clarify requirements where needed and translate them into scalable back-end data processes that support MI reporting needs. * Maintaining comprehensive technical documentation and collaborative workspaces in Confluence to improve transparency, knowledge sharing, and onboarding. * Working closely with other junior data engineers to provide guidance, share technical expertise, and actively support ongoing knowledge sharing. * Reviewing code developed by fellow engineers to ensure quality, maintainability, and adherence to DWP coding standards., Civil Servants applying on promotion will usually be appointed to the salary minimum or within 10% of existing salary. Any move to DWP from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may, however, be eligible for other government schemes, including Tax Free Childcare. Determine your eligibility at the Childcare Choices Website. For further information on reasonable adjustments, terms and conditions and how we recruit visit the How we recruit page. Feedback will only be provided if you attend an interview or assessment. Security Successful candidates must undergo a basic (or equivalent) criminal record check. Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check (opens in a new window) . See our vetting charter (opens in a new window) . People working with government assets must complete baseline personnel security standard (opens in new window) checks. Nationality requirements This job is broadly open to the following groups: * UK nationals * nationals of the Republic of Ireland * nationals of Commonwealth countries who have the right to work in the UK * nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS) (opens in a new window) * nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS) * individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020 * Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service Further information on nationality requirements (opens in a new window) Working for the Civil Service The Civil Service Code (opens in a new window) sets out the standards of behaviour expected of civil servants. We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles (opens in a new window) . The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria. The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy. Diversity and Inclusion The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan (opens in a new window) and the Civil Service Diversity and Inclusion Strategy (opens in a new window) . Apply and further information This vacancy is part of the Great Place to Work for Veterans (opens in a new window) initiative. The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window) . ## Related Videos - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Old tools, new tricks](https://www.wearedevelopers.com/videos/1916-old-tools-new-tricks) - [42 x 2 Canvases Later: Two Years, Two Minds, Many Lessons](https://www.wearedevelopers.com/videos/1458-42-x-2-canvases-later-two-years-two-minds-many-lessons) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Software Engineer Salary in The UK](https://www.wearedevelopers.com/magazine/231-software-engineer-salary-in-the-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)