Data Engineer

Our Data Engineering role will be responsible for building and maintaining data processing pipelines and also the transformation and optimisation of data for analytical use. As a Data Engineer, you’ll work with an experienced software development team in a cross-functional Agile environment.

Upon reviewing your application, we will discuss the development opportunities or challenges that are appropriate to your professional profile.

Due to the interesting work we do and the sector this team is working in, all candidates must hold current eDV clearance., * Build data pipelines that clean, transform, and aggregate data from disparate sources

• Collaborate with stakeholders and other engineers
• Contribute to the completion of milestones associated with your project
• Contribute to continuous improvement within your team
• Collaborate with your peers on technical direction within your team

• Strong analytic skills related to working with unstructured datasets
• Python (PySpark, Pandas, PyArrow)
• Distributed data processing (Apache Spark)
• Data ETL (Apache Airflow, AWS Step Functions, Apache NiFi)
• Cloud services (AWS, Azure or GCP)
• Messaging / Streaming (Kafka, AWS SQS, Other Cloud Queuing Native services)
• SQL and NoSQL databases and storage (HDFS, Iceberg, Elastic, S3, Data Lake)
• Containerisation and orchestration (Docker, Kubernetes, Openshift)
• Testing frameworks and best practices

• Contributory Pension Scheme (up to 10.5% company contribution)
• 6 times salary ‘Life Assurance’ with pension
• 25 days holiday (increasing with service) + statutory public holidays, plus opportunity to buy and sell up to 5 days
• Company bonus scheme (discretionary)
• Flexible Benefits scheme with extensive salary sacrifice options, including Health Cashplan, Dental, and Cycle to Work
• Enhanced sick pay

• Enhanced family friendly policies including enhanced maternity, paternity & shared parental leave

• 37-hour working week
• Early 1.30pm finish Friday, start your weekend early!
• Up to 5 paid days volunteering each year.
• Flexible working culture focused on output, with formal arrangements on request (assessed subject to role). Highlight any requests to the Talent Acquisition team.