Data Engineer

We’re looking for a talented Data Engineer to join a growing team delivering secure, large-scale data solutions for government and public sector customers.

This is an exciting opportunity to work on complex data platforms that support critical decision-making and operational outcomes. You’ll play a key role in building and enhancing secure batch and Real Time data processing capabilities, working with modern cloud technologies and industry-leading open-source tools.

As part of a collaborative Agile team, you’ll help design, develop, and optimise data pipelines, ensuring data is accurately processed, transformed, and delivered to downstream consumers. This role offers exposure to cutting-edge cloud and data technologies alongside the opportunity to work on meaningful projects within highly secure environments., * Develop, test, and maintain robust data pipelines that meet both functional and non-functional business requirements.

• Design, build, and optimise data processing solutions to support batch and Real Time analytics capabilities.
• Contribute to logical and physical database design activities across a range of data platforms.
• Expand and enhance platform capabilities, enabling the business to solve new and evolving data challenges.
• Analyse complex datasets and identify opportunities to improve data processing, quality, and performance.
• Ensure data models align with established data architecture standards, maintaining consistency across entities, relationships, and definitions.
• Perform root cause analysis on data-related issues, identifying and implementing effective solutions.
• Collaborate with engineers, architects, and stakeholders to deliver scalable and reliable data solutions.
• Support continuous improvement initiatives, helping to drive best practices across data engineering and platform development.
• Produce clear technical documentation covering data pipelines, processes, and system enhancements., * Experience developing and maintaining data pipelines using modern data processing technologies and frameworks.

• Knowledge of Agile delivery methodologies and experience working within fast-paced development environments.
• Strong understanding of data modelling, data structures, and their advantages and limitations across different use cases.
• Experience working with public cloud platforms, particularly AWS, including services such as EC2, RDS, Lambda, Athena, and Glue.
• Experience configuring and optimising Relational and NoSQL databases, including query tuning and performance optimisation.
• Proficiency in programming and Scripting languages such as Python.
• Experience applying Test-Driven Development (TDD) methodologies and associated testing frameworks.
• Strong analytical and problem-solving skills with the ability to work through complex data challenges.
• Excellent communication skills and the ability to work effectively within multidisciplinary teams.
• Bachelor’s degree or equivalent commercial experience in a relevant discipline.

Desirable

• Experience with CI/CD tooling such as Jenkins or Drone.
• Knowledge of configuration management tools including Ansible and Packer.
• Experience using Terraform for Infrastructure-as-Code deployments.
• Understanding of both Linux and Windows environments.
• Exposure to modern cloud-native architectures and DevOps practices.
• Experience working within government, defence, or other highly regulated environments.

What You’ll Bring

• A passion for data engineering and solving complex technical problems.
• A strong attention to detail and commitment to delivering high-quality solutions.
• A collaborative mindset and the ability to work effectively within Agile teams.
• A proactive approach to learning new technologies and expanding your technical skillset.
• The ability to translate business requirements into scalable and efficient data solutions.
• A desire to contribute to projects that deliver meaningful impact within secure and mission-critical environments.