> Markdown version of [/jobs/ext/3117244-lead-principle-data-engineer](https://www.wearedevelopers.com/jobs/ext/3117244-lead-principle-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead/Principle Data Engineer - **Company:** Sanderson Recruitment Plc - **Location:** Cheltenham, UK - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Agile Methodology, Amazon Web Services, Amazon Elastic Compute Cloud, Big Data, Cloud Engineering, Computer Programming, System Configuration, Continuous Integration, Data Architecture, Information Engineering, Data Structures, Data Systems, Linux, DevOps, Python (Programming Language), NoSQL, Open Source Technology, Performance Tuning, Query Optimization, Ansible, Software Deployment, Data Processing, Scripting, Infrastructure Automation Frameworks, Data Management, Terraform, Stream Processing, Data Pipelines, Jenkins - **Published:** September 28, 2026 - **Apply:** https://www.totaljobs.com/job/lead-data-engineer/sanderson-government-and-defence-job108051839 ## About the Role * Knowledge of Agile delivery methodologies and experience working within fast-paced development environments. * Strong understanding of data modelling, data structures, and their advantages and limitations across different use cases. * Experience working with public cloud platforms, particularly AWS, including services such as EC2, RDS, Lambda, Athena, and Glue. * Experience configuring and optimising Relational and NoSQL databases, including query tuning and performance optimisation. * Proficiency in programming and scripting languages such as Python. * Experience applying Test-Driven Development (TDD) methodologies and associated testing frameworks. * Strong analytical and problem-solving skills with the ability to work through complex data challenges. * Excellent communication skills and the ability to work effectively within multidisciplinary teams. * Bachelor's degree or equivalent commercial experience in a relevant discipline. Desirable * Experience with CI/CD tooling such as Jenkins or Drone. * Knowledge of configuration management tools including Ansible and Packer. * Experience using Terraform for Infrastructure-as-Code deployments. * Understanding of both Linux and Windows environments. * Exposure to modern cloud-native architectures and DevOps practices. * Experience working within government, defence, or other highly regulated environments. What You'll Bring * A passion for data engineering and solving complex technical problems. * A strong attention to detail and commitment to delivering high-quality solutions. * A collaborative mindset and the ability to work effectively within Agile teams. * A proactive approach to learning new technologies and expanding your technical skillset. * The ability to translate business requirements into scalable and efficient data solutions. * A desire to contribute to projects that deliver meaningful impact within secure and mission-critical environments. ## Description We're looking for a talented Data Engineer to join a growing team delivering secure, large-scale data solutions for government and public sector customers. This is an exciting opportunity to work on complex data platforms that support critical decision-making and operational outcomes. You'll play a key role in building and enhancing secure batch and real-time data processing capabilities, working with modern cloud technologies and industry-leading open-source tools. As part of a collaborative Agile team, you'll help design, develop, and optimise data pipelines, ensuring data is accurately processed, transformed, and delivered to downstream consumers. This role offers exposure to cutting-edge cloud and data technologies alongside the opportunity to work on meaningful projects within highly secure environments., * Develop, test, and maintain robust data pipelines that meet both functional and non-functional business requirements. * Design, build, and optimise data processing solutions to support batch and real-time analytics capabilities. * Contribute to logical and physical database design activities across a range of data platforms. * Expand and enhance platform capabilities, enabling the business to solve new and evolving data challenges. * Analyse complex datasets and identify opportunities to improve data processing, quality, and performance. * Ensure data models align with established data architecture standards, maintaining consistency across entities, relationships, and definitions. * Perform root cause analysis on data-related issues, identifying and implementing effective solutions. * Collaborate with engineers, architects, and stakeholders to deliver scalable and reliable data solutions. * Support continuous improvement initiatives, helping to drive best practices across data engineering and platform development. * Produce clear technical documentation covering data pipelines, processes, and system enhancements., * Experience developing and maintaining data pipelines using modern data processing technologies and frameworks. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis) - [Docker network without Docker](https://www.wearedevelopers.com/videos/1418-docker-network-without-docker) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [#90DaysOfDevOps - The DevOps Learning Journey](https://www.wearedevelopers.com/videos/548-90daysofdevops-the-devops-learning-journey) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [What’s the Difference between a Junior, Mid, and Senior Developer?](https://www.wearedevelopers.com/magazine/238-what-s-the-difference-between-a-junior-mid-and-senior-developer) - [Fully Remote Software Engineer Jobs](https://www.wearedevelopers.com/magazine/447-fully-remote-software-engineer-jobs)