> Markdown version of [/jobs/ext/2140645-secure-data-engineer](https://www.wearedevelopers.com/jobs/ext/2140645-secure-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Secure Data Engineer - **Company:** Capgemini - **Location:** Birmingham, UK (Remote available) - **Salary:** £46,793.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Airflow, Automation of Tests, Cloud Computing, Configuration Management, Continuous Integration, Information Engineering, Data Security, DevOps, Distributed Computing Environment, Github, Python (Programming Language), PostgreSQL, Object-Oriented Software Development, Ansible, Search Technologies, Software Engineering, Data Streaming, Workflow Management Systems, Data Logging, Data Ingestion, Large Language Models, Apache Spark, SC Clearance, Containerization, AI Platforms, Gitlab-ci, Kubernetes, Apache Kafka, Data Management, Terraform, Software Version Control, Data Pipelines, Docker - **Published:** August 20, 2026 - **Apply:** https://www.adzuna.co.uk/jobs/details/5847552144 ## About the Role To obtain SC clearance, the successful applicant must have resided continuously within the United Kingdom for the last 5 years, along with other criteria and requirements. Throughout the recruitment process, you will be asked questions about your security clearance eligibility such as, but not limited to, country of residence and nationality. Some posts are restricted to sole UK Nationals for security reasons; therefore, you may be asked about your citizenship in the application process., * Infrastructure as Code or configuration management (for example Terraform or Ansible). * Experience in secure, restricted or air-gapped environments, including Defence networks or MODCloud-aligned platforms. * Familiarity with Google Distributed Cloud (GDC) or other edge and on-premises platforms used in constrained or disconnected settings. * Exposure to GenAI and agentic building blocks: RAG, vector search, LLM orchestration, or LLM evaluation and observability. * Experience using AI coding assistants within strict data-handling and security boundaries, recognising that connected AI tooling is not always available on secure networks ## Description Developing modular, maintainable software components that process, transform and expose data for analytical, operational or AI-driven use cases, with testing and observability built in from the start. Streaming and real-time architecture Designing and implementing data ingestion and event-driven patterns that support real-time or near-real-time flows, keeping them reliable under demanding operational conditions. Workflow orchestration Defining data workflows programmatically and managing complex dependencies, scheduling and error-recovery behaviour within secure, assured environments. Operationalising Data flows for AI in secure environments Packaging, deploying and monitoring ML and, increasingly, GenAI and agentic data workloads inside assured environments. This can include self-hosted or open-weight models, retrieval (RAG) and vector search over governed data, and the guardrails, evaluation and human oversight that non-deterministic systems demand. Running these patterns where internet-connected AI services are not available is a core part of the role's future. Deployment and ownership Containerising your own services and deploying them into secure Kubernetes or cloud environments, using CI/CD principles adapted for Defence delivery. You'll own your applications in production and contribute to secure deployment patterns. Resilience, observability and compliance Implementing health monitoring, structured logging, metrics and lineage to meet Defence requirements for auditability, security and operational assurance, including observability for non-deterministic and agentic behaviour. You'll design systems that can self-heal or fail gracefully when needed. Infrastructure as Code Provisioning the resources your services depend on using Infrastructure as Code, working closely with platform teams to stay aligned with accredited Defence architectures What You Will Bring You'll be a strong, code-first data engineer who takes ownership of what you build and is energised by hard constraints rather than put off by them. You don't need to have shipped GenAI or agentic systems in secure environments already, but you should be genuinely interested in the problem and have the engineering foundations to take it on. Essential: * Core engineering. Expert-level Python and strong software engineering foundations: object-oriented design, automated testing and version control. * Data engineering stack. Building pipelines with streaming frameworks, distributed processing engines and relational or analytical storage (for example Kafka, Spark, PostgreSQL). * Orchestration. Defining and running data workflows with modern orchestration frameworks (for example Airflow, Dagster or Prefect). * Data quality and lineage. Tools and techniques for data testing, documentation and lineage (for example Great Expectations or dbt). * AI, MLOps and emerging LLMOps. Operationalising ML in production (model packaging, monitoring, controlled deployment), and an understanding of how serving GenAI and agentic systems differs, including evaluation, guardrails and observability for non-deterministic outputs. * Containerisation and Kubernetes. Confidence deploying applications in containerised environments, including defining services, pods and deployment configurations (for example Docker and Kubernetes). * DevOps mindset. Hands-on CI/CD experience and a belief in owning the services you build (for example GitLab CI, GitHub Actions or Argo).Hands-on experience delivering data pipelines and data platforms in production environments. ## Related Videos - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Building Sovereign AI: Lessons from Deploying Secure RAG Systems using Confidential Computing](https://www.wearedevelopers.com/videos/100108-building-sovereign-ai-lessons-from-deploying-secure-rag-systems-using-confidential-computing) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) - [Shipping Faster with Less: Render on Cloud Hosting, AI Workloads, and the Future of DevOps](https://www.wearedevelopers.com/videos/1894-shipping-faster-with-less-render-on-cloud-hosting-ai-workloads-and-the-future-of-devops) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [7 Cloud Computing Trends Coming in 2025 for Developers](https://www.wearedevelopers.com/magazine/412-7-cloud-computing-trends-coming-in-2025-for-developers) - [Dev Digest 134 - Where pixels sing?](https://www.wearedevelopers.com/magazine/477-dev-digest-134-where-pixels-sing) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)