Machine Learning Engineer

An opportunity is available for an experienced Senior Machine Learning Engineer to design, build and operationalise advanced machine learning solutions that directly support UK national security objectives.

This role sits within a multidisciplinary AI engineering environment, working closely with Data Scientists, Software Engineers, Product teams and government stakeholders. The Senior ML Engineer will own the journey from experimentation and hypothesis testing through to secure, production-grade deployment, using a modern AWS-based MLOps and LLMOps platform.

The Role The successful candidate will balance rapid experimentation with production readiness, prototyping and validating machine learning and generative AI approaches while ensuring successful models integrate seamlessly into live operational systems.

This is a high-impact role at a pivotal point in the adoption of AI, machine learning and large language models across critical national systems, offering the chance to deliver real-world outcomes at scale., * Designing, developing and optimising machine learning models across traditional ML use cases (forecasting, classification, anomaly detection) and GenAI / LLM solutions

• Leading experimentation cycles, including hypothesis definition, experimental design, evaluation and iteration, while complying with governance standards
• Transitioning validated experiments into production-ready ML services, collaborating closely with engineering teams on deployment and monitoring
• Building scalable ML pipelines using AWS services and modern experiment tracking frameworks
• Developing and integrating LLM-powered capabilities for evaluation, tracing and production monitoring
• Implementing robust experiment tracking, model versioning and reproducibility, ensuring full auditability
• Designing feature engineering strategies and contributing to feature store development
• Monitoring live models, analysing performance and driving continuous improvement
• Applying responsible AI principles, including explainability, robustness and fairness
• Communicating experimental results and production outcomes to stakeholders, highlighting operational and strategic value
• Mentoring junior engineers and promoting best practices across the team

The ideal candidate will bring strong hands-on experience in machine learning engineering, with the ability to translate experimental success into reliable, scalable systems.

Essential experience includes:

• Commercial experience developing and deploying machine learning models in Python
• Proficiency with ML frameworks such as scikit-learn, XGBoost, PyTorch or TensorFlow
• Strong experience delivering ML solutions using AWS services (e.g. SageMaker, Lambda, S3)
• Expertise in experiment design, including hypothesis formulation, A/B testing and statistical evaluation
• Proven experience moving models from experimentation into production with appropriate governance and quality controls
• Hands-on experience with MLOps tooling such as MLflow, Weights & Biases or Data Version Control
• Practical experience building LLM / GenAI applications, including prompt engineering and retrieval-augmented generation (RAG)
• Familiarity with LLMOps frameworks such as LangChain, LangSmith or LangGraph
• Understanding of model validation, evaluation techniques and production monitoring
• Experience working in cross-functional teams from problem definition through to delivery
• Strong communication skills, with the ability to explain complex concepts to non-technical audiences
• Sound judgement in applying AI appropriately and recognising when non-AI approaches are more suitable

Desirable Experience

• Advanced LLM techniques, including agents, tool use and agentic workflows
• Experience with vector databases (e.g. Pinecone, Weaviate, pgvector)
• Feature store technologies such as Feast or AWS Feature Store
• Containerisation and orchestration using Docker, Kubernetes or ECS
• Infrastructure as Code using Terraform or CloudFormation
• Large-scale data processing frameworks such as Spark or Dask
• Knowledge of data governance, compliance and regulated environments
• Experience delivering solutions within highly regulated industries such as government, finance or healthcare

Security Clearance This role requires UK Security Clearance. Applicants must already hold clearance or be eligible and willing to undergo the vetting process.