> Markdown version of [/jobs/ext/2174600-ai-evaluation-engineer-sc-eligible-4-months-london-bristol-or-manchester](https://www.wearedevelopers.com/jobs/ext/2174600-ai-evaluation-engineer-sc-eligible-4-months-london-bristol-or-manchester). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AI Evaluation Engineer - SC Eligible - 4 Months - London, Bristol or Manchester - **Company:** Allen Lane - **Location:** Bristol, UK - **Salary:** £182,000.0 - £195,000.0 - **Contract:** Temporary contract - **Skills:** Artificial Intelligence, Python (Programming Language), Machine Learning, Software Engineering, Strategies of Testing, Large Language Models, Generative AI, SC Clearance - **Published:** August 22, 2026 - **Apply:** https://www.careerboard.com/pt/en/find-jobs-in-United-Kingdom/-816EE0059D0D1B7DF0/ ## About the Role The successful candidate will have strong software engineering experience, a deep understanding of AI model and agentic layers, and practical experience with AI evaluation tools such as Ragas, DeepEval, or similar frameworks. Strong communication, leadership and stakeholder engagement skills are essential, alongside the ability to thrive in environments with ambiguity and rapid change., * Strong commercial experience developing AI, Machine Learning or Generative AI solutions * Proven experience designing and implementing AI evaluation frameworks and testing methodologies * Hands-on software engineering expertise, particularly using Python * Experience working with LLMs, AI model layers and agentic AI architectures * Practical knowledge of tools such as Ragas, DeepEval, Promptfoo, LangSmith or similar evaluation platforms * Demonstrable experience building AI harnesses, experimentation environments or benchmarking solutions * Strong stakeholder management and communication skills, with the ability to present technical concepts clearly * Comfortable operating in ambiguous environments and helping shape technical direction * Experience working across multiple teams and influencing wider technical communities * Public sector, government or highly regulated industry experience would be beneficial but is not essential, * Must be eligible for SC Clearance and meet Civil Service nationality requirements * Active or transferable SC Clearance highly desirable * Candidates can start on BPSS clearance while SC clearance progresses if required * Hybrid working with attendance expected approximately 2 days per week in London, Bristol or Manchester * Flexibility required for workshops, team days and stakeholder events across UK locations * Must be comfortable working in a dynamic, evolving environment with changing priorities ## Description A major public sector organisation is seeking an experienced AI Evaluation Engineer to join a fast-paced and highly innovative technology team shaping the future direction of AI adoption across government. Working within a small, ambitious team operating with a start-up mindset, you will be responsible for designing, building and implementing robust AI evaluation frameworks, assessing the quality and performance of Large Language Models (LLMs), agentic AI solutions and emerging AI technologies. This role requires a hands-on engineer who can build solutions from the ground up, develop evaluation harnesses, create automated testing frameworks, and confidently communicate outcomes to both technical and non-technical stakeholders. You will work across multiple government and cross-functional teams, helping identify opportunities, solve complex challenges, and establish best practices for AI evaluation and assurance., * Design, build and maintain AI evaluation frameworks for Large Language Models (LLMs) and agentic AI systems * Develop automated evaluation pipelines, testing harnesses and quality assurance methodologies for AI solutions * Measure model performance using quantitative and qualitative evaluation techniques * Implement and utilise tools such as Ragas, DeepEval and other AI evaluation frameworks * Build proof of concepts and production-ready Python-based solutions from the ground up * Assess AI system outputs, identify weaknesses and recommend improvements to model performance * Collaborate with multidisciplinary teams across government to understand business challenges and AI opportunities * Present findings, recommendations and technical solutions to a range of stakeholders * Provide technical leadership and contribute to strategic decision-making around AI adoption and evaluation * Work independently while maintaining accountability and ownership of deliverables in a fast-moving environment ## Related Videos - [Introduction to Responsible AI: Balancing Value and Risk](https://www.wearedevelopers.com/videos/1972-introduction-to-responsible-ai-balancing-value-and-risk) - [Introduction to Azure Machine Learning](https://www.wearedevelopers.com/videos/368-introduction-to-azure-machine-learning) - [Your imaginations is (no longer) the limit: how Generative AI empowers people to be creative](https://www.wearedevelopers.com/videos/741-your-imaginations-is-no-longer-the-limit-how-generative-ai-empowers-people-to-be-creative) - [Should we build Generative AI into our existing software?](https://www.wearedevelopers.com/videos/1129-should-we-build-generative-ai-into-our-existing-software) - [What non-automotive Machine Learning projects can learn from automotive Machine Learning projects](https://www.wearedevelopers.com/videos/397-what-non-automotive-machine-learning-projects-can-learn-from-automotive-machine-learning-projects) - [The shadows that follow the AI generative models](https://www.wearedevelopers.com/videos/624-the-shadows-that-follow-the-ai-generative-models) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts)