> Markdown version of [/jobs/ext/2951810-ai-evaluation-engineer](https://www.wearedevelopers.com/jobs/ext/2951810-ai-evaluation-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# AI Evaluation Engineer

- **Company:** Notethis
- **Location:** Bristol, UK
- **Contract:** Temporary contract
- **Skills:** Artificial Intelligence, Application Layers, Automation of Tests, Cloud Computing, DevOps, Python (Programming Language), Machine Learning, Large Language Models, Codebase
- **Published:** September 17, 2026
- **Apply:** https://www.apply4u.co.uk/jobs/ai-evaluation-engineer/47339813

## About the Role

code solutions, harnesses and testing capabilities from the ground up using Python. Define and execute evaluation strategies across model, agentic and application layers. Assess AI system quality, performance, reliability, safety and risk. Work closely with technical and non-technical stakeholders across multiple departments and organisations. Provide technical leadership and guidance on AI evaluation best practice. Skills Experience Strong commercial experience working with AI, Machine Learning and LLM-based systems in production environments. Deep understanding of AI evaluation methodologies, frameworks and quality engineering principles. Hands-on Python development experience with the ability to create and maintain your own codebase. Experience evaluating AI solutions across: Model layer Agentic layer Application layer Knowledge of AI evaluation tooling and frameworks such as: RAGAS DeepEval (DP-Eval) Other AI evaluation and testing frameworks Experience developing automated testing,
evaluation harnesses and assurance processes for AI systems. Proven leadership experience, including working across teams, departments and organisational boundaries. Strong stakeholder engagement and communication skills. Desirable Experience Public sector or government programme experience. Understanding of responsible AI and AI assurance frameworks. Please NoteThis opportunity is specifically seeking candidates with strong AI evaluation and engineering expertise. Experience primarily focused on cloud, infrastructure, platform or DevOps engineering without significant AI evaluation experience are unlikely to be suitable



## Related Videos

- [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform)
- [Getting to Know Your Legacy (System) with AI-Driven Software Archeology](https://www.wearedevelopers.com/videos/1437-getting-to-know-your-legacy-system-with-ai-driven-software-archeology)
- [Green Cloud Computing](https://www.wearedevelopers.com/videos/592-green-cloud-computing)
- [Testing AI Agents: Automated Evaluation for Chatbots & RAG Systems](https://www.wearedevelopers.com/videos/100300-testing-ai-agents-automated-evaluation-for-chatbots-rag-systems)
- [DevOps Maturity Check – a way to balance autonomy and alignment](https://www.wearedevelopers.com/videos/58-devops-maturity-check-a-way-to-balance-autonomy-and-alignment)
- [AI as a Test Designer: Transforming Experience into Automated Testing](https://www.wearedevelopers.com/videos/1984-ai-as-a-test-designer-transforming-experience-into-automated-testing)

## Related Articles

- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts)
- [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud)
- [The State of WebDev AI 2025 Results: What Can We Learn?](https://www.wearedevelopers.com/magazine/581-the-state-of-webdev-ai-2025-results-what-can-we-learn)