> Markdown version of [/jobs/ext/2419865-senior-qa-automation-engineer-sdet-genai-llm-testing-lorien-impellam](https://www.wearedevelopers.com/jobs/ext/2419865-senior-qa-automation-engineer-sdet-genai-llm-testing-lorien-impellam). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior QA Automation Engineer / SDET - GenAI & LLM Testing - Lorien Impellam

- **Company:** Lorien
- **Location:** London, UK
- **Experience:** Expert
- **Contract:** Temporary to permanent
- **Skills:** Artificial Intelligence, Amazon S3, Automation of Tests, Cloud Computing, Continuous Integration, Distributed Systems, Python (Programming Language), Message Oriented Middleware, Strategies of Testing, Large Language Models, Generative AI, Event Driven Architecture, Pytest, Kubernetes, Apache Kafka, Restful APIs, Api Management, SDET
- **Published:** August 24, 2026
- **Apply:** https://www.reed.co.uk/jobs/senior-qa-automation-engineer-sdet-genai-llm-testing-lorien-impellam/57271528

## About the Role

Core Automation

 * Strong Python 3.x development skills with experience building testing frameworks from the ground up.
 * Extensive experience with Pytest.
 * Strong API testing experience.
 * Experience creating test strategies and quality approaches from architectural diagrams and solution designs.

Cloud & Platform

Working knowledge of AWS services including:

 * Lambda
 * Bedrock
 * S3
 * EKS

Event-Driven Systems

 * Experience testing Kafka-based architectures.
 * Strong understanding of asynchronous messaging and event-driven systems.

GenAI / LLM Testing (Essential)

 * Proven experience testing LLM and GenAI applications.
 * Prompt testing and prompt evaluation.
 * Non-deterministic output validation.
 * AI guardrail verification and safety testing.
 * Hallucination and response quality assessment.
 * Experience establishing quality standards for AI solutions.

LLM Evaluation Tooling

Experience with one or more of:

 * DeepEval
 * Promptfoo
 * RAGAS
 * LangSmith
 * Similar LLM evaluation frameworks

Contract Testing

 * Hands-on experience with Pact Flow.
 * Consumer-driven contract testing within distributed systems.

Desirable

 * Banking or Financial Services experience.
 * Experience working in highly regulated environments.
 * Exposure to RAG architectures.
 * Knowledge of LLM observability and monitoring.
 * Kubernetes experience.
 * CI/CD pipeline integration.

## Description

We're supporting a leading banking client on a large-scale AI transformation programme and are looking for an experienced Senior QA Automation Engineer / SDET with a strong background in GenAI and LLM testing.

This role will focus on defining and implementing automated testing frameworks for AI-driven products, ensuring the quality, reliability and safety of LLM-powered solutions deployed within a highly regulated banking environment., * Design and develop scalable automated test frameworks using Python.

 * Create test strategies from architecture and solution design documentation.
 * Test RESTful APIs and microservice-based applications.
 * Define and execute testing approaches for event-driven architectures and Kafka-based systems.
 * Validate GenAI and LLM solutions through functional, non-functional and behavioural testing.
 * Develop automated prompt testing and evaluation frameworks.
 * Verify AI guardrails, safety controls and compliance requirements.
 * Implement validation approaches for non-deterministic LLM outputs.
 * Build and maintain contract tests using Pact Flow.
 * Collaborate closely with Engineering, Architecture, Data Science and Product teams.
 * Support CI/CD pipelines and quality engineering best practices.


## Related Videos

- [pytest: Simple, rapid and fun testing with Python](https://www.wearedevelopers.com/videos/213-pytest-simple-rapid-and-fun-testing-with-python)
- [WeAreDevelopers LIVE - CSS is DOOMed](https://www.wearedevelopers.com/videos/1838-wearedevelopers-live-css-is-doomed)
- [AI as a Test Designer: Transforming Experience into Automated Testing](https://www.wearedevelopers.com/videos/1984-ai-as-a-test-designer-transforming-experience-into-automated-testing)
- [Automagic Configuration in Python](https://www.wearedevelopers.com/videos/363-automagic-configuration-in-python)
- [Testing AI Agents: Automated Evaluation for Chatbots & RAG Systems](https://www.wearedevelopers.com/videos/100300-testing-ai-agents-automated-evaluation-for-chatbots-rag-systems)
- [Hate organising your photos? Try it with 5 Terabytes](https://www.wearedevelopers.com/videos/79-hate-organising-your-photos-try-it-with-5-terabytes)

## Related Articles

- [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models)
- [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)
- [13 AI Tools You Have to Try](https://www.wearedevelopers.com/magazine/219-13-ai-tools-you-have-to-try)
- [The 8 Best Code Testing Tools](https://www.wearedevelopers.com/magazine/402-the-8-best-code-testing-tools)
- [Dev Digest 132 - Binging WADFlix?](https://www.wearedevelopers.com/magazine/473-dev-digest-132-binging-wadflix)
- [ Dev Digest 213: Petrol Prices, Agentic Workflows, AI Skills and CODE100!](https://www.wearedevelopers.com/magazine/718-dev-digest-213-petrol-prices-agentic-workflows-ai-skills-and-code100)