> Markdown version of [/jobs/ext/61792-python-qa-automation-engineer](https://www.wearedevelopers.com/jobs/ext/61792-python-qa-automation-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Python QA Automation Engineer - **Company:** EPAM Systems, Inc. - **Location:** London, UK - **Contract:** Permanent contract - **Skills:** Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Automation of Tests, Cloud Engineering, Computer Programming, Continuous Integration, Distributed Systems, Python (Programming Language), Data Logging, Large Language Models, Pytest, Containerization, AI Platforms, Integration Tests, Cucumber (Software), Network Server, Automation Anywhere, Microservices - **Published:** May 21, 2026 - **Apply:** https://uk.indeed.com/viewjob?jk=96c380959b5d9b10 ## About the Role Do you have experience in Test automation?, Do you have a Master's degree?, * Strong programming experience in Python applied to test automation and evaluation * Expertise in Pytest and familiarity with BDD frameworks such as Behave or Cucumber * Knowledge of LLM evaluation approaches including RAGAS, DeepEval, or custom pipelines * Understanding of common agentic AI issues such as hallucination, tool misuse, and performance bottlenecks * Familiarity with automated testing of AI workflows, distributed systems, or microservices environments * Strong grasp of Agile delivery methodologies and CI/CD integration for quality checks * Excellent communication and problem-solving skills with a focus on accuracy and reliability Nice to have * Experience with Model Context Protocol (MCP) or other agent orchestration solutions * Exposure to observability, monitoring, or logging tools for AI systems * API and service integration testing background for multi-layered platforms * Knowledge of containerized environments and cloud-native architecture * Background in enterprise AI automation projects or intelligent platform engineering ## Description We're looking for a Python QA Automation Engineer to join EPAM in London, in an onsite working mode, contributing to an AI-driven enterprise project for one of our clients. This role focuses on exposing internal enterprise tooling to AI agents via Model Context Protocol (MCP), ensuring robust quality engineering and compliance across innovative intelligent automation solutions. As a Python QA Automation Engineer, you will be responsible for developing automated testing frameworks, evaluation pipelines, and quality controls for MCP Components within our AI platform. You will work closely with engineering and product teams in a Classic Agile environment to guarantee the reliability, accuracy, and performance of agent-driven workflows operating in enterprise-scale systems. This position offers a chance to define standards for testing AI models and tools in production-like environments while being a critical part of one of EPAM's most transformative projects. Responsibilities * Develop automated testing frameworks to validate MCP Servers and related AI systems * Design and implement evaluation strategies for LLM accuracy, safety, and reliability * Create automated tests using Python, Pytest, and BDD frameworks * Build quality gates into CI/CD pipelines to maintain continuous assurance * Identify and address agentic AI failure modes such as hallucination, latency, and incorrect tool usage * Collaborate with engineering, QA, and product teams to define quality metrics and acceptance criteria * Contribute to Agile ceremonies, ensuring testing practices align with sprint goals * Prepare detailed reporting on quality outcomes and improvement opportunities * Maintain documentation for test cases, evaluation pipelines, and validation strategies ## Related Videos - [AI as a Test Designer: Transforming Experience into Automated Testing](https://www.wearedevelopers.com/videos/1984-ai-as-a-test-designer-transforming-experience-into-automated-testing) - [pytest: Simple, rapid and fun testing with Python](https://www.wearedevelopers.com/videos/213-pytest-simple-rapid-and-fun-testing-with-python) - [Crypto-secure Data Management with In-Database Blockchain](https://www.wearedevelopers.com/videos/632-crypto-secure-data-management-with-in-database-blockchain) - [This App Reached 10,000 Users in One Week. Here's How.](https://www.wearedevelopers.com/videos/100329-this-app-reached-10-000-users-in-one-week-here-s-how) - [Testing AI Agents: Automated Evaluation for Chatbots & RAG Systems](https://www.wearedevelopers.com/videos/100300-testing-ai-agents-automated-evaluation-for-chatbots-rag-systems) - [Build Delightful Mobile Experiences with Kotlin, Realm, and Atlas Device Sync](https://www.wearedevelopers.com/videos/694-build-delightful-mobile-experiences-with-kotlin-realm-and-atlas-device-sync) ## Related Articles - [From Prototype to Production: Build AI Agents with This Free 4-Course Learning Path](https://www.wearedevelopers.com/magazine/655-from-prototype-to-production-build-ai-agents-with-this-free-4-course-learning-path) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [The 13 Best Python Libraries for Developers in 2025](https://www.wearedevelopers.com/magazine/371-the-13-best-python-libraries-for-developers-in-2025) - [What is Agentic Programming and Why Should Developers Care?](https://www.wearedevelopers.com/magazine/625-what-is-agentic-programming-and-why-should-developers-care) - [Dev Digest 137 - AI'm not sure about this](https://www.wearedevelopers.com/magazine/485-dev-digest-137-ai-m-not-sure-about-this) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline)