> Markdown version of [/jobs/ext/2694824-ai-automation-quality-engineer](https://www.wearedevelopers.com/jobs/ext/2694824-ai-automation-quality-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AI Automation Quality Engineer - **Company:** ScienceJobs.Org - **Location:** Boston, MA, United States - **Experience:** Experienced - **Contract:** Permanent contract - **Skills:** Testing (Software), Test Suite, Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Confluence, JIRA, Automation of Tests, Cloud Computing, Code Coverage, Software Quality, Continuous Integration, Cursor (Graphical User Interface Elements), DevOps, Django Web Framework, Python (Programming Language), Machine Learning, Microsoft UI Automation, Oracle (Applications), Systems Development Life Cycle, Student Information Systems, Selenium, Software Engineering, Systems Integration, Strategies of Testing, Web Applications, Openapi, Software Repository, Performance Testing, GitHub Copilot, Large Language Models, Multi-Agent Systems, Prompt Engineering, Cypress (Programming Language), Generative AI, Fastapi, Pytest, Git Flow, Kubernetes, Playwright, Graphql, Front End Software Development, Restful APIs, Docker - **Published:** September 3, 2026 - **Apply:** https://sciencejobs.org/job/job/apply?id=7369226 ## About the Role Basic Qualifications are theminimumthreshold a candidate must meetin order tobe considered for this role. * Minimum of two years' post-secondary education or relevant work experience, * 3+ years building automated test frameworks for enterprise web applications * Strong Python development experience * Experience testing Django and/or FastAPI applications * Experience with Playwright, Selenium, Cypress, pytest, or similar frameworks * Experience using Jira and Agile development processes * Understanding of Git workflows and CI/CD pipelines * Experience working with LLMs (OpenAI, Anthropic, Gemini, etc.), prompt engineering, and AI-assisted software development * Understanding of Retrieval-Augmented Generation (RAG), embeddings, vector databases, MCP, or agent frameworks such as LangGraph, CrewAI, AutoGen, Semantic Kernel, or similar * Strong analytical thinking and ability to translate business requirements into automated quality strategies * Knowledge of advanced IT project management principles (e.g. Agile) and software * Knowledge of SDLC and STLC The following qualifications are strongly preferred. If you meet some, but not all, you are still encouraged to apply; we value employees with a willingness to learn. * Experience developing AI agents that generate code or software tests * Experience integrating AI into SDLC workflows * Knowledge of GitHub Copilot, Cursor, Claude Code, Windsurf, or similar AI development environments * Experience with vector databases (Pinecone, pgvector, Chroma, Weaviate) * Experience with OpenTelemetry, observability, and evaluation frameworks for AI systems * Experience in higher education or enterprise ERP/student information systems * Knowledge of Oracle, REST APIs, GraphQL, Docker, Kubernetes, and cloud-native development * Experience measuring test coverage, mutation testing, and AI quality metrics * AI/ML engineering and agentic system design * Test automation and quality engineering best practices * Cross-functional collaboration with Business Analysts, developers, and product owners * Strong analytical thinking and problem-solving * Demonstrated team performance skills, service mindset approach, and the ability to act as a trusted advisor * Continuous improvement and adaptability to emerging AI and testing technologies Certificates and Licenses: * Completion of Harvard IT Academy specified foundational courses (or external equivalent) preferred ## Description The AI Automation Quality Engineer leads the next generation of quality engineering for the my.harvard portal, moving beyond traditional QA to design intelligent, agent-based testing capabilities that automatically analyze requirements, understand application code, generate reusable test assets, and continuously improve HUIT's testing strategy. * This role combines strong software testing experience with modern AI engineering concepts, including Retrieval-Augmented Generation (RAG), large language models (LLMs), agentic workflows, and test automation frameworks. Working closely with Business Analysts, developers, and product owners, the AI Automation Quality Engineer helps build an autonomous QA platform that accelerates delivery while improving software quality. * AI-Powered Test Generation: Design and implement AI agents that read Jira user stories, acceptance criteria, architectural documentation, and business rules to automatically generate comprehensive functional, integration, regression, accessibility, and API test scenarios * Code-Aware Testing: Build agents capable of analyzing Django, FastAPI, and frontend codebases to understand application behavior, identify impacted components, and generate reusable test cases with high coverage * RAG & Knowledge Engineering: Develop Retrieval-Augmented Generation (RAG) pipelines that leverage Jira, Confluence, Git repositories, existing test suites, architecture documentation, API specifications, and historical defects to improve test quality and reduce hallucinations * Human-in-the-Loop Validation: Implement workflows where generated test scenarios are reviewed and validated by Business Analysts and Product Owners before automation code is produced * Autonomous Test Automation: After approval, generate maintainable automated tests using Playwright, Selenium, pytest, or equivalent frameworks, validate results, and prepare commits or pull requests for inclusion in the shared test repository * Continuous Learning: Capture review feedback, production defects, and failed test patterns to improve prompts, retrieval strategies, reusable testing patterns, and agent performance over time * Quality Engineering: Maintain regression suites, API tests, UI automation, performance validation, and CI/CD integration while ensuring generated tests remain reliable and maintainable * Collaboration: Partner with developers, architects, BAs, and DevOps engineers to define AI-driven quality engineering standards and governance ## Related Videos - [Improving quality with Agentic AI with Rovo Dev and Xray](https://www.wearedevelopers.com/videos/2005-improving-quality-with-agentic-ai-with-rovo-dev-and-xray) - [AI as a Test Designer: Transforming Experience into Automated Testing](https://www.wearedevelopers.com/videos/1984-ai-as-a-test-designer-transforming-experience-into-automated-testing) - [pytest: Simple, rapid and fun testing with Python](https://www.wearedevelopers.com/videos/213-pytest-simple-rapid-and-fun-testing-with-python) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [Automagic Configuration in Python](https://www.wearedevelopers.com/videos/363-automagic-configuration-in-python) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [MLOps And AI Driven Development](https://www.wearedevelopers.com/magazine/82-mlops-and-ai-driven-development) - [From Prototype to Production: Build AI Agents with This Free 4-Course Learning Path](https://www.wearedevelopers.com/magazine/655-from-prototype-to-production-build-ai-agents-with-this-free-4-course-learning-path) - [Transforming Software Development: The Role of AI and Developer Tools](https://www.wearedevelopers.com/magazine/527-transforming-software-development-the-role-of-ai-and-developer-tools) - [What is Agentic Programming and Why Should Developers Care?](https://www.wearedevelopers.com/magazine/625-what-is-agentic-programming-and-why-should-developers-care)