> Markdown version of [/jobs/ext/2141474-ai-software-test-engineer](https://www.wearedevelopers.com/jobs/ext/2141474-ai-software-test-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # AI Software Test Engineer - **Company:** Aquent - **Location:** Ann Arbor, MI, United States - **Contract:** Permanent contract - **Skills:** Testing (Software), Java (Programming Language), JavaScript (Programming Language), Computer-Aided Design, Application Programming Interfaces (APIs), Agile Methodology, Artificial Intelligence, Automation of Tests, C Sharp (Programming Language), Information Systems, Continuous Delivery, Continuous Integration, Distributed Systems, Github, Junit, Python (Programming Language), Load Testing, Systems Development Life Cycle, Regression Testing, Software Safety, Selenium, Software Engineering, Systems Integration, Test Data, Strategies of Testing, TypeScript, Web Applications, Performance Testing, Postman, Spring Cloud, Large Language Models, Generative AI, Pytest, AI Platforms, Integration Tests, Kubernetes, Information Technology, Playwright, Enterprise Integration, Virtual Agents, Automation Anywhere, Api Management, Docker, Jenkins - **Published:** August 20, 2026 - **Apply:** https://find.jobs/jobs-near-me/apply/ats-redirect/?id=2935124042-2 ## About the Role * Bachelor's degree in Computer Science, Software Engineering, Information Systems, or a related technical field. * 3-6 years of experience in software testing, QA automation, or quality engineering. * Proven experience developing automated test solutions using Python, Java, JavaScript/TypeScript, or C#. * Demonstrated experience with API testing and automation tools. * Strong understanding of test automation principles, the Software Development Lifecycle (SDLC), Agile methodologies, and CI/CD pipelines. * Experience testing distributed systems, web applications, and APIs. * Proficiency in AI Testing concepts including Prompt Testing, Response Evaluation, Hallucination Detection, Agent Workflow Validation, RAG Validation, AI Safety Testing, AI Regression Testing, and AI Benchmarking. * Expertise in Quality Engineering practices such as Test Automation, API Testing, Integration Testing, Performance Testing, Load Testing, Security Testing, Defect Analysis, and Root Cause Investigation. * Familiarity with industry-standard tools and technologies like Selenium, Playwright, Postman, JUnit / PyTest, GitHub Actions, Jenkins, Docker, and Kubernetes. Nice-to-Have Qualifications * 1-3 years of exposure to AI/ML or Generative AI technologies. * Experience testing Generative AI applications, large language model (LLM)-based systems, AI agents, RAG applications, or complex platform integrations. * Familiarity with leading AI models and platforms. * Experience building evaluation and benchmarking frameworks for AI solutions. * Experience testing cloud-native applications on major cloud platforms. * Knowledge of responsible AI, AI governance, and AI risk management practices, understanding how AI initiatives align with enterprise governance, risk, privacy, and compliance requirements. ## Description Are you ready to dive into the exciting world of Artificial Intelligence and make a tangible impact on the next generation of intelligent systems? We are seeking a visionary and hands-on engineer to spearhead the quality assurance of groundbreaking AI-powered applications, machine learning systems, AI agents, and generative AI solutions. In this pivotal role, you will be instrumental in ensuring that our AI systems are not just innovative, but also reliable, safe, accurate, and performant, directly contributing to the trust and success of our advanced technology initiatives and the millions of users we serve. Your Impact and the Opportunity As a critical contributor, you will own the end-to-end testing and automation efforts for robust AI solutions. You'll collaborate closely with AI developers, architects, and product teams to guarantee production-quality AI, directly reducing regression defects, improving confidence in AI model behavior, and enabling the safe, scalable delivery of AI-powered features. This role offers significant opportunities for innovation and collaboration across diverse teams, directly shaping the reliability and success of cutting-edge AI technologies that meet stringent quality, security, and governance requirements. * Develop and implement comprehensive testing strategies for AI applications, platforms, and services. * Validate AI model outputs for accuracy, consistency, reliability, and safety, performing adversarial, negative, and edge-case testing to identify potential failures and hallucinations. * Design and execute functional, integration, end-to-end, regression, and performance tests specifically tailored for AI solutions. * Create intricate test cases for prompt-driven, agentic, and retrieval-based AI workflows, ensuring robust validation of AI guardrails, business rules, permissions, and governance controls. * Build and maintain advanced automated test frameworks and evaluation pipelines for AI responses and workflows, integrating AI testing seamlessly into continuous integration/continuous delivery (CI/CD) pipelines. * Implement automated quality scoring and regression detection mechanisms, and create reusable test data, mocks, simulators, and validation frameworks. * Test AI agents, workflows, APIs, complex system integrations, and tool-calling capabilities, validating integrations with external systems, data sources, and enterprise services. * Verify performance, reliability, scalability, and resiliency of AI workloads, including executing load and stress testing for AI services. * Collaborate proactively with software engineers, AI engineers, product owners, architects, and security teams, providing critical quality feedback during design reviews and development. * Contribute significantly to test strategy, quality standards, and best practices, and support production readiness reviews and defect triage activities. * Contribute to projects such as testing AI chat assistants and copilots, validating AI agent workflows, evaluating retrieval augmented generation (RAG) search quality, automating AI response evaluation frameworks, and performance testing AI services and orchestration platforms. ## Related Videos - [AI as a Test Designer: Transforming Experience into Automated Testing](https://www.wearedevelopers.com/videos/1984-ai-as-a-test-designer-transforming-experience-into-automated-testing) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [How Unit Testing Saved My Career](https://www.wearedevelopers.com/videos/1642-how-unit-testing-saved-my-career) - [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [Transforming Software Development: The Role of AI and Developer Tools](https://www.wearedevelopers.com/magazine/527-transforming-software-development-the-role-of-ai-and-developer-tools) - [Top AI Tools for Developers in 2025](https://www.wearedevelopers.com/magazine/560-top-ai-tools-for-developers-in-2025) - [What is Agentic Programming and Why Should Developers Care?](https://www.wearedevelopers.com/magazine/625-what-is-agentic-programming-and-why-should-developers-care)