AI Testing Architect - REMOTE
Role details
Job location
Tech stack
Job description
Typical task breakdown: - Define the AI vision and architecture for the testing function. - Identify testing activities that can be improved with AI, including: test case generation, test data creation, automated test maintenance, defect prediction, root cause analysis, log and failure analysis, requirements-to-test traceability, risk-based test prioritization, intelligent regression selection - Design and implement AI-enabled frameworks for: functional testing, API testing, UI automation, performance testing, security testing support, exploratory testing augmentation - Partner with QA, developers, DevOps, product managers, and data teams to embed AI into CI/CD and quality engineering workflows. - Evaluate and recommend AI/ML, GenAI, and test automation tools. - Establish standards for AI governance, privacy, security, explainability, and responsible usage in testing. - Create architecture for integrating LLMs, test repositories, defect systems, observability tools, and CI platforms.
- Define metrics to measure the effectiveness of AI in testing, such as:defect leakage, automation coverage, test execution efficiency, flaky test reduction - mean time to diagnose failures, release quality trends - Mentor QA engineers and SDETs on AI-assisted testing practices. - Lead pilots, proofs of concept, and scaled rollout of AI testing capabilities. - Ensure compliance with enterprise policies for data handling and model usage. Position's Contributions to Work Group: - We're looking for an AI Architect to lead how our testing organization uses AI to improve quality, speed, coverage, and insight across the software delivery lifecycle. - This person will define the AI strategy for QA and testing, design AI-enabled testing solutions, guide tool selection and integration, and help teams adopt AI responsibly in day-to-day testing activities.
Requirements
Hands-on experience with tools/languages such as Python, Java, JavaScript, SQL, and test tools like Selenium, Playwright, Cypress, Appium, JUnit, TestNG, PyTest, or similar. - Experience integrating with platforms such as Jira, Azure DevOps, GitHub, Jenkins, GitLab, or cloud test platforms. - Knowledge of test analytics, data pipelines, and observability/log analysis tools. - Strong architecture and design skills for scalable, secure enterprise solution (Desired) - Experience building AI copilots or assistants for QA teams. - Familiarity with vector databases, model evaluation, and MLOps/LLMOps concepts. - Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform. - Understanding of regulated environments and compliance requirements. - Certifications in test architecture, cloud, AI/ML, or enterprise architecture. - Hands on experience with Python - development experience in AWS Cloud technology - Experience in TDD, continuous integration, code review practice is strongly desired - Experience with Apigee or other API gateways is a plus - Experience with DevOps concepts and tools (e.g., CI/CD, Jenkins, Git) - At least 2 years working on an Agile team with a solid understanding of Agile/Lean practices. - Understanding of a micro service Architecture - Experience load and performance testing - Strong documentation skills
Benefits & conditions
This is a Contract position based out of Chicago, IL. Pay and Benefits The pay range for this position is $50.00 - $100.00/hr. Eligibility requirements apply to some benefits and may depend on your job classification and length of employment. Benefits are subject to change and may be subject to specific elections, plan, or program terms. If eligible, the benefits available for this temporary role may include the following: Medical, dental & vision Critical Illness, Accident, and Hospital 401(k) Retirement Plan - Pre-tax and Roth post-tax contributions available Life Insurance (Voluntary Life & AD&D for the employee and dependents) Short and long-term disability Health Spending Account (HSA) Transportation benefits Employee Assistance Program Time Off/Leave (PTO, Vacation or Sick Leave) Workplace Type This is a fully remote position.