AI Platform Test Lead

We are seeking a forward-thinking AI Platform Test Lead to design, build, and operationalize AI-driven Quality Engineering (QE) testing platforms that accelerate innovation across the Software Development Lifecycle (SDLC). This role will lead the strategy, implementation, and ongoing operation of AI-powered testing capabilities while enabling scalable, secure, and compliant adoption of AI technologies across enterprise software environments.

The ideal candidate will combine deep technical expertise in AI platforms and software quality engineering with the ability to evaluate emerging technologies and rapidly operationalize high-value capabilities within a regulated environment.

Key ResponsibilitiesAI Quality Engineering Platform Strategy & Architecture

• Define and evolve the enterprise AI QE testing platform architecture across cloud and data ecosystems.
• Design interoperable AI-driven testing solutions supporting both AI-enabled and traditional enterprise software applications.
• Build and support AI testing capabilities across:
• AWS AI services (Bedrock, SageMaker, model hosting, orchestration)
• Databricks / Mosaic AI (ML lifecycle, feature engineering, LLMOps)
• Enterprise LLM platforms such as Claude
• SaaS applications and internally developed software

AI Capability Engineering & Operations

• Design and operationalize reusable AI testing capabilities including:
• Prompt engineering, tool orchestration, and AI agent frameworks
• AI evaluation, monitoring, observability, and performance measurement
• Secure and compliant AI usage aligned with regulated industry requirements
• Implement AI governance, guardrails, auditability, and traceability across testing platforms.
• Design, develop, and maintain AI-driven defect analytics and quality metrics.
• Continuously evaluate emerging AI technologies and integrate high-value capabilities into enterprise testing practices.

AI-Driven Software Development Lifecycle

• Drive adoption of AI-enabled and agentic Software Development Lifecycle (SDLC) practices.
• Define frameworks for:
• Specification-driven AI-assisted development
• AI coding assistants and autonomous development agents
• Intelligent workflow automation across engineering processes
• Integrate AI-native development platforms into CI/CD, DevSecOps, and enterprise engineering workflows.

• Bachelor’s degree in Computer Science, Software Engineering, Information Systems, or a related technical discipline (or equivalent experience).
• 5+ years of experience in Software Quality Engineering, Test Automation, or AI/ML Engineering.
• 3+ years of hands-on experience designing or supporting AI testing platforms, preferably within AWS environments.
• Experience with one or more AI-enabled testing platforms such as:
• Tricentis Tosca
• Testim
• ACCELQ
• Mabl
• LambdaTest
• Katalon
• Experience working with enterprise large language models (LLMs) such as Claude, OpenAI, or similar platforms.
• Strong understanding of modern LLM concepts including:
• Retrieval-Augmented Generation (RAG)
• Fine-tuning
• Embeddings
• Vector databases
• Graph databases
• Multi-agent orchestration, * Experience supporting regulated environments with GxP validation processes for AI/ML systems.
• Exposure to AI development frameworks such as:
• LangChain
• Semantic Kernel
• Similar agent orchestration frameworks
• Experience with AI testing, evaluation, and model validation tooling.
• Knowledge of multi-cloud and hybrid cloud architectures.

Key Competencies

• Strategic mindset with strong hands-on technical execution.
• Ability to evaluate emerging AI technologies and translate them into scalable business solutions.
• Strong systems thinking across platforms, data architecture, AI workflows, and software engineering.
• Excellent communication and stakeholder management skills with the ability to influence both technical teams and senior leadership.
• Highly collaborative with a bias toward experimentation, continuous learning, and iterative delivery.
• Passion for advancing software quality through AI-enabled automation and intelligent engineering practices., * Bachelor’s (Required), * Software Quality, Test Automation or AI/ML Engineering: 5 years (Required)
• AI Testing Platform (design or support): 3 years (Required)
• AWS environment: 2 years (Preferred)

Pulled from the full job description

• 401(k)
• Health insurance
• Vision insurance
• Dental insurance, * 401(k)
• Dental insurance
• Health insurance
• Vision insurance