AI QE Test Platform Strategy & Architecture

• AI QE Test Platform Strategy & Architecture

o Help define and mature Exelixis’ enterprise AI QE Test platform architecture across cloud and data ecosystems

o Design interoperable AI driven QE test solutions to support software solutions to support

o AWS AI stack (Bedrock, SageMaker, model hosting, orchestration)

o Databricks / Mosaic AI (ML lifecycle, feature engineering, LLM ops)

o Claude for Enterprise (secure conversational AI and enterprise workflows)

o SaaS and in-house developed software products

• AI Capability Engineering & Operations

o Operationalize reusable AI capabilities:

§ Prompt, tool, and agent orchestration frameworks

§ Evaluation, monitoring, and observability pipelines

§ Enable secure, compliant AI usage (GxP, HIPAA where applicable)

o Implement AI platform guardrails

o Auditability and traceability

• Design and operationalize Defect Statistics
• Drive adoption of agentic software development lifecycle (SDLC) practices
• Define frameworks for

• 5+ years in software quality engineering and testing, AI/ML engineering
• 3+ years hands-on experience with AI Test platforms (AWS preferred)
• Proven experience with:

o Experience with one or more AI QE Testing Platforms: Tricentis Testim/Tosca, ACCELQ, Mabl, LambdaTest, Katalon

o Enterprise LLM platforms (e.g., Claude, OpenAI, or similar)

o Strong understanding of LLM architectures (RAG, fine-tuning, embeddings, Vector DBs, Graph DBs, Multi agent orchestration)

Preferred Qualifications

• Familiarity with:

o GxP validation processes for AI/ML systems

• Exposure to:

o Agent frameworks (LangChain, Semantic Kernel, etc.)

o AI testing and evaluation tooling

o Multi-cloud / hybrid architectures

Key Competencies

• Strategic + hands-on balance (thinks like an architect, executes like an engineer)
• Ability to translate emerging AI trends into enterprise value
• Strong systems thinking across platforms, data, and workflows
• Excellent stakeholder communication-able to influence senior leadership and engineering teams alike
• Bias for action-rapid experimentation and iterative delivery

TECHNICAL SKILLS

Must Have

• Agile
• Agile Application Development
• AI Test Platform
• Amazon Web Services (AWS)
• Anthropic Claude AI
• Copilot
• GitHub
• IT software QE Testing leveraging AI test tools and test platforms
• SDLC

We are seeking a forward-thinking AI Platform Test Lead to design, build, and operationalize the QE testing leveraging AI tools and utilities to accelerate innovation in IT’s Software Development Lifecycle. This role is W2 hourly with Roth Staffing, with all work taking place for our client in Alameda, CA.