> Markdown version of [/jobs/ext/3071792-lead-ii-software-testing](https://www.wearedevelopers.com/jobs/ext/3071792-lead-ii-software-testing). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead II - Software Testing

- **Company:** Amazon.com, Inc.
- **Location:** Bellevue, WA, United States
- **Experience:** Expert
- **Contract:** Temporary contract
- **Skills:** Testing (Software), 4G (Telecommunication), LTE (Telecommunication), Artificial Intelligence, Data Validation, Python (Programming Language), Machine Learning, Regression Testing, SQL Databases, Management of Software Versions, Chatbots, 5G NR, Large Language Models, AI Platforms, Low Latency, Data Pipelines, Api Management
- **Published:** September 25, 2026
- **Apply:** https://www.careerjet.com/jobad/us24ca56343a0f245a382a70278f1de7ff

## About the Role

Position Overview Job Title: Network Tester Education: Associate's degree in technical related field 2-5 years of experience in 3G, 4G, 5G, LTE preferred Job Type: Full-Time, …

 + 1 month ago

## Description

* Design and execute test plans for AI/ML-driven features, including model outputs, prompts, and integrated application behavior
 * Build and maintain automated test suites covering functional, regression, integration, and API testing
 * Evaluate model outputs for accuracy, consistency, bias, hallucination, and edge-case failures
 * Develop evaluation frameworks and golden datasets/test cases to benchmark model performance over time
 * Test prompt engineering changes, model version upgrades, and fine-tuning outputs for regressions
 * Perform adversarial and red-team style testing to surface safety, security, and robustness issues
 * Validate data pipelines feeding into AI models (data quality, schema, drift detection)
 * Collaborate with data scientists/ML engineers to define acceptance criteria and quality metrics for models
 * Test latency, scalability, and reliability of AI services under load
 * Contribute to CI/CD pipelines, integrating automated and model-evaluation tests
 * Hands-on experience testing LLM-based products (chatbots, copilots, RAG systems, AI agents) - designing test cases for non-deterministic, generative outputs
 * Practical experience with AI/LLM evaluation frameworks (e.g., Ragas, DeepEval, LangSmith, Promptfoo, OpenAI Evals, TruLens) - building eval suites, scoring rubrics, and golden datasets
 * Working knowledge of eval metrics for generative AI: hallucination rate, faithfulness/groundedness, relevance, answer correctness, toxicity/bias scoring, BLEU/ROUGE/semantic similarity where applicable
 * Experience with prompt regression testing - validating prompt changes and model/version upgrades against baseline eval sets
 * Strong proficiency in Python for writing eval scripts, test harnesses, and data validation logic
 * Familiarity with SQL and data validation techniques

TekWissen Group is an equal opportunity employer supporting workforce diversity.