> Markdown version of [/jobs/ext/4693678-practice-architect-testing](https://www.wearedevelopers.com/jobs/ext/4693678-practice-architect-testing). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Practice Architect-Testing - **Company:** TEKSYSTEMS INC. - **Location:** Dallas, TX, United States (Remote available) - **Experience:** Expert - **Salary:** $128,000 - $192,000 - **Contract:** Permanent contract - **Skills:** AI Evaluation, Java (Programming Language), JavaScript (Programming Language), Agile Methodology, Artificial Intelligence, Amazon Web Services, Automation of Tests, Microsoft Azure, Continuous Integration, Data Validation, Extract Transform Load (ETL), Data Security, Database Applications, Python (Programming Language), Machine Learning, Open Source Technology, Microsoft Copilot, Selenium, Software Engineering, Systems Integration, Strategies of Testing, TypeScript, Enterprise Software Applications, Retrieval-Augmented Generation, Large Language Models, Prompt Engineering, Hallucination Detection, Model Validation, Generative AI, Agentic-AI, AI Platforms, Information Technology, Playwright, RAGAS (Retrieval Augmented Generation Assessment), Data Management, Machine Learning Operations, DeepEval, Drift Detection, Azure AI, Restful APIs, Promptfoo - **Published:** November 12, 2026 - **Apply:** https://dejobs.org/x/x/C91DFFC40BC4484B8A1B0B29D1976055/job/ ## About the Role * 10+ years in software development or quality engineering, or system integration, including at least 3 years in an architect role (test, solution, or QE architect) at a recognized technology organization. * Proven track record designing enterprise test strategies, automation architectures, and frameworks from scratch using Java, Python, or JavaScript/TypeScript. * Demonstrated hands-on experience designing and executing testing strategies for AI/ML and GenAI systems, with a strong understanding of AI risks such as drift, bias, hallucination, and security. * Expertise in using open-source and commercial AI models, copilots, prompt engineering, and agentic AI to build testing solutions or accelerators. * Strong knowledge of UI and API automation (e.g., Playwright, Selenium, Rest Assured, Karate), CI/CD, and cloud platforms (AWS, Azure, or GCP). * Familiarity with model evaluation metrics, AI observability, and data validation strategies for data-driven applications. * Experience shaping Statements of Work, proposals, and estimates in a consulting or client-facing environment. * Excellent communication, presentation, and executive stakeholder skills; able to explain AI risk and quality trade-offs to technical and business audiences. * Bachelors degree in computer science or a related engineering field, or equivalent experience. * Ability to travel to the United States or Canada, and satisfactory completion of standard employment screening., * AI evaluation: Experience with evaluation frameworks such as DeepEval, Ragas, promptfoo, or Azure AI / Vertex AI evaluation tools. * AI platforms: Exposure to MLOps/LLMOps platforms, vector databases, and AI observability tools. * Specialization: Depth in one or more domains such as performance, ETL/data, security, or accessibility testing. * Industry: Experience in financial services, healthcare, telecommunications, energy, or other regulated industries. * Certifications: One AI testing or cloud certification (e.g., ISTQB CT-AI/CT-GenAI, Azure AI Engineer, AWS ML) and optionally SAFe. Valued, not required. ## Description We are seeking a Practice Architect to lead the design and delivery of AI-driven quality engineering solutions across client engagements. You will architect test strategies and frameworks for both traditional and AI-enabled systems, including ML models, GenAI, LLMs, RAG, and agentic applications, and apply AI to transform how testing itself is designed, automated, and governed. This is a senior , client-facing architecture role. You will shape solutions and Statements of Work, build reusable AI testing accelerators, guide delivery teams, and advise clients on trustworthy, responsible AI adoption. Proven architecture experience is required; AI is central to the role, grounded in strong quality engineering fundamentals., Solution Architecture and Practice Leadership * Architect end-to-end quality engineering solutions and reference architectures for enterprise applications, data platforms, and AI-enabled systems. * Provide architectural direction across engagements; review designs, set standards, and ensure solutions are scalable, secure, and maintainable. * Evaluate and recommend tools, platforms, and AI models based on fit, cost, risk, security, and client constraints. * Grow the CQ practice by identifying improvement opportunities, defining best practices, and documenting reusable patterns. * AI Quality Engineering * Design testing methodologies for deterministic and non-deterministic systems, covering data quality and lineage, model behavior and performance, bias, fairness, explainability, drift, robustness, and security. * Define evaluation approaches for GenAI, RAG, and agentic systems, including golden datasets, LLM-as-judge, rubric-based scoring, groundedness, hallucination, prompt injection, and tool-use validation. * Embed AI quality gates into CI/CD and MLOps/LLMOps workflows, with production monitoring, observability, and drift detection. * Define and track AI quality metrics (accuracy, precision/recall/F1, drift, bias indicators, latency, cost, production stability) and support AI governance aligned with client, regulatory, and risk requirements. AI-Driven Testing Accelerators * Architect and build reusable accelerators using LLMs, copilots, prompt engineering, agentic AI, and MCP-based integrations, such as AI-assisted test generation, self-healing automation, prompt validation, and AI regression suites. * Establish guardrails for AI-assisted delivery: human review, deterministic validation, traceability, secure data handling, and measurable ROI. * Client Advisory and Delivery Leadership * Serve as a trusted AI quality advisor through workshops, AI quality assessments, solution design, demos, proofs of concept, and proofs of value. * Lead creation and revision of Statements of Work, estimates, and proposals with clear assumptions, dependencies, and scope boundaries. * Provide technical leadership and mentorship across delivery teams, scaling AI testing capability and raising maturity in CQ principles, test automation, and quality architecture. * Partner with engineering, data science, product, and security teams to embed quality early and continuously across Agile, Waterfall, and hybrid delivery.