Senior Software Engineer, SQA - AI/ML- IND

We are seeking a Senior Software Engineer, SQA - AI/ML Products to ensure the quality, safety, and reliability of AI-driven capabilities across our digital products. This role focuses on validating LLM-powered applications such as chatbots, copilots, RAG-based search, recommendations, and agentic workflows, partnering closely with Product, Engineering, Data Science, and MLOps teams to define expected behaviors, edge cases, and quality standards., The Senior SQA Engineer will design and execute automated and AI-specific testing strategies across UI, APIs, and model-level interactions, integrating quality checks into CI/CD pipelines. Responsibilities include evaluating AI outputs for accuracy, grounding, hallucinations, bias, and safety; validating end-to-end RAG pipelines; monitoring production systems for drift, regressions, and performance issues; and supporting root-cause analysis and defect resolution. This role combines strong software quality engineering fundamentals with emerging GenAI evaluation techniques to ensure AI systems behave predictably and responsibly in production.

6+ years of experience in Software Quality Assurance or Test Automation, including experience testing AI/ML-based products.

-Strong proficiency in Python or JavaScript.

-Experience with cloud services preferably Azure.

-Experience testing RESTful APIs using tools such as Postman or Swagger.

-Hands-on experience with automation frameworks including Selenium, Cypress, Playwright, or PyTest.

-Working knowledge of LLM evaluation frameworks (e.g., LangSmith, RAGAS) and vector database APIs.

-Solid foundation in QA best practices: test planning, regression testing, CI/CD integration, and defect lifecycle management. -Experience testing RAG-based systems or agentic AI workflows at scale.

-Familiarity with model and data drift monitoring tools (e.g., Azure Monitor, Vertex AI monitoring, Fiddler).

-Exposure to ethical AI, bias testing, or responsible AI evaluation methodologies.* Experience working in regulated industries such as healthcare, life sciences, or financial

services.

-Understanding of MLOps or LLMOps concepts.