Test Engineering Analyst

Conduct deep technical analysis and testing of AI/ML and Generative AI systems, including validation and verification of model outputs, prompt behavior, and end-to-end application workflows.

• Participate in Business Requirements reviews and collaborate with systems analysts and developers to fully test application changes, including AI/ML model integrations and Gen AI features.
• Understand functional specifications and scope of releases to design robust testing harnesses that ensure quality delivery of AI-powered applications.
• Write complex SQL queries for retrieval of data from multi-database environments, including querying ML feature stores and model metadata. (Strong knowledge of Oracle database required).
• Apply expert systems analysis and design techniques to complex enterprise systems, with a focus on validating Gen AI outputs, LLM responses, and ML model predictions.
• Maintain broad knowledge of data sources/flow, interactions of complex systems, and the capabilities and limitations of AI/ML systems, including model drift, hallucination detection, and bias evaluation.
• Work closely with Product Owners, developers, Data Scientists, and stakeholders to understand AI/ML product specifics and create corresponding test strategies, including evaluation frameworks for generative outputs.
• Execute manual and exploratory testing on frontend (UI) and backend (API) systems, including AI/ML inference endpoints and Gen AI prompt pipelines.
• Develop and maintain testing suites for a large-scale Salesforce environment integrated with AI/ML capabilities such as Salesforce Einstein and external LLM services. Responsible for front-end and back-end testing for applications built on the Salesforce platform, including AI-driven features and automated decision systems.

Bachelor’s degree in Computer Science, IT, or related field is preferred

• 3+ years of experience in software testing and test automation, leveraging Selenium Java / BDD Framework
• Proven experience with AI tools for test generation, refactoring, and self-healing
• Experience building and scaling testing agents
• Knowledge of context engineering for AI-driven testing
• Experience with modern testing frameworks beyond Selenium
• Background in machine learning or AI applications
• Experience with microservices and containerized applications

Required Technical Skills

• Databases: Snowflake and Databricks
• Programming Languages: Python, Java/Selenium, SQL
• AI/ML Frameworks: LangChain, Pandas, and other popular Machine Learning libraries
• Test Automation: Selenium, Playwright, API testing frameworks
• Clouds: Any Cloud platform will suffice, AWS preferred (Bedrock, Sagemaker)
• CI/CD: CloudBees or Jenkins

Soft Skills

• Strong analytical and problem-solving abilities
• Excellent communication and collaboration skills
• Adaptability to rapidly evolving AI and automation technologies
• Strategic thinking with focus on testing efficiency and coverage
• Self-motivated with ability to work independently Apply Now