AI Test Engineer

SG2 Recruiting is supporting Trend Health in the search for an AI Test Engineer II who will help ensure the quality, reliability, and safety of AI-powered capabilities across the organization's healthcare platform. This is not a traditional software QA position. You'll be responsible for validating machine learning models, large language model (LLM) outputs, AI-driven workflows, and intelligent automation that directly support healthcare operations. Working alongside engineering, data, AI, and product teams, you'll develop comprehensive testing strategies that ensure AI systems produce accurate, explainable, and reliable results while meeting business objectives and clinical expectations. If you're excited about combining software quality engineering with emerging AI technologies and want to help build trustworthy AI solutions in healthcare, we'd like to hear from you. What You Will Be Doing AI Testing & Validation

• Designing comprehensive testing strategies for AI and machine learning models, including functional, regression, integration, and scenario-based testing
• Executing manual and automated validation of AI-generated outputs, including summaries, classifications, recommendations, and predictive results
• Developing test cases that evaluate edge cases, model drift, prompt behavior, and unexpected AI responses
• Measuring model performance against defined business, technical, and quality success criteria
• Documenting testing results and communicating findings to engineering and AI stakeholders

AI Workflow & Agent Validation

• Testing AI agent workflows, orchestration logic, automation pipelines, and multi-step AI processes
• Validating integrations between AI agents, APIs, data services, and downstream applications
• Identifying workflow failures, performance bottlenecks, and opportunities to improve reliability
• Verifying end-to-end functionality across AI-enabled business processes

Data Quality & Integrity

• Validating training, validation, and testing datasets for completeness, consistency, quality, and potential bias
• Performing data validation across ingestion and processing pipelines to ensure accurate inputs and outputs
• Partnering with Data Engineering teams to identify anomalies, inconsistencies, and data quality issues

Test Automation & Engineering

• Building and maintaining automated testing frameworks for AI-enabled applications
• Developing scripts and tooling that enable scalable, repeatable validation of AI-generated outputs
• Integrating automated testing into CI/CD pipelines to support continuous validation and rapid releases
• Improving testing efficiency through automation and reusable test assets

AI Safety & Governance

• Defining and validating AI guardrails, including accuracy thresholds, explainability standards, and fallback logic
• Detecting risks associated with hallucinations, bias, model drift, and unreliable outputs
• Collaborating with Engineering and AI teams to improve AI safety, reliability, and production readiness

Collaboration & Continuous Improvement

• Partnering with Engineering, Product, QA, AI, Data Engineering, and Operations teams throughout the software development lifecycle
• Participating in Agile ceremonies, sprint planning, backlog refinement, and release validation activities
• Contributing to testing standards, best practices, documentation, and quality processes
• Supporting root cause analysis for AI-related production issues and implementing long-term quality improvements, MANTECH seeks a highly motivated and collaborative Software Engineer to join our team in Columbia, MD. In this role, you will develop, maintain, and enhance complex and diverse sof…

• 12 days ago

• Bachelor's degree in Computer Science, Data Science, Engineering, or a related technical discipline (or equivalent professional experience)
• 3+ years of experience in software quality assurance, testing, or software engineering
• Experience testing APIs, distributed systems, data pipelines, or cloud-based applications
• Strong understanding of software testing methodologies, test planning, and quality engineering principles
• Experience developing automated testing solutions
• Strong SQL skills and experience validating complex datasets
• Experience working within Agile software development environments
• Familiarity with AI and machine learning concepts, including model evaluation, inference, prompt engineering, and model behavior
• Experience working with cloud technologies such as AWS or Databricks
• Strong analytical and troubleshooting skills with the ability to identify root causes of complex technical issues
• Excellent written and verbal communication skills with the ability to collaborate across technical and business teams

• Opportunity to shape the quality and safety of cutting-edge AI healthcare solutions
• Work with modern AI, cloud, and data technologies in a collaborative engineering environment
• Influence testing strategy for next-generation AI-powered products
• Collaborate with highly skilled engineering, data science, and product teams
• Competitive compensation and comprehensive benefits
• Professional growth in one of the fastest-growing areas of software engineering

About Trend Health Trend Health is leveraging artificial intelligence and advanced data technologies to transform healthcare operations through intelligent automation, data-driven insights, and innovative digital solutions. The company is committed to delivering secure, reliable, and clinically responsible AI capabilities that improve outcomes for healthcare organizations while maintaining the highest standards of quality, compliance, and patient trust.