Senior AI QA Engineer (Manual & Automation)

• Live Game Auditing: Monitor and audit live sports broadcasts (NBA, MLB, NFL, NHL) to ensure the AI/Inference Service correctly tracks, frames, and labels major moments (e.g., touchdowns, home runs, buzzer-beaters) in real-time
• Precision QA with Media Assets: Work directly with video frames, timecodes, transcriptions, captions, and JSON outputs to ensure pinpoint alignment between AI detections and actual broadcast moments
• Model Error Triage & Support: Act as the human-in-the-loop expert to catch AI hallucinations, misinterpretations of complex sports rules, or edge-case errors. Collaborate directly with AWS and core engineering teams to detail bugs and validate model resolutions
• Metadata & Brand Safety Validation: Review AI-generated labels/tags to ensure they align with sports context and meet advertising industry compliance standards (e.g., IAB guidelines), ensuring content is brand-safe for monetization
• QA Automation: Design and execute automation scripts to compare AI inference outputs against customer-provided ground truth data at scale
• Iterative Testing & Documentation: Keep meticulous records of inference service performance, track accuracy metrics, log defects, and help streamline iterative testing processes for ongoing model updates

We are seeking a highly detail-oriented AI QA Engineer with a mix of manual and automation testing skills to validate the performance of cutting-edge, AI-driven video analysis systems. In this role, you will focus on verifying how advanced AI, Large Language Models (LLMs), and computer vision models detect key moments and generate metadata for sports content.

This position sits at the intersection of traditional sports broadcasting and advanced artificial intelligence. The ideal candidate has a deep passion for sports (understanding rules, metrics, and context), strong technical QA scripting skills, and experience working with video assets, timecodes, and metadata., * Hybrid QA Experience: Proven experience in both manual and automation testing, ideally in video-focused or AI-driven environments

• Automation Scripting: Hands-on automation scripting skills (e.g., Python, Selenium, or similar testing frameworks) to validate JSON outputs and data payloads against ground truth datasets
• SDLC & QA Methodologies: Strong understanding of testing lifecycles, including detailed bug logging, defect triage, regression testing, and quality reporting
• LLM & CV Familiarity: Solid understanding of testing LLMs (workflows, prompt/response validation) and conceptual familiarity with computer vision and transcription analysis
• Media & Video Literacy: Experience working with video frame analysis, timecodes, subtitles/captions, and metadata structures
• Sports Domain Knowledge: A deep understanding of major sports leagues (NFL, NBA, MLB, NHL), including gameplay rules, terminology, and key metrics
• Communication: Strong verbal and written communication skills to act as a bridge between technical development teams and business stakeholders
• Analytical Mindset: An iterative, meticulous approach to data validation, checking model outputs for accuracy, bias, and context

Nice to have

• Prior experience with specialized video testing tools, media processing pipelines, or video player frameworks
• Professional background in sports analytics, sports media, or digital broadcasting technology

This Remote Position Cannot be Performed in New York City.

This posting includes a good faith range of the salary EPAM would reasonably expect to pay the selected candidate. The range provided reflects base salary only. Individual compensation offers within the range are based on a variety of factors, including, but not limited to: geographic location, experience, credentials, education, training; the demand for the role; and overall business and labor market considerations. Most candidates are hired at a salary within the range disclosed. Salary range: $100 - $105. In addition, the details highlighted in this job posting above are a general description of all other expected benefits and compensation for the position.

EPAM Systems, Inc. is an equal opportunity employer. We recognize the value of diversity and inclusion in creating success for our customers, business partners, shareholders, employees and communities. We are committed to recruiting, hiring, developing and promoting employees without discrimination. As a global employer, this commitment includes complying with all laws in the countries in which we operate. Nevertheless, we believe equal employment practices should not be limited to what the law requires. Equal opportunity and inclusion are essential to motivate, empower and recognize the best in everyone.

At EPAM, employment actions are based on individual qualifications, without regard to race, color, religion, creed, gender, pregnancy status, sexual orientation, gender identity, gender expression, marital or familial status, national origin, ancestry, genetics, age, disability status, veteran status, citizenship status when otherwise legally able to work, or any other characteristic protected by law.