Quality Engineer, Data & AI Platform

The Senior Quality Engineer - Data & AI sits at the intersection of data engineering, machine learning product delivery, and quality assurance. This role is responsible for validating AI-powered and data-driven features across M3's hospitality accounting platform, with primary ownership of quality for the Data & AI team's active products.

Unlike traditional QE roles focused on deterministic software, this position requires quality thinking applied to systems that produce probabilistic outputs - where "passing" is defined by accuracy thresholds, not binary correctness. The right candidate understands how AI products fail differently and is motivated to build the testing practice that prevents those failures from reaching customers.

This is an individual contributor role with significant cross-functional responsibility, including collaboration with Engineering, Product, Data, and external contractor teams.

Essential Duties:

The duties listed below are the essential functions of this position, and they may change as the needs of the company demand. All associates are expected to do what is necessary to get the work done and to cooperate fully with their supervisor's requests for additional or altered duties.

• Design and execute test strategies for AI/ML-enabled features; define acceptance thresholds (precision, recall, F1) in partnership with Engineering and Product.
• Validate AI-generated outputs - mappings, anomaly flags, LLM summaries - against domain-grounded acceptance criteria; build regression suites to detect concept drift, label drift, and feature degradation.
• Evaluate LLM outputs for accuracy, hallucination, and format compliance; build prompt regression test suites to catch behavior changes when model versions or system prompts are updated.
• Write data validation logic in Python or SQL against tables across Bronze, Silver, and Gold layers; design and automate data contract tests covering schema, null rates, referential integrity, and row counts.
• Implement automated validation checkpoints in Databricks pipelines; apply statistical testing methods to pipeline output validation.
• Define and instrument production monitoring checks for model performance: alert thresholds, confidence degradation, and data drift.
• Perform functional, regression, integration, and exploratory testing across releases, enhancements, and defect fixes using manual and automated approaches.
• Design, develop, maintain, and execute automated test scripts using frameworks such as Katalon and Playwright; support CI/CD quality gates.
• Develop and execute test cases for ETL processes, data warehouse workflows, and API validation.
• Review application logs and monitoring tools to identify and track defects; coordinate builds, deployments, and software migrations for planned releases and hotfixes.
• Maintain and utilize testing tools including Azure DevOps, SQL Server, Postman/Swagger/SoapUI, and automation frameworks.
• Participate in Agile ceremonies; collaborate with Product, Engineering, Data, and external contractor teams to ensure comprehensive coverage and aligned acceptance criteria.
• Communicate quality risks early and in writing; develop and maintain a QE runbook for AI/ML products.
• Adhere to secure testing practices and compliance requirements associated with the assigned Technical Security Level.
• Other duties as assigned., Are you legally authorized to work for any employer in the US without restriction?* Select... Will you now, or in the future, require immigration support for employment status?* Select...

Please answer this question accurately. At this time, we are unable to provide current or future sponsorship for employment authorization. This includes H-1B, OPT, STEM OPT extensions, or other visa classifications requiring employer sponsorship. Are you willing to undergo a background check, in compliance with applicable laws?* Select... What are you compensation expectations for you to consider a new role?*

Voluntary Self-Identification

For government reporting purposes, we ask candidates to respond to the below self-identification survey. Completion of the form is entirely voluntary. Whatever your decision, it will not be considered in the hiring process or thereafter. Any information that you do provide will be recorded and maintained in a confidential file.

As set forth in M3's Equal Employment Opportunity policy, we do not discriminate on the basis of any protected group status under any applicable law. Gender Select... Are you Hispanic/Latino? Select... Race & Ethnicity Definitions

If you believe you belong to any of the categories of protected veterans listed below, please indicate by making the appropriate selection. As a government contractor subject to the Vietnam Era Veterans Readjustment Assistance Act (VEVRAA), we request this information in order to measure the effectiveness of the outreach and positive recruitment efforts we undertake pursuant to VEVRAA. Classification of protected categories is as follows:

A "disabled veteran" is one of the following: a veteran of the U.S. military, ground, naval or air service who is entitled to compensation (or who but for the receipt of military retired pay would be entitled to compensation) under laws administered by the Secretary of Veterans Affairs; or a person who was discharged or released from active duty because of a service-connected disability.

A "recently separated veteran" means any veteran during the three-year period beginning on the date of such veteran's discharge or release from active duty in the U.S. military, ground, naval, or air service.

An "active duty wartime or campaign badge veteran" means a veteran who served on active duty in the U.S. military, ground, naval or air service during a war, or in a campaign or expedition for which a campaign badge has been authorized under the laws administered by the Department of Defense.

An "Armed forces service medal veteran" means a veteran who, while serving on active duty in the U.S. military, ground, naval or air service, participated in a United States military operation for which an Armed Forces service medal was awarded pursuant to Executive Order 12985. Veteran Status Select...

• Minimum of 5 years of experience in software testing or quality engineering, with at least 2 years focused on data-intensive or AI/ML-enabled products.
• Bachelor's Degree in Computer Science, IT, MIS, or equivalent combination of education and experience.
• 3+ years of hands-on experience with Microsoft SQL Server for data validation and testing; experience using Azure DevOps for test management and release coordination.
• 2+ years of API testing experience (Postman, Swagger, or SoapUI); experience writing and executing test cases for ETL processes and data warehouse environments.
• Hands-on experience with Databricks, Delta Lake, or equivalent Lakehouse platforms and Python for data validation strongly preferred.
• Experience validating ML model outputs and familiarity with LLM evaluation, prompt regression testing, or generative AI quality workflows preferred.
• Familiarity with dbt, Great Expectations, MLflow, or equivalent pipeline-level assertion and experiment tracking tools preferred.
• Working experience with automation frameworks such as Katalon or Playwright; familiarity with USALI and hospitality GL structure a plus.
• Understanding of Agile/Scrum methodologies; strong written and verbal communication skills.
• CSQA or ISTQB certification strongly preferred; certifications in AI/ML quality or data engineering are welcomed.