> Markdown version of [/jobs/ext/1229499-verification-validation-data-scientist](https://www.wearedevelopers.com/jobs/ext/1229499-verification-validation-data-scientist). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Verification & Validation Data Scientist - **Company:** RMA of New York, LLP - **Location:** Alexandria, VA, United States (Remote available) - **Experience:** Expert - **Salary:** $110,000.0 - $135,000.0 - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Audit Trail, Machine Learning, Large Language Models, Model Validation, Information Technology - **Published:** July 10, 2026 - **Apply:** https://www.careerjet.com/job/us7f477c3e3d1a328bdcfb117510c1c096/eaa ## About the Role ยท Bachelor's degree in computer science, mathematics, or engineering or relevant equivalent experience in lieu of degree. Master's degree preferred. Experience * Seven (7) or more years in data science, model validation, ML evaluation, or an independent V&V / QA function. * Demonstrated experience independently validating models or systems against ground-truth data, evaluation design, benchmarking, and error analysis. Certifications - NA Knowledge, Skills & Abilities * Strong applied statistics and metrics design; fluency in accuracy, bias, calibration, and uncertainty quantification. * Working understanding of LLMs and synthetic-respondent systems and their failure modes - enough to test them rigorously. * Independence and rigor: the temperament to challenge results and withhold sign-off, including from colleagues' work. * Clear communication: able to tell a non-technical executive, with evidence, why a result can or cannot be trusted. * Ability to effectively leverage artificial intelligence (AI) tools and technologies to streamline workflows, enhance productivity, and improve overall work quality. * Familiarity with survey/behavioral data as a validation target preferred. Physical Requirements This position operates in a typical office environment (which includes a home office setting) and requires the ability to perform essential job functions with or without reasonable accommodation. Physical requirements may include: * Prolonged periods of sitting at a desk and working on a computer. * Frequent use of hands and fingers for typing, handling documents, and using office equipment. * Occasional standing, walking, bending, and reaching. * Ability to lift and carry up to 30 pounds as needed. * Clear verbal and written communication skills for effective interaction with colleagues and stakeholders. ## Description This role exists to answer one question independently: do the synthetic-audience and automated systems actually perform as claimed? It is a verification-and-validation function drawn from data science, closer to model validation, ML evaluation, and the independent V&V discipline used in regulated and safety-critical software than to market research. The V&V Data Scientist will design and run the evaluation that benchmarks system output against ground-truth data, owns the accuracy metrics and error bounds, and certifies fitness for use, working independently of the engineer who builds the models. The separation of duties is the point: the builder does not grade their own work. Responsibilities * Design and run the independent validation methodology: benchmark synthetic-audience and automated output against real panel holdout and other ground-truth data. * Own the evaluation metrics, test harness design, and reported accuracy bounds; quantify where each system is and is not reliable. * Certify fitness for use, translate results into clear "validated for this use / not validated for that use" determinations. * Operate independently of the AI/Synthetic Engineer: verify the models and pipelines AI/Synthetic Engineer builds without having built them. * Monitor for model and data drift over time; re-validate as models, prompts, and source data change. * Maintain the audit trail and evidence base that supports the function's credibility with stakeholders and any external auditor. ## Related Videos - [AI in Regulated Industry - Validating AI-Enabled Products with PLM and Digital Twins](https://www.wearedevelopers.com/videos/2065-ai-in-regulated-industry-validating-ai-enabled-products-with-plm-and-digital-twins) - [Introduction to Responsible AI: Balancing Value and Risk](https://www.wearedevelopers.com/videos/1972-introduction-to-responsible-ai-balancing-value-and-risk) - [Resilient by Design: Building Robust Architectures in High-Stakes Financial Systems](https://www.wearedevelopers.com/videos/2106-resilient-by-design-building-robust-architectures-in-high-stakes-financial-systems) - [Introduction to Azure Machine Learning](https://www.wearedevelopers.com/videos/368-introduction-to-azure-machine-learning) - [Is my AI alive but brain-dead? How monitoring can tell you if your machine learning stack is still performing](https://www.wearedevelopers.com/videos/262-is-my-ai-alive-but-brain-dead-how-monitoring-can-tell-you-if-your-machine-learning-stack-is-still-performing) - [No Keys for the Robot: GitOps as the Control Plane for Autonomous Agents](https://www.wearedevelopers.com/videos/100095-no-keys-for-the-robot-gitops-as-the-control-plane-for-autonomous-agents) ## Related Articles - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [How to start an AI project for a good cause and boost your career](https://www.wearedevelopers.com/magazine/15-how-to-start-an-ai-project-for-a-good-cause-and-boost-your-career) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [What Are Large Language Models?](https://www.wearedevelopers.com/magazine/304-what-are-large-language-models) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [How machine learning can help us tell fact from fiction](https://www.wearedevelopers.com/magazine/509-how-machine-learning-can-help-us-tell-fact-from-fiction)