Sr. Data Validation Engineer

Insight Global
New York, United States of America
2 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

New York, United States of America

Tech stack

Agile Methodologies
Artificial Intelligence
Amazon Web Services (AWS)
Business Analytics Applications
Data analysis
Business Logic
Automation of Tests
Big Data
Google BigQuery
Cloud Database
Data Validation
Data Infrastructure
Data Integrity
ETL
Database Queries
Python
Standard Sql
SQL Databases
Data Streaming
Cloud Platform System
Sql Optimization
Large Language Models
Atlassian Tools
Tools for Reporting
Software Coding
Data Inconsistencies
GPT
Data Pipelines

Job description

Insight Global is seeking a Data Validation Engineer for an end client to join its QA Automation organization and serve as the team's subject matter expert for data quality and validation. This individual will be responsible for validating large-scale datasets, analytics platforms, and ETL pipelines that support Paramount's streaming and digital products. The ideal candidate will have strong Python and SQL skills, experience working with BigQuery and cloud-based data environments, and a proven background validating data accuracy, completeness, and consistency. This role will also leverage AI tools and large language models (LLMs) to enhance data validation processes, automate analysis, improve anomaly detection, and accelerate root cause investigation. The Data Validation Engineer will partner closely with QA Automation Engineers, Data Engineers, and business stakeholders to ensure trusted data is available for reporting, analytics, and downstream testing efforts.

Responsibilities

  • Own end-to-end data validation efforts across large-scale data pipelines and analytics platforms

  • Validate data movement through ETL/ELT processes and identify discrepancies between source and target systems

  • Develop and maintain Python-based data validation, reconciliation, and quality-checking scripts

  • Write SQL queries to analyze data, investigate issues, and verify business logic

  • Perform source-to-target validation across BigQuery and cloud-based data platforms

  • Troubleshoot data quality issues and perform root cause analysis

  • Leverage AI/LLM technologies (Claude, ChatGPT, Copilot, etc.) to generate and optimize validation rules, SQL queries, reconciliation logic, and analysis summaries

  • Utilize AI-assisted techniques for anomaly detection, trend analysis, and identification of data inconsistencies

  • Partner with QA Automation Engineers to ensure reliable, trusted data is available for downstream testing efforts

  • Support validation of analytics generated across web, mobile, and connected TV (CTV) platforms

  • Collaborate within Agile teams using Jira and Confluence

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.

Requirements

  • 5+ years of experience in Data Validation, Data Quality, Big Data QA, ETL Testing, or a related field

  • Strong Python development and writing code from scratch

  • Advanced SQL skills with hands-on experience writing complex queries

  • Experience validating large-scale datasets

  • Experience testing ETL/ELT data pipelines

  • Hands-on experience with BigQuery

  • Experience performing source-to-target data reconciliation

  • Experience troubleshooting data discrepancies and conducting root cause analysis

  • Strong understanding of data quality principles, including:

o Data accuracy

o Data completeness

o Data consistency

o Data integrity

o Duplicate detection

  • Experience working within GCP or AWS cloud data environments

  • Experience with reporting tools

  • Experience leveraging AI/LLM tools to improve data validation, analysis, or engineering workflows

  • Experience creating or enhancing validation logic, reconciliation processes, or data quality checks using AI-assisted approaches

Apply for this position