Data Quality Engineer

Benefits:W2 OPPORTUNITYCompetitive salaryOpportunity for advancementJob Title: Data Quality Engineer (Databricks, Kafka, AWS)Location: Dallas, TX (Hybrid - 3 days onsite)Job Type: Long-term ContractWork Authorization: Open - W2 opportunityInterview Process: In-person (Client interview- Mandatory)We are looking for a Data Quality Engineer to own validation across batch and streaming data pipelines. This role focuses on ensuring data correctness, reliability, and performance across platforms built on Databricks, Kafka, AWS, SQL, and Python.This is a hands-on role focused on building scalable data validation frameworks and ensuring production-grade data systems.Key ResponsibilitiesEnd-to-End Data Validation* Validate data pipelines for accuracy, completeness, consistency, and timeliness* Build SQL-based validations for business rules and transformations* Implement reconciliation between source and downstream systems* Ensure data lineage and traceabilityETL / ELT & Spark Testing* Test pipelines built on AWS (Glue, Lambda, EMR, Step Functions)* Validate transformations using SQL and Python* Test ingestion, transformation, aggregation, and serving layers* Handle backfills, reprocessing, and historical data loads* Validate Spark pipelines (PySpark/Scala) on DatabricksStreaming (Kafka)* Validate data integrity, ordering, and delivery guarantees* Test producer and consumer logic and serialization formats (Avro, JSON, Protobuf)* Validate topics, partitions, offsets, retention, and schema evolution* Simulate late events, duplicates, and failure scenariosAutomation & Frameworks* Build Python-based data testing frameworks* Develop reusable validation utilities and synthetic datasets* Integrate data tests into CI/CD pipelines* Enable automated alerts for data quality issuesPerformance & Reliability* Validate throughput, latency, and concurrency at scale* Test retry logic, idempotency, and recovery mechanisms* Perform regression, soak, and failover testingObservability* Validate logs, metrics, and alerts using tools such as CloudWatch, Prometheus, and Grafana* Define and monitor data SLAs and SLOs* Support incident response, root cause analysis, and postmortemsRequired Qualifications & Experience* 7+ years of total experience in QA, SDET, or Data Quality Engineering* Minimum 4-6 years of hands-on experience working with data platforms, data pipelines, or data engineering ecosystems* 3+ years of hands-on experience with Databricks and Apache Spark* Strong SQL skills for data validation, reconciliation, and complex analysis* Proficiency in Python for automation and data validation* Experience testing ETL/ELT pipelines (batch and streaming)* Hands-on experience with Kafka or similar streaming platforms* Strong understanding of AWS data services (S3, Glue, Lambda, Redshift, Athena)* Experience working with large-scale distributed data systems* Strong debugging, analytical, and problem-solving skillsNice to Have* Experience with data quality or observability tools such as Great Expectations or Monte Carlo* Knowledge of schema registry and data contracts* Experience with CI/CD tools such as GitHub Actions or JenkinsFlexible work from home options available. Compensation: $62.00 - $65.00 per hour About UsWe work to deliver profitability in your business - with effective communication, consulting, and interactive solutions. Following an Agile Work Approach, we make sure you get the ideal solutions at minimum expenses.Work ApproachOur PhilosophyOur Philosophy starts-and-ends at the Client-first approach. Be it understanding your business requirements to choosing the right technologies, we work as a collective team that takes all the possible steps to grow continuously towards our common goal.Work PolicyWe promote a collaborative work environment. We involve everyone working in the organization in community decisions and encourage them to think from a broader perspective. Our work process promotes flexibility and we maintain a high level of discipline at different levels of execution.The FutureSelectMinds have years of experience in the domain helps us understand the need-of-the-hour better. This understanding drives us to a better future with every minute ticking. We believe we will be taking off major businesses from their flagship positions, with the products we are eyeing today. Benefits:W2 OPPORTUNITYCompetitive salaryOpportunity for advancementJob Title: Data Quality Engineer (Databricks, Kafka, AWS)Location: Dallas, TX (Hybrid - 3 days onsite)Job Type: Long-term ContractWork Authorization: Open - W2 opportunityInterview Process: In-person (Client interview- Mandatory)We are looking for a Data Quality Engineer to own validation across batch and streaming data pipelines. This role focuses on ensuring data correctness, reliability, and performance across platforms built on Databricks, Kafka, AWS, SQL, and Python.This is a hands-on role focused on building scalable data validation frameworks and ensuring production-grade data systems.Key ResponsibilitiesEnd-to-End Data Validation* Validate data pipelines for accuracy, completeness, consistency, and timeliness* Build SQL-based validations for business rules and transformations* Implement reconciliation between source and downstream systems* Ensure data lineage and traceabilityETL / ELT & Spark Testing* Test pipelines built on AWS (Glue, Lambda, EMR, Step Functions)* Validate transformations using SQL and Python* Test ingestion, transformation, aggregation, and serving layers* Handle backfills, reprocessing, and historical data loads* Validate Spark pipelines (PySpark/Scala) on DatabricksStreaming (Kafka)* Validate data integrity, ordering, and delivery guarantees* Test producer and consumer logic and serialization formats (Avro, JSON, Protobuf)* Validate topics, partitions, offsets, retention, and schema evolution* Simulate late events, duplicates, and failure scenariosAutomation & Frameworks* Build Python-based data testing frameworks* Develop reusable validation utilities and synthetic datasets* Integrate data tests into CI/CD pipelines* Enable automated alerts for data quality issuesPerformance & Reliability* Validate throughput, latency, and concurrency at scale* Test retry logic, idempotency, and recovery mechanisms* Perform regression, soak, and failover testingObservability* Validate logs, metrics, and alerts using tools such as CloudWatch, Prometheus, and Grafana* Define and monitor data SLAs and SLOs* Support incident response, root cause analysis, and postmortemsRequired Qualifications & Experience* 7+ years of total experience in QA, SDET, or Data Quality Engineering* Minimum 4-6 years of hands-on experience working with data platforms, data pipelines, or data engineering ecosystems* 3+ years of hands-on experience with Databricks and Apache Spark* Strong SQL skills for data validation, reconciliation, and complex analysis* Proficiency in Python for automation and data validation* Experience testing ETL/ELT pipelines (batch and streaming)* Hands-on experience with Kafka or similar streaming platforms* Strong understanding of AWS data services (S3, Glue, Lambda, Redshift, Athena)* Experience working with large-scale distributed data systems* Strong debugging, analytical, and problem-solving skillsNice to Have* Experience with data quality or observability tools such as Great Expectations or Monte Carlo* Knowledge of schema registry and data contracts* Experience with CI/CD tools such as GitHub Actions or JenkinsFlexible work from home options available. Compensation: $62.00 - $65.00 per hour About UsWe work to deliver profitability in your business - with effective communication, consulting, and interactive solutions. Following an Agile Work Approach, we make sure you get the ideal solutions at minimum expenses.Work ApproachOur PhilosophyOur Philosophy starts-and-ends at the Client-first approach. Be it understanding your business requirements to choosing the right technologies, we work as a collective team that takes all the possible steps to grow continuously towards our common goal.Work PolicyWe promote a collaborative work environment. We involve everyone working in the organization in community decisions and encourage them to think from a broader perspective. Our work process promotes flexibility and we maintain a high level of discipline at different levels of execution.The FutureSelectMinds have years of experience in the domain helps us understand the need-of-the-hour better. This understanding drives us to a better future with every minute ticking. We believe we will be taking off major businesses from their flagship positions, with the products we are eyeing today. Benefits:W2 OPPORTUNITYCompetitive salaryOpportunity for advancement Benefits: Job Title: Data Quality Engineer (Databricks, Kafka, AWS)Location: Dallas, TX (Hybrid - 3 days onsite)Job Type: Long-term ContractWork Authorization: Open - W2 opportunityInterview Process: In-person (Client interview- Mandatory)We are looking for a Data Quality Engineer to own validation across batch and streaming data pipelines. This role focuses on ensuring data correctness, reliability, and performance across platforms built on Databricks, Kafka, AWS, SQL, and Python.This is a hands-on role focused on building scalable data validation frameworks and ensuring production-grade data systems.Key ResponsibilitiesEnd-to-End Data Validation* Validate data pipelines for accuracy, completeness, consistency, and timeliness* Build SQL-based validations for business rules and transformations* Implement reconciliation between source and downstream systems* Ensure data lineage and traceabilityETL / ELT & Spark Testing* Test pipelines built on AWS (Glue, Lambda, EMR, Step Functions)* Validate transformations using SQL and Python* Test ingestion, transformation, aggregation, and serving layers* Handle backfills, reprocessing, and historical data loads* Validate Spark pipelines (PySpark/Scala) on DatabricksStreaming (Kafka)* Validate data integrity, ordering, and delivery guarantees* Test producer and consumer logic and serialization formats (Avro, JSON, Protobuf)* Validate topics, partitions, offsets, retention, and schema evolution* Simulate late events, duplicates, and failure scenariosAutomation & Frameworks* Build Python-based data testing frameworks* Develop reusable validation utilities and synthetic datasets* Integrate data tests into CI/CD pipelines* Enable automated alerts for data quality issuesPerformance & Reliability* Validate throughput, latency, and concurrency at scale* Test retry logic, idempotency, and recovery mechanisms* Perform regression, soak, and failover testingObservability* Validate logs, metrics, and alerts using tools such as CloudWatch, Prometheus, and Grafana* Define and monitor data SLAs and SLOs* Support incident response, root cause analysis, and postmortemsRequired Qualifications & Experience* 7+ years of total experience in QA, SDET, or Data Quality Engineering* Minimum 4-6 years of hands-on experience working with data platforms, data pipelines, or data engineering ecosystems* 3+ years of hands-on experience with Databricks and Apache Spark* Strong SQL skills for data validation, reconciliation, and complex analysis* Proficiency in Python for automation and data validation* Experience testing ETL/ELT pipelines (batch and streaming)* Hands-on experience with Kafka or similar streaming platforms* Strong understanding of AWS data services (S3, Glue, Lambda, Redshift, Athena)* Experience working with large-scale distributed data systems* Strong debugging, analytical, and problem-solving skillsNice to Have* Experience with data quality or observability tools such as Great Expectations or Monte Carlo* Knowledge of schema registry and data contracts* Experience with CI/CD tools such as GitHub Actions or Jenkins Job Title: Data Quality Engineer (Databricks, Kafka, AWS)Location: Dallas, TX (Hybrid - 3 days onsite)Job Type: Long-term ContractWork Authorization: Open - W2 opportunityInterview Process: In-person (Client interview- Mandatory)We are looking for a Data Quality Engineer to own validation across batch and streaming data pipelines. This role focuses on ensuring data correctness, reliability, and performance across platforms built on Databricks, Kafka, AWS, SQL, and Python.This is a hands-on role focused on building scalable data validation frameworks and ensuring production-grade data systems.Key ResponsibilitiesEnd-to-End Data Validation* Validate data pipelines for accuracy, completeness, consistency, and timeliness* Build SQL-based validations for business rules and transformations* Implement reconciliation between source and downstream systems* Ensure data lineage and traceabilityETL / ELT & Spark Testing* Test pipelines built on AWS (Glue, Lambda, EMR, Step Functions)* Validate transformations using SQL and Python* Test ingestion, transformation, aggregation, and serving layers* Handle backfills, reprocessing, and historical data loads* Validate Spark pipelines (PySpark/Scala) on DatabricksStreaming (Kafka)* Validate data integrity, ordering, and delivery guarantees* Test producer and consumer logic and serialization formats (Avro, JSON, Protobuf)* Validate topics, partitions, offsets, retention, and schema evolution* Simulate late events, duplicates, and failure scenariosAutomation & Frameworks* Build Python-based data testing frameworks* Develop reusable validation utilities and synthetic datasets* Integrate data tests into CI/CD pipelines* Enable automated alerts for data quality issuesPerformance & Reliability* Validate throughput, latency, and concurrency at scale* Test retry logic, idempotency, and recovery mechanisms* Perform regression, soak, and failover testingObservability* Validate logs, metrics, and alerts using tools such as CloudWatch, Prometheus, and Grafana* Define and monitor data SLAs and SLOs* Support incident response, root cause analysis, and postmortemsRequired Qualifications & Experience* 7+ years of total experience in QA, SDET, or Data Quality Engineering* Minimum 4-6 years of hands-on experience working with data platforms, data pipelines, or data engineering ecosystems* 3+ years of hands-on experience with Databricks and Apache Spark* Strong SQL skills for data validation, reconciliation, and complex analysis* Proficiency in Python for automation and data validation* Experience testing ETL/ELT pipelines (batch and streaming)* Hands-on experience with Kafka or similar streaming platforms* Strong understanding of AWS data services (S3, Glue, Lambda, Redshift, Athena)* Experience working with large-scale distributed data systems* Strong debugging, analytical, and problem-solving skillsNice to Have* Experience with data quality or observability tools such as Great Expectations or Monte Carlo* Knowledge of schema registry and data contracts* Experience with CI/CD tools such as GitHub Actions or Jenkins Compensation: $62.00 - $65.00 per hour About UsWe work to deliver profitability in your business - with effective communication, consulting, and interactive solutions. Following an Agile Work Approach, we make sure you get the ideal solutions at minimum expenses.Work ApproachOur PhilosophyOur Philosophy starts-and-ends at the Client-first approach. Be it understanding your business requirements to choosing the right technologies, we work as a collective team that takes all the possible steps to grow continuously towards our common goal.Work PolicyWe promote a collaborative work environment. We involve everyone working in the organization in community decisions and encourage them to think from a broader perspective. Our work process promotes flexibility and we maintain a high level of discipline at different levels of execution.The FutureSelectMinds have years of experience in the domain helps us understand the need-of-the-hour better. This understanding drives us to a better future with every minute ticking. We believe we will be taking off major businesses from their flagship positions, with the products we are eyeing today. We work to deliver profitability in your business - with effective communication, consulting, and interactive solutions. Following an Agile Work Approach, we make sure you get the ideal solutions at minimum expenses. Work Approach Our PhilosophyOur Philosophy starts-and-ends at the Client-first approach. Be it understanding your business requirements to choosing the right technologies, we work as a collective team that takes all the possible steps to grow continuously towards our common goal. Work Policy We promote a collaborative work environment. We involve everyone working in the organization in community decisions and encourage them to think from a broader perspective. Our work process promotes flexibility and we maintain a high level of discipline at different levels of execution. The Future SelectMinds have years of experience in the domain helps us understand the need-of-the-hour better. This understanding drives us to a better future with every minute ticking. We believe we will be taking off major businesses from their flagship positions, with the products we are eyeing today.

• 7+ years of total experience in QA, SDET, or Data Quality Engineering
• Minimum 4-6 years of hands-on experience working with data platforms, data pipelines, or data engineering ecosystems
• 3+ years of hands-on experience with Databricks and Apache Spark
• Strong SQL skills for data validation, reconciliation, and complex analysis
• Proficiency in Python for automation and data validation
• Experience testing ETL/ELT pipelines (batch and streaming)
• Hands-on experience with Kafka or similar streaming platforms
• Strong understanding of AWS data services (S3, Glue, Lambda, Redshift, Athena)
• Experience working with large-scale distributed data systems
• Strong debugging, analytical, and problem-solving skills

Nice to Have

• Experience with data quality or observability tools such as Great Expectations or Monte Carlo
• Knowledge of schema registry and data contracts
• Experience with CI/CD tools such as GitHub Actions or Jenkins

Flexible work from home options available., SelectMinds have years of experience in the domain helps us understand the need-of-the-hour better. This understanding drives us to a better future with every minute ticking. We believe we will be taking off major businesses from their flagship positions, with the products we are eyeing today.

Compensation: $62.00 - $65.00 per hour