Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Automation of Tests Continuous Integration Data Validation Database Queries Database Testing Job Scheduling Python (Programming Language) Apache Spark Data Lakes Pyspark Data Pipelines Databricks

Requirements

• Strong Python and PySpark/Spark SQLdata testing experience, especially for data validation and comparison
• Hands-on Databricks experience, including notebooks, Delta Lake, Unity Catalog, and job scheduling
• Strong SQL skills for source-to-target data validation
• Experience building automated data testing frameworks, not just manual QA
• Experience with data validation such asrecord-level comparison, schema validation, count validation, data reconciliation, and column-level mismatch analysis
• Experience testing data pipelines and validating source-to-target mappings
• Ability to create automated test reports and investigate data discrepancies Nice-to-Have: Familiarity with Delta Lake and Unity Catalog Experience with CI/CD for data pipelines Understanding of vehicle data domains.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com

Prepare application

• Draft this with your agent
• Open in Claude
• Open in ChatGPT
• Copy prompt