> Markdown version of [/jobs/ext/1083276-qa-automation-engineer-enterprise-data-analytics-platform](https://www.wearedevelopers.com/jobs/ext/1083276-qa-automation-engineer-enterprise-data-analytics-platform). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # QA Automation Engineer - Enterprise Data & Analytics Platform - **Company:** Universal E-business Solutions, LLC - **Location:** United States - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Artificial Intelligence, Audit Trail, Automation of Tests, Microsoft Azure, Data Validation, Information Engineering, Data Governance, Data Visualization, DevOps, Python (Programming Language), Scaled Agile Framework, SQL Databases, Tableau (Software), Feature Engineering, Pyspark, Data Lineage, Collibra, Databricks - **Published:** June 9, 2026 - **Apply:** https://www.dice.com/job-detail/0f2b99f4-de4e-4798-9429-9957445e94e4 ## About the Role * Minimum of 5+ years of solid experience in Data Engineering with proven experience testing and validating data pipelines in Databricks, including medallion architecture. * Proficient in creating testing framework for validating Data Quality. * Proficient in Databricks notebook, PySpark, Python, SQL, and data quality testing. * Expert with testing AI/BI models, ensuring data quality from feature engineering through model scoring. * Experience in CI/CD pipelines (e.g., Azure DevOps) for automated test execution. * Strong knowledge of data governance (data lineage, audit trails, compliance testing). * Excellent problem-solving skills with the ability to work in a fast-paced environment. * Experience with tools such as Azure Purview and Profisee MDM is preferred. ## Description * Architect and implement robust automated testing frameworks leveraging PySpark and Databricks-native tools for data validation across Raw, Curated, and Mart layers. * Design and implement data quality validation frameworks, including checks on accuracy, completeness, and consistency across transformation layers. * Create advanced data quality KPIs, integrating them into automated dashboards to track quality trends across layers. * Design metadata-driven tests, integrating with CI/CD pipelines, with coverage on all transformation layers. * Lead development of QA user stories and acceptance criteria, precisely defining test scenarios for ingestion, transformation, and consumption layers. * Perform complex data reconciliation testing across 10+ source systems, ensuring accuracy, completeness, and consistency from source through Mart. * Own the end-to-end testing lifecycle (QA, Staging, Production), defining what and when to test at each stage and ensuring sign-off criteria are met. * Partner closely with data engineers to troubleshoot pipeline failures, connectivity issues, and performance bottlenecks. * Set standards for data lineage and auditability, ensuring every transformation step can be validated and traced. * Plan, facilitate, and manage User Acceptance Testing (UAT) involving business users for data visualization tools such as Tableau running on Databricks. * Prepare UAT test scenarios aligned with business use cases, guide users through testing, and gather actionable feedback. * Drive defect triage, resolution, and retesting, ensuring readiness for production release. * Work within a SAFe Agile framework, participating in PI planning, sprint ceremonies, and cross-team coordination. Collaborate with DevOps, Data Engineers, Data Scientists, and Product Owners to integrate QA into CI/CD pipelines. * Provide regular updates to project and senior management on progress of QA milestones and tasks. ## Related Videos - [Beyond GPT: Building Unified GenAI Platforms for the Enterprise of Tomorrow](https://www.wearedevelopers.com/videos/1525-beyond-gpt-building-unified-genai-platforms-for-the-enterprise-of-tomorrow) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [AI as a Test Designer: Transforming Experience into Automated Testing](https://www.wearedevelopers.com/videos/1984-ai-as-a-test-designer-transforming-experience-into-automated-testing) - [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning) ## Related Articles - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [What Industries Outside of AI Are Hiring The Most AI Experts?](https://www.wearedevelopers.com/magazine/98-what-industries-outside-of-ai-are-hiring-the-most-ai-experts) - [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)