QA Automation with Spark

We are looking for a skilled QA Automation Engineer with experience in Apache Spark and data testing to design, develop, and execute automated testing solutions for data-intensive applications and ETL/data pipelines. The ideal candidate will have strong knowledge of QA methodologies, automation frameworks, SQL, Python/Java, and Spark-based data processing., * Design and develop automated test scripts and frameworks for data pipelines and Spark applications.

• Perform functional, regression, integration, system, and end-to-end testing.
• Validate Apache Spark jobs, including data transformations, aggregations, joins, and business rules.
• Perform data validation and reconciliation between source and target systems using SQL.
• Test ETL/ELT pipelines and ensure data quality, accuracy, completeness, and consistency.
• Develop automation scripts using Python, Java, or Scala.
• Execute Spark jobs and validate input/output datasets, transformations, and processing results.
• Identify, analyze, and troubleshoot defects in applications and data pipelines.
• Integrate automated tests into CI/CD pipelines using tools such as Jenkins, GitHub Actions, or Azure DevOps.
• Work closely with developers, data engineers, business analysts, and product teams to resolve issues.
• Create and maintain test plans, test cases, automation scripts, and defect reports.
• Monitor test execution and provide quality metrics and status reports.
• Participate in Agile/Scrum ceremonies and contribute to continuous improvement of QA processes.

• 3+ years of experience in QA Automation / Software Testing.
• Hands-on experience with Apache Spark / PySpark.
• Strong SQL skills for data validation and backend testing.
• Experience with Python, Java, or Scala.
• Knowledge of automation frameworks such as Selenium, PyTest, TestNG, or similar.
• Experience testing ETL/data pipelines and large datasets.
• Understanding of Spark concepts such as DataFrames, transformations, actions, joins, partitions, and Spark SQL.
• Experience with Git and CI/CD tools.
• Good understanding of SDLC, STLC, defect management, and Agile methodologies.
• Strong analytical and problem-solving skills.

Ace IT Careers

• Seattle, WA

• $87,000-88,500 per year Ace IT Careers is seeking candidates interested in Junior QA Automation Engineer opportunities with our employer and staffing network. We are particularly interested in candidate…

• 25 days ago