Senior software engineer - 2026-apg-424

We are seeking a Senior Software Engineer - Data Quality Framework to design, build, and maintain a scalable data quality framework within Databricks using PySpark.

You will play a key role in developing reusable framework components that support data validation, rule execution, and contract-driven data quality checks across enterprise data environments. The role combines strong software engineering practices with data platform engineering in a modern cloud-based ecosystem (Azure Databricks).

You will work in a multidisciplinary Agile team responsible for delivering reliable and scalable data capabilities that support critical business and data products. Key Responsibilities Data Quality Framework Development

• Design, develop, and maintain a PySpark-based data quality framework in Databricks
• Build reusable components for rule execution, validation logic, and result handling
• Implement contract-driven data validation mechanisms across datasets
• Develop scalable and maintainable framework architecture

Engineering & Platform Ownership

• Ensure high engineering standards through:

• Code refactoring and modular design
• Unit and integration testing
• CI/CD pipeline implementation and maintenance
• Packaging and versioning of framework components

• Support multi-environment deployment strategies (dev/test/prod)

Databricks Platform Operations

• Develop and manage Databricks jobs, workflows, alerts, and notifications
• Support operational stability of data pipelines and framework execution
• Contribute to monitoring and reliability improvements

Stakeholder Collaboration

• Work closely with data engineers, analysts, and platform teams
• Translate data quality requirements into scalable technical solutions
• Contribute to documentation, standards, and best practices
• Support coaching and knowledge sharing within the team

Assignment Deliverables

• A scalable and reusable data quality framework in Databricks
• Robust PySpark-based validation and rule execution components
• Stable CI/CD and deployment processes for framework components
• Improved reliability and governance of data quality processes
• Documented standards, guidelines, and reusable engineering patterns
• Operational monitoring setup (alerts, triggers, notifications)

Education Level: Bachelor's degree in Computer Science, Software Engineering, Data Engineering, or related field Experience Required: Minimum 5 years Travel Requirement: 25% - 50% Language: English (Dutch is a plus depending on team setup) Role Overview, * Minimum 5 years of experience in software engineering, data engineering, or platform engineering

• Strong proficiency in Python and PySpark
• Experience working with Apache Spark in production environments
• Hands-on experience with Databricks (Azure preferred)
• Strong understanding of:

• Software design principles (OOP, modular design, maintainability)
• CI/CD pipelines and testing strategies
• Multi-environment cloud deployments

• Experience with version control, packaging, and release processes
• Strong communication skills in English
• Ability to work independently and take ownership of tasks

Preferred Experience

• Experience building or maintaining data quality frameworks or rule engines
• Knowledge of metadata-driven validation approaches
• Experience with data contracts and governance concepts
• Familiarity with Databricks alerts, workflows, and operational tooling
• Experience with Azure DevOps and Artifactory
• Experience developing reusable platform components or shared libraries
• Exposure to enterprise-scale cloud data platforms

Key Competencies

• Strong software engineering mindset
• High attention to code quality and maintainability
• Analytical and structured problem-solving ability
• Ownership and proactive delivery attitude
• Ability to work across technical and business stakeholders
• Strong collaboration in agile teams
• Platform thinking and scalability focus