Developer - Azure Databricks / Python / PySpark

We are looking for an experienced Developer / Data Engineer with strong hands-on expertise in Azure Databricks, Python, and PySpark. The ideal candidate will specialize in building robust, reliable, and maintainable data-processing solutions within a Databricks environment. This role is focused on data processing engineering within Databricks, rather than traditional Business Intelligence (BI), data enrichment, or reporting architectures. The successful candidate should have deep practical experience managing complex data processes and making sound technical decisions in a continuously evolving architecture., * Design, develop, and maintain robust data-processing solutions using Azure Databricks.

• Build and manage batch and streaming data-processing pipelines.
• Develop scalable solutions using Python, PySpark, and SQL.
• Design appropriate data models within the Databricks environment.
• Manage and control multiple parallel data processes working with shared source data.
• Ensure data-processing solutions are stable, reliable, maintainable, and scalable.
• Work with Azure cloud services and implement CI/CD pipelines using YAML.
• Develop and maintain infrastructure using Infrastructure as Code (IaC), particularly ARM/Bicep.
• Identify and resolve technical issues while minimizing technical debt.
• Contribute to continuous improvement of the data-processing environment and development practices.
• Collaborate closely with other engineers, architects, and stakeholders in an Agile environment.
• Make pragmatic technical decisions without unnecessary over-engineering or blindly applying design patterns.

• Strong, hands-on experience with Azure Databricks.
• Deep expertise in Python and PySpark.
• Strong SQL skills.
• Proven experience designing and managing data processing within Databricks.
• Experience with both streaming and batch data processing.
• Experience designing data models within Databricks.
• Experience managing multiple parallel data processes using shared source data.
• Solid experience with Microsoft Azure cloud services.
• Hands-on experience with CI/CD pipelines using YAML.
• Experience with Infrastructure as Code, particularly ARM and/or Bicep.
• Experience working in an Agile development environment.
• 8-10 years of relevant professional experience.

Nice-to-Have

• Broader understanding of end-to-end data solutions.
• Experience working within complex and evolving data architectures.
• Understanding of best practices for building highly reliable and maintainable data platforms.

Soft Skills & Professional Competencies The ideal candidate should demonstrate:

• Strong focus on quality, reliability, and maintainability.
• A proactive mindset and commitment to continuous improvement.
• Strong collaboration and communication skills.
• A structured and analytical approach to problem-solving.
• A pragmatic engineering mindset.
• Ability to make well-founded technical decisions.
• Ability to avoid unnecessary complexity and over-engineering.
• Strong ownership of solutions from development through to reliable operation., We are specifically looking for a pragmatic Azure Databricks/Python specialist with genuine, deep hands-on experience-not someone with only superficial exposure to Databricks. The successful candidate will be comfortable managing and controlling data within a Databricks environment, building stable and high-quality processing solutions, and working effectively within a complex technical architecture.