Lead Data Platform Engineer

As Lead Data Platform Engineer, you will:

• Develop and deploy Databricks Lakehouse platform solutions within Azure.

• Build and maintain automation for clusters, tagging, monitoring and platform operations.

• Architect and design end-to-end data platform solutions focused on scalability, security and reliability.

• Lead delivery using Databricks, PySpark, Spark SQL and Azure Data Factory.

• Drive DevOps best practices including CI/CD, automated testing and infrastructure as code.

• Collaborate closely with data engineers, ML engineers and wider technical teams to deliver robust platform capabilities.

• Take ownership of key data platform projects and influence engineering direction.

• Provide technical leadership and ensure standards, patterns and best practices are followed.

• Strong commercial experience in Azure and Databricks, with deep knowledge of Lakehouse architecture.

• Proficiency in PySpark, Python, SQL and Spark SQL.

• Strong DevOps engineering skills including CI/CD, automated testing and deployment pipelines.

• Experience with monitoring, logging and alerting solutions.

• Ability to design, build and optimise cloud platform solutions.

• Stakeholder management skills and confidence working across multiple teams.

• API development experience.

• Azure Data Factory knowledge is beneficial.

• Salary between £80,000 and £85,000 plus bonus.

• Comprehensive benefits package and annual pay review.

• Flexible by choice working environment with remote-first culture.

• 35-hour working week promoting genuine work-life balance.

• Opportunities to shape platform strategy and contribute to future AI and automation initiatives.

They are a large, well-established organisation known for combining deep technical expertise with a people-first culture. With a nationwide presence and a modern approach to flexible working, they invest heavily in technology, innovation and colleague development. Their data function is expanding and plays a central role in supporting transformation, decision-making and future AI initiatives. You will join a collaborative team that values engineering excellence, autonomy and continuous improvement.