Lead Data Platform Engineer

As a Lead Data Engineer you will build a Databricks platform with ownership, shaping how data is engineered, deployed, automated, and governed in Azure.

Specifically, you can expect to be involved in the following:

• Designing and developing end-to-end data platform solutions in Azure using Databricks
• Building and maintaining automation for cluster management, monitoring, tagging, and platform optimisation.
• Implementing CI/CD and DevOps best practices across data pipelines and platform components.
• Taking ownership of design decisions and long-term architectural direction.

The successful Lead Data Engineer will have the following skills and experience:

• Deep expertise with Azure Databricks, including development, deployment, and platform-level engineering.
• Strong software engineering capability across Pyspark, Spark SQL and Azure Data Factory.
• Solid understanding of DevOps practices including CI/CD and automated testing.
• Experience with monitoring, logging, and alerting solutions.
• Ability to work effectively with stakeholders across engineering functions.

The successful Lead Data Platform Engineer will receive the following benefits:

• Salary between £75,000 - £85,000 - depending on experience
• Competitive bonus and benefits.
• Remote work with a flexible 35-hour week that can be taken as a 4-day week.

This industry-leading organisation serves specialist, people-first knowledge to its clients and is looking to provide further expertise by improving its data capabilities.