> Markdown version of [/jobs/ext/2409670-lead-platform-engineer-databricks](https://www.wearedevelopers.com/jobs/ext/2409670-lead-platform-engineer-databricks). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Platform Engineer (Databricks)

- **Company:** Retail Limited
- **Location:** Richmond, UK
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Data Analysis, Microsoft Azure, Data Architecture, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), Python (Programming Language), Machine Learning, Cloud Services, SQL Databases, Data Streaming, Azure Service Bus, Apache Spark, Data Lakes, Information Technology, Bicep, Apache Kafka, Data Management, Machine Learning Operations, Terraform, Data Pipelines, Databricks
- **Published:** August 25, 2026
- **Apply:** https://uk.indeed.com/viewjob?jk=6656e0ca8ea50997

## About the Role

* Proven experience leading the design and delivery of scalable, production-grade data platforms.
 * Deep hands-on expertise with Databricks, Spark, Delta Lake, Python and SQL.
 * Strong understanding of cloud-native data engineering, including Azure and Infrastructure as Code (Terraform, Bicep or similar).
 * Expertise in data modelling, ETL/ELT, governance, security, quality and platform operations.
 * Experience mentoring engineers and collaborating with stakeholders to deliver high-quality technical solutions.

Even better if you have

 * Databricks certifications
 * Experience with Unity Catalog, MLflow or Databricks Asset Bundles
 * Familiarity with dbt or a similar transformation framework
 * Exposure to Kafka, Event Hubs or other real-time streaming technologies
 * Experience working with retail or consumer packaged goods data, including point-of-sale, inventory or supply-chain data
 * A degree in computer science, data engineering or a related subject, or equivalent practical experience is preferred

## Description

We're looking for a Lead Data Engineer to join our Data & Engineering team and take ownership of the architecture and development of our Databricks environment.

This is a senior, hands-on role where you'll design and build the scalable data infrastructure behind our products, while guiding a small team of engineers. You'll help define our platform standards, data architecture and engineering best practices, creating the foundations for analytics, machine learning and AI across the business.

If you enjoy staying close to the technology, solving complex data challenges and helping other engineers develop, this could be a brilliant opportunity.

What you'll be doing

 * Own and evolve our Databricks platform, making sure it's secure, scalable and built for the future.
 * Design modern data architectures and engineering standards that support analytics, AI and product innovation.
 * Lead the delivery of batch and real-time data pipelines using Databricks, Spark and Delta Lake.
 * Coach and mentor a team of engineers, promoting technical excellence and continuous improvement.
 * Work closely with Product, Analytics and Data Science teams to solve real business challenges through data.


## Related Videos

- [Back(end) to the Future: Embracing the continuous Evolution of Infrastructure and Code](https://www.wearedevelopers.com/videos/440-back-end-to-the-future-embracing-the-continuous-evolution-of-infrastructure-and-code)
- [Infrastructure as Code: The Developer's Secret Weapon](https://www.wearedevelopers.com/videos/1221-infrastructure-as-code-the-developer-s-secret-weapon)
- [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [OLTP in the Lakehouse: Redefining Data for AI Workloads](https://www.wearedevelopers.com/videos/2038-oltp-in-the-lakehouse-redefining-data-for-ai-workloads)
- [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [Where To Find Software Engineering Jobs](https://www.wearedevelopers.com/magazine/396-where-to-find-software-engineering-jobs)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)