> Markdown version of [/jobs/ext/2277989-databricks-lead-engineer-hybrid-contract](https://www.wearedevelopers.com/jobs/ext/2277989-databricks-lead-engineer-hybrid-contract). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Databricks Lead Engineer - Hybrid - Contract

- **Company:** Anson McCade
- **Location:** London, UK
- **Experience:** Expert
- **Salary:** £149,500.0
- **Contract:** Temporary contract
- **Skills:** Artificial Intelligence, Continuous Integration, Information Engineering, Data Masking, Data Security, Database Development, Machine Learning, Role-Based Access Control, Search Technologies, Data Streaming, Generative AI, Data Lakes, Pyspark, Machine Learning Operations, Databricks
- **Published:** August 28, 2026
- **Apply:** https://www.totaljobs.com/job/lead-engineer/anson-mccade-job107908650

## About the Role

* At least two years of recent, hands-on Databricks experience
 * Evidence of delivering Databricks solutions into production environments
 * An active Databricks Data Engineering, Machine Learning or Generative AI certification
 * Strong experience implementing Medallion Architecture across Bronze, Silver and Gold layers
 * Hands-on Unity Catalog experience covering catalog design, access controls, lineage and secure data sharing
 * Experience developing Declarative Pipelines with Expectations, including failure handling and observability
 * Knowledge of Auto Loader, Lakeflow Connect and batch, streaming or incremental ingestion patterns
 * Production-level PySpark and SQL development skills
 * Strong knowledge of Delta Lake, including MERGE, schema evolution, OPTIMIZE, ZORDER and VACUUM
 * Experience with Databricks Workflows, orchestration and CI/CD
 * Understanding of monitoring, alerting, replay, backfill and operational support strategies
 * Ability to mentor engineers and act as the technical escalation point
 * Experience with RBAC, data masking, Photon, cluster optimisation or cost management would be beneficial
 * Exposure to Mosaic AI, Vector Search, model serving, RAG or Lakebase would be advantageous

## Description

A hands-on Databricks Lead Engineer is required to lead the delivery of a modern enterprise data platform for a leading financial services organisation.

The role will take technical ownership of Databricks lakehouse solutions, covering ingestion through to curated serving layers. Responsibilities will include implementing Medallion Architecture, Unity Catalog, Declarative Pipelines and scalable Databricks-native ingestion patterns while mentoring engineers and driving production-quality delivery.


## Related Videos

- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [OLTP in the Lakehouse: Redefining Data for AI Workloads](https://www.wearedevelopers.com/videos/2038-oltp-in-the-lakehouse-redefining-data-for-ai-workloads)
- [Enabling intelligent logistics automation: home-grown Industrial IoT platform at Austrian Post](https://www.wearedevelopers.com/videos/2018-enabling-intelligent-logistics-automation-home-grown-industrial-iot-platform-at-austrian-post)

## Related Articles

- [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Dev Digest 132 - Binging WADFlix?](https://www.wearedevelopers.com/magazine/473-dev-digest-132-binging-wadflix)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer)
- [Everything a Developer Needs to Know About MCP with Neo4j](https://www.wearedevelopers.com/magazine/604-everything-a-developer-needs-to-know-about-mcp-with-neo4j)