> Markdown version of [/jobs/ext/4666119-databricks-technical-lead](https://www.wearedevelopers.com/jobs/ext/4666119-databricks-technical-lead). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Databricks Technical Lead

- **Company:** Method Resourcing
- **Location:** Birmingham, United Kingdom (Remote available)
- **Experience:** Expert
- **Salary:** £234,000 - £260,000
- **Contract:** Temporary contract
- **Skills:** Artificial Intelligence, Business Logic, Automation of Tests, Microsoft Azure, Code Generation, Continuous Integration, Information Engineering, Dimensional Modeling, Python (Programming Language), SAP ERP, Power BI, Standard Sql, Azure Data Lake, Reverse Engineering, Salesforce.Com, SAP (Applications), Systems Integration, Azure Data Factory, Large Language Models, Agentic-AI, Data Lakes, Pyspark, SAP S/4HANA, Virtual Agents, Terraform, Data Pipelines, Databricks
- **Published:** November 11, 2026
- **Apply:** https://www.methodresourcing.co.uk/job.asp?OP=b&VID=8805

## About the Role

* Extensive hands-on experience delivering and technically leading enterprise solutions on Azure Databricks
 * Demonstrable use of AI, LLMs or agents to reverse-engineer data pipelines or reporting logic
 * Strong Python, PySpark and SQL
 * Deep knowledge of Delta Lake, Unity Catalog, Workflows and medallion architecture
 * Automated validation, reconciliation and testing experience
 * Azure data engineering (ADLS Gen2, ADF)
 * Databricks CI/CD (Terraform, Asset Bundles or Azure DevOps)
 * SAP ECC or S/4HANA integration experience
 * Dimensional modelling and Power BI semantic model familiarity
 * Proven team leadership while staying hands-on

Desirable

 * Databricks certification (Data Engineer Professional or Solutions Architect)
 * Platform migration, carve-out or separation experience
 * Reusable AI agent frameworks for data engineering
 * Strong governance and data quality experience
 * Consulting background

## Description

Method Resourcing have been asked by one of our consulting partners to find a hands-on Databricks Technical Lead to design and deliver a new Azure Databricks platform for a major life sciences organisation replacing its enterprise data and analytics stack.

The programme uses AI and LLM tooling to reverse-engineer existing pipelines, business logic and reports into documented, tested code. You'll mentor three senior data engineers, own the technical architecture and stay hands-on in development.

What you'll do

 * Architect and build the platform on Azure Databricks (Delta Lake, Unity Catalog, medallion architecture)
 * Use AI agents and LLMs to reverse-engineer legacy reports, queries and transformations, and build repeatable AI-assisted processes for code generation, testing and migration
 * Build automated testing and reconciliation to validate migrated data against existing reporting
 * Integrate SAP, Azure Data Lake, Salesforce and other systems, and design gold-layer models for Power BI
 * Set engineering standards (Workflows, CI/CD, IaC, governance) and mentor the team through to UAT and cutover