> Markdown version of [/jobs/ext/4678282-azure-databricks-technical-lead](https://www.wearedevelopers.com/jobs/ext/4678282-azure-databricks-technical-lead). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Azure Databricks Technical Lead

- **Company:** Capgemini
- **Location:** London, United Kingdom (Remote available)
- **Experience:** Expert
- **Salary:** £56,235
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, User Authentication, Microsoft Azure, Software as a Service, Cloud Storage, Databases, Information Engineering, Extract Transform Load (ETL), Database Development, Python (Programming Language), Performance Tuning, RStudio, Software Deployment, Business Intelligence Development Studio, Enterprise Software Applications, Autoscaling, Retrieval-Augmented Generation, Large Language Models, Change Data Capture, Build Management, Data Lakes, Pyspark, Enterprise Integration, Restful APIs, Serverless Computing, Key Vault, Databricks
- **Published:** November 12, 2026
- **Apply:** https://www.adzuna.co.uk/jobs/details/5962144640

## About the Role

* Hands-on implementation and troubleshooting of Azure Databricks
 * Databricks Serverless configuration and workload optimisation
 * Databricks SQL and Delta Lake development
 * Cluster management, compute policies, autoscaling, and workload monitoring

Data Engineering

 * Strong Python, PySpark, and SQL development skills
 * Design and build ingestion and transformation pipelines
 * Full and incremental data loading
 * CDC (Change Data Capture) implementation
 * Data quality controls, schema evolution, reconciliation, and error handling

Integration Development

 * REST API integration experience
 * Integration with databases, SaaS applications, cloud storage, and enterprise systems
 * Secure authentication and credential management

Azure Platform Services

 * Azure Identity and Access Management
 * Azure Networking and Private Endpoints
 * Azure Security and Key Vault
 * Azure Monitoring and Operational Support

FinOps & Cost Optimisation

 * Resource tagging implementation
 * Cost monitoring and reporting
 * Budget controls and usage optimisation

Essential Skills

Platform Operations

 * Experience supporting production Azure Databricks environments
 * Monitoring, incident management, troubleshooting, and performance tuning
 * Operational governance and platform standards implementation

Discovery Zone & Migration Support

 * Experience enabling application deployment within Databricks
 * BI tooling integration and scheduling
 * Support for POSIT/RStudio migration activities

Collaboration

 * Working with Data Engineers, Data Scientists, and Platform Teams
 * Technical documentation and knowledge transfer
 * Agile delivery experience

Desirable Skills

 * POSIT/RStudio migration experience
 * AI/ML and LLM integration
 * RAG and Vector Database exposure
 * Regulated industry experience