> Markdown version of [/jobs/ext/2816182-databricks-data-ai-engineer](https://www.wearedevelopers.com/jobs/ext/2816182-databricks-data-ai-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Databricks Data & AI Engineer - **Company:** Deloitte - **Location:** London, UK - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Sql Data Warehouse, Artificial Intelligence, Airflow, Amazon Web Services, Business Analytics Applications, Data Analysis, Automation of Tests, Microsoft Azure, BigQuery, Cloud Computing Security, Continuous Integration, Data Architecture, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), Data Systems, Software Design Patterns, DevOps, Apache Hive, Identity and Access Management, Python (Programming Language), Machine Learning, Performance Tuning, DataOps, Search Technologies, Software Deployment, SQL Databases, Data Streaming, Cloud Platform System, Feature Engineering, Azure Data Factory, Snowflake, Apache Spark, Generative AI, Containerization, Data Lakes, Pyspark, Kubernetes, Infrastructure Automation Frameworks, Data Management, Machine Learning Operations, Virtual Agents, Terraform, Software Version Control, Data Pipelines, Docker, Amazon Redshift, Databricks - **Published:** September 10, 2026 - **Apply:** https://apply.deloitte.co.uk/UKCareers/Login?jobId=25086 ## About the Role Overall, you're a curious problem solver and hands-on engineer who enjoys working collaboratively to solve complex data challenges. You thrive in diverse teams, enjoy learning from others, and bring a pragmatic approach to delivering high-quality data solutions that create measurable business value. You'll have strong technical skills including some or all of the following: * Proven experience in data engineering, data modelling, and architecture, preferably on a major cloud platform (Azure, AWS, or GCP). * Strong experience designing and building scalable data platforms, data products and analytics solutions using modern lake house or cloud data warehouse technologies * Proficiency writing advanced, highly optimised SQL and Spark (PySpark/Scala), with a focus on performance optimisation and maintainability. * Designing and building solutions on a cloud data warehouse or lakehouse platform (e.g., Databricks, Snowflake, BigQuery, Redshift). * Strong understanding of data modelling principles for analytics, reporting, machine learning and AI use cases. * Experience applying modern engineering practices, including CI/CD, automated testing, infrastructure-as-code and data observability. * Knowledge of cloud-native security, governance and access management practices, including technologies such as Unity Catalog and cloud IAM frameworks. * Experience working with Databricks Lakehouse Platform, including Delta Lake, Databricks Workflows, Unity Catalog , Lakeflow and Databricks SQL. * Experience building & enabling AI workloads using Databricks AI capabilities such as Agent Bricks, MLflow, Feature Engineering, Vector Search, Mosaic AI and model serving, with awareness of agent orchestration, RAG and production grade AI agents. Required * Strong understanding of Databricks Lakehouse architecture, Medallion design patterns and Delta Lake principles. * Hands-on experience delivering at least one enterprise-scale Databricks implementation from design through production deployment. * Proven experience developing and supporting production-grade data pipelines and data products. * Strong proficiency in Python or Scala for data engineering. * Experience working directly with business and technical stakeholders to gather requirements and communicate solution approaches. * Strong analytical, problem-solving and troubleshooting skills Preferred * Databricks Certified Data Engineer (Associate or Professional) or Databricks Certified Generative AI Engineer Associate * Experience with orchestration technologies such as Azure Data Factory, Airflow or Databricks Workflows. * Knowledge of containerisation and platform engineering technologies such as Docker and Kubernetes. * Experience enabling machine learning, GenAI or advanced analytics workloads using technologies such as MLflow, Feature Engineering, Vector Search and Mosaic AI capabilities. * Exposure to DevOps, platform automation and Infrastructure-as-Code tools such as Terraform. ## Description We're expanding our AI & Data Engineering practice and are looking for people who are natural initiative-takers, who bring out the best in others, are brilliant listeners, and can grow our business without compromising standards, integrity, or culture. You will work with outstanding data, AI, and cloud talent across multiple disciplines to innovate and create powerful solutions for iconic brands and complex organisations. We are seeking a skilled Databricks Data & AI Engineer who combines strong data engineering capability with practical experience in enabling advanced analytics, machine learning and AI-enabled business solutions with strategic consulting acumen and problem-solving skills. The ideal candidate should possess strong knowledge of the Databricks Data Intelligence Platform and the modern data & AI engineering practices, complemented by a keen ability to translate complex business requirements into robust, scalable Data & AI solutions that create trusted data foundations, AI-ready products and measurable business value. As a key member of our team, you will: * Design, build, test and deploy scalable batch and streaming data pipelines, AI-ready data products, feature pipeline and analytics solutions on the Databricks Platform using Delta Lake, Spark SQL/PySpark, Unity Catalog and Databricks SQL. * Deliver solutions across the full data & AI engineering lifecycle, from requirements analysis and solution design through development, testing, deployment, operational readiness and continuous improvement. * Work directly with clients and stakeholders to understand business, data,AI and technical requirements, facilitate design workshops and recommend practical solution options. * Communicate technical concepts clearly, explaining solution choices, trade-offs, dependencies and recommendations to both technical and non-technical audiences. * Design and implement robust data models and ETL/ELT patterns that support enterprise reporting, analytics, machine learning and AI use cases. * Optimise data pipelines and workloads for performance, reliability, scalability and cost, identifying engineering risks and implementing appropriate mitigations. * Embed security, governance, lineage, access control, data quality and Responsible AI considerations into solutions, including cataloguing and policy enforcement through Unity Catalog. * Enable AI and machine learning solutions by preparing trusted, governed and reusable datasets, supporting feature engineering, experimentation, model deployment and integration with downstream AI applications. * Apply modern engineering practices, including version control, automated testing, CI/CD, infrastructure-as-code, monitoring and data observability. * Produce and maintain high-quality technical artefacts, including solution designs, data models, interface specifications, deployment documentation and operational runbooks. * Support and guide other engineers, contributing to technical reviews, reusable engineering standards and quality assurance across delivery teams. * Enable advanced analytics, machine learning Gen AI & Agentic AI solutions by creating trusted, governed , discoverable and production-ready data foundations. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [DevOps Maturity Check – a way to balance autonomy and alignment](https://www.wearedevelopers.com/videos/58-devops-maturity-check-a-way-to-balance-autonomy-and-alignment) - [Docker build without Docker](https://www.wearedevelopers.com/videos/100114-docker-build-without-docker) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [How to Become an AI Engineer](https://www.wearedevelopers.com/magazine/331-how-to-become-an-ai-engineer) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)