> Markdown version of [/jobs/ext/3256903-senior-data-engineer-databricks-and-delta](https://www.wearedevelopers.com/jobs/ext/3256903-senior-data-engineer-databricks-and-delta). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Engineer: Databricks and Delta

- **Company:** Eaglecliff Recruitment
- **Location:** London, UK
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** Artificial Intelligence, Microsoft Azure, Computer Programming, Data Architecture, Extract Transform Load (ETL), Data Structures, Python (Programming Language), Performance Tuning, SAP Project System, SQL Databases, Data Processing, Data Ingestion, Data Lakes, Qlikview, Spark Streaming, Machine Learning Operations, Video Streaming, Stream Analytics, Data Pipelines, Databricks
- **Published:** September 4, 2026
- **Apply:** https://www.careerboard.com/pt/en/find-jobs-in-United-Kingdom/-117C22014DB26BCB66/

## About the Role

We are looking for a highly skilled Senior Data Engineer with strong analytical capabilities and hands-on experience in building scalable data solutions.

The ideal candidate will have expertise in Databricks, streaming architectures, data ingestion frameworks, and dashboarding, along with experience working with SAP data sources.

You will be responsible for designing and implementing scalable data pipelines and enabling Real Time and batch analytics across the organization., * Strong experience with Databricks- DLT, Delta Lake concepts, UC governance

 * Solid understanding of streaming technologies (eg, Spark Structured Streaming, Autoloader)
 * Programming skills in Python, SQL, or Scala
 * Proficiency in data modelling, ETL/ELT processes, and data architecture
 * Experience in MS Copilot, AI Foundry, Databricks (MosiacAI, MLflow, Agentbricks, Genie)
 * Strong analytical background with problem-solving skills
 * Performance tuning concepts like watermarking, late data handling, parallelism & checkpointing
 * Hands-on expertise in ADF, and Qlik Replicate for data ingestion and replication
 * Experience working in Azure cloud environments

## Description

* Design and implement scalable data pipelines using Databricks and Delta Live Tables (DLT)
 * Develop and manage streaming data solutions for Real Time analytics and decision-making
 * Ingest data from various sourcesincluding SAP systems using Qlik Replicate and ADF to ingest data from various sources ensuring data quality and consistency
 * Perform data modelling and optimize data structures for analytics and reporting
 * Collaborate with cross-functional teams to understand data requirements and deliver robust solutions
 * Monitor, troubleshoot, and enhance data workflows for performance and reliability
 * Maintain technical documentation and contribute to data engineering best practices


## Related Videos

- [Fully Orchestrating Databricks from Airflow](https://www.wearedevelopers.com/videos/336-fully-orchestrating-databricks-from-airflow)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)
- [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering)
- [Parquet, Delta, Iceberg & Ducklake - An introduction for developers](https://www.wearedevelopers.com/videos/100075-parquet-delta-iceberg-ducklake-an-introduction-for-developers)
- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)