> Markdown version of [/jobs/ext/2797616-senior-data-engineer-databricks-azure-data-platform](https://www.wearedevelopers.com/jobs/ext/2797616-senior-data-engineer-databricks-azure-data-platform). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Senior Data Engineer Databricks & Azure Data Platform

- **Company:** Acuity INC
- **Location:** UK
- **Experience:** Expert
- **Contract:** Permanent contract
- **Skills:** A/B Testing, Artificial Intelligence, Microsoft Azure, Continuous Integration, Information Engineering, Data Systems, Python (Programming Language), Search Technologies, SQL Databases, Unstructured Data, Cloud Platform System, Azure Data Factory, Large Language Models, Generative AI, Pyspark, Api Design, Terraform, Data Pipelines, Databricks
- **Published:** September 8, 2026
- **Apply:** https://www.collegerecruiter.com/job/2856354083-senior-data-engineer-databricks--azure-data-platform

## About the Role

* Hands-on experience with Databricks.
 * Advanced Python, PySpark and SQL.
 * Strong experience building scalable data pipelines and data models.
 * Experience with Azure or another major cloud platform.
 * Experience building APIs and exposing data to applications or customers.
 * Experience with data products or customer-facing data solutions.
 * Exposure to Generative AI / LLMs and real-world AI use cases.
 * Familiarity with RAG, embeddings, vector databases or semantic search.
 * Strong communication skills and confidence working with technical, product and commercial stakeholders.
 * A product mindset - you think about how data creates value, not just how it moves from A to B., * Airbyte
 * Terraform
 * A/B testing or experimentation
 * AI governance
 * Life sciences or pharmaceutical experience

## Description

We're looking for a Senior Data Engineer who wants to do more than build pipelines.

You'll help turn complex data into commercial, customer-facing products, combining strong Data Engineering skills with emerging AI and LLM technologies.

Working across Databricks, Azure, Python and PySpark, you'll build scalable data solutions, explore new AI capabilities and work directly with product and commercial teams to bring ideas into production.

This is a hands-on role with plenty of ownership: approximately 60% engineering, 20% mentoring and 20% exploration and innovation., * Work with structured and unstructured data, embeddings and semantic search.

 * Help evolve our Databricks and Azure data platform.
 * Improve data quality, testing, observability and CI/CD.
 * Partner with product and commercial teams to turn business opportunities into technical solutions.
 * Mentor engineers and help shape our growing Data & AI capability.


## Related Videos

- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [Infrastructure as Code: The Developer's Secret Weapon](https://www.wearedevelopers.com/videos/1221-infrastructure-as-code-the-developer-s-secret-weapon)
- [Cutting LLM Costs Without Cutting Quality: How to Beat Proprietary LLMs with Fine-Tuned Open Source](https://www.wearedevelopers.com/videos/100151-cutting-llm-costs-without-cutting-quality-how-to-beat-proprietary-llms-with-fine-tuned-open-source)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [OLTP in the Lakehouse: Redefining Data for AI Workloads](https://www.wearedevelopers.com/videos/2038-oltp-in-the-lakehouse-redefining-data-for-ai-workloads)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Graph and AI Trends 2026: Why Is AI Running but Not Yet Delivering?](https://www.wearedevelopers.com/magazine/680-graph-and-ai-trends-2026-why-is-ai-running-but-not-yet-delivering)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers)
- [Dev Digest 132 - Binging WADFlix?](https://www.wearedevelopers.com/magazine/473-dev-digest-132-binging-wadflix)