Senior Data Engineer Databricks & Azure Data Platform

Acuity INC
UK
4 days ago
Apply on www.apply4u.co.uk
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours

Tech stack

A/B Testing Application Programming Interfaces (APIs) Artificial Intelligence Data Analysis Microsoft Azure Continuous Integration Data Systems Python (Programming Language) Search Technologies SQL Databases Unstructured Data Cloud Platform System
+8 more
Azure Data Factory Large Language Models Generative AI Pyspark Api Design Terraform Data Pipelines Databricks

Job description

commercially valuable datasets. Build and integrate APIs to expose data and power products. Apply LLMs, RAG and Generative AI to classification, enrichment, search and insight generation. Work with structured and unstructured data, embeddings and semantic search. Help evolve our Databricks and Azure data platform. Improve data quality, testing, observability and CI/CD. Partner with product and commercial teams to turn business opportunities into technical solutions. Mentor engineers and help shape our growing Data & AI capability. What We’re Looking For Hands-on experience with Databricks. Advanced Python, PySpark and SQL. Strong experience building scalable data pipelines and data models. Experience with Azure or another major cloud platform. Experience building APIs and exposing data to applications or customers. Experience with data products or customer-facing data solutions. Exposure to Generative AI / LLMs and real-world AI use cases. Familiarity with RAG, embeddings, vector

Requirements

databases, or semantic search. Strong communication skills and confidence working with technical, product and commercial stakeholders. A product mindset - you think about how data creates value, not just how it moves from A to B. Nice to Have Airbyte Terraform A/B testing or experimentation AI governance Life sciences or pharmaceutical experience Why Join Us? You’ll have the opportunity to build products, not just infrastructure, shape how AI is used within real-world data solutions and have genuine influence over technical and product direction. You’ll work with a modern Data & AI stack, have space to explore new technologies and help build solutions that deliver measurable customer impact. Learn more about life at Acuity Analytics on our Working Here page. #J-18808-Ljbffr

About the company

Ascent has recently joined Acuity Analytics, bringing together deep industry expertise, technology and data capabilities to help organisations solve complex challenges and drive meaningful change. About the Role We’re looking for a Senior Data Engineer who wants to do more than build pipelines. You’ll help turn complex data into commercial, customer-facing products, combining strong Data Engineering skills with emerging AI and LLM technologies. Working across Databricks, Azure, Python and PySpark, you’ll build scalable data solutions, explore new AI capabilities and work directly with product and commercial teams to bring ideas into production. This is a hands-on role with plenty of ownership: approximately 60% engineering, 20% mentoring and 20% exploration and innovation. What You’ll Be Doing Build scalable data products and customer-facing data solutions. Develop production-ready pipelines using Databricks, Python, PySpark and SQL. Transform raw data into structured, enriched and

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.apply4u.co.uk
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

2:27 min

Managing traffic and tracking costs with Databricks Unity Catalog

Viktoria Semaan Viktoria Semaan · World Congress 2026 Europe

1:34 min

Essential commands for running and testing Terraform configurations

Hennie Francis · LIVE

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

1:59 min

Key takeaways and accessing the Databricks developer toolkit

Viktoria Semaan Viktoria Semaan · World Congress 2026 Europe

1:12 min

Building intelligent applications and enhancing developer productivity experiences

Alexander Wallner Alexander Wallner +3 · World Congress 2024

Videos

See all

Related articles

See all