Data Engineer (Databricks)

Sanderson Recruitment Plc
London, UK
29 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours

Tech stack

Application Programming Interfaces (APIs) Amazon Web Services Amazon S3 Cloud Computing Information Engineering Data Infrastructure Data Migration Identity and Access Management Power BI Data Streaming Cloud Platform System Data Ingestion
+3 more
Collibra Data Pipelines Databricks

Job description

  • Evangelise and drive the introduction and adoption of Databricks across
  • Support and shape the strategy for migrating data pipelines, data feeds, and associated services to Databricks
  • Work with architects and engineering teams to define and evolve the target data platform architecture
  • Establish best practices for data engineering, governance, and platform usage
  • Contribute to the design of the target operating model for the data platform
  • Lead the design, testing, and validation of use cases and hypotheses to demonstrate platform value
  • Define and support learning and development pathways to upskill internal teams
  • Facilitate workshops, discovery sessions, and stakeholder engagement activities

Requirements

We are seeking an experienced Databricks expert and professional to support the introduction and adoption of the Databricks platform across the organisation.

This role will play a key part in helping us to shape our approach to alpha phase of the project, our data platform transition strategy, supporting the migration of existing data products, and establishing best practices, governance, and operating models.

The successful candidate will combine strong technical expertise with the ability to engage stakeholders, translate business needs into solutions, and help with defining what is required to upskill internal teams., Databricks & Data Engineering

  • Strong understanding of Databricks architecture, security, and core services
  • Experience using Databricks Workflows, Jobs, and Notebooks
  • Proven experience delivering at least one end-to-end Databricks implementation or migration
  • Experience designing and building scalable data pipelines (batch and streaming)
  • Experience with data modelling approaches, including medallion architecture (bronze/silver/gold)
  • Experience in large-scale enterprise data platform transformations

Cloud & Platform Experience

  • Experience working with cloud-based data platforms (preferably AWS, including S3, Glue, IAM)

Data Migration & Architecture

  • Experience supporting data migration and modernisation initiatives
  • Ability to collaborate with architects and engineers to define target architecture, including:

  • Data ingestion layers
  • Processing frameworks
  • Serving layers (BI and APIs)

Ability to balance performance, cost, and governance considerations

Stakeholder Engagement & Consulting

  • Ability to communicate Databricks concepts clearly to technical audiences
  • Ability to engage effectively with non-technical stakeholders
  • Experience translating business use cases into scalable data solutions
  • Experience running workshops, discovery sessions, and engagement activities
  • Proven experience operating in a senior data engineering, architecture, or consulting role
  • Experience supporting platform adoption and organisational change initiatives

Desirable

  • Experience with data governance tools or frameworks
  • Knowledge of BI tools such as Power BI
  • Relevant Databricks or cloud certifications

Working Style

  • Collaborative and consultative
  • Comfortable working across business and technical teams
  • Able to operate in a fast-evolving environment with multiple stakeholders

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.totaljobs.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

1:24 min

Moving the semantic layer upstream to avoid vendor lock-in

Piotr Menclewicz Piotr Menclewicz · Europe 2026 Virtual

3:43 min

The enduring legacy of the amazon S3 storage API

Chris Heilmann +3 · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

3:48 min

Standardizing data access schemas with OData

Florian Bader Florian Bader · WWC Europe 2026

3:09 min

Balancing data science skillings alongside systems engineering rigor

Nico Schmidt · LIVE

3:44 min

Automating storage savings with S3 intelligent tiering

Sébastien Stormacq · WWC 2021

Videos

See all

Related articles

See all