Data Engineer (Databricks)
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+3 more
Job description
- Evangelise and drive the introduction and adoption of Databricks across
- Support and shape the strategy for migrating data pipelines, data feeds, and associated services to Databricks
- Work with architects and engineering teams to define and evolve the target data platform architecture
- Establish best practices for data engineering, governance, and platform usage
- Contribute to the design of the target operating model for the data platform
- Lead the design, testing, and validation of use cases and hypotheses to demonstrate platform value
- Define and support learning and development pathways to upskill internal teams
- Facilitate workshops, discovery sessions, and stakeholder engagement activities
Requirements
We are seeking an experienced Databricks expert and professional to support the introduction and adoption of the Databricks platform across the organisation.
This role will play a key part in helping us to shape our approach to alpha phase of the project, our data platform transition strategy, supporting the migration of existing data products, and establishing best practices, governance, and operating models.
The successful candidate will combine strong technical expertise with the ability to engage stakeholders, translate business needs into solutions, and help with defining what is required to upskill internal teams., Databricks & Data Engineering
- Strong understanding of Databricks architecture, security, and core services
- Experience using Databricks Workflows, Jobs, and Notebooks
- Proven experience delivering at least one end-to-end Databricks implementation or migration
- Experience designing and building scalable data pipelines (batch and streaming)
- Experience with data modelling approaches, including medallion architecture (bronze/silver/gold)
- Experience in large-scale enterprise data platform transformations
Cloud & Platform Experience
- Experience working with cloud-based data platforms (preferably AWS, including S3, Glue, IAM)
Data Migration & Architecture
- Experience supporting data migration and modernisation initiatives
-
Ability to collaborate with architects and engineers to define target architecture, including:
- Data ingestion layers
- Processing frameworks
- Serving layers (BI and APIs)
Ability to balance performance, cost, and governance considerations
Stakeholder Engagement & Consulting
- Ability to communicate Databricks concepts clearly to technical audiences
- Ability to engage effectively with non-technical stakeholders
- Experience translating business use cases into scalable data solutions
- Experience running workshops, discovery sessions, and engagement activities
- Proven experience operating in a senior data engineering, architecture, or consulting role
- Experience supporting platform adoption and organisational change initiatives
Desirable
- Experience with data governance tools or frameworks
- Knowledge of BI tools such as Power BI
- Relevant Databricks or cloud certifications
Working Style
- Collaborative and consultative
- Comfortable working across business and technical teams
- Able to operate in a fast-evolving environment with multiple stakeholders
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.totaljobs.comGood distractions
Talks and stories from around this role — technically off-topic, practically not.
Moments
Explore playlistsVideos
See allRelated articles
See all
Data Analyst Salary in the UK
Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production
Making Data Warehouses Fast: A Developer’s Story
Top Big Data Technologies That You Need to Know