Databricks Consultant

Databricks Architect/Consultant 3-Month Contract - Discovery Engagement - £750 per day Outside IR35

A leading international organisation within the building materials and distribution sector is undertaking a strategic modernisation of its data platform and is introducing Databricks as a core capability within its future data architecture.

To support this initiative, we are seeking an experienced Databricks Architect/Consultant to deliver a 3-month discovery and architecture engagement working closely with internal Data Engineering teams.

The purpose of this engagement is to assess the current data landscape and design the target-state Databricks architecture, alongside a clear and pragmatic migration roadmap.

The existing data environment includes Azure Synapse, Informatica, and FOCUS, and the consultant will play a key role in determining how these capabilities evolve as the organisation transitions to a modern lakehouse architecture.

This role is discovery and architecture focused, setting the foundation for a subsequent delivery programme., * Conduct a structured assessment of the current data platform landscape including Azure Synapse, Informatica, and FOCUS

• Review existing data pipelines, transformation processes, orchestration, storage, and reporting layers
• Evaluate the fit of Databricks components within the existing architecture
• Identify technical debt, performance limitations, and opportunities for optimisation
• Engage with business and technical stakeholders to understand analytical and operational data requirements
• Document the current-state architecture and key platform dependencies

Target Architecture Design

• Define the target-state data platform architecture using Databricks
• Design a scalable lakehouse architecture aligned with enterprise data and analytics needs
• Define patterns for:

• Data ingestion and integration
• Data transformation and processing
• Data modelling and data serving layers
• Environment management (Dev/Test/Prod)
• CI/CD and platform operations

• Design the security and data governance model, including Unity Catalogue
• Establish architectural principles and engineering standards for adoption of Databricks

Migration Strategy & Roadmap

• Define the migration approach for workloads currently operating within Synapse and Informatica
• Assess which workloads should be migrated, re-engineered, or retained
• Develop a phased migration roadmap prioritised by business value and complexity
• Provide high-level estimates and sequencing guidance
• Identify key risks, dependencies, and mitigation strategies

Collaboration & Knowledge Transfer

• Work closely with internal data engineering and architecture teams
• Provide architectural guidance and best practice recommendations
• Support internal teams in understanding the Databricks platform and lakehouse principles

• Proven experience architecting Databricks-based data platforms (Azure Databricks preferred)
• Experience delivering data platform discovery and architecture engagements
• Strong knowledge of lakehouse architecture, Delta Lake, and Spark-based processing
• Experience working within Azure data platform environments
• Experience with or migrating from Azure Synapse and/or Informatica
• Experience designing security and governance models within Databricks (including Unity Catalogue)
• Strong stakeholder engagement and documentation capability