Databricks Champion Architect

NTT DATA
London, UK
8 days ago
Apply on www.apply4u.co.uk
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Experienced
Experience required
2 years minimum
Working hours
Regular working hours

Tech stack

Application Programming Interfaces (APIs) Artificial Intelligence Amazon Web Services Microsoft Azure Continuous Integration Data Governance Data Sharing High-Level Architecture Information Lifecycle Management Key Management Machine Learning Query Optimization
+11 more
Runbook Search Technologies Strategies of Testing Data Classification Retrieval-Augmented Generation Concurrency Apache Spark Generative AI Data Lakes Machine Learning Operations Databricks

Job description

Own the end-to-end Databricks solution architecture across ingestion, transformation, governance, serving, and operations.Define and enforce Medallion Architecture (Bronze/Silver/Gold) standards and reference designs (including data quality, SCD/CDC patterns, performance, and retention).Architect ingestion approaches using Lakeflow Connect, AutoLoader, batch/streaming/CDC strategies, and enterprise integration patterns.Establish platform governance using Unity Catalog: Namespace strategy, access models, lineage, data sharing, and secure-by-design patterns.Define the approach for Declarative Pipelines with Expectations (DLT): Data quality strategy, validation rules, quarantine patterns, and observability standards.Champion “production-ready” engineering standards: CI/CD approach, environment strategy (dev/test/prod), monitoring, runbooks, and operational readiness.Drive stakeholder alignment: Work with security, enterprise architecture, data governance, platform teams, and delivery squads.Enable and upskill teams: Provide playbooks, templates, architecture decision records (ADRs), design reviews, and technical coaching.Contribute to pre-sales and roadmap: High-level designs, estimation, risk/dependency management, and technical proposals.

What experience you’ll bring:

Minimum Experience

2+ years of hands-on Databricks experience gained recently (e.g., within the last 2-3 years) with evidence of modern feature usage and production delivery.Demonstrable architecture ownership (solution design, governance models, or platform standards), even if overall industry experience is greater than 2 years., We are an equal opportunities employer. We believe in the fair treatment of all our employees and commit to promoting equity and diversity in our employment practices. We are also a proud Disability Confident Committed Employer - we are committed to creating a diverse and inclusive workforce. We actively collaborate with individuals who have disabilities and long-term health conditions which have an effect on their ability to do normal daily activities, ensuring that barriers are eliminated when it comes to employment opportunities. In line with our commitment, we guarantee an interview to applicants who declare to us, during the application process, that they have a disability and meet the minimum requirements for the role. If you require any reasonable adjustments during the recruitment process, please let us know. Join us in building a truly diverse and empowered team.

Requirements

A minimum bar is 2+ years of Databricks experience in recent years, demonstrating strong familiarity with modern capabilities and current platform patterns (not legacy Spark-only experience)., Databricks Certified Data Engineer Associate/ProfessionalDatabricks Certified Machine Learning Associate/ProfessionalDatabricks Certified Generative AI Engineer (Associate)Medallion Architecture design ownership: Proven ability to design Bronze/Silver/Gold layers, data contracts, data lifecycle, and serving patterns (BI, APIs, ML features).Unity Catalog architecture experience: Governance model design (catalog strategy, grants), lineage, and secure access patterns.Declarative Pipelines with Expectations (DLT): Architectural understanding plus practical experience implementing expectations, failure handling, and quality reporting.Ingestion architecture & delivery: Hands-on experience designing ingestion using AutoLoader and/or Lakeflow Connect (including schema evolution, incremental loads, and operational patterns).Databricks experience with latest capabilities: Candidate can clearly articulate what modern features they used recently and why (platform evolution awareness is essential for this role).

Should Have

Strong command of Delta Lake and performance patterns: MERGE/CDC strategies, partitioning/file sizing, OPTIMIZE/ZORDER, VACUUM, table maintenance automation.Production architecture practices: Environment separation, secrets management, job orchestration standards (Databricks Workflows), testing strategy, and observability.Cloud platform integration understanding (one or more of Azure/AWS/GCP): Identity, networking, storage, and security integration considerations.Ability to influence and standardise across teams: Templates, reference repos, reusable components, and governance guardrails.

Nice to Have

Lakeflow ingestion connectors experience (specific connector implementations).RBAC / masking expertise: Row-level security, column masking, sensitive data classification patterns in Unity Catalog.GenAI capabilities on Databricks: Mosaic AI, Vector Search, model serving, RAG architectures, AI Functions.Lakebase awareness or hands-on experience.Workload & query optimisation at scale: Photon adoption patterns, cluster policy design, skew/shuffle tuning, streaming tuning, concurrency strategies.Cost governance: Understanding cost drivers (DBUs/compute types), chargeback/showback approaches, cluster policies, and cost controls.

About the company

We’re a business with a global reach that empowers local teams, and we undertake hugely exciting work that is genuinely changing the world. Our advanced portfolio of consulting, applications, business process, cloud, and infrastructure services will allow you to achieve great things by working with brilliant colleagues, and clients, on exciting projects.

Our inclusive work environment prioritises mutual respect, accountability, and continuous learning for all our people. This approach fosters collaboration, well-being, growth, and agility, leading to a more diverse, innovative, and competitive organisation. We are also proud to share that we have a range of Inclusion Networks such as: the Women’s Business Network, Cultural and Ethnicity Network, LGBTQ+ & Allies Network, Neurodiversity Network and the Parent Network.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.apply4u.co.uk
Prepare application

Good distractions

Loading talks and stories from around this role…