Principal Data Consultant - AWS / Databricks

Scale Factory
London, UK
9 days ago
Apply on www.totaljobs.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
10 years minimum
Working hours
Regular working hours

Tech stack

Amazon Web Services Cloud Computing Code Review Continuous Integration Data Architecture Information Engineering Data Governance Data Vault Modeling Python (Programming Language) Machine Learning SQL Databases Data Streaming
+8 more
Retrieval-Augmented Generation Apache Spark SC Clearance Event Driven Architecture Apache Kafka Machine Learning Operations Terraform Databricks

Job description

You’ll be the architect on our highest-impact data engagements. You will define the vision with senior clients, defend your design in front of CTOs and engineering teams, and stay grounded in the codebase to prove it works. When the project demands it, you’ll step up as Tech Lead. Off-project, you’ll help shape our practice: taking real-world learnings and building the reference architectures, repeatable propositions, and delivery standards that set the benchmark for everything we do next., Client engagement leadership

  • Lead the architecture work on client engagements from discovery through to production
  • Run discovery and target-architecture workshops with client CTOs, heads of data and engineering leads
  • Translate commercial goals into an architecture and a sequenced roadmap the client can fund
  • Be the senior technical voice the client trusts when trade-offs, cost or risk are contested

Architecture ownership

  • Set and own the target data architecture: ingestion, storage, transformation, serving, governance
  • Make and document the significant decisions (lakehouse vs warehouse, batch vs streaming, build vs buy) with the trade-offs written down
  • Define non-functional requirements, cost models and FinOps guardrails alongside the functional design
  • Build in security, data protection, lineage and quality from the start, not as a later workstream

Hands-on technical leadership

  • Take the tech lead role on engagements when the shape of the team requires it
  • Write and review code where it matters: reference implementations, spikes, thorny pipelines, infrastructure-as-code
  • Set engineering standards for the practice - testing, CI/CD, environments, code review, observability
  • Coach and unblock engineers; raise the standard of the people around you

Practice building and pre-sales

  • Spot and qualify follow-on work inside live accounts
  • Build reusable assets: reference architectures, accelerators, delivery playbooks
  • Help define the hiring bar and interview the next data engineers and architects
  • Work with the existing cloud infrastructure and testing practices so data engagements pull them through, and vice versa, * Databricks and/or AWS certification at professional level, and experience supporting a firm’s partner status (competency badges, certified-headcount targets, co-sell with partner account teams)
  • Streaming and event-driven architecture in production (Kafka/MSK, Kinesis, Structured Streaming)
  • Data products, mesh or federated ownership models applied on a real programme, not just in theory
  • ML and AI platform work: feature stores, MLflow, model serving, and the data foundations behind RAG or agentic use cases
  • Regulated-sector exposure - financial services, healthcare, public sector - and the assurance that comes with it

How you work

  • Opinionated but not dogmatic: recommends the simplest architecture that meets the requirement, and says so when a client’s preferred tool is the wrong one
  • Comfortable being the only data person in the room and building the case from scratch
  • Writes clearly - decision records, options papers and diagrams a client executive can act on

Requirements

  • 10+ years in data engineering and architecture, with at least 3 as the lead architect on complex, multi-team programmes
  • Designed and delivered modern data platforms end to end on AWS and/or Databricks
  • Still writes production-grade code: strong Python and SQL, Spark, dbt or equivalent, infrastructure-as-code (Terraform preferred)
  • Track record in a consultancy, systems integrator or other client-facing setting - comfortable being accountable to a paying client, not only an internal stakeholder
  • Data modelling depth across dimensional, Data Vault and lakehouse/medallion approaches, with a view on when each is most appropriate
  • Data governance in practice: catalogue, lineage, access control, quality, and Regulatory obligations
  • Owned platform cost: sizing, forecasting and bringing a runaway bill back down
  • Estimating and scoping work, and shaping proposals that were won
  • SC clearance, or eligibility for it

Benefits & conditions

  • Commercially aware: understands utilisation, margin and how a scope conversation protects both sides
  • Generous with knowledge; wants to grow a team rather than be its single point of failure
  • Happy with the ambiguity of an early-stage practice inside an established business

What’s in It for You?

At Scale Factory, we believe in recognising and rewarding great work. Here’s what you can expect:

  • 25 Days of Annual Leave: Plus 1 extra day every year for the first three years, with a holiday buying scheme.
  • Pension Plan: Employer-matched contributions up to 5%.
  • Health Benefits: Comprehensive coverage for medical, dental, optical, and alternative therapies.
  • Fitness Support: Subsidised gym memberships and a bike-to-work scheme.
  • Commuter Benefits: Season ticket loans to make travel easier.
  • Free Annual Rail Cards
  • Electric vehicle salary sacrifice scheme
  • Flexible Work Model: Hybrid working that fits your lifestyle.
  • Social Events: From annual kick-offs and Christmas parties to team socials and sporting events, we love celebrating success together.
  • Training & Development: Tailored learning opportunities and full support for certifications to keep your skills sharp.

About the company

We are an AI enablement consultancy. Delivering quality-led AI-first foundations, embedding capability and excellence into the foundational fabric of the companies we work with. We give leaders the confidence to move faster, take smarter risks and pursue transformation.

Formerly Ten10, we have deep roots in Quality which means we don’t just build for the future, we ensure it actually works. We combine this engineering discipline with cutting-edge AI expertise to deliver robust, scalable foundations that turn ambitious transformation into reliable reality.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.totaljobs.com
Prepare application

Good distractions

Loading talks and stories from around this role…