Senior Data Engineer (Databricks) - London

Auquan
London, United Kingdom
2 days ago
Apply on www.adzuna.co.uk
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
5 years minimum
Compensation
£54,395
Working hours
Regular working hours

Tech stack

Artificial Intelligence Airflow Amazon Web Services Microsoft Azure Spreadsheets Customer Data Management Information Engineering Distributed Computing Environment Python (Programming Language) Raw Data Cloud Services SQL Databases
+5 more
Data Streaming Apache Spark Change Data Capture Apache Kafka Databricks

Job description

ALFA builds autonomous AI agents for finance workflows (credit, investment, ESG and risk) for banks, private equity and private credit firms, and asset managers. Agents are only as good as the data they run on, and that data arrives messy: filings, portfolio packs, CRM exports, spreadsheets and PDFs.

The Data Engineer owns the path from a customer’s raw data to clean, secure, well-modeled data that our agentic workflows and deliverables can trust. You will work in pods alongside Outcome Engineers and Product Engineers, and turn one-off customer data work into reusable pipelines and primitives as our platform grows.

Requirements

  • 5+ years in data engineering, building production pipelines end to end.
  • Strong Python and SQL.
  • Hands-on experience with Spark or another distributed processing engine.
  • Experience with orchestration tools (Airflow, Dagster, Prefect or similar) and cloud data platforms (AWS, GCP or Azure).
  • Solid grasp of data modelling, partitioning, and incremental or change-data-capture (CDC) patterns.
  • You take ownership, communicate clearly, and like working closely with the people who use the data.

Nice to have

  • Databrtalog,Workflows, DLT) or similar lakehouse work on Snorg/Hudi.
  • Experience with financial or market data.
  • Experi or AIsystems: document parsing, vector stores, retrie
  • Streaming (Kafka, Kinesis) and dbt.
  • Customer empathy: you think like an analyst or risk officer and know which numbers drive their decisions
  • Structured thinker: you turn noisy data into clear schemas, steps and validation rules
  • Comfortable in ambiguity: you make progress with imperfect data and bring structure where there is none
  • Proactive problem-solver: you spot a broken feed before the client does
  • Hustle and ownership: you own outcomes end to end, not just tickets
  • Curious, fast learner: you keep up with data tooling, AI and finance developments
  • Clear communicator: you explain data trade-offs to senior customer stakeholders
  • Commercial instinct: you see where a pipeline built for one customer can serve the next

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.adzuna.co.uk
Prepare application

Good distractions

Loading talks and stories from around this role…