Lead Data Engineer

Oliver James
London, CA, United States
4 days ago
Apply on jobs.y-axis.com
Prepare application

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
2 years minimum
Working hours
Regular working hours
Languages
French

Tech stack

Artificial Intelligence Microsoft Azure Continuous Integration Information Engineering Python (Programming Language) Scrum Methodology Standard Sql Retrieval-Augmented Generation Large Language Models Apache Spark Data Lakes Virtual Agents
+1 more
Databricks

Job description

A major international travel business is looking for a Lead Data Engineer to run a team of 5-7 engineers building its Databricks data platform and data products.

The role is split evenly: half leading people, half leading the engineering. You’ll stay hands-on in code, design and review while coaching the team and acting as the main technical contact for business areas such as Commercial, Operations and Customer.

What you’ll do

  • Lead, coach and develop a team of data engineers, and work with the Engineering Manager, Scrum Master and product owners on priorities and delivery
  • Set and enforce coding, testing, review and CI/CD standards
  • Build reusable project templates (Databricks Asset Bundles), shared libraries and CI/CD pipelines that every new data product follows
  • Make data AI-ready: rich metadata, semantic definitions and curated Genie spaces
  • Set standards for, and review the work of, analytics engineers in the business

Requirements

  • 7+ years in data engineering, including at least 2 years leading or mentoring engineers
  • Deep hands-on experience with Databricks and Spark (Unity Catalog, Delta Lake, Lakeflow / DLT), plus strong Python and SQL
  • Strong data modelling skills and experience building data as a product
  • A track record of building templates, libraries or CI/CD that other engineers adopted
  • Working knowledge of AI data agents such as Databricks Genie
  • Clear communication with both technical and business stakeholders

Nice to have: dbt, Azure, FinOps on Databricks, data contracts, AI agent architecture (LLM, RAG, MCP), and experience leading teams across the UK and Europe. French is an advantage but not required. *

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on jobs.y-axis.com
Prepare application

Good distractions

Loading talks and stories from around this role…