Lead Data Engineer

developrec
Solihull, UK
4 days ago
Apply on www.totaljobs.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours

Tech stack

Airflow Computing Platforms Automation of Tests BigQuery Cloud Storage Software Quality Code Review Continuous Integration Information Engineering Data Systems Data Vault Modeling Dimensional Modeling
+15 more
Data Flow Control Python (Programming Language) Standard Sql Cloudera Software Engineering SQL Databases Data Streaming Apache Spark Technical Debt Git Infrastructure Automation Frameworks Data Management Terraform Looker Analytics Data Pipelines

Job description

We’re looking for an experienced Lead Data Engineer to provide hands-on technical leadership within a data engineering team for a large retail brand. You’ll lead the design and delivery of scalable, resilient data platforms and pipelines, set engineering standards, shape technical direction, and mentor engineers. You’ll work closely with engineering, data and business stakeholders to solve complex problems and deliver high-quality data solutions., * Lead the technical design and architecture of data pipelines, models and platform components.

  • Establish standards across code quality, testing, security, observability and documentation.
  • Drive technical discovery, design sessions, code reviews and resolution of technical debt.
  • Design and deliver scalable batch and streaming data solutions.
  • Mentor and coach engineers, supporting technical development and knowledge sharing.
  • Make informed recommendations on tools, technologies and architectural approaches.
  • Champion data quality, governance, security and compliance by design.
  • Collaborate with data, product and technology teams to deliver joined-up solutions.
  • Contribute to technical hiring, onboarding and team capability development.

Requirements

  • Deep experience with GCP, particularly BigQuery, Dataflow, Pub/Sub, Cloud Storage and Cloud Composer/Airflow.
  • Advanced Python and SQL skills with a strong software engineering mindset.
  • Proven experience building complex, scalable batch and event-driven pipelines.
  • Strong data modelling experience, including dimensional modelling, Data Vault or equivalent.
  • Experience with Dataform or similar SQL-based transformation tooling.
  • Understanding of CI/CD, Git, automated testing and infrastructure as code, particularly Terraform.
  • Experience implementing data quality, monitoring, observability and alerting.
  • Ability to lead architecture decisions and communicate technical trade-offs clearly.

Desirable: Spark/DataProc, Looker or similar BI tools, analytics engineering, data mesh/platform architecture, GCP cost optimisation, and experience within retail or e-commerce environments.

Benefits & conditions

  • Performance-related bonus
  • Healthcare and employer pension contributions
  • 25 days’ holiday plus birthday and bank holidays
  • Flexible benefits, including EV, dental, cycle-to-work and holiday trading schemes
  • Employee discounts and wellbeing support
  • Enhanced family leave and life assurance
  • Onsite gym membership, lunch and coffee facilities

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.totaljobs.com
Prepare application

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon ¡ World Congress 2026 Europe

6:21 min

Investigating push inefficiencies with upstream Git experts

Jonathan Creamer ¡ Coffee With Developers

2:30 min

Leveraging BigQuery ML for scalable SQL-based segmentation experiments

Julian Joseph ¡ LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou ¡ Coffee With Developers

56 sec

Favorite git commands and the importance of patch commits

Eileen Uchitelle Eileen Uchitelle +1 ¡ Coffee With Developers

3:05 min

Audience questions on AI agents and pipeline vectorization

Joy Joy ¡ World Congress 2024

Videos

See all

Related articles

See all