Senior Data Engineer

Understanding Recruitment
Greater London, United Kingdom
3 days ago
Apply on www.understandingrecruitment.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Working hours
Regular working hours

Tech stack

Artificial Intelligence Amazon Web Services Apache HTTP Server Systems Engineering Cloud Computing Data Architecture Information Engineering Data Infrastructure Extract Transform Load (ETL) Distributed Computing Environment Python (Programming Language) Machine Learning
+13 more
Operational Databases Raw Data Cloud Services Standard Sql Parquet Data Ingestion Large Language Models Event Driven Architecture Data Lakes AWS Data Analytics Apache Kafka Machine Learning Operations Data Pipelines

Job description

Senior Data Engineer

We’re hiring a Senior Data Engineer to take technical ownership of a modern, AWS-native data platform, spanning everything from raw data ingestion and transformation to building scalable, reliable data pipelines that power production machine learning and AI systems.

This is a hands-on, high-autonomy role within a lean engineering team, where you’ll work directly with senior technical leadership and have a genuine say in how the data architecture and platform evolve.

The Engineering Challenge

You’ll own and evolve an AWS-native data environment, improving the reliability, scalability and performance of existing data infrastructure while building the foundations for future ML and AI initiatives.

The current stack includes:

  • Data Engineering: Apache Iceberg, Kafka, Parquet and Amazon Athena
  • Cloud Infrastructure: AWS-native data lake architecture, data ingestion, processing and storage
  • Architecture: AWS-based data lake (~5TB), with opportunities to modernise pipelines, optimise performance and improve platform maturity
  • ML & AI Integration: Supporting production ML workflows through SageMaker, with future opportunities around GenAI and LLM-powered diagnostic capabilities

What You’ll Own

  • Designing, building and maintaining scalable, production-grade data pipelines across batch and streaming workloads
  • Evolving the AWS data lake architecture, improving data quality, reliability, performance and accessibility
  • Building and optimising data ingestion, transformation and processing workflows using modern data engineering technologies
  • Working directly with data scientists, ML engineers and architects to ensure reliable, high-quality data is available for production ML systems
  • Influencing architectural decisions, establishing engineering best practices and shaping the next generation of data infrastructure

What We’re Looking For

  • Strong hands-on experience building and operating production data engineering systems
  • Deep familiarity with AWS and cloud-native data architectures
  • Strong Python and SQL skills, with solid software engineering fundamentals
  • Experience designing and maintaining scalable ETL/ELT pipelines, data lakes or modern lakehouse architectures
  • Understanding of distributed data processing, streaming architectures and data pipeline optimisation
  • Someone who enjoys end-to-end ownership, technical problem-solving and working without layers of bureaucracy
  • Experience with Apache Iceberg, Kafka, Parquet or Athena is highly desirable, while exposure to SageMaker or ML infrastructure is a bonus - production data engineering expertise is the priority.

Requirements

  • Strong hands-on experience building and operating production data engineering systems
  • Deep familiarity with AWS and cloud-native data architectures
  • Strong Python and SQL skills, with solid software engineering fundamentals
  • Experience designing and maintaining scalable ETL/ELT pipelines, data lakes or modern lakehouse architectures
  • Understanding of distributed data processing, streaming architectures and data pipeline optimisation
  • Someone who enjoys end-to-end ownership, technical problem-solving and working without layers of bureaucracy
  • Experience with Apache Iceberg, Kafka, Parquet or Athena is highly desirable, while exposure to SageMaker or ML infrastructure is a bonus - production data engineering expertise is the priority.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.understandingrecruitment.com
Prepare application

Good distractions

Loading talks and stories from around this role…