> Markdown version of [/jobs/ext/3904069-lead-data-engineer-perm-aws-migration](https://www.wearedevelopers.com/jobs/ext/3904069-lead-data-engineer-perm-aws-migration). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Lead Data Engineer - Perm - AWS Migration

- **Company:** Lorien Resourcing
- **Location:** Manchester, UK
- **Experience:** Expert
- **Salary:** £85,000.0 - £100,000.0
- **Contract:** Permanent contract
- **Skills:** Amazon Web Services, Amazon S3, Apache HTTP Server, Big Data, Code Review, Continuous Integration, Information Engineering, Data Infrastructure, Data Warehousing, Python (Programming Language), Microsoft SQL Server, SQL Databases, Data Streaming, Data Lakes, Pyspark, Apache Flink, AWS Data Analytics, Apache Kafka, Spark Streaming, Terraform, Data Pipelines
- **Published:** October 24, 2026
- **Apply:** https://www.apply4u.co.uk/jobs/lead-data-engineer-perm-aws-migration/53589779

## About the Role

An independent, high-profile betting and gaming business is looking for a Lead Data Engineer who's happy to stay hands-on, set the standard and still get stuck into the code.
You'll be the senior technical authority for the data platform, working as the hiring manager's technical number two. The immediate focus is the final stages of a large-scale migration from on-premises SQL Server to an AWS lakehouse, alongside designing the real-time streaming architecture that will take report latency from around 24 hours down to minutes. You'll have real authority over architecture and standards, backed by code reviews, design guidance and leading by example. There are no direct reports, so your time goes on engineering rather than admin.
You'll work closely with data scientists, analysts and other engineering leads, and represent data engineering in architecture and planning conversations with senior stakeholders. Engineers across the business already use enterprise AI tooling, and there's room to go further.
What they're looking for:
Previous experience as technical lead on a large-scale data platform or data warehouse migration (cutover, reconciliation, parallel running, decommissioning)
Deep, hands-on AWS data experience: Glue, Lambda, S3, Redshift and EMR
Advanced Python, PySpark and SQL
Production experience with Kafka or Amazon MSK, including Spark Structured Streaming
Hands-on experience with Apache Iceberg or Delta Lake
Strong data modelling and lakehouse design (bronze, silver, gold)
Terraform, CI/CD and pipeline orchestration
A track record of setting standards and mentoring engineers through review, pairing and design sessions
Confidence communicating with technical and non-technical stakeholders, and challenging constructively
Desirable: SQL Server or on-prem to cloud migration, Apache Flink, regulated industry experience