> Markdown version of [/jobs/ext/1542614-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/1542614-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** CreateFuture - **Location:** Edinburgh, UK (Remote available) - **Experience:** Expert - **Salary:** £63,954.0 - **Contract:** Permanent contract - **Skills:** Clean Code Principles, Airflow, Amazon Web Services, Amazon S3, Automation of Tests, Microsoft Azure, Big Data, BigQuery, Cloud Engineering, Code Review, Customer Data Management, Data Governance, Data Integrity, Data Systems, DevOps, Data Flow Control, Python (Programming Language), Uptime, SQL Databases, Data Streaming, AWS Cdk, Delivery Pipeline, Snowflake, Apache Spark, Event Driven Architecture, Apache Flink, AWS Glue, Apache Kafka, Spark Streaming, Data Management, Terraform, Data Pipelines, Legacy Systems, Amazon Redshift, Databricks - **Published:** July 15, 2026 - **Apply:** https://www.adzuna.co.uk/jobs/details/5801190136 ## About the Role * Solid experience with Python, SQL, and Spark - and pipeline tools such as dbt or Airflow * Comfortable with data quality frameworks and testing - Great Expectations, dbt tests, or custom validation layers * Experience with infrastructure-as-code (Terraform, AWS CDK) is a plus * Familiar with broader cloud platforms - Azure (Data Factory, Synapse) or GCP (Dataflow, BigQuery) - is beneficial but not essential, Depending on the role, we might also ask you to do a short presentation, a practical or technical task or have a values focused conversation. We will explain what is involved before anything happens. ## Description We're looking for an experienced Senior Data Engineer to join our growing Data Practice, specialising in iGaming. You'll work as part of a multidisciplinary team delivering data solutions for some of the world's largest betting and gaming operators - organisations processing millions of events per day across sportsbook, casino, and exchange platforms. This is a role for someone who understands the pressures of the iGaming data landscape: sub-second odds feeds, high-stakes data integrity requirements, complex regulatory reporting, and large-scale platform migrations from legacy on-premises estates to cloud-native architectures on AWS. What you'll be doing * Deliver large-scale data migrations from on-premises or legacy platforms to cloud-native AWS architectures, including Lakehouse implementations using Databricks or Snowflake. * Design, build, and optimise low-latency data pipelines that process real-time betting events, odds updates, and market data at scale. * Implement robust data quality, validation, reconciliation, and audit controls across betting, trading, and customer data domains. * Develop streaming and micro-batch data solutions using AWS Kinesis, Apache Kafka, or Amazon MSK to support real-time sportsbook and in-play data feeds. * Build and maintain data solutions using AWS services including S3, Glue, Lambda, Athena, Redshift, Lake Formation, Step Functions, and MWAA. * Support regulatory and compliance reporting by ensuring data pipelines are reliable, traceable, and meet audit and lineage requirements for the UK Gambling Commission and other international regulators. * Collaborate with architects, platform engineers, and cross-functional teams to design scalable, resilient, and high-performing data platforms. * Champion engineering best practices through high-quality code, automated testing, documentation, and peer code reviews, while mentoring junior engineers where appropriate. * Contribute technical expertise during solution design sessions, architecture discussions, and client workshops, providing input into delivery approaches and technical decisions., * You've worked in or closely with sportsbook, exchange, or casino platforms - and understand the data models behind markets, selections, bets, and settlements * You've built or maintained low-latency data feeds - odds ingestion, in-play event streams, price change feeds - where milliseconds matter * You understand the data integrity challenges specific to betting: reconciliation between trading and settlement systems, detection of data drift, and auditability for regulatory purposes * You've delivered or contributed to large-scale data migrations - moving high-volume transactional or event data from legacy estates to the cloud without losing fidelity or uptime * AWS is your primary cloud - you're confident across Glue, Lambda, Athena, S3, Redshift, RDS, Kinesis, MSK, and Step Functions * You've built real-time or near-real-time pipelines using streaming frameworks - Kinesis Data Streams, Kafka, Flink, or Spark Streaming * You understand AWS cost models and how to optimise high-throughput workloads - Spot compute, reserved capacity, and right-sizing for event-driven architectures * AWS certifications are a bonus, AWS - Glue, Lambda, Athena, S3, Redshift, RDS, Kinesis, MSK, Lake Formation, Step Functions, MWAA Streaming AWS Kinesis, Apache Kafka / MSK, Apache Flink, Spark Streaming Languages Python, SQL, Spark Pipeline tools dbt, Airflow / AWS MWAA, AWS Glue Platforms Databricks, Snowflake, Amazon Redshift Data quality Great Expectations, dbt tests, custom reconciliation frameworks IaC / DevOps Terraform, AWS CDK (desirable) What we'll offer you: We trust people to do their best work. That means flexibility over rigid rules, impact over activity, and real investment in your growth both professionally and personally. You'll be part of a supportive, and friendly culture, surrounded by smart, curious people who care deeply about what they do. We offer flexible working, including hybrid and remote options. Our office hubs are located in Edinburgh, Leeds, Manchester, London and Bulgaria, with occasional travel to client sites or CreateFuture offices when needed. We trust you to manage your time balancing collaboration with client time and focused work. What matters is the impact you have, not how busy you look. Our hiring process We try to keep our hiring process clear, fair and respectful of your time. We aim to get back to everyone who applies and we will be upfront about where you are in the process. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [From DevOps to Scaled DevOps: How We’re Rebuilding Continuous Delivery as a Platform](https://www.wearedevelopers.com/videos/100018-from-devops-to-scaled-devops-how-we-re-rebuilding-continuous-delivery-as-a-platform) - [DevOps at Netflix](https://www.wearedevelopers.com/videos/270-devops-at-netflix) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) - [DevOps Maturity Check – a way to balance autonomy and alignment](https://www.wearedevelopers.com/videos/58-devops-maturity-check-a-way-to-balance-autonomy-and-alignment) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [A Guide to Green Tech and Green IT Careers](https://www.wearedevelopers.com/magazine/374-a-guide-to-green-tech-and-green-it-careers) - [Dev Digest 121 - AI goes offline](https://www.wearedevelopers.com/magazine/456-dev-digest-121-ai-goes-offline) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market)