> Markdown version of [/jobs/ext/99101-senior-data-engineer-ingestion-platform](https://www.wearedevelopers.com/jobs/ext/99101-senior-data-engineer-ingestion-platform). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - Ingestion & Platform - **Company:** traffic label - **Location:** UK (Remote available) - **Experience:** Expert - **Contract:** Temporary contract - **Skills:** Application Programming Interfaces (APIs), Airflow, Automation of Tests, Databases, Continuous Integration, Information Engineering, Data Infrastructure, Extract Transform Load (ETL), File Systems, Github, Python (Programming Language), Metadata, OAuth, Systems Integration, Data Ingestion, Snowflake, Build Management, Sendgrid, Api Design, Restful APIs, Terraform, Pagination - **Published:** May 20, 2026 - **Apply:** https://uk.indeed.com/viewjob?jk=8813fce9ae4c8b09 ## About the Role Must have: * 6+ years in data engineering. * Advanced Python: API clients, file processing, schema validation, integration logic. * Deep Airflow experience: DAG design, dynamic generation, sensors, retry patterns, backfill, pools, connections management. * dbt (mid+): bronze/silver models, source freshness, tests, materialisation strategy. * Snowflake (mid): external stages, COPY INTO, storage integration, basic optimisation. * API integration experience: authentication (OAuth, API keys), pagination, rate limiting, retry with backoff, error classification. * Proven experience building reusable pipeline frameworks/templates that others use daily. * Reverse ETL or outbound integration experience: pushed data from the warehouse to external systems. * Writes clear documentation: architecture decisions, pipeline templates, and onboarding guides for the team. * CI/CD pipeline design (GitHub Actions)., * Terraform basics. * Metadata-driven / config-driven pipeline experience. * Experience with ESP platforms (DotDigital, SendGrid, Braze, or similar). * Gambling, marketing, or multi-tenant data platform experience. * Comfortable using AI coding assistants. ## Description We are building a Customer Data Platform (CDP) that ingests data from 10+ casino partners, builds a unified player profile, and activates marketing campaigns through multiple ESPs (DotDigital, Adestra, FastTrack). The platform is live: 5 sources in production, 30-40 dbt models in the silver layer, working email campaigns through DotDigital. You'll join a Data Platform team of 5-10 engineers (Senior DE, QA, DA, PM) reporting to the Data Engineering Lead. You will own the ingestion layer, source integrations, and platform tooling. Your job: make it so that connecting a new data source takes days, not months. You'll design and build reusable pipeline templates and own integrations with external systems (both inbound and outbound). This is not a "write one pipeline and move on" role. This is "build the system that generates pipelines" + own the integrations that feed and consume data. Why this role Connecting a new data source currently takes 2-3 months. Your job: make it 5 days. This is a data platform engineering challenge, not a pipeline writing task. This is the group's flagship project. The CDP improves how the company works with players across all brands and directly generates revenue through better marketing activation. The platform is 4 months old, modern stack, zero legacy. But already in production with real traffic, real campaigns, real money. You're scaling a working system, not prototyping in isolation. The core challenge is designing a system where adding a new data partner is a configuration. You'll solve problems across the full spectrum: API design, schema evolution, orchestration patterns, data quality, and delivery to downstream consumers., * Design and build ingestion pipelines for REST API, database sources, etc., using Python * Own ETL and Reverse ETL end-to-end, including implementation, deployment, operations, and incident handling. * Understand how external systems work (casino platforms, ESPs, affiliate tools) to build reliable integrations with them. * Design DAG patterns: dependency management, partial retry, backfill procedures. * Implement and maintain authentication flows, token refresh handling, and secrets management. * Monitor, troubleshoot, and resolve integration issues in production environments. * Write and maintain automated tests to ensure the reliability and stability of integration services. * Collaborate closely with Engineering, Product, and Data teams to support platform scalability and operational efficiency. * Contribute to infrastructure improvements and Infrastructure-as-Code initiatives. * Write Architecture Decision Records (ADRs) for significant design choices., Data Engineer Locations Remote Europe, Remote UK Remote status Fully Remote ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Keeping applications secure by evolving OAuth 2.0 and OpenID Connect](https://www.wearedevelopers.com/videos/100152-keeping-applications-secure-by-evolving-oauth-2-0-and-openid-connect) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) - [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Dev Digest 120 - Apple and peers](https://www.wearedevelopers.com/magazine/455-dev-digest-120-apple-and-peers) - [Fullstack Developer Salary UK](https://www.wearedevelopers.com/magazine/251-fullstack-developer-salary-uk)