> Markdown version of [/jobs/ext/2786168-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/2786168-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** Dotdigital - **Location:** Greater London, UK - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Agile Methodology, Artificial Intelligence, Airflow, Amazon Web Services, Microsoft Azure, BigQuery, Cloud Computing, Continuous Integration, Data as a Services, Information Engineering, Data Governance, Data Infrastructure, Data Security, Data Structures, Data Systems, Data Warehousing, Python (Programming Language), Microsoft Message Queuing, Microsoft SQL Server, MongoDB, Online Analytical Processing, NoSQL, Online Transaction Processing, SQL Databases, Data Streaming, Data Processing, Real Time Systems, Snowflake, Apache Spark, Event Driven Architecture, Build Management, Pyspark, Cassandra, Apache Kafka, Data Management, Vertica, Data Pipelines, Databricks - **Published:** September 8, 2026 - **Apply:** https://www.collegerecruiter.com/job/2840674725-senior-data-engineer ## About the Role Technical Expertise * Significant experience delivering Python-based projects for data engineering. * Experience building and tuning Spark pipelines that run at scale across large quantities of data. * Strong hands-on experience with SQL and NoSQL databases (e.g. SQL Server, MongoDB, Cassandra). * Proven experience with modern data warehousing and large-scale processing (e.g. Snowflake, DBT, BigQuery, ClickHouse). * Proficient with data orchestration tools such as Airflow, Dagster, or Prefect. * Experience with cloud platforms (Azure, AWS, or GCP) for data processing and storage. * Practical experience with Kafka or equivalent event-driven architectures (e.g. AWS SQS, Azure EventHubs, AWS Kinesis). * Good understanding of data modelling for OLAP and OLTP workloads. * Familiar with agile methodologies and CI/CD processes in the context of data solutions. * Experienced as a senior team member on complex data engineering projects. * Able to design and optimise data structures for high-volume systems. * Experienced in assisting with data platform modernisation or migration to the cloud. * Takes initiative to solve challenging data issues and drive projects forward. Bonus * Experience using ClickHouse as part of a data pipeline and analytics solution. * Experience using Databricks or similar data platforms. ## Description We are seeking a Senior Data Engineer to help evolve and enhance our data platform and capabilities. You'll work on building robust, scalable, and intelligent data systems that power real-time services, insights, and decisions across Dot Digital., As a Senior Data Engineer, you will be responsible for delivering key data platform features, designing and building data pipelines, and collaborating with product, analytics, and data science teams to unlock the value of data. You will have the opportunity to influence technology choices, optimise processes, and ensure the consistent, reliable, and secure movement and storage of data across the organisation., * Design and build scalable, reliable, and secure data pipelines for streaming, batch, and real-time processing. * Work in partnership with the Data Science teams to build scalable PySpark workloads that can be leveraged to generate advanced models. * Implement and optimise data models and storage solutions using Python and SQL with orchestration tools in a cloud environment. * Leverage AI to automate both data processing and engineering processes. * Advocate and uphold best practices for data governance, security, and monitoring. * Collaborate cross-functionally with engineers, analysts, and data scientists to deliver impactful data solutions. * Mentor and support junior engineers in data engineering principles and practices. * Evaluate and recommend new tools and technologies to strengthen data services. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) - [NoSQL Data Modeling for Front-end Developers](https://www.wearedevelopers.com/videos/297-nosql-data-modeling-for-front-end-developers) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Data Analyst Salary in the UK](https://www.wearedevelopers.com/magazine/278-data-analyst-salary-in-the-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [Fullstack Developer Salary UK](https://www.wearedevelopers.com/magazine/251-fullstack-developer-salary-uk)