> Markdown version of [/jobs/ext/2794920-senior-data-engineer](https://www.wearedevelopers.com/jobs/ext/2794920-senior-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Senior Data Engineer - **Company:** Retelligence - **Location:** London, UK - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Airflow, Amazon Web Services, Databases, Information Engineering, Data Infrastructure, Fault Tolerance, Python (Programming Language), NoSQL, Performance Tuning, Standard Sql, Data Streaming, Workflow Management Systems, Data Logging, Scripting, Google Cloud, Cloud Platform System, Data Ingestion, Kubernetes, Real Time Data, Apache Kafka, Video Streaming, Data Pipelines - **Published:** September 8, 2026 - **Apply:** https://www.collegerecruiter.com/job/2840673187-senior-data-engineer ## About the Role * Production Experience: Extensive hands-on experience in Data Engineering, specifically a proven track record of successfully architecting, building, and managing production-grade real-time data pipelines across multiple large-scale initiatives. * Cloud Platform Specialization: Mandatory deep, practical experience leveraging Google Cloud Platform (GCP) and its associated tools for the deployment and management of real-time data ingestion and processing services. (While AWS knowledge is valuable, GCP expertise is primary.) * Distributed Streaming: Strong familiarity with distributed streaming technologies, such as Kafka or similar platforms. * Programming & Scripting: Expert proficiency in Python for development, coupled with the ability to optimize and refactor data pipelines for improved performance and scalability. * Database Expertise: Advanced knowledge of SQL is mandatory, alongside a strong understanding of various database technologies (NoSQL, time-series databases) and deep competence in data modeling principles. * Workflow Orchestration: Strong working knowledge of industry-standard data orchestration tools (e.g., Apache Airflow or Kubernetes)., We are seeking a highly collaborative professional with strong problem-solving skills, capable of thriving in a fast-paced environment. Exceptional written and verbal communication skills in English are required for effective technical leadership and documentation. ## Description * System Architecture & Development: Lead the design, development, and maintenance of scalable, high-performance, real-time data pipelines and core infrastructure components within the cloud environment. * Data Integration: Ensure seamless, low-latency data flow by integrating and unifying diverse data sources across the organization. * Data Modeling & Quality: Build and optimize robust data models specifically for complex querying and analytical use cases, and ensure data quality, integrity, and security across all systems. * Reliability & Monitoring: Construct highly available, fault-tolerant systems for high-volume data ingestion and processing, implementing effective monitoring, logging, and alerting mechanisms. * Performance Optimization: Continuously monitor, tune, and optimize pipeline scalability and performance to meet stringent throughput and latency standards. * Cross-Functional Alignment: Partner directly with technical and business teams to ensure the data infrastructure directly supports strategic objectives. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Leveraging Real time data in FSIs](https://www.wearedevelopers.com/videos/806-leveraging-real-time-data-in-fsis) - [Kubernetes and Microservices with Multi-Model Databases](https://www.wearedevelopers.com/videos/382-kubernetes-and-microservices-with-multi-model-databases) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [Tomorrow's cloud data platforms - fully managed database-as-a-service (DBaaS)](https://www.wearedevelopers.com/videos/254-tomorrow-s-cloud-data-platforms-fully-managed-database-as-a-service-dbaas) - [Maximising Cassandra's Potential: Tips on Schema, Queries, Parallel Access, and Reactive Programming](https://www.wearedevelopers.com/videos/1167-maximising-cassandra-s-potential-tips-on-schema-queries-parallel-access-and-reactive-programming) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Software Engineer Salary London](https://www.wearedevelopers.com/magazine/252-software-engineer-salary-london) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)