> Markdown version of [/jobs/ext/2796018-advanced-data-engineer](https://www.wearedevelopers.com/jobs/ext/2796018-advanced-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Advanced Data Engineer - **Company:** Your World Recruitment Group - **Location:** Cardiff, UK - **Salary:** £54,080.0 - £58,240.0 - **Contract:** Temporary to permanent - **Skills:** Airflow, Automation of Tests, Microsoft Azure, Big Data, BigQuery, Cloud Computing, Cloud Engineering, Code Review, Continuous Integration, Information Engineering, Data Governance, Data Integration, Extract Transform Load (ETL), Data Profiling, Data Warehousing, Relational Databases, DevOps, Github, Python (Programming Language), Meta-Data Management, Cloud Services, DataOps, Cloudera, Software Engineering, SQL Databases, Google Cloud, Apache Spark, Gitlab, Git, Containerization, Information Technology, Data Lineage, Production Code, Data Management, Physical Data Models, Hardware Infrastructure, Api Design, Software Version Control, Data Pipelines, Docker - **Published:** September 8, 2026 - **Apply:** https://www.yourworld.com/uk/professions/non-medical-non-clinical/it/advanced-data-engineer-m6md ## About the Role * Technical Core: Strong proficiency in SQL and Python; working knowledge or experience with R is highly desirable. * Cloud & Infrastructure: Hands-on experience designing cloud-native solutions (preferably GCP), migrating on-premise systems to the cloud, and managing relational databases/data warehouses. * Data Engineering Stack: Demonstrated experience with big data processing (Spark, Dataproc, BigQuery), orchestration (Airflow/Cloud Composer/Prefect), and API-driven data integration. * DevOps & Quality: Proven background with Git (GitHub/GitLab/Azure DevOps), Docker, CI/CD automation, and modern DataOps principles. * Education & Certifications: + Undergraduate degree in Computer Science, a related technical field, or equivalent practical work experience. + GCP Data Engineer or GCP Cloud Architect certification is highly desirable. ## Description We are seeking an Advanced Data Engineer to design, build, and optimize our cloud-native data architecture and scalable processing pipelines. In this role, you will lead complex data integration projects, drive on-premise to cloud migrations-with a strong focus on Google Cloud Platform (GCP)-and implement modern DataOps practices. You will collaborate closely with multidisciplinary teams to translate business requirements into secure, high-performing data products while upholding software engineering and data governance standards., * Data Engineering & Cloud Architecture: Design, build, test, and maintain scalable ETL/ELT pipelines and production-ready data products. Lead the migration of legacy data platforms, pipelines, and services from on-premises infrastructure to GCP. * Pipeline Orchestration & Optimization: Monitor, troubleshoot, and optimize pipeline performance for large-scale datasets using distributed processing tools (Spark, BigQuery) and workflow orchestrators (Airflow, Cloud Composer, Dagster). * Software & DataOps Best Practices: Apply CI/CD, containerization (Docker), Git-based version control, and automated testing to build production-quality code. Conduct peer code reviews, refactor existing code, and maintain robust technical documentation. * Data Modeling & Governance: Develop conceptual, logical, and physical data models. Implement metadata management, data lineage, data profiling, and validation frameworks to maintain high data quality. Ensure full compliance with UK GDPR, the Data Protection Act 2018, and ethical data guidelines. * Collaboration & Delivery: Work in an Agile environment to translate non-technical business requirements into robust technical solutions. Communicate architectural and data concepts clearly to both technical peers and key business stakeholders. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [WeAreDevelopers LIVE - Modern DevOps for IoT Devices and More](https://www.wearedevelopers.com/videos/1805-wearedevelopers-live-modern-devops-for-iot-devices-and-more) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [Git for Code Reviews](https://www.wearedevelopers.com/videos/429-git-for-code-reviews) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [The Most Popular IT Jobs on the Market](https://www.wearedevelopers.com/magazine/376-the-most-popular-it-jobs-on-the-market) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)