> Markdown version of [/jobs/ext/2304936-consultant-data-engineer-irc302341](https://www.wearedevelopers.com/jobs/ext/2304936-consultant-data-engineer-irc302341). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Consultant Data Engineer IRC302341 - **Company:** GlobalLogic Inc. - **Location:** Greater London, UK - **Experience:** Expert - **Contract:** Permanent contract - **Skills:** Airflow, Amazon Web Services, Amazon S3, Apache HTTP Server, Cloud Computing, Continuous Integration, Directed Acyclic Graph (Directed Graphs), Information Engineering, Data Transformation, Data Migration, Data Warehousing, Software Debugging, Identity and Access Management, Python (Programming Language), SQL Databases, Teradata SQL, Data Build Tool (dbt), Apache Spark, Git, Pyspark, Data Lineage, Data Delivery, Terraform, Data Pipelines, Legacy Systems, Databricks - **Published:** August 30, 2026 - **Apply:** https://www.collegerecruiter.com/job/2830497174-consultant-data-engineer-irc302341 ## About the Role * Cloud Platforms: Hands-on engineering experience within the AWS ecosystem (S3, IAM, Glue) and deep expertise in Databricks (SparkSQL, PySpark, governed cataloging). * Modern Data Stack: Strong proficiency in dbt (Core or Cloud) for data transformation and understanding of modern table formats, specifically Apache Iceberg. * Orchestration: Experience building and maintaining data pipelines using Apache Airflow (Astronomer experience is a strong plus). * Languages: Expert-level SQL (ability to read Teradata SQL/BTEQ and write highly optimised Spark/Databricks SQL) and strong Python skills for pipeline development and Airflow DAGs. * Legacy Systems: Familiarity with Teradata architecture and utilities (FastExport, BTEQ) to effectively understand the source systems being decommissioned., * Experience: 7+ years of dedicated Data Engineering experience, with a proven track record in complex data migration projects (On-Premise to Cloud). * Financial Services Background: Experience working in a highly regulated banking or financial services environment is strongly preferred, particularly within Risk or Finance domains. * Engineering Best Practices: Strong grasp of CI/CD for data pipelines, Git version control, data modelling (Kimball/Inmon), and infrastructure as code (Terraform). * Communication: Ability to work collaboratively in a fast-paced, cross-functional agile team, effectively communicating technical blockers to technical leads and project managers. ## Description We are seeking a highly skilled Сonsultant Data Engineer to be the driving force behind this massive architectural shift. You will be hands-on in moving a highly complex, highly regulated data ecosystem from an on-premise Teradata environment to a cutting-edge AWS "Lakehouse" architecture., We are seeking a highly skilled Сonsultant Data Engineer to be the driving force behind this massive architectural shift. You will be hands-on in moving a highly complex, highly regulated data ecosystem from an on-premise Teradata environment to a cutting-edge AWS "Lakehouse" architecture. In this role, you will focus on building robust data pipelines, modernizing legacy code, and ensuring flawless data delivery. You will work closely with System Analysts, Automation Engineers, and Data Architects to convert hundreds of legacy scripts into modern dbt models, orchestrate them via Astronomer (Airflow), and establish governed warehousing in Databricks., * Lakehouse Engineering: Build and maintain scalable data pipelines to migrate Consumer Risk datasets from Teradata to Amazon S3, utilizing Apache Iceberg for the lakehouse layer and Databricks for governed warehousing. * Code Modernisation (BTEQ to dbt): Perform complex code conversions, transforming up to 500 legacy Teradata BTEQ scripts into modular, version-controlled, and highly performant dbt (data build tool) models. * Pipeline Orchestration: Design, develop, and monitor complex Directed Acyclic Graphs (DAGs) using Astronomer (Apache Airflow) to orchestrate end-to-end data workflows. * Data Lineage & Transparency: Integrate pipelines with OpenLineage to ensure complete, end-to-end data traceability, providing the transparency required for regulatory compliance. * Reconciliation & Parity: Work alongside the QA/Automation team to execute automated reconciliation frameworks, debugging and resolving data discrepancies to ensure strict attribute-level parity and timing equivalence during governed parallel runs. ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Modern Data Architectures need Software Engineering](https://www.wearedevelopers.com/videos/1030-modern-data-architectures-need-software-engineering) - [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data) - [How a Small Team Shrank a Microsoft Monorepo by 94%](https://www.wearedevelopers.com/videos/1236-how-a-small-team-shrank-a-microsoft-monorepo-by-94) - [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [A Guide to Green Tech and Green IT Careers](https://www.wearedevelopers.com/magazine/374-a-guide-to-green-tech-and-green-it-careers) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)