> Markdown version of [/jobs/ext/2256192-cloud-data-loading-architect-gcp-and-bigquery](https://www.wearedevelopers.com/jobs/ext/2256192-cloud-data-loading-architect-gcp-and-bigquery). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Cloud Data Loading Architect (GCP and BigQuery) - **Company:** Insight - **Location:** Leeds, UK - **Contract:** Permanent contract - **Skills:** Java (Programming Language), Application Programming Interfaces (APIs), Airflow, Automation of Tests, BigQuery, Cloud Computing, Cloud Computing Security, Cloud Database, Cloud Engineering, Cluster Analysis, Databases, Continuous Integration, Data as a Services, Data Integration, Extract Transform Load (ETL), Fault Tolerance, Data Flow Control, Github, Identity and Access Management, JSON, Python (Programming Language), Meta-Data Management, Performance Tuning, Standard Sql, Cloudera, Data Streaming, Unstructured Data, Workflow Management Systems, Parquet, Data Logging, Pulumi, Google Cloud, Data Ingestion, Cloud Monitoring, Delivery Pipeline, Build Server, Gitlab, Avro, Terraform, Apache Beam, Legacy Systems, Jenkins - **Published:** August 26, 2026 - **Apply:** https://www.collegerecruiter.com/job/2815180143-cloud-data-loading-architect-gcp-and-bigquery ## About the Role 1. Deep Expertise in Google Cloud Data Services: BigQuery, GCS, Dataflow (Apache Beam), Pub/Sub, Dataproc, Cloud Composer, Storage Write API. 2. Data Ingestion Engineering Mastery: Hands-on experience designing frameworks to load data from APIs, files, databases, event streams, and mainframe/legacy systems into cloud stores. 3. Strong SQL & BigQuery Optimisation skills: Partitioning, clustering, materialised views, cost-efficient query design, columnar processing. Experience building transformation pipelines using Airflow, Dataflow, dbt, or equivalent orchestration tools. Ability to work with Parquet, Avro, ORC, JSON, CSV, nested/repeated structures, and schema evolution. 4. Strong Python and/or Java skills used to build Dataflow pipelines, ingestion utilities, automation scripts. 5. Cloud Security & Governance awareness: IAM roles, least-privilege models, VPCSC, service accounts, artifact signing, audit. Cloud Build, GitHub Actions, Terraform, Cloud Deployment Manager or Pulumi. 6. Data Quality & Observability mindset: Experience implementing validation frameworks, anomaly detection, reconciliation rules, logging/monitoring (e.g., Cloud Logging, Cloud Monitoring). 7. Excellent Architectural Communication Skills: Ability to document, diagram, and communicate ingestion patterns to stakeholders at technical and non-technical levels. ## Description Role: Cloud Data Loading Architect (GCP and BigQuery) Location: Halifax or Leeds (Hybrid) Job Type: Contract Role Summary * We are seeking an experienced Cloud Data Loading Architect to design, build, and optimise automated pipelines that ingest structured, semi-structured, and unstructured datasets into Google Cloud Platform (GCP), specifically BigQuery. * This role will lead end-to-end data ingestion design-from source discovery and schema mapping, through transformation and data quality, to scalable, secure loads into cloud-native analytical warehouses. * The ideal candidate combines strong cloud engineering skills with hands-on data integration experience and a deep understanding of BigQuery performance optimisation. Key Responsibilities * Design and implement high-throughput, fault-tolerant ingestion pipelines for batch and streaming data landing in BigQuery, using Dataflow, Dataproc, Composer (Airflow), Pub/Sub, BigQuery Storage Write API, and related services. * Define data loading frameworks, mapping rules, schema evolution strategy, and metadata management. * Create reusable ingestion blueprints that ensure governance, lineage, and auditability. * Establish data quality checks, validation rules, reconciliation logic, and SLAs. * Optimize BigQuery cost, storage, partitioning, clustering, and access patterns. * Collaborate with security & platform teams to ensure IAM, service accounts, VPCSC, and encryption policies are fully applied. Use GitHub, GitLab or Jenkins for CI/CD. * Produce detailed technical documentation and coach engineering squads. * Troubleshoot ingestion failures, performance bottlenecks, and cross-platform data integration issues. Qualifications 1. Deep Expertise in Google Cloud Data Services: BigQuery, GCS, Dataflow (Apache Beam), Pub/Sub, Dataproc, Cloud Composer, Storage Write API. 2. Data Ingestion Engineering Mastery: Hands-on experience designing frameworks to load data from APIs, files, databases, event streams, and mainframe/legacy systems into cloud stores. 3. Strong SQL & BigQuery Optimisation skills: Partitioning, clustering, materialised views, cost-efficient query design, columnar processing. Experience building transformation pipelines using Airflow, Dataflow, dbt, or equivalent orchestration tools. Ability to work with Parquet, Avro, ORC, JSON, CSV, nested/repeated structures, and schema evolution. 4. Strong Python and/or Java skills used to build Dataflow pipelines, ingestion utilities, automation scripts. 5. Cloud Security & Governance awareness: IAM roles, least-privilege models, VPCSC, service accounts, artifact signing, audit. Cloud Build, GitHub Actions, Terraform, Cloud Deployment Manager or Pulumi. 6. Data Quality & Observability mindset: Experience implementing validation frameworks, anomaly detection, reconciliation rules, logging/monitoring (e.g., Cloud Logging, Cloud Monitoring). 7. Excellent Architectural Communication Skills: Ability to document, diagram, and communicate ingestion patterns to stakeholders at technical and non-technical levels. ## Related Videos - [Tips and Tricks for Working with JSON](https://www.wearedevelopers.com/videos/1229-tips-and-tricks-for-working-with-json) - [From event streaming to event sourcing 101](https://www.wearedevelopers.com/videos/91-from-event-streaming-to-event-sourcing-101) - [WeAreDevelopers LIVE - Modern DevOps for IoT Devices and More](https://www.wearedevelopers.com/videos/1805-wearedevelopers-live-modern-devops-for-iot-devices-and-more) - [Alibaba Big Data and Machine Learning Technology](https://www.wearedevelopers.com/videos/37-alibaba-big-data-and-machine-learning-technology) - [Introducing JSON Structure](https://www.wearedevelopers.com/videos/100219-introducing-json-structure) - [Green Cloud Computing](https://www.wearedevelopers.com/videos/592-green-cloud-computing) ## Related Articles - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [7 Cloud Computing Trends Coming in 2025 for Developers](https://www.wearedevelopers.com/magazine/412-7-cloud-computing-trends-coming-in-2025-for-developers) - [Got AI ideas but no money? Here are 10 free ways to level up your AI skills with Google Cloud](https://www.wearedevelopers.com/magazine/600-got-ai-ideas-but-no-money-here-are-10-free-ways-to-level-up-your-ai-skills-with-google-cloud) - [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know) - [What Are The Top Skills Required For Azure Developers?](https://www.wearedevelopers.com/magazine/77-what-are-the-top-skills-required-for-azure-developers)