GCP Data Engineer/Architect

McClure Engineering Co.

United States

about 1 month ago

Apply on www.indeed.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Working hours

Regular working hours

Job source

www.indeed.com

Tech stack

Java (Programming Language) Airflow Big Data BigQuery Cloud Computing Cloud Computing Security Cloud Database Cluster Analysis Computer Programming Continuous Integration Data Architecture Data Security

+16 more

Data Systems Data Flow Control Identity and Access Management Python (Programming Language) Performance Tuning Query Optimization SQL Databases Data Processing Freeform SQL Google Cloud Data Storage Technologies Git Pyspark Software Version Control Data Pipelines Apache Beam

Job description

We are seeking a highly skilled Data Engineer with deep expertise in Google Cloud Platform (GCP) and modern data architecture. The ideal candidate will have hands-on experience designing scalable data pipelines, implementing Medallion Architecture, and building robust enterprise-grade data solutions.
This role requires strong technical proficiency in BigQuery, PySpark, Dataflow, and Airflow, along with a solid understanding of cloud data governance, performance optimization, and CI/CD practices., * Design, develop, and maintain scalable batch and real-time data pipelines on GCP
Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data processing
Build high-performance data transformations using Python and PySpark
Develop and optimize complex SQL queries for analytical workloads
Work extensively with BigQuery for large-scale data processing and performance tuning
Develop and deploy pipelines using Cloud Dataflow
Orchestrate workflows using Cloud Composer (Apache Airflow)
Manage data storage and lifecycle using Google Cloud Storage (GCS)
Implement version control and CI/CD pipelines using Git-based tools
Ensure data security, governance, and access control using GCP IAM
Optimize data solutions for performance, scalability, reliability, and cost-efficiency

Requirements

Strong hands-on experience with Google Cloud Platform (GCP)
Expertise in BigQuery (partitioning, clustering, query optimization)
Proven experience implementing Medallion Data Architecture
Hands- on Experience with Java Development
Strong programming skills in Python and PySpark
Advanced proficiency in SQL (complex joins, window functions, performance tuning)
Hands-on experience with Cloud Dataflow
Experience with Cloud Composer (Airflow) for orchestration
Experience working with Google Cloud Storage (GCS)
Knowledge of version control systems (Git) and CI/CD practices
Strong understanding of GCP IAM and cloud security best practices