GCP Data Engineer

Signitives Technologies LLC

United States

2 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

indeed.com

Tech stack

Airflow Big Data BigQuery Cloud Computing Cloud Computing Security Cluster Analysis Computer Programming Continuous Integration Data Architecture Data Governance Data Security Data Systems

+19 more

Data Warehousing Data Flow Control Identity and Access Management Python (Programming Language) Performance Tuning Scrum Methodology Query Optimization SQL Databases Data Streaming Enterprise Data Management Data Processing Freeform SQL Google Cloud Data Storage Technologies Git Data Lakes Pyspark Software Version Control Data Pipelines

Job description

We are seeking a highly skilled Data Engineer with deep expertise in Google Cloud Platform (GCP) and modern data architecture. The ideal candidate will have hands-on experience designing scalable data pipelines, implementing Medallion Architecture, and building robust enterprise-grade data solutions., * Design, develop, and maintain scalable batch and real-time data pipelines on GCP

Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data processing
Build high-performance data transformations using Python and PySpark
Develop and optimize complex SQL queries for analytical workloads
Work extensively with BigQuery for large-scale data processing and performance tuning
Develop and deploy pipelines using Cloud Dataflow
Orchestrate workflows using Cloud Composer (Apache Airflow)
Manage data storage and lifecycle using Google Cloud Storage (GCS)
Implement version control and CI/CD pipelines using Git-based tools
Ensure data security, governance, and access control using GCP IAM
Optimize data solutions for performance, scalability, reliability, and cost-efficiency

Requirements

Do you have experience in Version control systems?, * Strong hands-on experience with Google Cloud Platform (GCP)

Expertise in BigQuery (partitioning, clustering, query optimization)
Proven experience implementing Medallion Data Architecture
Strong programming skills in Python and PySpark
Advanced proficiency in SQL (complex joins, window functions, performance tuning)
Hands-on experience with Cloud Dataflow
Experience with Cloud Composer (Airflow) for orchestration
Experience working with Google Cloud Storage (GCS)
Knowledge of version control systems (Git) and CI/CD practices
Strong understanding of GCP IAM and cloud security best practices

Preferred Qualifications