Google Cloud Platform Data Engineer

VST Consulting, Inc

San Francisco, CA, United States

1 day ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Experienced

Experience required

4 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Airflow Big Data BigQuery Cloud Storage Code Review Continuous Integration Data Governance Extract Transform Load (ETL) Data Transformation Data Systems Data Warehousing Distributed Computing Environment

+16 more

Python (Programming Language) Operational Databases Cloudera SQL Databases Data Logging Google Cloud Cloud Platform System Infrastructure as Code (IaC) Git Containerization Pyspark Information Technology Apache Kafka Terraform Data Pipelines Docker

Job description

We are looking for a skilled Google Cloud Platform Data Engineer with proven experience building and maintaining scalable data pipelines and cloud-based data platforms on Google Cloud Platform (Google Cloud Platform). The ideal candidate should have strong hands-on expertise in Python, SQL, PySpark, BigQuery, Apache Airflow, and dbt, along with experience working with modern ETL/ELT frameworks and cloud data solutions. The role involves collaborating closely with Data Analysts, Data Scientists, business stakeholders, and engineering teams to deliver reliable, scalable, and high-quality data solutions.

Responsibilities

* Design, develop, and maintain scalable data pipelines using Python and PySpark. * Build and optimize ETL/ELT workflows using Apache Airflow. * Develop data transformation models and workflows using dbt. * Work with BigQuery for data warehousing, querying, and optimization. * Write efficient and optimized SQL queries for large-scale data processing. * Ensure data quality, integrity, and reliability across data pipelines. * Collaborate with Data Analysts, Data Scientists, and business stakeholders to understand data requirements. * Monitor and troubleshoot production data workflows and proactively resolve issues. * Implement best practices for logging, monitoring, testing, and CI/CD. * Optimize cloud resource usage and cost efficiency on Google Cloud Platform. * Participate in code reviews and follow engineering best practices.

Requirements

* Strong hands-on experience with Python and SQL. * Experience with PySpark and distributed data processing. * Strong knowledge of BigQuery and Google Cloud Platform services. * Hands-on experience with Apache Airflow for workflow orchestration. * Hands-on experience with dbt for data transformation and modeling. * Understanding of ETL/ELT frameworks and data warehousing concepts. * Experience with Git and CI/CD practices. * Strong analytical and problem-solving skills.

Google Cloud Platform Services

Experience with or familiarity with: * BigQuery * Cloud Storage * Dataproc * Cloud Composer * Pub/Sub

Good to Have * Experience with streaming pipelines using Kafka or Pub/Sub. * Understanding of data governance and security concepts. * Experience with Terraform or Infrastructure as Code (IaC). * Knowledge of Docker and Kubernetes/containerization. * Google Cloud Platform certification is a plus.

Preferred Qualification

* Bachelor s degree in Computer Science, Engineering, or a related field. * Google Cloud Platform certification is preferred but not mandatory.