Data Engineer Google Cloud Platform

Projas Technologies, LLC

Oakland, United States

25 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

4 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Airflow Big Data BigQuery Continuous Integration Information Engineering Data Governance Data Integrity Extract Transform Load (ETL) Dataspaces Data Systems Data Flow Control Python (Programming Language)

+14 more

DataOps SQL Databases Data Streaming Google Cloud Apache Spark Git Pyspark Infrastructure Automation Frameworks Data Lineage Operational Systems Terraform Stream Processing Software Version Control Data Pipelines

Job description

You will be responsible for designing, building, and maintaining scalable data pipelines, ensuring high data quality and reliability across our analytics and operational systems., * Design and implement scalable ETL/ELT pipelines using Dataflow, Composer, and Spark/PySpark

Optimize and manage large-scale datasets in BigQuery
Monitor and improve data quality using Monte Carlo
Collaborate with data scientists, analysts, and business stakeholders to deliver reliable data solutions
Automate workflows and orchestrate pipelines using Apache Airflow (via Composer)
Ensure data governance, lineage, and observability across the data ecosystem
Troubleshoot and resolve data pipeline issues in production environments

Requirements

We are seeking a highly skilled and motivated Senior Data Engineer to join our data platform team. The ideal candidate will have hands-on experience with Google Cloud Platform (Google Cloud Platform) services, including BigQuery, Cloud Composer, Dataflow, and Apache Spark/PySpark, along with a strong understanding of data quality frameworks using Monte Carlo., * 4+ years of experience in data engineering or related field

Strong proficiency in Google Cloud Platform services: BigQuery, Dataflow, Cloud Composer
Expertise in Apache Spark and PySpark
Experience with Monte Carlo or similar data observability tools
Proficiency in SQL and Python
Familiarity with CI/CD pipelines and version control (e.g., Git)
Excellent problem-solving and communication skills, * Experience with real-time data processing and streaming
Knowledge of data modeling and warehousing best practices
Familiarity with Terraform or Infrastructure as Code (IaC) tools
Certification in Google Cloud Platform Data Engineering is a plus

data engineer, Google Cloud Platform, Google Cloud Platform, BigQuery, Cloud Composer, Dataflow, Apache Spark, PySpark, Monte Carlo, data quality, data observability, Airflow, ETL pipelines, ELT pipelines, data pipeline, data engineering jobs, cloud data engineer, data reliability, data governance, data lineage, Python, SQL, CI/CD, Terraform, remote data engineer job