GCP Data Engineer

Sumeru Solutions
Philadelphia, PA, United States
4 days ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours

Tech stack

Query Performance Java (Programming Language) Airflow Apache HTTP Server BigQuery Cloud Computing Cloud Storage Directed Acyclic Graph (Directed Graphs) Data Validation Software Debugging Data Flow Control Data Streaming
+7 more
Google Cloud Sql Optimization Software Troubleshooting Core Data Data Management Api Management Apache Beam

Job description

  • GCP Core Data Stack Handson production experience building maintaining and supporting scalable batch and streaming data pipelines on Google Cloud Platform specifically BigQuery and Google Cloud Storage GCS

Requirements

GCP (Good Clinical Practices), GCP (Good Clinical Practices), Data Management, Cloud Computing, Cloud Storage, Identify Issues, Schedule Development, Apache, Writing Skills, Java, * Cloud Composer Apache Airflow Proven ability to author schedule troubleshoot and optimize complex Airflow DAGs in a managed Cloud Composer environment custom operators task dependencies retries backfills log diagnostics and runtimeSLA tuning

  • Google Cloud Dataflow Java Handson experience building maintaining and debugging Apache Beam pipelines written in Java on Google Cloud Dataflow including job monitoring and performance troubleshooting
  • Dataform SQLX Practical experience developing and debugging SQLX ELT transformations data quality assertions and dependency graphs in Dataform
  • Advanced BigQuery SQL Optimization Deep SQL proficiency in BigQuery including partitioned PARTITIONTIME and clustered tables query performance tuning slotcost optimization and querying operational metadatatelemetry tables
  • Python for Data Engineering Strong python3 skills for writing and debugging productiongrade Airflow tasks API integrations and automation scripts
  • Production Support RCA Data Validation Demonstrated ability to own production reliabilitymonitoring daily DAGtask runs performing deep rootcause analysis RCA on failures or data discrepancies building automated data validationreconciliation checks and deploying permanent fixes within SLA
  • Data Modelling OnshoreOffshore Collaboration Solid grasp of dimensional modelling and modern ELT patterns for highvolume datasets eg ecommerce or consumeraudience data paired with clear communication for daily onshoreoffshore handoffs and stakeholder alignment

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…