Google Cloud Platform Data and platform Engineer

Amaze Systems Inc
United States
1 day ago
Apply on www.dice.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
7 years minimum
Working hours
Regular working hours
Job source

Tech stack

Adobe Analytics Adobe Experience Manager Airflow Audit Trail BigQuery Cloud Storage Extract Transform Load (ETL) Disaster Recovery Data Flow Control Standard Sql Parquet Feature Store
+8 more
Google Cloud Apache Spark Pyspark Core Data Data Lineage Apache Kafka Machine Learning Operations Looker Analytics

Job description

  1. Ingestion: Stream health, benefits, news and click-through files (CV/Parquet) through Kafka and Dataflow into Cloud Storage and BigQuery-
  2. Transformation: Run ETL/ELT jobs on Dataflow and Datapros Spark to produce clean, normalized BigQuery datasets.
  3. Curated marts: Build 360Β° denormalized views that combine prescription (Rx), benefits, behavioral and clinical signals.
  4. Feature Store and ML datasets: Load the Google Cloud Platform Feature Store, build batch and online feature paths, create train/eval splits and version datasets. Monitor data drift and feature freshness with Vertex Al.

  5. Adobe Analytics: Bring in behavioral event data and line it up with clinical and benefits data.
  6. Data quality and BigQuery operations: Set up SLAs, alerting, schema validation and anomaly detection. Manage partitioning, schema changes and cost.
  7. Disaster recovery: Set recovery time and recovery point targets (RTO/RPO), replicate BigQuery across regions, maintain DR buckets and Feature Store snapshots, and test failover runbooks.
  8. Archival: Set BigQuery table expiry, move older data to cheaper Cloud Storage tiers (Nearline, Coldline, Archive) and meet HIPAA retention rules.

  9. HIPAA and patient health information (PHI): Apply field-level encryption and masking and de-identify data (Safe Harbor or Expert Determination) before training. Enforce access controls and audit logging.

  10. Documentation: Write up data contracts, data lineage, Feature Store definitions, DR runbooks and archival policies.

Requirements

  • Core data skills: BigQuery (advanced), Kafka, Dataflow/Beam, Spark (PySpark or Scala), Pub/Sub, Composer/Airflow, plus strong SQL and data modeling (Lakehouse).
  • ML data work: Feature Stores, Vertex Al Pipelines or similar MLOps tools, and the ML lifecycle.
  • Proven experience with: DR design, archival and retention for regulated data, and hands-on HIPAA/PHI controls.

Nice to have

Adobe Experience Platform, Looker, Google Cloud Platform customer-managed encryption keys (CMEK), HIPAA business associate agreement (BAA) awareness,

NIST/HITRUST, and healthcare data experience (Rx, clinical or benefits).

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com
Prepare application

Good distractions

Loading talks and stories from around this role…