Remote Full Time Role_Google Cloud Platform Data Engineer (Platform, ML & Analytics)_Remote, USA

Amaze Systems Inc
United States
2 days ago
Apply on www.dice.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
1 year minimum
Working hours
Regular working hours
Job source

Tech stack

Adobe Analytics Adobe Experience Manager Artificial Intelligence Airflow Data Analysis Audit Trail BigQuery Cloud Storage Information Engineering Extract Transform Load (ETL) Disaster Recovery Failover
+19 more
Data Flow Control Python (Programming Language) Machine Learning Standard Sql SQL Databases Management of Software Versions Parquet Feature Store Google Cloud Cloud Platform System Feature Engineering Large Language Models Apache Spark Pyspark Core Data Data Lineage Apache Kafka Machine Learning Operations Looker Analytics

Job description

The Senior Data Engineer (Platform, ML & Analytics) at Client will be the only data engineering owner on the Al Insight & Next Best Action (NBA) engine team. They own the whole data lifecycle, from raw ingestion through curated marts to the Feature Store and ML training datasets, and they’re also responsible for HIPAA compliance, disaster recovery (DR) and archival., * Kafka/Dataflow ingestion, Spark ETL/ELT, BigQuery marts, data quality and SLAs, partitioning and cost, Python/SQL, data modeling

  • Data engineering (platform governance): 20% HIPAA/PHI controls, disaster recovery (DR) zones and failover, archival and retention, documentation
  • ML data engineering: 25% Feature Store population, training dataset pipelines (train/eval splits, versioning), feature engineering on Spark, drift and freshness monitoring, Vertex Al Pipelines and MLOps
  • AI: 5% Vertex Al appears only as a pipeline tool or downstream consumer. The JD doesn’t ask for model building, GenAl or LLM work

Key asks

  • Ingestion: Stream health, benefits, news and click-through files (CV/Parquet) through Kafka and Dataflow into Cloud Storage and BigQuery-
  • Transformation: Run ETL/ELT jobs on Dataflow and Datapros Spark to produce clean, normalized BigQuery datasets.
  • Curated marts: Build 360Β° denormalized views that combine prescription (Rx), benefits, behavioral and clinical signals.
  • Feature Store and ML datasets: Load the Google Cloud Platform Feature Store, build batch and online feature paths, create train/eval splits and version datasets.
  • Monitor data drift and feature freshness with Vertex Al.
  • Adobe Analytics: Bring in behavioral event data and line it up with clinical and benefits data.
  • Data quality and BigQuery operations: Set up SLAs, alerting, schema validation and anomaly detection. Manage partitioning, schema changes and cost.
  • Disaster recovery: Set recovery time and recovery point targets (RTO/RPO), replicate BigQuery across regions, maintain DR buckets and Feature Store snapshots, and test failover runbooks.
  • Archival: Set BigQuery table expiry, move older data to cheaper Cloud Storage tiers (Nearline, Coldline, Archive) and meet HIPAA retention
  • rules.
  • HIPAA and patient health information (PHI): Apply field-level encryption and masking and de-identify data (Safe Harbor or Expert
  • Determination) before training. Enforce access controls and audit logging.
  • Documentation: Write up data contracts, data lineage, Feature Store definitions, DR runbooks and archival policies.

Requirements

  • Core data skills: BigQuery (advanced), Kafka, Dataflow/Beam, Spark (PySpark or Scala), Pub/Sub, Composer/Airflow, plus strong SQL and data modeling (Lakehouse).
  • ML data work: Feature Stores, Vertex Al Pipelines or similar MLOps tools, and the ML lifecycle.
  • Proven experience with: DR design, archival and retention for regulated data, and hands-on HIPAA/PHI controls.

Nice to have

  • Adobe Experience Platform, Looker, Google Cloud Platform customer-managed encryption keys (CMEK), HIPAA business associate agreement (BAA) awareness,
  • NIST/HITRUST, and healthcare data experience (Rx, clinical or benefits).

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com
Prepare application

Good distractions

Loading talks and stories from around this role…