Remote Full Time Role_Google Cloud Platform Data Engineer (Platform, ML & Analytics)_Remote, USA
Amaze Systems Inc
United States
2 days ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
1 year minimum
Working hours
Regular working hours
Job source
Tech stack
Adobe Analytics
Adobe Experience Manager
Artificial Intelligence
Airflow
Data Analysis
Audit Trail
BigQuery
Cloud Storage
Information Engineering
Extract Transform Load (ETL)
Disaster Recovery
Failover
+19 more
Data Flow Control
Python (Programming Language)
Machine Learning
Standard Sql
SQL Databases
Management of Software Versions
Parquet
Feature Store
Google Cloud
Cloud Platform System
Feature Engineering
Large Language Models
Apache Spark
Pyspark
Core Data
Data Lineage
Apache Kafka
Machine Learning Operations
Looker Analytics
Job description
The Senior Data Engineer (Platform, ML & Analytics) at Client will be the only data engineering owner on the Al Insight & Next Best Action (NBA) engine team. They own the whole data lifecycle, from raw ingestion through curated marts to the Feature Store and ML training datasets, and theyβre also responsible for HIPAA compliance, disaster recovery (DR) and archival., * Kafka/Dataflow ingestion, Spark ETL/ELT, BigQuery marts, data quality and SLAs, partitioning and cost, Python/SQL, data modeling
- Data engineering (platform governance): 20% HIPAA/PHI controls, disaster recovery (DR) zones and failover, archival and retention, documentation
- ML data engineering: 25% Feature Store population, training dataset pipelines (train/eval splits, versioning), feature engineering on Spark, drift and freshness monitoring, Vertex Al Pipelines and MLOps
- AI: 5% Vertex Al appears only as a pipeline tool or downstream consumer. The JD doesnβt ask for model building, GenAl or LLM work
Key asks
- Ingestion: Stream health, benefits, news and click-through files (CV/Parquet) through Kafka and Dataflow into Cloud Storage and BigQuery-
- Transformation: Run ETL/ELT jobs on Dataflow and Datapros Spark to produce clean, normalized BigQuery datasets.
- Curated marts: Build 360Β° denormalized views that combine prescription (Rx), benefits, behavioral and clinical signals.
- Feature Store and ML datasets: Load the Google Cloud Platform Feature Store, build batch and online feature paths, create train/eval splits and version datasets.
- Monitor data drift and feature freshness with Vertex Al.
- Adobe Analytics: Bring in behavioral event data and line it up with clinical and benefits data.
- Data quality and BigQuery operations: Set up SLAs, alerting, schema validation and anomaly detection. Manage partitioning, schema changes and cost.
- Disaster recovery: Set recovery time and recovery point targets (RTO/RPO), replicate BigQuery across regions, maintain DR buckets and Feature Store snapshots, and test failover runbooks.
- Archival: Set BigQuery table expiry, move older data to cheaper Cloud Storage tiers (Nearline, Coldline, Archive) and meet HIPAA retention
- rules.
- HIPAA and patient health information (PHI): Apply field-level encryption and masking and de-identify data (Safe Harbor or Expert
- Determination) before training. Enforce access controls and audit logging.
- Documentation: Write up data contracts, data lineage, Feature Store definitions, DR runbooks and archival policies.
Requirements
- Core data skills: BigQuery (advanced), Kafka, Dataflow/Beam, Spark (PySpark or Scala), Pub/Sub, Composer/Airflow, plus strong SQL and data modeling (Lakehouse).
- ML data work: Feature Stores, Vertex Al Pipelines or similar MLOps tools, and the ML lifecycle.
- Proven experience with: DR design, archival and retention for regulated data, and hands-on HIPAA/PHI controls.
Nice to have
- Adobe Experience Platform, Looker, Google Cloud Platform customer-managed encryption keys (CMEK), HIPAA business associate agreement (BAA) awareness,
- NIST/HITRUST, and healthcare data experience (Rx, clinical or benefits).
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.dice.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦