GCP Data Engineer

Interon IT Solutions LLC
Chantilly, VA, United States
about 2 months ago
Apply on www.careerjet.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
4 years minimum
Working hours
Regular working hours

Tech stack

Airflow BigQuery Cloud Storage Continuous Integration Data Validation Information Engineering Extract Transform Load (ETL) Data Migration Data Warehousing Database Development Distributed Computing Environment Data Flow Control
+21 more
Python (Programming Language) Operational Data Store Cloudera Software Engineering SQL Stored Procedures SQL Databases Data Streaming Data Processing Google Cloud Apache Spark Software Troubleshooting Git Data Lakes Gitlab-ci Kubernetes Terraform Looker Analytics Data Pipelines Apache Beam Docker Jenkins

Job description

Our client is looking for an experienced GCP Data Engineer to build and support cloud-based data solutions for healthcare, pharmacy, claims, member, and operational data. This is a hands-on development role. The engineer will work with architects, analysts, product owners, and other engineering teams to build reliable batch and real-time data pipelines on Google Cloud Platform. Responsibilities Design, develop, and maintain scalable data pipelines using GCP services. Build batch and streaming solutions using Dataflow, Pub/Sub, BigQuery, Cloud Storage, Dataproc, and Cloud Composer. Develop data-processing applications using Python, SQL, Spark, and Apache Beam. Create BigQuery tables, views, stored procedures, and data models. Apply partitioning and clustering strategies to improve BigQuery performance and cost. Ingest and transform healthcare, pharmacy, claims, member, provider, and operational data. Build data validation, reconciliation, error-handling, and monitoring processes. Troubleshoot pipeline failures, data-quality issues, and performance problems. Support data migration from legacy and on-premises platforms to GCP. Protect PHI, PII, and other sensitive information using appropriate security and access controls. Develop automated build and deployment pipelines using Git, Jenkins or GitLab CI/CD, and Terraform.

Requirements

8+ years of data engineering or software development experience. 4+ years of hands-on experience with GCP data services. Strong experience with BigQuery, Dataflow, Pub/Sub, Cloud Storage, Dataproc, and Cloud Composer. Advanced Python and SQL development skills. Experience with Spark, Apache Beam, and Airflow. Strong understanding of ETL/ELT, data warehousing, data lakes, and distributed data processing. Experience building both batch and streaming data pipelines. Experience with Docker, Kubernetes, Terraform, Git, and CI/CD. Strong troubleshooting and communication skills. Preferred Qualifications Healthcare, pharmacy, PBM, claims, or health insurance experience. Understanding of HIPAA, PHI, and PII requirements. Experience with dbt, Dataplex, Data Catalog, Cloud Run, Cloud Functions, or Looker. Google Professional Data Engineer certification.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerjet.com
Prepare application

Good distractions

Loading talks and stories from around this role…