GCP Data Engineer

Interon IT Solutions LLC

Chantilly, VA, United States

13 days ago

Apply on www.careerjet.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

4 years minimum

Working hours

Regular working hours

Job source

www.careerjet.com

Tech stack

Airflow BigQuery Cloud Storage Continuous Integration Data Validation Information Engineering Extract Transform Load (ETL) Data Migration Data Warehousing Database Development Distributed Computing Environment Data Flow Control

+21 more

Python (Programming Language) Operational Data Store Cloudera Software Engineering SQL Stored Procedures SQL Databases Data Streaming Data Processing Google Cloud Apache Spark Software Troubleshooting Git Data Lakes Gitlab-ci Kubernetes Terraform Looker Analytics Data Pipelines Apache Beam Docker Jenkins

Job description

Our client is looking for an experienced GCP Data Engineer to build and support cloud-based data solutions for healthcare, pharmacy, claims, member, and operational data. This is a hands-on development role. The engineer will work with architects, analysts, product owners, and other engineering teams to build reliable batch and real-time data pipelines on Google Cloud Platform. Responsibilities Design, develop, and maintain scalable data pipelines using GCP services. Build batch and streaming solutions using Dataflow, Pub/Sub, BigQuery, Cloud Storage, Dataproc, and Cloud Composer. Develop data-processing applications using Python, SQL, Spark, and Apache Beam. Create BigQuery tables, views, stored procedures, and data models. Apply partitioning and clustering strategies to improve BigQuery performance and cost. Ingest and transform healthcare, pharmacy, claims, member, provider, and operational data. Build data validation, reconciliation, error-handling, and monitoring processes. Troubleshoot pipeline failures, data-quality issues, and performance problems. Support data migration from legacy and on-premises platforms to GCP. Protect PHI, PII, and other sensitive information using appropriate security and access controls. Develop automated build and deployment pipelines using Git, Jenkins or GitLab CI/CD, and Terraform.

Requirements

8+ years of data engineering or software development experience. 4+ years of hands-on experience with GCP data services. Strong experience with BigQuery, Dataflow, Pub/Sub, Cloud Storage, Dataproc, and Cloud Composer. Advanced Python and SQL development skills. Experience with Spark, Apache Beam, and Airflow. Strong understanding of ETL/ELT, data warehousing, data lakes, and distributed data processing. Experience building both batch and streaming data pipelines. Experience with Docker, Kubernetes, Terraform, Git, and CI/CD. Strong troubleshooting and communication skills. Preferred Qualifications Healthcare, pharmacy, PBM, claims, or health insurance experience. Understanding of HIPAA, PHI, and PII requirements. Experience with dbt, Dataplex, Data Catalog, Cloud Run, Cloud Functions, or Looker. Google Professional Data Engineer certification.