GCP Data Engineer

Tech Inc
Woodbridge Township, NJ, United States
25 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Compensation
$124,800.0
Working hours
Regular working hours
Job source

Tech stack

Airflow Apache HTTP Server BigQuery Continuous Integration Data Governance Data Transformation Data Flow Control Data Streaming Google Cloud Deployment Automation Data Delivery Data Pipelines
+1 more
Apache Beam

Job description

We are seeking a GCP-focused Data Engineer to support the data ecosystem behind our Collections Application, which is used by agents for inbound and outbound outreach to customers struggling with payments before they transition into full collections. This role centers on ingesting and processing upstream data from many different sources, typically delivered as files, and transforming these datasets into high-quality, reliable structures within BigQuery and Iceberg. You will build scalable pipelines using Airflow and Dataflow, ensuring timely, accurate data delivery for analytics, agent operations, and compliance functions.

Day-to-Day Responsibilities:

  • Ingest and process multiple upstream data sources, often delivered as recurring files from internal teams, partner systems, or operational applications.
  • Build, maintain, and optimize data pipelines using Apache Airflow for scheduling, orchestration, and monitoring.
  • Develop and run transformation jobs using Dataflow (Apache Beam) to clean, normalize, and prepare data for downstream use.
  • Create and manage lakehouse tables in Apache Iceberg, ensuring robustness, schema evolution support, and reliable metadata handling.
  • Load processed datasets into BigQuery for operational reporting, analytics, and collections-specific business use cases.
  • Collaborate with analytics, risk, operations, and compliance teams to ensure data accuracy, lineage clarity, and timely availability.
  • Troubleshoot ingestion issues, file anomalies, schema drift, and pipeline failures across a multi-source environment.
  • Document workflows, data flows, and ingestion patterns for internal consistency and audit/regulatory needs.

Requirements

  • Hands-on experience with Google Cloud Platform (GCP).
  • Strong experience with BigQuery for large-scale data storage and querying.
  • Proficiency with Dataflow (Apache Beam) for data transformations.
  • Experience designing and scheduling pipelines with Apache Airflow.
  • Practical experience working with Apache Iceberg for lakehouse table management.
  • Experience handling multiple upstream data sources, especially file-based ingestion into GCS.
  • Ability to work with complex, sensitive financial datasets in a regulated environment.
  • Strong problem-solving skills around ingestion failures, schema inconsistencies, and multi-source data environments.

Preferred / Plus Skills:

  • Experience supporting collections, delinquency, or financial servicing environments.
  • Familiarity with CI/CD for data pipelines and automated deployment frameworks.
  • Exposure to data governance, data quality frameworks, and lineage tools.
  • Experience handling sensitive customer financial data with appropriate security controls.

Benefits & conditions

Pay: From $60.00 per hour

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

1:09 min

Evaluating mature stream processing frameworks for production systems

Soroosh Khodami Soroosh Khodami · WWC 2024

2:15 min

Empowering domain teams with an open data platform

Sandhya Menon Sandhya Menon · WWC Europe 2026

2:57 min

Core technical practices for robust data engineering

Sandhya Menon Sandhya Menon · WWC Europe 2026

3:27 min

Explaining query execution overhead and caching limitations in BigQuery

Adnan Rahic · JS Congress

4:23 min

Comparing performance and complexity against standard microservice architectures

Soroosh Khodami Soroosh Khodami · WWC 2024

2:30 min

Leveraging BigQuery ML for scalable SQL-based segmentation experiments

Julian Joseph · LIVE

Videos

See all

Related articles

See all