GCP ETL Engineer

OpenKyber LLC

United States

3 months ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

indeed.com

Tech stack

Data Analysis Big Data BigQuery Data Integration Extract Transform Load (ETL) Data Transformation DevOps Apache Hive Performance Tuning Query Optimization SQL Stored Procedures SQL Databases

+6 more

Enterprise Data Management Data Ingestion Apache Spark Caching Data Lakes Real Time Data

Job description

Design, develop, and optimize large-scale data processing pipelines using Apache Spark (Spark SQL, Spark Core, and DataFrames) for high-volume batch and real-time data workloads.

Write complex and high-performance SQL queries, stored procedures, and data transformations to support data ingestion, cleansing, aggregation, and reporting requirements.

Perform advanced performance tuning and troubleshooting of Spark jobs and SQL workloads, including partitioning, joins, caching, and query optimization for improved efficiency.

Build and maintain ETL/ELT workflows by integrating data from multiple structured and unstructured sources into enterprise data platforms and data lakes.

Work with BigQuery for data analysis, query optimization, and large-scale analytical reporting; knowledge of schema design and cost-efficient query execution is an added advantage.

Collaborate with cross-functional teams including Data Architects, Analysts, and DevOps teams to ensure scalable, secure, and reliable big data solutions aligned with business requirements.

For applications and inquiries, contact: hirings@openkyber.com