GCP BigQuery+Nifi+Kafa Data Engineer

We are seeking a highly skilled GCP BigQuery Data Engineer with hands-on experience building cloud-native data platforms and scalable data pipelines. The ideal candidate will have strong expertise in Google BigQuery, Apache NiFi, Apache Kafka, Cloud Dataflow, Pub/Sub, SQL, and ETL/ELT development, with experience designing enterprise data warehouses and Data Lake solutions., · Design, develop, and optimize enterprise data warehouse solutions using Google BigQuery.

· Design, develop, and maintain real-time data ingestion pipelines using Apache NiFi to capture, transform, and route streaming data into Apache Kafka topics.

· Build scalable event-driven streaming architectures using Apache NiFi, Apache Kafka, and Google BigQueryfor high-throughput, fault-tolerant, and low-latency data processing and analytics.

· Configure and optimize Apache NiFi processors for data extraction, transformation, routing, filtering, schema validation, error handling, retry mechanisms, and reliable delivery to Kafka topics.

· Develop streaming ingestion solutions enabling Google BigQuery to consume Kafka event streams and transform near real-time data into analytical tables and enterprise reporting datasets.

· Design data models and implement ETL/ELT processes to move data from raw to curated and published data layers.

· Design, create, and manage large-scale BigQuery datasets, including temporary/permanent and internal/external tables.

· Optimize BigQuery workloads through query tuning, partitioning, clustering, and cost optimization techniques to improve performance and reduce cloud costs.

· Monitor, troubleshoot, and optimize streaming workloads by tuning NiFi flows, Kafka topics/partitions, and BigQuery streaming ingestion to ensure high availability, data integrity, minimal latency, and cost-efficient processing.

· Build scalable Data Lake frameworks and ingestion pipelines using cloud-native GCP technologies.

· Develop reporting and visualization solutions using Looker, Looker Studio (Data Studio), Connected Sheets, and other BigQuery reporting tools.

· Collaborate with data architects, modelers, developers, DevOps engineers, project managers, and business stakeholders to deliver scalable enterprise analytics solutions and continuous data platform improvements.

· 7+ years of experience in Data Engineering, Data Warehousing, or Big Data platforms.

· Mush-Have Skills: Apache NiFi, Apache Kafka.

· Strong hands-on experience with Google Cloud Platform (GCP), including BigQuery, Cloud Dataflow, Pub/Sub, and Google Cloud Storage (GCS).

· Hands-on experience designing and supporting real-time streaming pipelines using Apache NiFi, Apache Kafka, and Google BigQuery.

· Experience using BigQuery Console/Query Editor for data management, performance tuning, and SQL development.

· Strong experience designing scalable data models, ETL/ELT pipelines, Data Lake architectures, and enterprise data warehouse solutions.

· Experience with batch and streaming data ingestion using GCP services.

· Thorough understanding of BigQuery cost structure, including storage, ingestion, and query costs, with experience implementing query optimization, partitioning, clustering, and other cost optimization techniques.

· Experience managing large-scale datasets, including temporary/permanent and internal/external BigQuery tables.

· Experience developing reporting and visualization solutions using Looker, Looker Studio (Data Studio), and Connected Sheets.

· Excellent verbal and written communication skills with the ability to collaborate effectively with technical teams, business stakeholders, senior management, and executive leadership.

· Experience working in Agile environments and collaborating with cross-functional teams to deliver enterprise data engineering solutions.

Required Technical Skills

GCP: BigQuery, Cloud Storage (GCS), Cloud Dataflow, Pub/SubData Engineering: Apache NiFi, Apache Kafka, Python, SQL, Spark, ETL/ELT, Data Modeling, Data Warehousing, Data Lakes, Batch & Streaming Pipelines.Visualization: Looker, Looker Studio, Connected SheetsTools & Practices: CI/CD, DevOps, Agile/Scrum

Preferred Qualifications

· Google Cloud certifications.

· Experience delivering enterprise-scale analytics and cloud data solutions.

$100,000 - $110,000 a year - Full-time, Contract