GCP Data Engineer - Remote / Telecommute

CYNET SYSTEMS INC.

Austin, TX, United States

11 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Compensation

$104,000.0 - $114,400.0

Working hours

Regular working hours

Job source

www.careerjet.com

Tech stack

Airflow Big Data BigQuery Cloud Storage Databases Information Engineering Extract Transform Load (ETL) Data Security Data Warehousing Data Flow Control Identity and Access Management Python (Programming Language)

+14 more

Microsoft SQL Server Performance Tuning Scrum Methodology SQL Stored Procedures SQL Databases Data Processing Freeform SQL Google Cloud Data Ingestion Git Google Bigquery Software Version Control Data Pipelines Apache Beam

Job description

Design, develop, and maintain scalable ETL/ELT pipelines using GCP Dataflow and Python.
Develop and optimize BigQuery datasets, tables, views, partitioning, and clustering strategies.
Create and maintain complex SQL queries, stored procedures, functions, and database objects in MS SQL Server.
Migrate and integrate data from on-premises databases and various source systems into GCP.
Implement data quality, validation, reconciliation, and monitoring frameworks.
Work with Cloud Storage (GCS), BigQuery, Dataflow, and Composer/Airflow for end-to-end data processing.
Analyze performance bottlenecks and optimize large-scale data workloads.
Build reusable Python components for data ingestion, transformation, and automation.
Collaborate with business stakeholders, architects, and data analysts to understand requirements.
Ensure data security, governance, and compliance standards are followed., Job Category: Product and Research Job Description: As a Senior Software Engineer on the Identity and Access Management (IAM) team, you’ll design, build, and maintain secure, r…
2 days ago +

Requirements

Strong experience in MS SQL Server (SQL, Stored Procedures, Views, Functions, Performance Tuning).
Proficiency in Python for data engineering and automation.
Hands-on experience with Google BigQuery.
Experience developing data pipelines using Google Cloud Dataflow (Apache Beam).
Knowledge of GCP services such as Cloud Storage, Pub/Sub, Cloud Composer, IAM, and Monitoring.
Strong understanding of ETL/ELT, Data Warehousing, and Data Modeling concepts.
Experience with source control tools such as Git.
Knowledge of Agile/Scrum methodologies.