IT - Technology Architect | ETL & Data Quality | ETL & Data Quality - ALL

Cloudspace LLC
Houston, United States of America
15 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior
Compensation
$ 125K

Job location

Houston, United States of America

Tech stack

Unit Testing
Azure
Big Data
Computer Programming
Continuous Integration
Information Engineering
ETL
Data Transformation
Python
Query Optimization
Azure
SQL Databases
Data Streaming
Parquet
Data Storage Technologies
Azure
GIT
Modularization
Pytest
Data Lake
PySpark
Git Flow
Software Coding
Databricks

Requirements

Experience: 5 8 years in data engineering with at least 3+ years on Azure. Azure Data Factory: Pipelines, data flows (mapping/wrangling), triggers, integration runtimes, parameterization, error handling & retries. Azure Databricks: Databricks Notebooks, Jobs/Workflows, PySpark/Scala, Delta Lake, cluster configuration and optimization. Programming: Advanced Python or Scala (at least one), strong coding standards, modularization, and unit testing (e.g., pytest). SQL: Expert-level SQL (CTEs, window functions, query tuning), working with large datasets. Data Storage & Formats: ADLS Gen2, Delta Lake, Parquet, partitioning, Z-Ordering, OPTIMIZE/VACUUM. Ops & CI/CD: Git, branching strategies, CI/CD for ADF/Databricks; environment promotion; Infrastructure-as-Code exposure is a plus.

Benefits & conditions

Minimum years of experience: >10 years Certifications Needed: No Top 3 responsibilities you would expect the Subcon to shoulder and execute: Coding AND Development Migration activities Unit testing Interview Process (Is face to face required?) No, Project Code :GPTI core India

Apply for this position