ADF Data Engineer

Siri InfoSolutions Inc

Houston, TX, United States

5 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

3 years minimum

Working hours

Regular working hours

Job source

www.careerjet.com

Tech stack

Unit Testing Microsoft Azure Big Data Computer Programming Continuous Integration Information Engineering Data Transformation Python (Programming Language) Query Optimization Azure Data Lake SQL Databases Data Streaming

+11 more

Parquet Data Storage Technologies Azure Data Factory Git Modularization Pytest Data Lakes Pyspark Git Flow Software Coding Databricks

Job description

Reporting directly to the Global Manufacturing Manager, the Advanced Manufacturing Machining Engineer is responsible for executing the manufacturing roadmap. This role integrates…

25 days ago, Powell Industries, Inc. is currently seeking qualified candidates for a Senior R&D Engineer to assist in developing and validating new product designs for power distribution equipm…
8 days ago

Requirements

Experience: 5 8 years in data engineering with at least 3+ years on Azure. Azure Data Factory: Pipelines, data flows (mapping/wrangling), triggers, integration runtimes, parameterization, error handling & retries. Azure Databricks: Databricks Notebooks, Jobs/Workflows, PySpark/Scala, Delta Lake, cluster configuration and optimization. Programming: Advanced Python or Scala (at least one), strong coding standards, modularization, and unit testing (e.g., pytest). SQL: Expert-level SQL (CTEs, window functions, query tuning), working with large datasets. Data Storage & Formats: ADLS Gen2, Delta Lake, Parquet, partitioning, Z-Ordering, OPTIMIZE/VACUUM. Ops & CI/CD: Git, branching strategies, CI/CD for ADF/Databricks; environment promotion; Infrastructure-as-Code exposure is a plus.