.gif)
BB
Benedikt Bischof
Making Data Warehouses Fast: A Developer’s Story
about 4 years ago
ADF Data Engineer
Diverse Lynx LLC
Houston, TX, United States
1 day ago
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Apply on www.careerjet.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
3 years minimum
Compensation
$165,000.0 - $194,000.0
Working hours
Regular working hours
Job source
Tech stack
Unit Testing
Microsoft Azure
Big Data
Computer Programming
Continuous Integration
Information Engineering
Data Transformation
Python (Programming Language)
Query Optimization
Azure Data Lake
SQL Databases
Data Streaming
+11 more
Parquet
Data Storage Technologies
Azure Data Factory
Git
Modularization
Pytest
Data Lakes
Pyspark
Git Flow
Software Coding
Databricks
Requirements
Experience: 5-8 years in data engineering with at least 3+ years on Azure. Azure Data Factory: Pipelines, data flows (mapping/wrangling), triggers, integration runtimes, parameterization, error handling & retries. Azure Databricks: Databricks Notebooks, Jobs/Workflows, PySpark/Scala, Delta Lake, cluster configuration and Eligible to workimization. Programming: Advanced Python or Scala (at least one), strong coding standards, modularization, and unit testing (e.g., pytest). SQL: Expert-level SQL (CTEs, window functions, query tuning), working with large datasets. Data Storage & Formats: ADLS Gen2, Delta Lake, Parquet, partitioning, Z-Ordering, Eligible to workIMIZE/VACUUM. Ops & CI/CD: Git, branching strategies, CI/CD for ADF/Databricks; environment promotion; Infrastructure-as-Code exposure is a plus. Minimum years of experience: 8-10 years
Benefits & conditions
bp
- Houston, TX
- $165,000-194,000 per year
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Apply on www.careerjet.com
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Talks and stories from around this role — technically off-topic, practically not.
Moments
Explore playlists6:08 min
Applying software engineering environments and testing to data pipelines
Matthias Niehoff · World Congress 2024
4:32 min
Harnessing Spark with Python using PySpark and Py4J
Ayon Roy · LIVE
3:05 min
Tagging and organizing execution scenarios with pytest markers
Florian Bruhin · World Congress 2021
6:21 min
Investigating push inefficiencies with upstream Git experts
Jonathan Creamer · Coffee With Developers
3:03 min
Career evolution in data engineering and AI platforms
Maria Apazoglou · Coffee With Developers
3:37 min
Scaling machine learning pipelines from prototypes to petabytes
Julian Joseph · LIVE
Videos
See allRelated articles
See all
AJ
Austin Joy
What Are The Top Skills Required For Azure Developers?
over 4 years ago
EM
Eli McGarvie
Highest Paying Tech Companies for Developers
over 3 years ago
EM
Eli McGarvie
Data Engineer Salary UK
over 3 years ago
DS
Dhannush Subramani
Top Big Data Technologies That You Need to Know
about 4 years ago
CH
Chris Heilmann
Dev Digest 132 - Binging WADFlix?
about 2 years ago