Data Engineer with SQL and Databricks Exp.-In Person Interview

EApps Tech LLC
Chicago, United States of America
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Compensation
$ 146K

Job location

Chicago, United States of America

Tech stack

Java
.NET
API
Unit Testing
Azure
Code Coverage
Continuous Integration
Data Validation
ETL
Software Debugging
Python
SQL Azure
Performance Tuning
Azure
SQL Databases
Systems Integration
TypeScript
Data Logging
Pulumi
Azure
Spring-boot
Data Lake
Angular
PySpark
Integration Tests
Azure
Databricks

Requirements

  • ETL / ELT Concepts: Strong understanding of pipeline patterns, incremental loads, data validation, and troubleshooting.

  • SQL: Advanced querying (CTEs, views, joins, complex query logic) and performance tuning for transformations and validation. Python: Production-quality development (modular code, testing, logging, integration with APIs/files, CICD, Unit Test/Integration test automation, Code Coverage).

  • PySpark: Distributed transformations and performance optimization (joins, partitions, debugging), CICD, Unit Test/Integration test automation, Code Coverage.

  • Azure Data Factory (ADF): Build/operate ADF pipelines, parameterization, triggers, monitoring, retry/error handling; integrate with Databricks/ADLS.

  • Databricks: Develop and operationalize notebooks/jobs/workflows; Delta Lake patterns; basic cluster/job configuration.

  • Azure Fundamentals + Pulumi: Hands-on with ADLS Gen2, Azure Portal, Storage Explorer, Resource Groups, Azure SQL, and familiarity integrating with Azure OpenAI. Able to use/maintain Pulumi scripts for provisioning and managing Azure resources across environments

Nice to have skills: -

  • Ability to support/translate validation rules with SQL scripts and create data quality reports. TypeScript: Useful for pulumi pipeline to create Azure components.

  • Java: Useful for integration with existing services/components.

  • .NET: Useful for integration with existing services/components. Angular / Spring Boot: Minor troubleshooting or coordination with app teams.

Apply for this position