Databricks / Azure Engineer

IT Associates
Harwood Heights, United States of America
3 days ago

Role details

Contract type
Temporary contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior
Compensation
$ 131K

Job location

Remote
Harwood Heights, United States of America

Tech stack

Agile Methodologies
Azure
Big Data
Data Architecture
Data Validation
Information Engineering
Relational Databases
Database Queries
Distributed Computing Environment
Python
Operational Databases
Performance Tuning
Scrum
Data Processing
Freeform SQL
Cloud Platform System
Data Ingestion
Spark
Data Lake
PySpark
Data Management
Azure
Data Pipelines
Databricks

Job description

  • Design, develop, and maintain scalable cloud-based data pipelines using Azure Databricks and/or Azure Synapse
  • Implement data processing solutions using Spark (PySpark and/or Scala) and Python
  • Build and optimize data ingestion, transformation, and aggregation workflows in Azure environments
  • Work with data lakes, Delta Lake, and distributed processing frameworks to support analytics and downstream use cases
  • Write and optimize complex SQL queries for data validation, transformation, and reporting
  • Participate in solution design discussions, technical reviews, and Agile ceremonies (stand-ups, sprint planning, retrospectives)
  • Take ownership of assigned data engineering solutions and deliver end-to-end implementations with minimal oversight
  • Troubleshoot, optimize, and support production workloads, ensuring performance, reliability, and scalability

Requirements

We are looking for a skilled and self-driven Databricks/Azure Engineer to design, build, and support scalable data solutions in the Azure cloud. This role focuses on developing high-performance data pipelines using Azure Databricks and/or Azure Synapse, leveraging Spark and Python to deliver reliable, production-ready data platforms., * 5+ years of professional data engineering experience

  • Hands-on experience with Azure Databricks and/or Azure Synapse Analytics
  • Strong expertise in Apache Spark (PySpark and/or Scala) and Python for large-scale data processing
  • Experience building and maintaining scalable data pipelines in cloud environments (Azure preferred)
  • Solid understanding of data lakes, Delta Lake, and distributed data processing concepts
  • Strong SQL skills and experience working with relational databases
  • Ability to work independently, manage priorities, and deliver high-quality solutions with minimal guidance

Preferred Skills

  • Experience supporting enterprise production data platforms
  • Familiarity with data architecture best practices and performance tuning
  • Strong problem-solving and troubleshooting abilities

Work Environment

  • Agile, collaborative team setting
  • Opportunity to work on modern cloud data platforms
  • Combination of independent ownership and cross-team collaboration

Benefits & conditions

The anticipated hourly rate range for this position is ($60-$63/hr). Actual hourly rate will be based on a variety of factors including relevant experience, knowledge, skills and other factors permitted by law. A range of medical, dental, retirement and/or other benefits are available after a waiting period.

Apply for this position