Data Engineer (Tech Lead) Databricks / Azure

Pioneer IT Systems LLC
1 month ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Remote

Tech stack

Agile Methodologies
Unit Testing
Azure
Data Validation
Data Systems
Power BI
Azure
SQL Databases
Performance Testing
Azure
Spark
Integration Tests
Data Management
Data Pipelines
Databricks

Job description

Seeking a Senior Data Engineer / Tech Lead to support enterprise data initiatives, focusing on building and optimizing scalable data pipelines within the Azure ecosystem.

This individual will play a key leadership role in designing, developing, and ensuring the reliability of data solutions that power business reporting and analytics., * Lead the design and development of scalable data pipelines using Databricks and Azure services

  • Move and transform data from source systems into Azure Data Lake for reporting and analytics
  • Ensure data quality through proper unit testing and system/integration testing
  • Validate that all required files and transactions are accurately captured and processed
  • Collaborate with cross-functional teams in an Agile environment
  • Provide technical leadership, mentorship, and best practices to the engineering team
  • Work with Unity Catalog and implement/manage Delta Tables within Databricks

Requirements

  • 5+ years of Databricks experience (Certification required)
  • 5+ years of Azure Data Factory experience
  • 5+ years of strong SQL development experience
  • Proven experience in a Tech Lead or senior-level engineering role
  • Strong understanding of Agile methodologies
  • Excellent verbal communication skills

Additional Requirements (Select Roles)

  • Experience with Power BI development
  • Ability to support performance testing and optimization of reporting solutions

Preferred Experience

  • Strong experience with Apache Spark programming
  • Hands-on experience with data validation, testing frameworks, and pipeline monitoring

Familiarity with enterprise-scale data platforms and governance practices

Apply for this position