Data Engineer (Tech Lead) Databricks / Azure
Pioneer IT Systems LLC
1 month ago
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
SeniorJob location
Remote
Tech stack
Agile Methodologies
Unit Testing
Azure
Data Validation
Data Systems
Power BI
Azure
SQL Databases
Performance Testing
Azure
Spark
Integration Tests
Data Management
Data Pipelines
Databricks
Job description
Seeking a Senior Data Engineer / Tech Lead to support enterprise data initiatives, focusing on building and optimizing scalable data pipelines within the Azure ecosystem.
This individual will play a key leadership role in designing, developing, and ensuring the reliability of data solutions that power business reporting and analytics., * Lead the design and development of scalable data pipelines using Databricks and Azure services
- Move and transform data from source systems into Azure Data Lake for reporting and analytics
- Ensure data quality through proper unit testing and system/integration testing
- Validate that all required files and transactions are accurately captured and processed
- Collaborate with cross-functional teams in an Agile environment
- Provide technical leadership, mentorship, and best practices to the engineering team
- Work with Unity Catalog and implement/manage Delta Tables within Databricks
Requirements
- 5+ years of Databricks experience (Certification required)
- 5+ years of Azure Data Factory experience
- 5+ years of strong SQL development experience
- Proven experience in a Tech Lead or senior-level engineering role
- Strong understanding of Agile methodologies
- Excellent verbal communication skills
Additional Requirements (Select Roles)
- Experience with Power BI development
- Ability to support performance testing and optimization of reporting solutions
Preferred Experience
- Strong experience with Apache Spark programming
- Hands-on experience with data validation, testing frameworks, and pipeline monitoring
Familiarity with enterprise-scale data platforms and governance practices