Data Engineer Azure Databricks (Corporate Vertical Developer

High Bridge Consulting
New York, United States of America
2 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

New York, United States of America

Tech stack

Application Integration Architecture
Azure
Big Data
Business Software
C Sharp (Programming Language)
Information Systems
Data as a Services
Information Engineering
Data Governance
Data Hub
Data Infrastructure
Data Integration
ETL
Distributed Computing Environment
Python
Meta-Data Management
Metadata Repositories
SQL Azure
Performance Tuning
Powershell
Cloud Services
Systems Integration
Enterprise Data Management
Data Processing
Enterprise Software Applications
Azure
System Availability
Spark
Event Driven Architecture
Data Lake
Information Technology
Data Analytics
Data Pipelines
Serverless Computing
Workday
ServiceNow
Databricks

Job description

We are seeking an experienced Data Engineer Azure Databricks (Corporate Vertical Developer) to support multiple corporate business functions, including RESO, HCG, Compliance, Facilities, and Finance. This role will be responsible for designing and building scalable data solutions, enabling enterprise-wide analytics, and driving data integration initiatives across the organization., * Design, develop, and maintain scalable data pipelines using Microsoft Azure and Azure Databricks.

  • Build and optimize ETL/ELT processes for both structured and unstructured data sources.
  • Partner with stakeholders across corporate functions to gather requirements and deliver data-driven solutions.
  • Design and implement data models, architectures, and frameworks that support reporting, analytics, and business intelligence initiatives.
  • Integrate enterprise platforms such as ServiceNow, Dataverse, Workday, and other business applications into centralized data environments.
  • Ensure adherence to data quality, governance, security, and compliance standards.
  • Monitor, troubleshoot, and optimize data workflows to ensure high availability and performance.
  • Contribute to the ongoing enhancement of enterprise data platforms and recommend innovative technologies and best practices.
  • Support data modernization initiatives and enterprise-wide analytics strategies.

Requirements

The ideal candidate will have strong expertise in Azure cloud technologies, Databricks, data engineering, and enterprise data integration, with the ability to collaborate effectively across business and technical teams. Location: NYC Area (Hybrid 3 days onsite per week), * Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent experience).

  • 5+ years of experience in Data Engineering or a related discipline.
  • Strong hands-on experience with Microsoft Azure data services.
  • Extensive experience with Azure Databricks, including Apache Spark and distributed data processing.
  • Advanced SQL development and optimization skills with large-scale data environments.
  • Experience building and orchestrating data pipelines using Azure Data Factory (ADF) or similar ETL/ELT tools.
  • Strong experience with Azure SQL Database, including performance tuning and optimization.
  • Proficiency in Python and/or Scala for data engineering and automation.
  • Experience with C# and PowerShell for integration and automation tasks.
  • Experience developing and deploying Azure Functions for data processing and system integrations.
  • Experience integrating APIs, event-driven architectures, and enterprise applications.
  • Strong understanding of scalable, secure, and cloud-native data architecture principles.
  • Excellent communication and collaboration skills with the ability to work across multiple business domains., * Experience with Azure DevOps and CI/CD pipeline implementation.
  • Knowledge of Delta Lake and modern Lakehouse architectures.
  • Familiarity with data governance, metadata management, and data catalog solutions.
  • Experience working within Agile/Scrum development environments.
  • Relevant Microsoft Azure and/or Databricks certifications.

Apply for this position