Technology Lead - Azure Data Factory (ADF) & Azure Databricks Operations

Ascii Group, LLC

Philadelphia, PA, United States

9 days ago

Apply on www.dice.com

Prepare application

Role details

Contract type

Temporary contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

6 years minimum

Compensation

$135,200.0

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Data Infrastructure Queue Management Systems Release Management DataOps Runbook Enterprise Data Management Azure Data Factory System Availability Mttr Cloud Integration Servicenow Databricks

Job description

Seeking a Technology Lead - Azure Data Factory (ADF) & Azure Databricks Operations with 8-10+ years of data operations experience for a 6-month contract in Philadelphia, PA (local candidates highly preferred; relocation accepted if ready by Day 1). The lead will oversee onshore data operations across the Azure data platform, act as the escalation owner for P1/P2 incidents, and drive EDWH/BI pipeline reliability. Key responsibilities include troubleshooting Databricks job and pipeline failures, root cause analysis, developing Runbooks/SOPs, managing SLAs, and leading incident/change coordination. Note: Virtual interview process., * Data Platform Operations & Escalations: Lead onshore Azure data platform operations, serving as primary escalation owner for P1/P2 incidents, security compliance, and change coordination.

Pipeline & Databricks Troubleshooting: Analyze and resolve Databricks job/pipeline failures, perform deep root cause analysis (RCA), and maintain high system availability/MTTR targets.
Incident, Problem & Change Management: Oversee queue management, backlog prioritization, ServiceNow change requests, and release management across enterprise EDWH/BI workloads.
Operations Governance & SOPs: Develop, update, and maintain Runbooks, SOPs, and operational metrics reporting for continuous service improvement.
Cross-Functional Leadership: Coordinate across multi-disciplinary engineering, business, and operations teams to streamline enterprise data integration workflows.

Requirements

Core Experience: 8-10+ years of experience in data operations with a dedicated focus on Azure Databricks, Enterprise Data Warehousing (EDWH), and Business Intelligence (BI).
Azure Integration Stack: Strong hands-on experience with Azure Data Factory (ADF), Databricks pipeline operations, execution troubleshooting, and monitoring.
Operational Excellence: Demonstrated expertise in Root Cause Analysis (RCA), Runbook/SOP creation, SLA management, and P1/P2 incident resolution.
Education & Certifications: 6+ minimum years required; relevant Azure Data Engineer or Databricks certifications preferred.

Preferred Qualifications