Databricks Platform Engineer (Azure) Hybrid

We’re looking for a Databricks Platform Engineer to help run and improve our Databricks platform on Microsoft Azure.

This role is focused on making sure Databricks is secure, stable, well-governed, and easy for data teams to use. You’ll be responsible for how Databricks is set up, automated, secured, and integrated with the rest of the Azure platform. You’ll work closely with data engineers, security teams, and cloud infrastructure teams to support analytics at scale. This is a hands-on Databricks platform role, not a pure data engineering position.

What You’ll Be Doing

• Databricks Platform Operations
• Run and support Databricks workspaces across multiple environments (dev, test, prod)
• Manage clusters, jobs, and platform configuration
• Administer Unity Catalog, including access controls and data governance
• Act as a go-to person for Databricks platform issues and incidents
• Automation & Infrastructure
• Automate Databricks and Azure infrastructure using Terraform
• Build and maintain CI/CD pipelines (Azure DevOps)
• Use Databricks APIs to automate platform operations
• Create reusable modules, documentation, and runbooks
• Azure & Networking
• Deploy and maintain VNet-integrated Databricks workspaces
• Configure Private Endpoints, firewalls, and secure network access
• Ensure data traffic between Databricks, storage, and other systems is locked down and compliant
• Data Access & Integrations
• Integrate Databricks with Azure Blob Storage / ADLS Gen2
• Support connections to external databases (JDBC) and internal/external APIs
• Configure identity and access management using Azure AD, SCIM, and role-based permissions
• Monitoring, Security & Compliance
• Support monitoring and observability using Datadog, Grafana, and Dragster
• Work with security teams on data encryption, audit logging, GDPR and compliance requirements
• Help monitor platform costs and performance

• Essential Experience
• Strong hands-on experience operating Databricks, ideally in Azure
• Experience managing Databricks clusters, jobs, and Unity Catalog
• Solid understanding of Azure networking (VNets, Private Endpoints, firewalls)
• Experience with Terraform and infrastructure automation
• Familiarity with Azure AD, authentication, and access control
• Comfortable scripting in Python and/or Shell
• Experience supporting Databricks in a production / enterprise setup
• Nice to Have
• Experience with multi-workspace Databricks setups
• Exposure to API Management or API integrations
• Experience with containers (Docker, AKS)
• Background in regulated industries (energy, finance)
• Familiarity with orchestration tools (Airflow, Dagster)

Technologies You’ll Work With

• Databricks, Spark, Delta Lake, Unity Catalog
• Microsoft Azure (VNets, Private Endpoints, Blob Storage, AAD)
• Terraform, Azure DevOps, CI/CD
• Python, REST APIs, JDBC, SCIM
• Datadog, Grafana, Dragster
• Linux, Docker