Azure Data Platform Engineer (Databricks Focus)

• Engineer, operate, and evolve enterprise-grade Azure Databricks platforms across multiple subscriptions and environments (dev, test, prod).
• Design and implement Databricks workspace configurations, cluster policies, pools, and job orchestration with a strong focus on scalability, reliability, and cost efficiency.
• Engineer and maintain Azure platform components supporting Databricks, including networking, identity, storage, and security services.
• Monitor platform health, performance, and usage; proactively design improvements and resolve operational issues.

Databricks-Centric Data & Analytics Enablement

• Engineer and optimize Databricks integrations with Azure services, including:

• Azure Data Lake Storage (ADLS Gen2)
• Delta Lake and Unity Catalog
• Azure SQL, Azure Synapse, Event Hubs, and other data sources

• Support batch and streaming data pipelines and analytics workloads running on Databricks.
• Partner with Data Engineering, Analytics, and AI teams to ensure performant, reliable, and secure production workloads.

Security, Governance & Compliance Engineering

• Design and implement security and governance controls across Databricks and Azure, including:

• Azure Active Directory integration, RBAC, and SCIM provisioning
• Unity Catalog, data access policies, and cluster governance

• Engineer secure integrations using Azure Key Vault, Private Endpoints, and network security constructs.
• Ensure platform designs comply with organizational security, governance, and audit requirements.

Infrastructure as Code & Platform Automation

• Design, develop, and maintain Terraform-based IaC for Databricks and supporting Azure infrastructure (workspaces, permissions, networking, storage).
• Build and evolve CI/CD pipelines using Azure DevOps YAML pipelines for platform and infrastructure deployments.
• Promote engineering best practices for IaC, including modularization, versioning, automated testing, and environment consistency.

Collaboration & Platform Leadership

• Act as a subject matter expert for Databricks on Azure , guiding teams on platform usage, architecture, and operational best practices.
• Work cross-functionally with Cloud, Architecture, CICD, Data, AI, and Application teams.
• Create and maintain platform documentation, reference architectures, and operational runbooks.

Do you have experience in Unity?, * 5+ years of hands-on experience engineering solutions on Microsoft Azure , with a strong focus on Azure Databricks platforms.

• Deep expertise in:

• Azure Databricks (clusters, jobs, pools, Delta Lake, Unity Catalog, workspace engineering)
• Azure core services (Storage, Networking, Identity & Access Management)
• Cost optimization, monitoring, and observability for data platforms

• Strong experience with Terraform , including multi-subscription and multi-environment deployments.
• Proficiency with CI/CD pipelines, preferably Azure DevOps .
• Solid understanding of cloud security, governance, and platform engineering practices.

Preferred Certifications Azure & Databricks Certifications

• Microsoft Certified: Azure Administrator Associate
• Microsoft Certified: Azure Solutions Architect Expert
• Databricks Certified Data Platform Engineer or Administrator, * Strong communication and documentation skills.
• Ability to explain complex Azure and Databricks platform designs to diverse technical audiences.
• Engineering mindset with a strong focus on automation, reliability, and continuous improvement.
• Collaborative, proactive, and adaptable in a fast-evolving cloud and data landscape.