TELECOMMUTE Senior Azure Databricks (Platform) Architect

Platform Assessment & Migration Planning

• Assess & document. Audit workspaces, clusters, jobs, secrets, identities, networking, storage, and metastore/Unity Catalog; identify dependencies, risks, and migration blockers.

• Plan the migration. Define sequencing, cutover approach, validation plan, rollback procedure, and post-migration support model.

Databricks Architecture & Target Environment Design

• Design target-state architecture. Workspace strategy, network topology, governance model, and Unity Catalog/metastore strategy (external locations, storage credentials, catalog/schema structure, access control) in the Client’s tenant.

• Establish platform standards. Cluster policies, compute patterns, naming conventions, environment promotion, monitoring, and cost management.

Security, Identity & Governance

• Rebuild identity & access. Microsoft Entra ID, SSO/SCIM, RBAC, service principals, managed identities, and least-privilege access.

• Re-establish security controls & governance. Key Vault, private endpoints/Private Link, VNet peering, NSGs, storage firewalls, and Unity Catalog governance policies with auditability.

Tenant-to-Tenant Migration Execution

• Lead migration execution. Move or recreate workspaces, notebooks, jobs, cluster configs, secrets, CI/CD assets, and integrations in the target tenant.

• Manage cutover & validate readiness. Coordinate migration windows, go/no-go decisions, hypercare, and post-migration validation of access, job execution, security, and monitoring.

Infrastructure Automation & DevOps

• Automate & rebuild pipelines. Provision environments with Terraform (preferred) or Bicep/ARM; rebuild CI/CD in Azure DevOps or GitHub Actions for platform, notebook, and job deployment.

Data Engineering Support (Secondary)

• Support pipeline migration. Partner with data engineering on ETL/ELT, Databricks Workflows, and Delta Live Tables affected by the separation; validate data parity and job reliability.

Stakeholder Coordination & Handoff

• Coordinate & document. Manage dependencies and cutover readiness across infrastructure, security, application, and business teams; produce architecture diagrams, runbooks, and handoff documentation.

We’re seeking a hands-on Senior Azure Databricks Architect to lead this migration and separation effort end-to-end - covering platform architecture, tenant-to-tenant migration strategy, Unity Catalog and governance, security and networking, Infrastructure as Code, CI/CD, and cutover execution. The ideal candidate has led Azure Databricks migrations, cloud carve-outs, or tenant separation initiatives; data engineering experience is a plus, but deep Databricks architecture and migration expertise is the core requirement., Platform Architecture, Migration & Tenant Separation

• 8+ years in cloud data platform architecture or enterprise platform operations, including 5+ years hands-on Azure Databricks architecture, administration, and operations.

• Proven migration leadership on Azure Databricks tenant-to-tenant migrations, cloud carve-outs, divestitures, or comparable enterprise-scale platform migrations.

• Cutover and rollback expertise, including migration plans, runbooks, validation checklists, and hypercare - with strong risk management and stakeholder coordination for live production transitions.

• Deep governance expertise with Unity Catalog or Hive metastore - catalog/schema design, external locations, permissions, and lineage.

• Proven ability to design scalable, secure, cost-aware Databricks environments across Dev/Test/UAT/Prod with monitoring, runbooks, and support models.

Azure Infrastructure, Security & Automation

• Strong Azure networking and security experience (VNets, Private Link, NSGs, Key Vault, managed identities, Entra ID).

• Strong Terraform experience (ARM/Bicep acceptable); CI/CD experience with Azure DevOps or GitHub Actions.

Communication & Leadership

• Ability to lead technical decisions across platform, security, and data teams, and communicate migration risks, tradeoffs, and readiness decisions to technical and business stakeholders.