Databricks Admin SME

Client is seeking an experienced Databricks Admin Subject Matter Expert (SME) to manage, optimize, and secure enterprise-scale Databricks environments. The ideal candidate will be responsible for administration, governance, performance optimization, security implementation, and operational excellence of the Databricks platform while supporting business-critical data engineering and analytics workloads. This role requires deep expertise in Databricks workspace administration, cloud integrations, user access governance, security best practices, and platform scalability across large enterprise environments., Databricks Platform Administration Manage and administer Databricks workspaces across development, QA, and production environments. Configure clusters, cluster policies, job scheduling, and workspace settings. Monitor system performance, usage trends, and optimize platform costs. Implement workspace provisioning, maintenance, upgrades, and troubleshooting.

Security & Governance Design and manage role-based access control (RBAC), Unity Catalog, and data governance frameworks. Integrate Databricks with enterprise identity providers (Azure AD, AWS IAM, Okta, etc.). Ensure compliance with organizational security policies and industry standards. Configure secure networking, private link, VNET injection, and encryption protocols.

Cloud Integration & Infrastructure Collaborate with cloud infrastructure teams to manage Databricks deployments on Azure, AWS, or Google Cloud Platform. Configure storage integrations (ADLS, S3, Blob Storage). Support CI/CD automation using Terraform, ARM templates, or Infrastructure as Code (IaC). Maintain disaster recovery and backup strategies.

Monitoring & Optimization Set up monitoring dashboards, logging, and alerting for proactive issue resolution. Optimize cluster sizing, auto-scaling, and job performance. Conduct capacity planning and cost governance initiatives. Troubleshoot platform, networking, and integration issues.

Collaboration & Leadership Act as the primary SME for Databricks administration and operational support. Partner with data engineers, architects, security teams, and business stakeholders. Provide best practices, technical guidance, and mentoring to internal teams. Support enterprise data modernization initiatives.

Bachelor's degree in Computer Science, Information Technology, or related field. 15+ years of experience in data platform administration or cloud data engineering. 4+ years of hands-on Databricks administration experience. Strong expertise with: Databricks Workspace Administration Unity Catalog Cluster Management Security & Governance Azure/AWS/Google Cloud Platform cloud ecosystems Terraform / IaC Identity and Access Management Experience with: Spark Delta Lake SQL Python Monitoring tools DevOps processes Knowledge of networking, cloud security, and compliance standards.