Azure Databricks Admin

Cognizant is seeking a highly skilled Azure Databricks Administrator to join our growing Cloud, Data & AI practice. This role is responsible for the administration, governance, security, optimization, and operational support of enterprise-scale Azure Databricks environments.

The ideal candidate will possess deep expertise in Azure Databricks, Unity Catalog, Azure networking, security, data governance, and platform operations. This individual will serve as a technical leader and trusted advisor, partnering with engineering, infrastructure, security, and operations teams to ensure a scalable, secure, and high-performing data platform.

Work Model

Remote - United States

Responsibilities

Azure Databricks Administration & Optimization

• Design, deploy, and administer Azure Databricks workspaces utilizing VNet injection, private endpoints, and secure cluster connectivity.

• Configure and maintain cluster policies, Databricks Runtime versions, autoscaling, and workload management standards.

• Optimize platform performance through cluster tuning, Spark configuration management, and Delta Lake best practices.

• Perform capacity planning and resource forecasting to support enterprise growth.

• Drive cost optimization initiatives including right-sizing, auto-termination policies, reserved instances, and spot instance utilization.

• Troubleshoot and resolve complex platform, cluster, job execution, and connectivity issues.

Data Governance & Unity Catalog

• Design, implement, and administer Unity Catalog metastores, catalogs, schemas, external locations, and volumes.

• Establish and enforce fine-grained access controls across enterprise data assets.

• Support data lineage tracking, auditing, and governance requirements.

• Govern managed and external data assets while promoting best practices for data management.

Security & Identity Management

• Integrate Azure Databricks with Microsoft Entra ID for authentication and Single Sign-On (SSO).

• Configure and manage SCIM provisioning for automated user and group lifecycle management.

• Implement secure secret management using Azure Key Vault-backed secret scopes.

• Define and manage access controls across Unity Catalog, Databricks workspaces, and Azure Data Lake Storage Gen2.

• Support security audits, compliance reviews, and governance initiatives.

Networking & Connectivity

• Architect and maintain secure networking solutions utilizing VNets, NSGs, private endpoints, and firewall controls.

• Ensure secure connectivity between Databricks, Azure services, and on-premises environments.

• Manage DNS resolution, routing configurations, and network security best practices.

Data Integration & Automation

• Design and implement Azure Data Factory (ADF) integration patterns with Azure Databricks.

• Support orchestration of enterprise data pipelines and notebook execution workflows.

• Partner with DevOps teams to establish and maintain CI/CD pipelines.

• Utilize Databricks REST APIs and automation frameworks to streamline platform administration.

Storage Management

• Provide guidance on Azure Data Lake Storage Gen2 architecture and configuration.

• Implement secure access patterns using service principals, credential passthrough, and Microsoft Entra ID integration.

• Troubleshoot storage connectivity and access management issues.

Operations & Platform Reliability

• Implement monitoring and alerting using Azure Monitor, Log Analytics, and Databricks native monitoring tools.

• Monitor service availability, platform health, resource utilization, and job performance.

• Support disaster recovery, business continuity, and resiliency planning efforts.

• Plan and coordinate Databricks Runtime upgrades and platform maintenance activities.

• Communicate service impacts and operational risks to stakeholders.

Documentation & Enablement

• Develop and maintain architecture documentation, diagrams, standards, and operational runbooks.

• Establish best practices for platform governance, security, cost optimization, and performance management.

• Provide technical guidance, training, and knowledge transfer to internal teams.

Stakeholder Engagement

• Collaborate with data engineering, data science, infrastructure, networking, security, and operations teams.

• Act as the subject matter expert for Databricks platform administration and governance.

• Participate in architecture reviews and provide technical leadership for strategic initiatives.

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent work experience.

• 5+ years of experience supporting cloud-based data and analytics platforms.

• Hands-on experience administering Azure Databricks in enterprise environments.

• Strong experience with:

• Azure Databricks

• Azure Data Factory (ADF)

• Azure Data Lake Storage Gen2 (ADLS Gen2)

• Microsoft Entra ID

• Azure Key Vault

• Azure Networking (VNets, NSGs, Private Endpoints)

• Azure Monitor and Log Analytics

• Experience implementing and managing Unity Catalog.

• Strong understanding of cloud security, RBAC, IAM, encryption, and governance principles.

• Experience with SCIM-based identity provisioning.

• Proficiency in Python, SQL, PowerShell, and Azure CLI.

• Experience supporting CI/CD initiatives using Azure DevOps, GitHub Actions, or similar platforms.

• Excellent analytical, troubleshooting, and problem-solving skills.

• Strong verbal and written communication skills.

Preferred Qualifications

• Experience supporting large-scale enterprise Data & AI platforms.

• Knowledge of Spark architecture, Delta Lake optimization, and Databricks APIs.

• Experience with Infrastructure as Code (IaC) tools and cloud automation frameworks.

• Experience working within regulated or highly governed environments.

• Microsoft Certified: Azure Solutions Architect Expert.

• Databricks Certified Data Engineer Associate or Professional certification.

The annual salary for this position is between $110,000 and $160,000 , depending on experience, qualifications, geographic location, and other factors permitted by law.

This position is also eligible for Cognizant's discretionary annual incentive compensation program, based on performance and subject to the terms of the applicable plan.

Benefits

Cognizant offers a comprehensive benefits package that includes:

• Medical, Dental, Vision, and Life Insurance

• Short-Term and Long-Term Disability Coverage

• 401(k) Plan with Company Contributions

• Health Savings Account (HSA) and Flexible Spending Accounts (FSA)

• Paid Time Off (PTO), Company Holidays, and Sick Leave

• Paid Parental Leave

Cognizant (Nasdaq: CTSH) engineers modern businesses. We help our clients modernize technology, reimagine processes, and transform experiences so they can stay ahead in a rapidly changing world. Together, we deliver digital solutions that drive measurable business outcomes across industries. At Cognizant, you'll work with industry-leading professionals, cutting-edge technologies, and some of the world's most recognized organizations while advancing your career in a collaborative and inclusive environment.