AWS Databricks Platform Administrator
Quantum World Technologies Inc
Somerville, United States of America
2 days ago
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English Experience level
SeniorJob location
Somerville, United States of America
Tech stack
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Amazon Web Services (AWS)
Bash
Continuous Integration
Information Engineering
Data Governance
Identity and Access Management
Python
Release Management
Runbook
Enterprise Data Management
Cloud Platform System
System Availability
Amazon Web Services (AWS)
Infrastructure Automation Frameworks
Optimization Algorithms
Cloudwatch
Terraform
Databricks
Job description
We are looking for an experienced AWS Databricks Platform Administrator with 5 plus years of experience in cloud data platform administration with strong expertise in Databricks on AWS The role is responsible for managing securing automating and optimizing the Databricks platform to support enterprise scale data engineering analytics and governance needs The ideal candidate should have hands on experience in AWS infrastructure IAM Unity Catalog cost optimization platform monitoring and production support, * Administer and maintain Databricks workspaces in AWS environments
- Configure and maintain secure access to data stored in Amazon S3 using IAM roles instance profiles and trust policies
- Implement and manage Unity Catalog including catalogs schemas tables storage credentials and external locations
- Define and enforce cluster policies access controls and workspace standards to ensure security and compliance
- Monitor cluster performance usage trends and platform health and drive cost optimization through autoscaling tagging and policy controls
- Automate platform provisioning and configuration using Infrastructure as Code tools such as Terraform
- Troubleshoot production issues related to clusters permissions storage access networking and job failures
- Collaborate with data engineering security cloud infrastructure and support teams to ensure platform reliability and governance
- Support release management upgrades new feature enablement and platform best practices adoption
- Create and maintain operational documentation SOPs and runbooks for platform administration
Requirements
- 5-7 years of overall IT experience with strong hands on experience in Databricks administration on AWS
- Good understanding of AWS services including IAM S3 EC2 VPC CloudWatch and related security controls
- Hands on experience with Unity Catalog and enterprise data governance practices
- Strong knowledge of Databricks cluster management workspace administration job monitoring and access management
- Experience in Infrastructure as Code using Terraform and scripting using Python Shell or Bash
- Knowledge of CICD release management and environment migration processes for platform changes
- Experience in monitoring incident management root cause analysis and production support
- Understanding of cost management and optimization techniques for Databricks and AWS resources
- Strong communication and stakeholder management skills, * Experience with Databricks E2 architecture and multi workspace administration
- Exposure to charge back show back tagging strategies and usage reporting
- Familiarity with Databricks SQL dashboards and platform observability practices
- AWS certifications such as AWS Certified Solutions Architect SysOps Administrator or Security Specialty
Experience working in enterprise data platform or analytics transformation programs