Senior Databricks DevOps / Platform Engineer
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+27 more
Job description
- Provision and manage Databricks workspaces on AWS with Terraform (Databricks and AWS providers), including cross-account IAM roles, customer-managed VPCs, S3 root storage and PrivateLink.
- Administer Unity Catalog: metastores, catalogs, external locations, storage credentials, grants and data lineage.
- Define and enforce cluster policies, instance pools, SQL warehouse settings and job compute standards for performance, security and cost.
- Manage identity and access with SSO, SCIM provisioning, service principals, groups and least-privilege permissions across workspaces.
- Build and maintain CI/CD pipelines (Jenkins, GitLab CI) for Databricks code and infrastructure using Databricks Asset Bundles, the Databricks CLI and Git folders.
- Embed security gates and compliance controls within DevSecOps pipelines to support FISMA/FedRAMP requirements, ATO reassessments and POA&M resolution.
- Manage supporting AWS infrastructure, including S3, IAM, KMS, Lambda, Glue, Aurora, CloudWatch and VPC networking.
- Monitor platform health, usage and spend with Databricks system tables, CloudWatch and alerting; drive cost optimization and chargeback reporting.
- Plan and run platform upgrades, Databricks Runtime migrations, backup and disaster recovery for workspaces and metadata.
- Support data engineering teams running PySpark, Spark SQL and Delta Lake workloads, including Medallion Architecture pipelines; troubleshoot and tune complex Spark jobs.
- Collaborate within cross-functional Agile/Scrum teams in multi-contractor federal environments, providing technical leadership and production on-call support.
Requirements
Satsyil Corp is seeking a highly skilled and motivated Senior Databricks DevOps / Platform Engineer to join our team. In this role, you will build, secure and operate the enterprise Databricks platform on AWS. Using Terraform, CI/CD and Unity Catalog, you will automate workspace provisioning, enforce governance and compliance controls, and keep the platform reliable and cost-efficient for the data engineering and analytics teams supporting mission-critical public sector and enterprise environments. Title Senior Databricks DevOps / Platform Engineer Platform Databricks on AWS (Terraform, CI/CD, Unity Catalog) Experience 6+ years of hands-on Databricks experience, including platform administration and DevOps Clearance Ability to obtain and maintain a Public Trust, Required experience and technical skills
- Citizenship: U.S. citizenship or Green Card (lawful permanent resident) status required.
- Experience: 6+ years of hands-on Databricks experience, including workspace administration, platform automation and DevOps.
- Databricks platform: Production experience administering Databricks on AWS: Unity Catalog, cluster policies, identity and access, networking and the Databricks REST API and CLI.
- Infrastructure as code: Production experience with Terraform, including the Databricks and AWS providers, modules and remote state.
- CI/CD and DevSecOps: Jenkins, GitLab CI or equivalent; Databricks Asset Bundles; security scanning and policy gates in pipelines.
- Cloud ecosystem: Strong experience in AWS services including IAM, S3, KMS, VPC, Lambda, Glue, Aurora and CloudWatch.
- Big data stack: Working proficiency with PySpark, Spark SQL, Apache Spark and Delta Lake for supporting and troubleshooting workloads.
- Languages: Deep proficiency in Python, SQL and Bash scripting; working knowledge of REST APIs.
- Clearance: Ability to obtain and maintain a Public Trust.
-
Methodology and education: Hands-on experience in Agile/Scrum delivery environments. Bachelorβs degree in CS, IT, Data Science or a related engineering discipline. Preferred qualifications
- Hands-on experience with Delta Live Tables (DLT), MLflow and Model Registry platform support.
- Experience with observability tooling such as Datadog, Splunk or Grafana.
- Experience in Federal Government contracting and compliance frameworks (FISMA / FedRAMP environments).
-
Active Public Trust. Preferred certifications Candidates holding any of the following certifications will be prioritized:
- Databricks: Certified Data Engineer Associate or Professional; Databricks Platform Administrator accreditation.
- AWS: Certified DevOps Engineer - Professional; Solutions Architect - Associate or Professional; SysOps Administrator - Associate; Security - Specialty.
- Infrastructure as code: HashiCorp Certified: Terraform Associate.
- Containers: Certified Kubernetes Administrator (CKA).
Benefits & conditions
Databricks on AWS * $140K - $175K * U.S. citizens or Green Card holders, $140,000 - $175,000 per year, The salary range for this position is $140,000 - $175,000 per year, based on experience and qualifications.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦