Senior Databricks DevOps / Platform Engineer

Satsyil Corporation
Tysons, VA, United States
13 days ago
Apply on www.clearancejobs.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
6 years minimum
Compensation
$140,000 - $175,000
Working hours
Regular working hours

Tech stack

Unity 3d Microsoft Access Amazon Web Services Amazon S3 Bash Shell Big Data Continuous Integration Information Engineering DevOps Apache Hive Identity and Access Management Python (Programming Language)
+27 more
Metadata Scrum Methodology Cloud Services SQL Databases Datadog Grafana Apache Spark Amazon Virtual Private Cloud (VPC) Git Data Lakes Pyspark Gitlab-ci Kubernetes Infrastructure Automation Frameworks Data Lineage Hashicorp Machine Learning Operations Model Registry Cloudwatch Restful APIs Terraform Splunk Devsecops Jenkins Plan of Action and Milestones Databricks Vulnerability Analysis

Job description

  • Provision and manage Databricks workspaces on AWS with Terraform (Databricks and AWS providers), including cross-account IAM roles, customer-managed VPCs, S3 root storage and PrivateLink.
  • Administer Unity Catalog: metastores, catalogs, external locations, storage credentials, grants and data lineage.
  • Define and enforce cluster policies, instance pools, SQL warehouse settings and job compute standards for performance, security and cost.
  • Manage identity and access with SSO, SCIM provisioning, service principals, groups and least-privilege permissions across workspaces.
  • Build and maintain CI/CD pipelines (Jenkins, GitLab CI) for Databricks code and infrastructure using Databricks Asset Bundles, the Databricks CLI and Git folders.
  • Embed security gates and compliance controls within DevSecOps pipelines to support FISMA/FedRAMP requirements, ATO reassessments and POA&M resolution.
  • Manage supporting AWS infrastructure, including S3, IAM, KMS, Lambda, Glue, Aurora, CloudWatch and VPC networking.
  • Monitor platform health, usage and spend with Databricks system tables, CloudWatch and alerting; drive cost optimization and chargeback reporting.
  • Plan and run platform upgrades, Databricks Runtime migrations, backup and disaster recovery for workspaces and metadata.
  • Support data engineering teams running PySpark, Spark SQL and Delta Lake workloads, including Medallion Architecture pipelines; troubleshoot and tune complex Spark jobs.
  • Collaborate within cross-functional Agile/Scrum teams in multi-contractor federal environments, providing technical leadership and production on-call support.

Requirements

Satsyil Corp is seeking a highly skilled and motivated Senior Databricks DevOps / Platform Engineer to join our team. In this role, you will build, secure and operate the enterprise Databricks platform on AWS. Using Terraform, CI/CD and Unity Catalog, you will automate workspace provisioning, enforce governance and compliance controls, and keep the platform reliable and cost-efficient for the data engineering and analytics teams supporting mission-critical public sector and enterprise environments. Title Senior Databricks DevOps / Platform Engineer Platform Databricks on AWS (Terraform, CI/CD, Unity Catalog) Experience 6+ years of hands-on Databricks experience, including platform administration and DevOps Clearance Ability to obtain and maintain a Public Trust, Required experience and technical skills

  • Citizenship: U.S. citizenship or Green Card (lawful permanent resident) status required.
  • Experience: 6+ years of hands-on Databricks experience, including workspace administration, platform automation and DevOps.
  • Databricks platform: Production experience administering Databricks on AWS: Unity Catalog, cluster policies, identity and access, networking and the Databricks REST API and CLI.
  • Infrastructure as code: Production experience with Terraform, including the Databricks and AWS providers, modules and remote state.
  • CI/CD and DevSecOps: Jenkins, GitLab CI or equivalent; Databricks Asset Bundles; security scanning and policy gates in pipelines.
  • Cloud ecosystem: Strong experience in AWS services including IAM, S3, KMS, VPC, Lambda, Glue, Aurora and CloudWatch.
  • Big data stack: Working proficiency with PySpark, Spark SQL, Apache Spark and Delta Lake for supporting and troubleshooting workloads.
  • Languages: Deep proficiency in Python, SQL and Bash scripting; working knowledge of REST APIs.
  • Clearance: Ability to obtain and maintain a Public Trust.
  • Methodology and education: Hands-on experience in Agile/Scrum delivery environments. Bachelor’s degree in CS, IT, Data Science or a related engineering discipline. Preferred qualifications

  • Hands-on experience with Delta Live Tables (DLT), MLflow and Model Registry platform support.
  • Experience with observability tooling such as Datadog, Splunk or Grafana.
  • Experience in Federal Government contracting and compliance frameworks (FISMA / FedRAMP environments).
  • Active Public Trust. Preferred certifications Candidates holding any of the following certifications will be prioritized:

  • Databricks: Certified Data Engineer Associate or Professional; Databricks Platform Administrator accreditation.
  • AWS: Certified DevOps Engineer - Professional; Solutions Architect - Associate or Professional; SysOps Administrator - Associate; Security - Specialty.
  • Infrastructure as code: HashiCorp Certified: Terraform Associate.
  • Containers: Certified Kubernetes Administrator (CKA).

Benefits & conditions

Databricks on AWS * $140K - $175K * U.S. citizens or Green Card holders, $140,000 - $175,000 per year, The salary range for this position is $140,000 - $175,000 per year, based on experience and qualifications.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.clearancejobs.com
Prepare application

Good distractions

Loading talks and stories from around this role…