Databricks DevOps Engineer

Satsyil Corp is seeking an experienced Senior Databricks DevOps Engineer to design, automate, and maintain enterprise-scale Databricks environments. In this role, you will build the platform foundations that enable data engineers, data scientists, and analysts to work efficiently and securely. You will own CI/CD pipelines, infrastructure as code, workspace governance, and operational reliability across cloud-based Lakehouse environments, while serving as a technical leader and mentor on the team.

1. Key Roles & Responsibilities

· Design, deploy, and manage Databricks workspaces across AWS, Azure, or Google Cloud Platform using infrastructure as code (Terraform, Databricks Terraform Provider, ARM/Bicep, or CloudFormation).

· Build and maintain CI/CD pipelines for notebooks, jobs, Delta Live Tables, ML models, and libraries using Databricks Asset Bundles, Azure DevOps, GitHub Actions, GitLab CI, or Jenkins.

· Implement and administer Unity Catalog, including metastore configuration, catalog and schema design, access controls, data lineage, and audit logging.

· Configure and optimize clusters, SQL warehouses, cluster policies, and job compute to balance performance and cost.

· Establish security best practices, including network isolation (VNet/VPC injection, Private Link), secrets management, SCIM provisioning, SSO integration, and role-based access control.

· Develop monitoring, alerting, and observability solutions for platform health, job performance, and cost tracking using tools such as Datadog, Azure Monitor, CloudWatch, or Grafana.

· Automate environment promotion (dev, test, prod) and enforce release management, version control, and change management standards.

· Support MLOps workflows using MLflow, Model Registry, and Model Serving.

· Troubleshoot platform, pipeline, and performance issues, and lead root-cause analysis for production incidents.

· Partner with data engineering, security, and architecture teams to define platform standards and reference architectures.

· Mentor junior engineers and contribute to technical documentation, runbooks, and knowledge sharing.

· Ensure compliance with organizational and regulatory requirements (e.g., FedRAMP, NIST 800-53, FISMA, HIPAA, or SOC 2, as applicable).

· Experience: 8+ years of experience in DevOps, cloud infrastructure, or platform engineering, with at least 3 years working hands-on with Databricks.

· IaC & Cloud Infrastructure: Strong experience with Terraform, the Databricks Terraform Provider, and deep expertise in at least one major cloud platform (Azure, AWS, or Google Cloud Platform) including networking, IAM, and storage.

· CI/CD & Automation: Proven experience building CI/CD pipelines for data platforms; proficiency in Python, Bash, or PowerShell for automation, plus familiarity with Databricks REST API and CLI.

· Architecture & Governance: Solid understanding of Apache Spark, Delta Lake, Lakehouse architecture, and hands-on experience with Unity Catalog administration and data governance.

· Workflows: Strong knowledge of Git-based workflows and branching strategies.

· Visa Type: US Ctizenship

· Clearance & Citizenship: U.S. Citizenship required; must possess or have the ability to obtain Public Trust / Secret clearance.

· Communication & Education: Excellent communication skills to work with technical and non-technical stakeholders. Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field.

Preferred Qualifications & Nice-to-Haves:

· Experience with Databricks Asset Bundles (DABs), Delta Live Tables (DLT), containerization (Docker, Kubernetes), and MLOps practices using MLflow at scale.

· Familiarity with FinOps and cloud cost optimization for data workloads.

· Prior experience supporting federal government programs or regulated industries.

1. Preferred Certifications

Candidates possessing any of the following certifications will be prioritized:

· Databricks Certified Data Engineer Professional

· Databricks Certified Platform Administrator

· AWS Certified DevOps Engineer - Professional

· HashiCorp Certified: Terraform Associate