Databricks admin/platform engineer with FinOps experience

SPARTAN SOLUTIONS, LLC
United States
9 days ago
Apply on www.dice.com
Prepare application

Role details

Contract type
Contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Job source

Tech stack

Artificial Intelligence Amazon Web Services Microsoft Azure Information Engineering Data Governance Extract Transform Load (ETL) Metadata Repositories Search Technologies SQL Databases Systems Integration Data Logging Feature Store
+8 more
Google Cloud Data Ingestion Autoscaling Apache Spark Generative AI Data Lakes Machine Learning Operations Databricks

Job description

Design, implementation, and optimization of end-to-end solutions on Databricks, including Unity Catalog, Delta Lake, Databricks SQL, Workflows, Jobs, and Mosaic AI. Build, and maintain scalable data ingestion, transformation, and orchestration pipelines using Databricks tools (Spark, Delta Live Tables, Auto Loader, etc.). Identify and resolve performance bottlenecks in Spark jobs, clusters, and SQL queries. Implement cost optimization strategies, auto-scaling, and efficient resource management. Establish and enforce Databricks best practices for security, data governance, compliance (e.g., GDPR, HIPAA), and Unity Catalog implementation across the organization. Support data science and machine learning teams by implementing MLflow, Feature Store, Vector Search, and generative AI capabilities on Databricks. Act as the escalation point for complex technical issues, providing root cause analysis and rapid resolution for production workloads. Mentor data engineers, analysts, and developers on Databricks capabilities. Conduct training sessions, create documentation, and build internal centers of excellence. Design seamless integrations between Databricks and other technologies (cloud platforms like AWS/Azure/Google Cloud Platform, ETL tools, BI tools, data catalogs, and orchestration systems). Implement robust monitoring, logging, alerting, and dashboarding for Databricks workloads using Databricks features and external tools. Stay current with the latest Databricks features and roadmap. Evaluate new capabilities and recommend adoption strategies to drive business value. Work closely with stakeholders to translate business requirements into technical solutions and deliver high-impact data & AI projects.

Requirements

Strong exp. in FinOps end to end tagging, dashboards, finops data model. Need to understand connectivity, compute, resources at granular level. Data engineering background is a plus.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.dice.com
Prepare application

Good distractions

Loading talks and stories from around this role…