Databricks Architect/Admin

JOB DIVA INC
Hartford, United States of America
4 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior
Compensation
$ 26K

Job location

Hartford, United States of America

Tech stack

Artificial Intelligence
Amazon Web Services (AWS)
Data analysis
Computing Platforms
JIRA
Azure
Bash
Information Engineering
ETL
Python
Machine Learning
Oracle Applications
Ansible
Shell Script
SQL Databases
Large Language Models
Generative AI
Ab Initio
GIT
Data Lake
PySpark
Data Management
Machine Learning Operations
Terraform
Data Pipelines
ServiceNow
Databricks

Job description

  • Architect and manage Databricks environments, Unity Catalog, and Delta Lake
  • Design and optimize data pipelines, workflows, and automation frameworks
  • Monitor DBU usage, forecasting, and cost optimization strategies
  • Support Unix/Linux administration, shell scripting, and platform operations
  • Collaborate with Data Engineering, Analytics, Infrastructure, and Security teams
  • Implement governance, monitoring, HA/DR, and compliance best practices

Requirements

Seeking a senior Databricks Architect/Admin with strong expertise in Databricks platform administration, automation, pipelines, and cost optimization. This role is more consultative and will support an established Databricks team with platform architecture, forecasting, automation, and operational improvements., * 7+ years in Data Engineering/Data Platforms with 4+ years of Databricks experience

  • Expertise in Databricks, PySpark, Python, Delta Lake, DLT, and SQL Warehouses
  • Strong Unix/Linux administration and Bash scripting experience
  • Experience with AWS/Azure/GCP cloud platforms
  • Knowledge of MLflow, AutoML, AI/ML concepts, and automation frameworks
  • Experience with Oracle, Git, Jira, ServiceNow, and ETL tools like Fivetran/Ab Initio

Preferred:

  • Experience with Generative AI/LLM workflows
  • Terraform/Ansible and enterprise orchestration tools
  • Insurance or financial domain experience.

Apply for this position