Databricks Engineer

Insight Global
Birmingham, United States of America
3 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English

Job location

Birmingham, United States of America

Tech stack

Artificial Intelligence
Data analysis
System Configuration
Continuous Integration
Information Engineering
Data Governance
DevOps
Github
Python
SQL Databases
YAML
Multi-Agent Systems
Generative AI
Data Lake
Software Version Control
Databricks

Requirements

The Databricks Engineer / Developer must demonstrate strong hands-on experience with the following core capabilities:

  • Proficiency in SQL, YAML, and Databricks Notebooks for data engineering, analytics, and automation

  • Deep understanding and practical application of Medallion Architecture (Bronze, Silver, Gold patterns)

  • Experience designing, managing, and optimizing Delta Lake tables

  • Strong knowledge of Unity Catalog, including data governance, permissions, lineage, and security best practices

  • Hands-on experience building and maintaining AI/BI Dashboards and Reports within Databricks

  • Experience configuring and enabling Databricks Genie Rooms

  • Practical experience with Natural Language to SQL capabilities and semantic modeling to support business users

  • Strong proficiency in Python for data engineering, analytics, and AI workloads


3.2 Version Control, CI/CD, and Environment Management

The Engineer must have demonstrated experience with modern DevOps practices in Databricks environments, including:

  • Implementing and managing Databricks Asset Bundles using GitHub

  • Applying source control best practices for notebooks, jobs, and configuration

  • Promoting and managing solutions across development, UAT, and production environments

  • Supporting CI/CD pipelines for Databricks workloads, ensuring repeatability and governance · Experience with Databricks Agents / Agent Bricks

  • Designing and implementing agentic workflows for AI-driven automation and decision-making

  • Familiarity with retrieval-augmented generation (RAG) or AI orchestration patterns within Databricks

Apply for this position