Databricks Developer

We are seeking an Application Developer to design, build, and deploy interactive data and AI applications on the Databricks platform using Databricks Apps. This role sits at the intersection of application engineering and the Lakehouse - building user-facing tools that put governed data, ML models, and GenAI capabilities directly into the hands of business users.

We are looking for a modern, AI-native developer: someone with strong full-stack fundamentals (Python backend focus) who leverages AI coding assistants such as Claude, GitHub Copilot, and Databricks Assistant to accelerate delivery - and who has the engineering judgment to critically review, test, and secure AI-generated code. Framework-specific experience matters less than fundamentals; a strong engineer with sound API, authentication, and data-access instincts can ramp quickly on any supported framework.

Key Responsibilities

Application Development

Design and develop production-grade applications on Databricks Apps using Python frameworks such as Streamlit, Dash, Gradio, Flask, or FastAPI (or React/Node.js where appropriate)

Build intuitive, responsive UIs for data exploration, ML model interaction, GenAI/agentic workflows, and operational dashboards

Integrate applications with lakehouse assets - Delta tables, Unity Catalog-governed data, SQL Warehouses, Model Serving endpoints, and Vector Search indexes

Implement app-level state management, caching, and performance optimization for concurrent multi-user workloads

Platform Integration & Data Access

Develop against Databricks REST APIs and SDKs (Python SDK, Databricks Connect) for jobs, serving endpoints, and workspace resources

Build secure data access patterns using service principals, on-behalf-of-user authorization, and Unity Catalog permissions

Connect applications to backend services including Lakebase (Postgres), SQL Warehouses, and external APIs where required

Security, Governance & Deployment

Apply enterprise security standards: OAuth/SSO integration, secret management via Azure Key Vault-backed secret scopes, and least-privilege access design

Manage application lifecycle across development, staging, and production environments using CI/CD (Databricks Asset Bundles, Azure DevOps, or GitHub Actions)

Ensure applications comply with data governance, audit, and access-control policies defined in Unity Catalog

AI-Accelerated Engineering

Use AI coding assistants (Claude, GitHub Copilot, Databricks Assistant) as a core part of the development workflow to accelerate design, implementation, testing, and documentation

Apply rigorous review to AI-generated code: validate security posture, data-access boundaries, error handling, and performance before promotion

Develop precise, well-structured specifications and prompts that translate business requirements into working application components

Champion responsible AI-assisted development practices across the team, including standards for testing and validating generated code

Collaboration & Support

Partner with Data Engineering, Data Science, and BI teams to expose pipelines, models, and analytics through application interfaces

Gather requirements from business stakeholders and translate them into functional application designs

Provide production support, monitoring, and iterative enhancement of deployed applications

Bachelor's degree in Computer Science, Engineering, or a related field

4+ years of software/application development experience, including full-stack development with a Python backend focus (APIs, authentication, databases, web application fundamentals)

Demonstrated experience using AI coding assistants (Claude, GitHub Copilot, Databricks Assistant, or similar) in a professional development workflow, with the judgment to critically review, test, and secure AI-generated code

Strong grasp of REST API design, request/response lifecycles, and authentication/authorization patterns (OAuth 2.0, service principals, RBAC)

Strong SQL skills and experience working with relational databases and/or Delta Lake / lakehouse architectures

Working experience with Databricks (Azure Databricks preferred): notebooks, SQL Warehouses, Unity Catalog, Model Serving

Experience with Git-based workflows and CI/CD pipelines

Preferred Qualifications

Direct experience building and deploying Databricks Apps in production

Hands-on experience with one or more supported frameworks: Streamlit, Dash, Gradio, Flask, FastAPI, or React/Node.js (specific frameworks are learnable; fundamentals are required)

Experience integrating GenAI capabilities - LLM endpoints, RAG patterns, Vector Search, or agent frameworks (LangGraph, LangChain)

Familiarity with Lakebase or Postgres-backed application state

Experience with Azure services: Key Vault, Entra ID, Azure Data Factory, Azure DevOps

Databricks certification (Data Engineer Associate, Generative AI Engineer Associate, or similar)

Experience in regulated or engineering-driven industries (maritime, energy, manufacturing)