Databricks Engineer

We are seeking an AI Engineer with hands-on experience designing, building, and deploying enterprise Generative AI solutions on Databricks.

The ideal candidate will have expertise in Retrieval-Augmented Generation (RAG), custom AI agents, document intelligence, vector search, evaluation frameworks, prompt engineering, and model serving.

The role involves working closely with business and technical stakeholders to develop scalable AI solutions that support real-world enterprise use cases., * Design and implement Generative AI and Agentic AI solutions.

• Build and optimize RAG architectures and vector search solutions.
• Develop custom AI agents and agent workflows.
• Implement document parsing, information extraction, and document intelligence capabilities.
• Create and maintain evaluation frameworks to assess AI solution quality, accuracy, and reliability.
• Design prompts and optimize prompt engineering strategies.
• Deploy and manage models using Databricks Model Serving and related MLOps capabilities.
• Integrate AI solutions with enterprise data platforms and applications.
• Collaborate with Data Engineers and Full Stack Developers to deliver end-to-end AI-powered products.

Required Skills

AI / GenAI

• Retrieval-Augmented Generation (RAG)
• Custom AI agents / Agentic workflows
• Prompt engineering
• Evaluation frameworks (LLM-as-a-Judge, RAG evaluation, AI testing)
• Information extraction and AI document parsing
• Vector search
• Model serving and inference APIs
• AI governance and quality assessment

Databricks

• Databricks AI/ML Platform
• Databricks Model Serving
• MLflow
• Delta Lake
• Vector Search
• Unity Catalog
• PySpark
• Databricks SQL
• Lakehouse architecture

Technical Stack

• Python
• SQL
• PySpark
• LangChain / LangGraph / Semantic Kernel (preferred)
• Azure and/or AWS
• REST APIs
• Cloud-native AI deployments

• Experience building production-grade AI solutions.
• Experience deploying GenAI applications in regulated or enterprise environments.
• Databricks certifications (Machine Learning, GenAI, Lakehouse Fundamentals).
• Experience with multi-agent systems and enterprise AI architectures.
• Strong understanding of evaluation, observability, and governance of AI solutions.

The strongest match appears to be someone with a blend of:

• Enterprise AI architecture experience.
• Hands-on RAG and vector search implementation.
• Databricks AI/ML platform expertise.
• Custom AI agent development.
• AI evaluation and governance frameworks.
• Production deployment experience on Azure and/or AWS.