Data Bricks Architect
- Discuss this with your agent
- Open in Claude
- Open in ChatGPT
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Role details
Tech stack
+38 more
Job description
TIXY is seeking an experienced Databricks Architect with specialized hands-on expertise in Databricks Genie AI / Genie Agents to lead the architecture, design, and AI modernization of our enterprise data platform in St. Louis, MO. In this role, you will lead the implementation of modern multi-cloud Lakehouse solutions (Azure/AWS), drive enterprise data governance using Unity Catalog, and pioneer AI-ready data capabilities. You will enable natural-language analytics across enterprise datasets by deploying Databricks Genie and integrating Agentic AI capabilities (LangChain, LangGraph, LLM integration) directly into modern data workflows., * Architecture & Lakehouse Design: Design and execute scalable, resilient Medallion Architecture (Bronze/Silver/Gold) on Azure Databricks / AWS Databricks utilizing Delta Lake and Delta Live Tables (DLT).
- Genie AI & Agentic AI Modernization: Architect and deploy Databricks Genie and Genie Agents to enable natural-language data analysis, automated querying, and self-service BI for business stakeholders. Integrate LLM tools, RAG architectures, and AI frameworks (LangChain, LangGraph, FAISS/vector stores) into data pipelines.
- Pipeline Engineering & Performance Tuning: Architect PySpark/Spark SQL streaming and batch pipelines (Auto Loader, Structured Streaming, CDC). Perform advanced Spark UI diagnostics, query optimization, Z-Ordering, partitioning, and table maintenance on large-scale datasets.
- Data Governance & Security: Architect fine-grained data security using Unity Catalog (catalog/schema hierarchies, row-level security, column masking, dynamic data masking, dynamic lineage, and ABAC). Support integrations with external governance tools such as Alation and Microsoft Purview.
- Infrastructure as Code & DevOps: Design automated deployment workflows using Terraform (Databricks provider), remote state locking, and CI/CD pipelines (Azure DevOps / GitHub Actions) across Dev, Staging, and Production environments.
- Cross-Cloud & Enterprise Integration: Lead cross-cloud data flows (ADLS Gen2, AWS S3, Kinesis, Event Hubs, DynamoDB Streams) and ensure seamless orchestration with enterprise schedulers (Airflow / Cloud Composer / Azure Data Factory).
- Technical Leadership & Mentorship: Collaborate with cross-functional business analysis, data engineering, and data science teams to translate business needs into technical Lakehouse architecture blueprints.
Requirements
- Experience: 10+ years of progressive experience in Data Architecture, Enterprise Data Engineering, and Cloud Data Platform design, with at least 5-6+ years of daily, hands-on experience in Databricks and Delta Lake.
- Databricks Genie AI Expertise: Direct hands-on experience setting up, tuning, and modernizing platforms with Databricks Genie and Genie Agents for natural-language analysis.
- Generative AI / Agentic AI: Exposure to Agentic AI workflows, LangChain, RAG architectures, Vector Search, and AI-assisted tools (e.g., GitHub Copilot, Claude API, LLM tool-calling).
-
Core Tech Stack:
- Databricks & Spark: PySpark, Spark SQL, Delta Live Tables (DLT), Auto Loader, Structured Streaming.
- Governance: Unity Catalog, Microsoft Purview, Alation.
- Cloud Platforms: Azure Databricks (ADLS Gen2, ADF) and/or AWS Databricks (S3, Glue, Athena, Kinesis).
- Data Modeling: Medallion Lakehouse, SCD Type 1 & 2, Star/Snowflake schemas, dbt on Databricks.
- IaC & CI/CD: Terraform (Databricks provider), Azure DevOps, GitHub, Docker.
Certifications (Preferred):
- Databricks Certified Data Engineer Professional / Databricks Certified Machine Learning Professional
- Cloud Certifications (Azure Solutions Architect, AWS Certified Solutions Architect, or Azure Data Engineer Associate DP-203)
Education: Bachelorβs degree in computer science, Electrical Engineering, Information Technology, or related quantitative field.
Apply for this position
This job is hosted externally. Click below to view the full posting and apply.
Prepare application
- Draft this with your agent
- Open in Claude
- Open in ChatGPT
Good distractions
Loading talks and stories from around this roleβ¦