Databricks Architect (Resident Solution Architect)

• Design and implement scalable, cloud-native data platforms using Databricks.
• Lead end-to-end Databricks implementations, from architecture and design through development and production deployment.
• Develop and optimize large-scale ETL/ELT pipelines using Databricks, Delta Lake, Structured Streaming, and modern data engineering frameworks.
• Design enterprise data architecture aligned with banking business requirements and technical standards.
• Optimize Apache Spark workloads through partitioning strategies, execution plan analysis, memory management, and performance tuning.
• Implement and manage Databricks Lakehouse capabilities, including Unity Catalog, Workflows, MLflow, and Delta Live Tables.
• Establish DevOps and CI/CD practices for production-grade data engineering solutions.
• Apply MLOps principles to support machine learning lifecycle management, model deployment, and monitoring.
• Troubleshoot complex data platform challenges and recommend scalable, reliable, and cost-efficient architecture solutions.
• Provide technical leadership and architectural guidance to engineering and delivery teams.

We are seeking experienced Databricks Solution Architects with strong banking industry experience and expertise in enterprise data engineering, cloud data platforms, and modern data warehouse solutions. The ideal candidates will have successfully delivered multiple end-to-end Databricks implementations and possess hands-on architecture and development experience. This role requires deep knowledge of the Databricks Lakehouse Platform, Apache Spark, Delta Lake, cloud-native data solutions, and enterprise data platform modernization.

Primary Skills

Databricks Architecture

Databricks Lakehouse Platform

Apache Spark

Data Engineering

Delta Lake

Cloud Data Platforms

Secondary Skills

Structured Streaming

ETL/ELT Pipelines

Unity Catalog

Databricks Workflows

MLflow

Delta Live Tables

MLOps

DevOps and CI/CD

AWS

Microsoft Azure

Google Cloud Platform (Google Cloud Platform), * 12-15+ years of experience in data engineering, data platforms, data analytics, and modern data warehouse solutions.

• 10+ years of consulting and client-facing delivery experience.
• Demonstrated success delivering at least 6-8 end-to-end Databricks implementations.
• Hands-on experience as a Databricks developer, technical lead, or solution architect.
• Strong expertise in cloud-native data platform architecture across AWS, Azure, or Google Cloud Platform.
• Deep hands-on proficiency in at least one major cloud platform.
• Advanced knowledge of Apache Spark, including performance optimization, partitioning, execution plans, memory management, and runtime internals.
• Extensive experience with Databricks, Delta Lake, Structured Streaming, and scalable ETL/ELT pipelines.
• Experience implementing CI/CD and DevOps practices using Azure DevOps, GitHub Actions, GitLab CI/CD, or Jenkins.
• Working knowledge of MLOps, machine learning lifecycle management, model deployment, and monitoring.
• Strong understanding of enterprise data architecture, distributed processing, and performance optimization.
• Relevant experience in the banking or financial services industry.
• Strong problem-solving and solution architecture capabilities., * Databricks Data Engineering Professional certification or an equivalent advanced Databricks certification.
• Completion of recommended Databricks learning paths and coursework.
• Databricks Solutions Architect Champion program participation.
• Experience with advanced Databricks Lakehouse Platform capabilities and enterprise implementations.