Azure Databricks Architect

HMG America

Seattle, WA, United States

4 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

5 years minimum

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Artificial Intelligence Microsoft Azure Cloud Computing Code Review Continuous Integration Data Architecture Data Governance Extract Transform Load (ETL) Data Security Data Systems Disaster Recovery Apache Hive

+29 more

Python (Programming Language) Performance Tuning Role-Based Access Control Power BI Azure Data Lake SQL Databases Data Streaming Azure Service Bus Data Logging Azure Data Factory Apache Spark Caching Git Microsoft Fabric Data Lakes Pyspark Infrastructure Automation Frameworks Information Technology Data Lineage Bicep Apache Kafka Data Management Machine Learning Operations Video Streaming Software Coding Terraform Azure Synapse Analytics Data Pipelines Databricks

Job description

We are seeking an experienced Azure Databricks Architect to lead the design, implementation, and optimization of enterprise-scale data platforms on Azure. The ideal candidate will have deep expertise in Azure Databricks, Spark, Delta Lake, and modern Lakehouse architecture, with strong experience designing secure, scalable, and high-performance data solutions. Knowledge of Unity Catalog, Medallion Architecture, and Infrastructure as Code is highly preferred. These are all considered current best practices for enterprise Azure Databricks environments., * Design and architect enterprise-scale Azure Lakehouse solutions.

Define end-to-end data architecture, ingestion, transformation, and serving layers.
Lead Azure Databricks implementation using best practices.
Design secure data governance using Unity Catalog and role-based access controls.
Optimize Spark workloads for performance and cost.
Implement reusable frameworks, coding standards, and CI/CD pipelines.
Work closely with business, engineering, and cloud infrastructure teams.
Provide architectural guidance, code reviews, and technical leadership.
Ensure scalability, reliability, security, and disaster recovery across the platform.

Requirements

12+ years of IT experience with 5+ years as a Data/Cloud Architect.
Strong hands-on experience with Azure Databricks.
Expertise in PySpark, Spark SQL, Python, and SQL.
Strong knowledge of Delta Lake, Lakehouse Architecture, and Medallion (Bronze/Silver/Gold) Architecture.
Experience with Unity Catalog, data governance, RBAC, and data lineage.
Experience designing scalable ETL/ELT pipelines.
Hands-on experience with Azure Data Factory (ADF).
Strong experience with Azure Data Lake Storage (ADLS Gen2).
Experience with Azure Synapse Analytics or Microsoft Fabric is a plus.
Experience with CI/CD using Azure DevOps, Git, and Infrastructure as Code (Terraform/Bicep/ARM).
Strong understanding of performance tuning, cluster optimization, partitioning, caching, and cost optimization.
Experience implementing data quality, monitoring, logging, and observability.
Knowledge of streaming technologies (Structured Streaming/Event Hubs/Kafka) is preferred.
Excellent communication and stakeholder management skills., * Databricks Certified Professional/Architect certification.
Azure Solutions Architect or Azure Data Engineer certification.
Experience with MLflow, MLOps, or AI/GenAI workloads on Databricks.
Experience with Power BI integration.