Data Architect (Azure + Databricks)

CORE4 TECHNOLOGIES INCORPORATED

Milpitas, CA, United States

2 months ago

Apply on www.indeed.com

Prepare application

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Experience level

Expert

Experience required

8 years minimum

Working hours

Regular working hours

Job source

www.indeed.com

Tech stack

Amazon Web Services Microsoft Azure Batch Processing Code Review Computer Programming Continuous Integration Data Architecture Information Engineering Data Governance Data Infrastructure Data Vault Modeling DevOps

+26 more

Dimensional Modeling Python (Programming Language) Machine Learning Meta-Data Management Azure Data Lake Scala (Programming Language) SQL Databases Data Streaming Azure Service Bus Azure Data Factory Apache Spark Multi-Cloud Data Lakes Pyspark Infrastructure Automation Frameworks Collibra Apache Kafka Data Management Machine Learning Operations Data Lakehouse Cloud Migration Cloud Optimization Terraform Azure Synapse Analytics Data Pipelines Databricks

Job description

We are hiring an experienced Data Architect with deep expertise in Azure, Databricks, and Apache Spark to join a high-impact enterprise data transformation initiative. This is a hands-on architecture role where you’ll be responsible for designing scalable data platforms while actively building, coding, and optimizing production-grade solutions., * Design scalable, secure, and cost-efficient enterprise data architectures on Azure and Databricks.

Build and implement modern data lakehouse solutions using Delta Lake and Medallion Architecture.
Develop production-grade data pipelines using PySpark, Scala, SQL, and Apache Spark.
Optimize Databricks clusters, Spark workloads, and platform performance.
Design streaming and batch processing solutions using Event Hubs, Kafka, and Structured Streaming.
Implement data governance, security, and metadata management using Unity Catalog and Azure Purview.
Conduct architecture reviews, code reviews, and mentor engineering teams.
Collaborate with business and technical stakeholders to translate requirements into scalable solutions.
Drive best practices around DevOps, Infrastructure as Code, CI/CD, testing, and observability.

Requirements

If you’re an architect who enjoys coding or a senior data engineer ready to lead architecture, we’d love to hear from you., * 8+ years of Data Engineering/Data Architecture experience.

Expert-level experience with Azure Data Services:
Azure Data Lake Storage (ADLS)
Azure Data Factory (ADF)
Azure Synapse Analytics
Azure Event Hubs
Azure Purview
Strong expertise in Databricks, including:
Delta Lake
Unity Catalog
Medallion Architecture
Cluster Optimization
Advanced Apache Spark performance tuning and optimization.
Strong coding experience in PySpark, Scala, Python, and advanced SQL.
Experience designing enterprise-scale data platforms and modern data lakehouse architectures.
Strong understanding of dimensional modeling, normalized models, and Data Vault.
Experience with Infrastructure as Code (Terraform/ARM), CI/CD, and DevOps practices.

Preferred Qualifications

Azure Solutions Architect Expert Certification.
Databricks Certified Data Engineer Professional or Solutions Architect.
Experience with MLOps and Machine Learning platforms.
Cloud migration and modernization experience.
FinOps and cloud cost optimization knowledge.
Multi-cloud exposure (AWS/GCP) is a plus.
Consulting or advisory experience., * Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field.
Master’s degree preferred.