Data Architect (Azure + Databricks)

CORE4 TECHNOLOGIES INCORPORATED
Milpitas, CA, United States
20 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Expert
Experience required
8 years minimum
Working hours
Regular working hours
Job source

Tech stack

Amazon Web Services Microsoft Azure Batch Processing Code Review Computer Programming Continuous Integration Data Architecture Information Engineering Data Governance Data Infrastructure Data Vault Modeling DevOps
+26 more
Dimensional Modeling Python (Programming Language) Machine Learning Meta-Data Management Azure Data Lake Scala (Programming Language) SQL Databases Data Streaming Azure Service Bus Azure Data Factory Apache Spark Multi-Cloud Data Lakes Pyspark Infrastructure Automation Frameworks Collibra Apache Kafka Data Management Machine Learning Operations Data Lakehouse Cloud Migration Cloud Optimization Terraform Azure Synapse Analytics Data Pipelines Databricks

Job description

We are hiring an experienced Data Architect with deep expertise in Azure, Databricks, and Apache Spark to join a high-impact enterprise data transformation initiative. This is a hands-on architecture role where you’ll be responsible for designing scalable data platforms while actively building, coding, and optimizing production-grade solutions., * Design scalable, secure, and cost-efficient enterprise data architectures on Azure and Databricks.

  • Build and implement modern data lakehouse solutions using Delta Lake and Medallion Architecture.
  • Develop production-grade data pipelines using PySpark, Scala, SQL, and Apache Spark.
  • Optimize Databricks clusters, Spark workloads, and platform performance.
  • Design streaming and batch processing solutions using Event Hubs, Kafka, and Structured Streaming.
  • Implement data governance, security, and metadata management using Unity Catalog and Azure Purview.
  • Conduct architecture reviews, code reviews, and mentor engineering teams.
  • Collaborate with business and technical stakeholders to translate requirements into scalable solutions.
  • Drive best practices around DevOps, Infrastructure as Code, CI/CD, testing, and observability.

Requirements

If you’re an architect who enjoys coding or a senior data engineer ready to lead architecture, we’d love to hear from you., * 8+ years of Data Engineering/Data Architecture experience.

  • Expert-level experience with Azure Data Services:
  • Azure Data Lake Storage (ADLS)
  • Azure Data Factory (ADF)
  • Azure Synapse Analytics
  • Azure Event Hubs
  • Azure Purview
  • Strong expertise in Databricks, including:
  • Delta Lake
  • Unity Catalog
  • Medallion Architecture
  • Cluster Optimization
  • Advanced Apache Spark performance tuning and optimization.
  • Strong coding experience in PySpark, Scala, Python, and advanced SQL.
  • Experience designing enterprise-scale data platforms and modern data lakehouse architectures.
  • Strong understanding of dimensional modeling, normalized models, and Data Vault.
  • Experience with Infrastructure as Code (Terraform/ARM), CI/CD, and DevOps practices.

Preferred Qualifications

  • Azure Solutions Architect Expert Certification.
  • Databricks Certified Data Engineer Professional or Solutions Architect.
  • Experience with MLOps and Machine Learning platforms.
  • Cloud migration and modernization experience.
  • FinOps and cloud cost optimization knowledge.
  • Multi-cloud exposure (AWS/GCP) is a plus.
  • Consulting or advisory experience., * Bachelor’s degree in Computer Science, Engineering, Mathematics, or a related field.
  • Master’s degree preferred.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.indeed.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

2:17 min

Mapping the maturity roadmap for scaled devops adoption

Dominik Krichbaum Dominik Krichbaum · WWC Europe 2026

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

3:18 min

Scaling global network engineering through DevOps culture

Stuart Clark · LIVE

3:38 min

Architecting a unified data and machine learning workbench

Kapil Gupta Kapil Gupta · WWC 2025

3:37 min

Scaling machine learning pipelines from prototypes to petabytes

Julian Joseph · LIVE

Videos

See all

Related articles

See all