Data Bricks Architect

UNIFYX LLC

United States

25 days ago

Role details

Contract type

Permanent contract

Employment type

Full-time (> 32 hours)

Working hours

Regular working hours

Job source

www.dice.com

Tech stack

Application Programming Interfaces (APIs) Artificial Intelligence Amazon Web Services Amazon S3 Application Frameworks Continuous Integration Data Architecture Information Engineering Data Governance DevOps Python (Programming Language) Role-Based Access Control

+20 more

Standard Sql Data Streaming Data Processing Google Cloud Azure Data Factory Fast Healthcare Interoperability Resources Apache Spark Electronic Medical Records Git Data Lakes Pyspark Infrastructure Automation Frameworks Data Lineage Deployment Automation Health Level Seven International Software Coding Terraform Stream Processing Data Pipelines Databricks

Job description

Design and implement end-to-end Databricks Lakehouse architectures for healthcare data platforms.
Define data architecture patterns for batch and real-time data processing.
Design ingestion frameworks for healthcare data from EHR/EMR, claims, clinical, pharmacy, lab, HL7, FHIR, APIs, and other sources.
Develop scalable data pipelines using Apache Spark, PySpark, Delta Lake, Delta Live Tables/Lakeflow, and Databricks Workflows.
Design and implement Medallion Architecture (Bronze, Silver, and Gold layers).
Implement data governance, cataloging, lineage, and fine-grained access controls using Unity Catalog.
Ensure architecture and data processing practices support HIPAA, PHI, PII, and other healthcare security and compliance requirements.
Optimize Databricks workloads for performance, scalability, reliability, and cost.
Define data modeling strategies for analytics, reporting, population health, clinical analytics, and AI/ML use cases.
Collaborate with healthcare business stakeholders, data engineers, data scientists, analysts, security, compliance, and enterprise architecture teams.
Establish architecture standards, best practices, coding standards, and reusable frameworks.
Lead technical design reviews and provide mentorship to data engineering teams.
Support CI/CD, DevOps, infrastructure automation, and deployment strategies for Databricks environments.

Requirements

Strong hands-on experience with Databricks Lakehouse Platform.
Expert knowledge of Apache Spark and PySpark.
Strong experience with Delta Lake, Delta Live Tables/Lakeflow, Databricks Workflows, and Databricks SQL.
Experience with Unity Catalog, data governance, data lineage, RBAC, and access control.
Strong SQL and Python skills.
Experience with one or more cloud platforms:
Azure Databricks / ADLS / Azure Data Factory
AWS Databricks / S3 / Glue / Lambda
Google Cloud Platform Databricks / GCS / Pub/Sub
Experience designing batch and streaming data pipelines.
Experience with Git, CI/CD, Terraform, and DevOps practices.