Azure Databricks Architect - Remote with Travel to NJ

The Dignify Solutions LLC
Jersey City, NJ, United States
2 days ago
Apply on www.careerbuilder.com
Prepare application

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours

Tech stack

Unity 3d Data Analysis Business Logic Microsoft Azure Continuous Delivery Continuous Integration Information Engineering Software Debugging DevOps Dimensional Modeling High-Level Architecture Python (Programming Language)
+16 more
Metadata SQL Azure Performance Tuning Windows PowerShell Role-Based Access Control Power BI SQL Databases Scripting Azure Data Factory Apache Spark Microsoft Fabric Data Lakes Pyspark Data Management Terraform Databricks

Job description

Data Engineering Execution

  • Build and maintain ingestion frameworks (ADF / Databricks / Spark)
  • Implement Bronze -> Silver transformations aligned to architecture
  • Architect data quality checks, schema validation, and contract rules
  • Optimize pipelines for performance and cost

DevOps & Automation

  • Own CI/CD pipelines (Azure DevOps)
  • Automate onboarding (repos, pipelines, policies, templates)
  • Maintain IaC and scripting (Terraform, PowerShell / Python)
  • Troubleshoot pipeline failures and permission issues

Data Modeling - Design and implement curated data models in Gold layer aligned to business use cases Build:

  • Dimensional models (star/snowflake schemas)
  • Fact tables, dimensions, surrogate keys, SCD handling

Translate Silver datasets into:

  • Analytics-ready models
  • Consistent KPI definitions and business logic

Ensure:

  • Consistency across domains (common dimensions, conformed models)
  • Reusability and scalability of models
  • Alignment with enterprise semantic layer (Power BI / Fabric / downstream tools)

Handle changes such as:

  • Column derivation logic changes (not just schema changes)
  • Backward compatibility and impact analysis

Partner with business / data stewards to:

  • Define business rules
  • Validate metrics and edge cases
  • Support regression scenarios

Governance & Data Quality Implementation Implement governance patterns defined by architecture

Enforce:

  • Unity Catalog standards
  • naming conventions, metadata policies
  • access controls (RBAC/ABAC)

Architecture Translation Automation first thinking and design approach. Convert high-level architecture into:

  • low-level designs
  • reusable templates
  • engineering playbooks

Ensure consistency across projects and teams

Team Enablement

  • Support delivery teams with:
  • onboarding
  • debugging
  • design reviews
  • Provide hands-on guidance
  • Create reusable assets to reduce repeat questions

Requirements

Microsoft Windows Azure, Willing to Travel, Data Quality, Validation Testing, Performance Tuning/Optimization, Microsoft Windows Azure, DevOps, Continuous Deployment/Delivery, Continuous Integration, Onboarding, Core Platform

  • Azure Data Platform (ADF, ADLS, Azure SQL)
  • Databricks (DLT, Delta Lake, Spark, Workflows, Unity Catalog)
  • Microsoft Fabric (preferred)

Engineering

  • PySpark / Python (advanced)
  • SQL (strong)
  • Pipeline design (batch + streaming)

DevOps / Automation

  • Azure DevOps (pipelines, repos, policies
  • PowerShell automation (must-have for your setup)
  • CI/CD + Infrastructure-as-Code concepts

Governance / Data Quality

  • Schema enforcement, data contracts, DQ frameworks
  • Metadata-driven design understanding

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.careerbuilder.com
Prepare application

Good distractions

Loading talks and stories from around this role…