> Markdown version of [/jobs/ext/3789407-data-engineer-principal](https://www.wearedevelopers.com/jobs/ext/3789407-data-engineer-principal). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer, Principal

- **Company:** Horizontal Talent
- **Location:** Golden Valley, MN, United States
- **Experience:** Expert
- **Salary:** $101,920.0 - $176,800.0
- **Contract:** Temporary contract
- **Skills:** Application Programming Interfaces (APIs), Business Logic, Audit Trail, Automation of Tests, Microsoft Azure, Code Review, Data Deduplication, Information Engineering, Data Governance, Extract Transform Load (ETL), File Systems, Github, Monitoring of Systems, Apache Hive, Python (Programming Language), Microsoft SQL Server, SQL Azure, Oracle (Applications), Power BI, Standard Sql, Azure Data Lake, Reverse Engineering, Salesforce.Com, SQL Databases, Data Streaming, Systems Integration, Enterprise Data Management, Data Processing, Apache Spark, Technical Debt, Change Data Capture, Data Lakes, Pyspark, Collibra, Apache Kafka, Video Streaming, Terraform, Azure Synapse Analytics, Data Pipelines, Dynatrace, Docker, Databricks
- **Published:** October 16, 2026
- **Apply:** https://www.jofdav.com/jobs/60126651-data-engineer-principal

## About the Role

* 7+ years of hands-on data engineering experience, including work on complex enterprise platforms or migration initiatives.
 * Strong experience with Azure Synapse Analytics, including pipelines, Spark, and dedicated SQL pools.
 * Solid understanding of ADLS Gen2, Delta Lake, and modern lakehouse architecture patterns.
 * Hands-on knowledge of Databricks, PySpark, Spark SQL, and Delta Lake capabilities such as MERGE, CDC, schema enforcement, and time travel.
 * Advanced Python and SQL skills for building and supporting ETL/ELT solutions at scale.
 * Deep familiarity with data governance, audit, reconciliation, control frameworks, and data quality validation.
 * Experience working with enterprise data patterns such as SCD Type 0/1/2, soft and hard deletes, and retroactive changes.
 * Ability to analyze existing implementations, determine intent, and make sound re-engineering decisions.
 * Clear communication skills and the ability to work effectively with technical and non-technical stakeholders.

Preferred Skills

 * Experience with Azure Data Lake Storage Gen2 and Oracle GoldenGate for real-time integration.
 * Exposure to Azure Analysis Services and Power BI consumption-layer patterns.
 * Experience implementing governance features in Databricks, including Unity Catalog and audit logging.
 * Familiarity with Azure DevOps, GitHub Actions, Terraform, ARM, or Docker.
 * Knowledge of automated testing approaches for data pipelines, including reconciliation-based validation.
 * Experience integrating sources such as Oracle, SQL Server, Azure SQL Database, file systems, Salesforce, or APIs.
 * Exposure to Azure Purview, monitoring tools such as Dynatrace, or streaming technologies such as Event Hub, Kafka, or Kinesis.
 * Background in regulated or enterprise-scale environments.

Horizontal is committed to building an inclusive workplace where different perspectives, experiences, and approaches are valued. We welcome candidates who share our commitment to diversity, equity, and inclusion and who are excited to contribute in a collaborative, respectful environment.

## Description

We are seeking a hands-on Principal Data Engineer to help re-engineer an enterprise data platform and support the migration of validated workloads from Azure Synapse Analytics to Databricks. This role is ideal for someone who enjoys reverse-engineering complex solutions, strengthening data quality and governance, and ensuring business-critical data flows remain accurate throughout modernization.
Responsibilities

 * Assess and re-engineer existing data platform components across ingestion, transformation, storage, and consumption layers.
 * Review current pipeline logic, data models, orchestration patterns, and governance controls to identify gaps or technical debt.
 * Validate end-to-end data processing behavior, including change data capture, slowly changing dimensions, deduplication, and reconciliation routines.
 * Support the migration of workloads to a Databricks lakehouse environment while preserving business logic and control framework behavior.
 * Build and refine pipelines using modern lakehouse patterns and ensure output consistency during parallel-run and cutover activities.
 * Recreate ingestion, transformation, and orchestration solutions on target platforms with strong auditability and quality checks.
 * Collaborate with business, architecture, and governance partners to confirm rules, controls, and data requirements.
 * Contribute to code reviews, technical mentoring, and planning for legacy component retirement.