> Markdown version of [/jobs/ext/4633059-data-platform-engineer-principal_999](https://www.wearedevelopers.com/jobs/ext/4633059-data-platform-engineer-principal_999). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Platform Engineer Principal_999

- **Company:** Allianz
- **Location:** Edina, MN, United States (Remote available)
- **Experience:** Experienced
- **Salary:** $130,000 - $170,000
- **Contract:** Permanent contract
- **Skills:** Unity 3d, Query Performance, Audit Trail, Microsoft Azure, Bash Shell, Cloud Computing, Cluster Analysis, Data Architecture, Information Engineering, Data Infrastructure, Data Sharing, Disaster Recovery, Distributed Systems, Github, Identity and Access Management, Python (Programming Language), Key Management, Network Security, Log Analysis, OAuth, Performance Tuning, Windows PowerShell, Query Optimization, Role-Based Access Control, Reliability Engineering, Azure Data Lake, Security Assertion Markup Language (SAML), Security Information and Event Management, Datadog, Azure Data Factory, Cloud Monitoring, Snowflake, Apache Spark, Caching, Data Lakes, Data Lineage, Data Management, Database Replication, Drift Detection, Terraform, Api Management, Cisco, Databricks
- **Published:** November 8, 2026
- **Apply:** https://www.careerbuilder.com/job-details/data-platform-engineer-principal-999-minneapolis-mn--45f93a1f-4200-4816-b633-da2b0d853351

## About the Role

Topology, Cisco Unity, Multitasking, Injections, Microsoft Windows Azure, Cleanroom, Performance Tuning/Optimization, Warehousing, Housekeeping/Cleaning, Cost Modeling, * 5+ years in cloud infrastructure/platform engineering, with 3+ years focused on data platforms

 * Deep hands-on expertise with Databricks: Unity Catalog, Delta Lake internals, cluster/job architecture, performance tuning at scale
 * Advanced Microsoft Azure engineering experience: networking (VNets, Private Link, ExpressRoute), IAM/Entra ID, ADLS Gen2, Azure Data Factory/Synapse
 * Advanced Snowflake engineering experience: warehouse/query optimization, RBAC design, data sharing, Snowpipe/Streams/Tasks
 * Strong Infrastructure-as-Code expertise: Terraform (module authoring, state management, provider-level configuration for Databricks/Azure/Snowflake)
 * Strong scripting/automation: Python (API integrations, automation frameworks), PowerShell, Bash
 * Solid understanding of distributed systems, query engines (Spark internals), and data lake/lakehouse architecture patterns
 * Experience designing for compliance in regulated environments (DORA, GDPR, data residency/lineage requirements)
 * Proven incident response and reliability engineering experience (SLO design, postmortems, DR testing)

Preferred Qualifications

 * Advanced certifications: Databricks Certified Data Engineer Professional, SnowPro Advanced Architect, Azure Solutions Architect Expert
 * Experience in financial services/insurance with hands-on DORA/GDPR implementation
 * Experience with policy-as-code tooling (OPA, Sentinel, Checkov) and GitOps workflows
 * Familiarity with data lineage/catalog tools (Unity Catalog, Purview) and observability platforms (Azure Monitor, Datadog)
 * Experience building internal developer platforms or self-service data infrastructure tooling

## Description

* Design and implement Databricks architecture: workspace topology, Unity Catalog metastore design, cluster policies, compute pools, Delta Lake optimization (Z-ordering, OPTIMIZE/VACUUM scheduling, liquid clustering), and job orchestration strategy (Workflows, multi-task jobs, Delta Live Tables)
 * Architect Azure data landing zones: subscription/resource group hierarchy, hub-spoke networking, private endpoints, VNet integration for Databricks (VNet injection/secure cluster connectivity), ADLS Gen2 hierarchical namespace design, and Azure Data Factory/Synapse pipeline orchestration
 * Design Snowflake account-level architecture: multi-cluster warehouse strategy, resource monitor hierarchies, query performance tuning, data sharing/clean rooms, Snowpipe/Streams/Tasks for ingestion, and cost attribution models across business units
 * Define reference architectures and golden patterns for new data platform use cases; act as technical authority in architecture review boards

Access Management & Security Engineering

 * Design fine-grained authorization models: Unity Catalog row/column-level security and dynamic views, Snowflake RBAC hierarchies with functional/access roles separation, Azure RBAC + PIM for just-in-time privileged access
 * Engineer identity federation across platforms: Entra ID SCIM provisioning into Databricks/Snowflake, OAuth/SAML SSO, service principal and managed identity lifecycle management, secrets management (Azure Key Vault, Databricks secret scopes)
 * Lead security hardening: private connectivity (Private Link/Service Endpoints), network security group design, customer-managed encryption keys (CMK), audit log pipeline design (Unity Catalog audit logs, Snowflake ACCOUNT_USAGE, Azure Activity Logs) feeding into SIEM
 * Own technical response to security findings and support DORA/GDPR compliance evidence generation, including data residency and data lineage requirements

Cost & Performance Engineering

 * Build cost observability tooling/dashboards correlating Azure cost management, Databricks DBU consumption, and Snowflake credit usage against business units and workloads
 * Engineer compute efficiency: cluster autoscaling policies, spot/low-priority instance strategies, Photon acceleration adoption, warehouse auto-suspend tuning, query profiling and optimization (pruning, caching, clustering keys)
 * Implement FinOps automation: policy-as-code guardrails (cluster size limits, idle timeout enforcement), chargeback/showback models, anomaly detection on spend

Reliability Engineering & Operations

 * Define and operate SLOs/SLIs for platform availability and job reliability; build observability stacks (metrics, logs, traces) using Azure Monitor, Log Analytics, and Databricks system tables
 * Design disaster recovery architecture: cross-region replication for Delta Lake/ADLS, Snowflake database replication and failover groups, RTO/RPO-driven DR runbooks and tested failover procedures
 * Build CI/CD pipelines for infrastructure and platform configuration using Terraform (modules for Databricks/Azure/Snowflake providers), GitHub Actions/Azure DevOps, with automated drift detection and policy validation (OPA/Conftest, Checkov)
 * Lead incident response for Sev1/2 platform incidents; drive blameless postmortems and reliability improvements

Technical Leadership & Governance

 * Mentor junior/mid-level cloud administrators and data engineers on platform best practices
 * Define data platform engineering standards, IaC module libraries, and reusable automation frameworks
 * Partner with data engineering, security, and enterprise architecture to shape platform roadmap
 * Represent the data platform team in architecture governance and change advisory processes