Senior Data Engineer

Experteer Overview In this Senior Data Engineer role, you will shape data solutions within Deloitte’s Project Delivery Model, focusing on scalable data pipelines and cloud-based architectures. You will collaborate across cross-functional teams and client interfaces to drive design, governance, and optimization of data platforms. You’ll tackle high-volume data challenges, mentor engineers, and evaluate new technologies to keep our data capabilities cutting-edge. This position offers meaningful client work with limited travel and the chance to impact mission-critical data transformations. Compensation / Benefits * Coordinate with Engagement Managers and technical teams to align on issues and escalate where needed * Design, develop and optimize ETL/ELT pipelines using Azure Data Factory and Databricks * Author and tune PySpark / Spark SQL transformations for large-scale data * Architect end-to-end data solutions across development, UAT, and production with Unity Catalog * Lead design discussions with client architects and counterparts * Collaborate on data contracts and schema agreements across teams * Design and optimize high-volume data pipelines and implement performance improvements * Define data engineering standards, including naming, partitioning, cluster configurations, and Spark tuning * Develop cluster policies, autoscaling, and cost-optimization strategies * Root cause analysis of production incidents and implement durable fixes * Mentor junior and mid-level engineers through code reviews and pair programming * Evaluate new technologies (e.g., DABs, DLT, Auto Loader, Serverless Compute, event hubs) and recommend adoption Tasks * Python, PySpark, Spark SQL, SQL Server * Azure (ADF, ADLS Gen2, Key Vault, Azure Monitor) * Databricks (Delta Lake, Unity Catalog, Workflows) * Apache Airflow * Git / Azure DevOps * Deep Spark internals (DAG optimization, spill analysis, skew handling) * Delta Lake advanced features (time travel, deletion vectors, predictive I/O) * Unity Catalog governance (row/column security, external locations, system tables) * IaC - Terraform, Azure ARM templates * Bachelor’s degree in a related IT discipline; or equivalent experience * Limited immigration sponsorship may be available * Ability to travel 10% on average Key requirements *

focusing with client architects and counterparts * Collaborate on data contracts and schema agreements across teams * Design and optimize high-volume data pipelines and implement performance improvements * Define data engineering standards, including naming, partitioning, cluster configurations, and Spark tuning * Develop cluster policies, autoscaling, and cost-optimization strategies * Root cause analysis of production incidents and implement durable fixes * Mentor junior and mid-level engineers through code reviews and pair programming * Evaluate new technologies (e.g., DABs, DLT, Auto Loader, Serverless Compute, event hubs) and recommend adoption Tasks * Python, PySpark, Spark SQL, SQL Server * Azure (ADF, ADLS Gen2, Key Vault, Azure Monitor) * Databricks (Delta Lake, Unity Catalog, Workflows) * Apache Airflow * Git / Azure DevOps * Deep Spark internals (DAG optimization, spill analysis, skew handling) * Delta Lake advanced features (time travel, deletion vectors, predictive I/O) H _ Unity Catalog governance (row/column security, external locations, system tables) * IaC - Terraform, Azure ARM templates * Bachelor’s degree in a related IT discipline; or equivalent experience * Limited immigration sponsorship may be available * Ability to travel 10% on average Key requirements *