Azure Data Engineer_ Remote

Senior Data Engineer to join their Data Group, responsible for designing, developing, and optimizing enterprise-scale data ingestion pipelines, integration frameworks, and storage solutions using Databricks across cloud and hybrid environments. This is a hands-on technical leadership role focused on driving data-driven decision making, ensuring secure and efficient data ingestion, and enforcing governance, metadata, and lineage standards.

This individual will lead architecture and development of ETL/ELT pipelines using Databricks (Spark, Delta Lake, workflows), implement Lakehouse design patterns across bronze/silver/gold layers, and support both batch and near real-time data processing. The role also includes designing integration patterns (API, event-driven, system-to-system), building data quality checks, and ensuring compliance with enterprise security and governance standards.

Additional responsibilities include optimizing Databricks performance (jobs, clusters, workloads), designing cloud and on-prem storage solutions, and partnering with platform operations and IT teams to ensure reliability, SLA adherence, and production readiness. This person will also act as a technical mentor, establish engineering standards, and collaborate with architects, analytics teams, and business stakeholders.

Required experience includes 8-10+ years in data engineering, strong expertise in Databricks, Apache Spark, Delta Lake, Python, and SQL, and experience building enterprise-scale pipelines, integrating data via APIs and event platforms, and working within cloud or hybrid environments. Candidates should also have a strong understanding of data governance, metadata, lineage, and enterprise data platforms.

Preferred experience includes Databricks Unity Catalog, Lakehouse architecture, monitoring/alerting for data pipelines, and prior leadership or mentorship experience.