Databricks Platform Engineer

OverviewIn this role you will own and evolve Zurich’s Databricks platform to support the Global Finance Data Hub, ensuring scalable, secure, and cost-conscious data solutions.You will build and maintain ingestion and processing pipelines, govern data with Unity Catalog, and implement CI/CD and IaC practices in a collaborative, tech-driven environment.You’ll work with cross-functional teams to deliver a robust enterprise data platform that accelerates Zurich’s data strategy and operational efficiency.This is a chance to shape a world-class data foundation within a global insurer.Compensaciones / Beneficiosflexible working modelabroad up to 25 days yearlyhome office allowancewide trainings including language classeshealth insurancelife and retirement plansResponsabilidadesOwn and evolve the Databricks platform underpinning the Global Finance Data Hub with focus on scalability, reliability, security, and cost efficiencyDesign, build, and maintain data ingestion frameworks for batch, streaming, API, and CDC across multiple sourcesDevelop and manage Delta Live Tables pipelines with data quality, schema enforcement, and CDCAdminister and optimize Delta Lake environments (performance, schema evolution, time travel, VACUUM, OPTIMIZE)Implement Unity Catalog governance (access controls, lineage, classification, namespaces)Build, deploy, and support Databricks Workflows for complex pipelines and event-driven processingEstablish CI/CD pipelines using Git and Databricks Asset Bundles across dev/test/prodCollaborate with cloud/infrastructure teams on IaC using CloudFormation or CDKManage authentication/authorization (IAM roles, profiles, service principals, secrets)Monitor platform health and costs with Databricks system tables, CloudWatch, GrafanaDefine cluster policies, compute strategies, and DBU governance for optimizationEnsure security (PrivateLink, VPC endpoints, row/column security)Partner with data engineering, finance, architecture, and product teams to deliver scalable data platformDrive best practices, automation, and continuous improvement across the data ecosystemRequisitos principales5-8 years in Data Engineering/Platform Engineering/Cloud Data ArchitectureMinimum 3 years owning and managing Databricks platforms in enterprise environmentsStrong expertise in Databricks services (Unity Catalog, Delta Lake, Delta Live Tables, Workflows, Asset Bundles)Experience designing/supporting batch, streaming, API, and CDC data ingestion frameworksAdvanced PySpark, Python, and Databricks SQL for scalable pipelinesDevOps practices including Git, CI/CD, automated testing, and IaC (CloudFormation or CDK)AWS experience (S3, IAM, networking, security, secrets)Experience with large-scale data platforms in regulated industries; financial services desirablePreferred: Databricks/AWS certifications; Airflow, Kafka/MSK; data governance frameworkscollaborationeffective communicationproblem-solvingDatabricks Unity CatalogDelta LakeDelta Live Tables

Requisitos principales5-8 years in Data Engineering/Platform Engineering/Cloud Data Architecture Minimum 3 years owning and managing Databricks platforms in enterprise environments Strong expertise in Databricks services (Unity Catalog, Delta Lake, Delta Live Tables, Workflows, Asset Bundles) Experience designing/supporting batch, streaming, API, and CDC data ingestion frameworks Advanced PySpark, Python, and Databricks SQL for scalable pipelines DevOps practices including Git, CI/CD, automated testing, and IaC (CloudFormation or CDK) AWS experience (S3, IAM, networking, security, secrets) Experience with large-scale data platforms in regulated industries; financial services desirable Preferred: Databricks/AWS certifications; Airflow, Kafka/MSK; data governance frameworks collaboration effective communication problem-solving