platform engineer for financial data platforms

Описание

Zurich Insurance Group is a global insurance company. Its Technology Delivery Center in Barcelona provides technological solutions and support to local and global business units, covering software development, data analytics, cybersecurity, and IT infrastructure.

Задачи

• Own and evolve the Databricks platform supporting the Zurich Global Finance Data Hub, ensuring scalability, reliability, security, and cost efficiency;
• Design, build, and maintain batch, streaming, API, and CDC data ingestion frameworks across multiple source systems;
• Develop and manage Delta Live Tables pipelines with data quality controls, expectations, schema enforcement, and change data capture;
• Administer and optimise Delta Lake environments, including performance tuning, schema evolution, time travel, VACUUM, and OPTIMIZE operations;
• Implement and manage Unity Catalog governance practices, including access controls, data lineage, data classification, and namespace management;
• Build, deploy, and support Databricks Workflows for complex multi-task pipeline orchestration, dependency management, retries, and event-driven processing;
• Establish and maintain CI/CD pipelines using Git-based development practices and Databricks Asset Bundles across development, testing, and production environments;
• Collaborate with cloud and infrastructure teams to implement Infrastructure as Code solutions using AWS CloudFormation or CDK;
• Manage authentication and authorisation mechanisms, including AWS IAM roles, instance profiles, service principals, secret management, and credential rotation;
• Monitor platform health, performance, operational metrics, and costs using Databricks system tables, CloudWatch, Grafana, and related observability tools;
• Define and enforce cluster policies, compute strategies, and DBU governance controls to optimise platform utilisation and expenditure;
• Ensure platform security through PrivateLink, VPC endpoints, row-level security, and column-level security controls;
• Partner with data engineering, finance, architecture, and product teams to deliver a robust and scalable enterprise data platform;
• Drive platform best practices, technical standards, automation, and continuous improvement across the data ecosystem.

Требования

• 5-8 Years of experience in Data Engineering, Data Platform Engineering, or Cloud Data Architecture;
• At least 3 years of hands-on experience owning and managing Databricks platforms in enterprise environments;
• Strong expertise in Databricks services, including Unity Catalog, Delta Lake, Delta Live Tables, Workflows, and Asset Bundles;
• Proven experience designing and supporting batch, streaming, API, and CDC data ingestion frameworks;
• Advanced proficiency in PySpark, Python, and Databricks SQL for scalable data pipeline development;
• Solid knowledge of DevOps practices, including Git-based development, CI/CD pipelines, automated testing, and Infrastructure as Code;
• Strong AWS experience with S3, IAM, networking, security controls, and secrets management;
• Experience with large-scale data platforms in regulated industries;
• Nice to have: Financial services experience, Databricks and/or AWS certifications, Airflow, Kafka/MSK, enterprise data governance frameworks.

Условия

• Flexible working model;
• Option to work abroad for up to 25 days per year;
• Over 300 euros for home office setup and an additional monthly home office allowance;
• Internal and external training, including English, German, and Spanish classes depending on business needs;
• Ticket restaurant and health insurance with the option to exchange them for other benefits;
• Life and accident insurance;
• Collective life retirement plan;
• 2000 Referral bonus for referring talented employees;
• Special banking and insurance conditions and exclusive employee discounts;
• Functional diversity benefits;
• Stock options and mortgage benefits;
• Work location: Barcelona, Poblenou.

• 5-8 Years of experience in Data Engineering, Data Platform Engineering, or Cloud Data Architecture;
• At least 3 years of hands-on experience owning and managing Databricks platforms in enterprise environments;
• Strong expertise in Databricks services, including Unity Catalog, Delta Lake, Delta Live Tables, Workflows, and Asset Bundles;
• Proven experience designing and supporting batch, streaming, API, and CDC data ingestion frameworks;
• Advanced proficiency in PySpark, Python, and Databricks SQL for scalable data pipeline development;
• Solid knowledge of DevOps practices, including Git-based development, CI/CD pipelines, automated testing, and Infrastructure as Code;
• Strong AWS experience with S3, IAM, networking, security controls, and secrets management;
• Experience with large-scale data platforms in regulated industries;
• Nice to have: Financial services experience, Databricks and/or AWS certifications, Airflow, Kafka/MSK, enterprise data governance frameworks.

• Option to work abroad for up to 25 days per year;
• Over 300 euros for home office setup and an additional monthly home office allowance;
• Internal and external training, including English, German, and Spanish classes depending on business needs;
• Ticket restaurant and health insurance with the option to exchange them for other benefits;
• Life and accident insurance;
• Collective life retirement plan;
• 2000 Referral bonus for referring talented employees;
• Special banking and insurance conditions and exclusive employee discounts;
• Functional diversity benefits;
• Stock options and mortgage benefits;
• Work location: Barcelona, Poblenou.

Zurich Insurance Group is a global insurance company. Its Technology Delivery Center in Barcelona provides technological solutions and support to local and global business units, covering software development, data analytics, cybersecurity, and IT infrastructure.