Senior Data Engineer

Allianz Group
Barcelona, Spain
15 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours

Tech stack

Microsoft Azure Cloud Engineering Profiling Code Review Continuous Integration Data Centers Data Governance Data Integrity Extract Transform Load (ETL) Data Vault Modeling Github Python (Programming Language)
+26 more
Network Security Machine Learning Performance Tuning Scrum Methodology Query Optimization Raw Data Reference Data Power BI Elearning Standard Sql SAP (Applications) SQL Databases Data Streaming Data Storage Technologies Apache Spark Pyspark Information Technology Data Lineage SAP MDG Data Management Terraform Azure Synapse Analytics Software Version Control Data Pipelines Serverless Computing Databricks

Job description

Senior Data Engineer (m/f/d) About the Job The SimpliFi Data Pool is a central data repository hosted on Azure, designed to unify OE financial data and enhance cost efficiency.It serves as the primary source of accounting and reporting data, ensuring compliance with mandatory reporting requirements.The Data Pool is a key component of the SimpliFi program, integrated with SAP, providing a harmonized data model with Common Core components, including a globally harmonized Chart of Accounts and standardized accounting and reporting processes.The ecosystem is designed to streamline finance data management and reporting through key components: organized data storage (Staging Area, Raw Data Vault, Business Data Vault), tailored feeder system integrations, SAP connectivity via Azure functions, and a PowerBI-enabled consumption layer for detailed analysis and data lineage transparency.It incorporates robust Master and Reference Data Management (Informatica IDMC), rigorous data quality checks, and the SimpliFi Studio interface for manual uploads and pipeline monitoring, ensuring seamless operations, high data integrity, and informed decision-making.What you do Implement data pipelines in line with established engineering standards - pipeline design patterns, naming conventions, and modelling approaches (e.G. Data Vault).Apply architecture decisions made at Lead level and translate them into working, maintainable solutions.Build and maintain end-to-end data pipelines across ingestion, transformation, and loading.In SimpliFi terms, this means developing and owning flows across Raw Data Vault, Business Data Vault, UJT, and CORE layers - working hands-on with tools such as Databricks, Azure Synapse and IDMC.Responsible for data quality within their pipeline scope - implementing checks, identifying issues early, and resolving them before they propagate downstream.Escalate systemic or cross-domain issues to the Lead.Review code from junior and mid-level engineers, share best practices and contribute to raising team capability.Act as a day-to-day technical reference for less experienced colleagues.Work alongside Data Modellers, MDM/RDM specialists, and IT component leads.Translate technical constraints and findings into clear inputs for planning and design discussions.Ensure pipelines are built in line with data governance requirements - lineage, classification, and access controls.Support gate processes (e.G. DAB/CCP) by providing accurate technical evidence and documentation.Actively participate in sprint planning and delivery cycles - providing effort estimates, flagging technical dependencies, and keeping work moving within their delivery lane.What you bring Hands-on expertise building and operating large-scale pipelines - batch and streaming.Proficiency in tools like Apache Spark, databricks and cloud-native equivalents.Experience with ETL/ELT patterns at enterprise scale.Practical experience with modern lakehouse/warehouse platforms - Databricks, Azure Synapse, or equivalents.Understanding of partitioning, clustering, query optimisation.Deep working knowledge of at least one major cloud Azure - compute, storage, networking, managed services.In enterprise contexts like SimpliFi, Azure is typically dominant (ADLS, ADF, Azure Databricks).Solid grasp of modelling paradigms - Medallion, dimensional (Kimball) and Data Vault 2.0.Ability to review and contribute to models, not just implement them.Know how to implement metadata capture, data lineage (e.G. via IDMC or Purview) and access controls.Experience navigating governance frameworks and gate processes.Strong SQL (complex transformations, performance tuning), Python (pipeline logic, data quality scripts) and familiarity with PySpark for distributed workloads.Experience implementing DQ rules, profiling and monitoring - ideally with tools like DQX, Great Expectations..Understanding of master and reference data flows - how golden records are created, maintained and consumed.Experience with platforms like SAP MDG or Informatica MDM is a strong plus.CI/CD for data pipelines (GitHub Actions), version control discipline and infrastructure-as-code basics (Terraform).Able to read and contribute to architecture documents and understand layers, zones and data flow patterns across a platform.Can engage meaningfully with architects without needing hand-holding.Familiarity with Machine learning is a strong advantage.What we offer We offer a hybrid work model which recognizes the value of striking a balance between in-person collaboration and remote working incl.up to 25 days per year working from abroad.We believe in rewarding performance and our compensation and benefits package includes a company bonus scheme, pension, employee shares program and multiple employee discounts (details vary by location).From career development and digital learning programs to international career mobility, we offer lifelong learning for our employees worldwide and an environment where innovation, delivery and empowerment are fostered.Flexible working, health and wellbeing offers (including healthcare and parental leave benefits) support to balance family and career and help our people return from career breaks with experience that nothing else can teach.About Allianz Technology With its headquarters in Munich, Germany, Allianz Technology is Allianz’s global IT service provider and delivers IT solutions that drive the group’s digitalization.With more than 11,000 employees in over 20 countries around the world, Allianz Technology is tasked to run, optimize, transform and innovate the infrastructure, applications and services together with Allianz companies to co-create the best customer experience.We service the entire spectrum of digitalization - from one of the industry’s largest IT infrastructure projects that spans data centres, networks and security, to application platforms ranging from workplace services to digital interaction.In short: We deliver comprehensive end-to-end IT solutions for Allianz in the digital age.We are the backbone of Allianz.Find us at:.Commitment to Integrity, Fairness & Inclusion Allianz Group is one of the most trusted insurance and asset management companies in the world.Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer.Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us.We at Allianz believe in a strong inclusive culture that encourages people to speak their minds, get involved and question the status quo.We are proud to be an equal opportunity employer and encourage you to bring your whole self to work, no matter where you are from, what you look like, who you love, or what you believe in.We therefore welcome applications regardless of race, ethnicity or cultural background, age, gender, nationality, religion, social class, disability, sexual orientation, or any other characteristics protected under applicable local laws and regulations.To Recruitment Agencies Allianz Technology has an in-house recruitment team that sources great candidates directly.Therefore, Allianz Technology does not accept unsolicited resumes from agencies or search firm recruiters.When we do work with recruitment agencies, that engagement is formalized by a contract.Fees will only be paid when there is a contract in place.Without a contract in place, we will not accept invoices on unsolicited resumes, even if the candidate was ultimately employed by Allianz. IT & Tech Engineering Professional Non-Executive Allianz Technology Full-Time Permanent

Requirements

Practical experience with modern lakehouse/warehouse platforms - Databricks, Azure Synapse, or equivalents. Understanding of partitioning, clustering, query optimisation. Deep working knowledge of at least one major cloud Azure - compute, storage, networking, managed services. In enterprise contexts like SimpliFi, Azure is typically dominant (ADLS, ADF, Azure Databricks). Solid grasp of modelling paradigms - Medallion, dimensional (Kimball) and Data Vault 2.0. Ability to review and contribute to models, not just implement them. Know how to implement metadata capture, data lineage (e.G. via IDMC or Purview) and access controls. Experience navigating governance frameworks and gate processes. Strong SQL (complex transformations, performance tuning), Python (pipeline logic, data quality scripts) and familiarity with PySpark for distributed workloads. Experience implementing DQ rules, profiling and monitoring - ideally with tools like DQX, Great Expectations. . Understanding of master and reference data flows - how golden records are created, maintained and consumed. Experience with platforms like SAP MDG or Informatica MDM is a strong plus. CI/CD for data pipelines (GitHub Actions), version control discipline and infrastructure-as-code basics (Terraform). Able to read and contribute to architecture documents and understand layers, zones and data flow patterns across a platform. Can engage meaningfully with architects without needing hand-holding. Familiarity with Machine learning is a strong advantage.

Benefits & conditions

What we offer We offer a hybrid work model which recognizes the value of striking a balance between in-person collaboration and remote working incl. up to 25 days per year working from abroad. We believe in rewarding performance and our compensation and benefits package includes a company bonus scheme, pension, employee shares program and multiple employee discounts (details vary by location).

About the company

From career development and digital learning programs to international career mobility, we offer lifelong learning for our employees worldwide and an environment where innovation, delivery and empowerment are fostered. Flexible working, health and wellbeing offers (including healthcare and parental leave benefits) support to balance family and career and help our people return from career breaks with experience that nothing else can teach. About Allianz Technology With its headquarters in Munich, Germany, Allianz Technology is Allianz’s global IT service provider and delivers IT solutions that drive the group’s digitalization. With more than 11,000 employees in over 20 countries around the world, Allianz Technology is tasked to run, optimize, transform and innovate the infrastructure, applications and services together with Allianz companies to co-create the best customer experience. We service the entire spectrum of digitalization - from one of the industry’s largest IT infrastructure projects that spans data centres, networks and security, to application platforms ranging from workplace services to digital interaction. In short: We deliver comprehensive end-to-end IT solutions for Allianz in the digital age. We are the backbone of Allianz. Find us at:. Commitment to Integrity, Fairness & Inclusion Allianz Group is one of the most trusted insurance and asset management companies in the world. Caring for our employees, their ambitions, dreams and challenges, is what makes us a unique employer. Together we can build an environment where everyone feels empowered and has the confidence to explore, to grow and to shape a better future for our customers and the world around us. We at Allianz believe in a strong inclusive culture that encourages people to speak their minds, get involved and question the status quo. We are proud to be an equal opportunity employer and encourage you to bring your whole self to work, no matter where you are from, what you look like, who you love, or what you believe in. We therefore welcome applications regardless of race, ethnicity or cultural background, age, gender, nationality, religion, social class, disability, sexual orientation, or any other characteristics protected under applicable local laws and regulations. To Recruitment Agencies Allianz Technology has an in-house recruitment team that sources great candidates directly. Therefore, Allianz Technology does not accept unsolicited resumes from agencies or search firm recruiters. When we do work with recruitment agencies, that engagement is formalized by a contract. Fees will only be paid when there is a contract in place. Without a contract in place, we will not accept invoices on unsolicited resumes, even if the candidate was ultimately employed by Allianz. | IT & Tech Engineering | Professional | Non-Executive | Allianz Technology | Full-Time | Permanent

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.buscojobs.com.es

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

6:36 min

Funding open source through GitHub Accelerator and Sponsors

Stormy Peters · WWC 2023

4:32 min

Harnessing Spark with Python using PySpark and Py4J

Ayon Roy · LIVE

3:03 min

Career evolution in data engineering and AI platforms

Maria Apazoglou · Coffee With Developers

2:40 min

Using GitHub primitives for internal documentation and corporate operations

Kyle Daigle · Coffee With Developers

3:14 min

Structuring career paths and localized data architectures

Ulrich Wurstbauer +1 · LIVE

3:37 min

Scaling machine learning pipelines from prototypes to petabytes

Julian Joseph · LIVE

Videos

See all

Related articles

See all