> Markdown version of [/jobs/ext/1681055-lead-data-engineer](https://www.wearedevelopers.com/jobs/ext/1681055-lead-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md). --- # Lead Data Engineer - **Company:** Allianz Group - **Location:** Barcelona, Spain - **Contract:** Permanent contract - **Skills:** Airflow, Microsoft Azure, Cloud Engineering, Software Quality, Continuous Integration, Data Cleansing, Information Engineering, Data Governance, Extract Transform Load (ETL), Data Flow Control, Github, Python (Programming Language), Raw Data, Reference Data, Standard Sql, Data Streaming, Apache Spark, Containerization, Pyspark, Kubernetes, SAP MDG, Apache Kafka, Data Management, Terraform, Azure Synapse Analytics, Data Pipelines, Docker, Databricks - **Published:** July 27, 2026 - **Apply:** https://www.buscojobs.com.es/lead-data-engineer-m-f-d-en-barcelona-ID-364750583 ## About the Role Hands-on experience building and operating large-scale batch and streaming pipelines - Proficiency with Spark, Kafka, Airflow; cloud-native equivalents (ADF, Glue, Dataflow) - Experience with ETL/ELT at enterprise scale and modern lakehouse/warehouse platforms (Databricks, Azure Synapse) - Strong SQL and Python; familiarity with Scala or PySpark for distributed workloads - Experience with data quality tools (e.G., Acceldata, Great Expectations, Monte Carlo) - Knowledge of master/reference data flows; SAP MDG or Informatica MDM is a plus - CI/CD for data pipelines (GitHub Actions, Azure DevOps); IaC (Terraform); containerization (Docker/Kubernetes) - Ability to read/contribute to architecture documents and discuss data governance and metadata (IDMC or Purview)Requisitos principales ## Description Experteer Overview In this role, you design and operate large-scale data pipelines for the SimpliFi Data Pool, a central Azure-hosted data repository that harmonizes accounting data and powers reporting.You will own end-to-end data flows across Raw Data Vault, Business Data Vault, and CORE layers, collaborating with data modellers, architects, and IT leads.You drive data quality, governance, and lineage, enabling reliable insights and compliant reporting.This is a hands-on, architecture-influencing role that helps scale finance data capabilities across the organization.Compensaciones / Beneficios- Define and enforce data engineering standards and architecture decisions- Design and oversee end-to-end data pipelines: ingestion, transformation, loading across Raw Data Vault, Business Data Vault, UJT, and CORE layers- Own data quality at the pipeline level: define checks, monitor, resolve data issues- Mentor and review work of junior/mid engineers to raise code quality- Collaborate with Data Modellers, MDM/RDM specialists, Data Architects, and IT leads to align technical solutions with business needs- Ensure pipelines meet data governance requirements (lineage, classification, access controls) and support gate processes- Act as technical proxy in planning sessions to size effort, flag dependencies, and unblock delivery cycles- Contribute to architecture documentation and understand data flow patterns across platformsResponsabilidades- Hands-on experience building and operating large-scale batch and streaming pipelines- Proficiency with Spark, Kafka, Airflow; cloud-native equivalents (ADF, Glue, Dataflow)- Experience with ETL/ELT at enterprise scale and modern lakehouse/warehouse platforms (Databricks, Azure Synapse)- Strong SQL and Python; familiarity with Scala or PySpark for distributed workloads- Experience with data quality tools (e.G., Acceldata, Great Expectations, Monte Carlo)- Knowledge of master/reference data flows; SAP MDG or Informatica MDM is a plus- CI/CD for data pipelines (GitHub Actions, Azure DevOps); IaC (Terraform); containerization (Docker/Kubernetes)- Ability to read/contribute to architecture documents and discuss data governance and metadata (IDMC or Purview)Requisitos principales- hybrid work model- bonus scheme- pension- employee shares program- lifelong learning opportunities- flexible working arrangements and healthcare benefits ## Related Videos - [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works) - [Innovating Developer Tools with AI: Insights from GitHub Next](https://www.wearedevelopers.com/videos/1268-innovating-developer-tools-with-ai-insights-from-github-next) - [Docker Compose: Rediscovered](https://www.wearedevelopers.com/videos/1978-docker-compose-rediscovered) - [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt) - [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment) - [Bringing AI Model Testing and Prompt Management to Your Codebase with GitHub Models](https://www.wearedevelopers.com/videos/1536-bringing-ai-model-testing-and-prompt-management-to-your-codebase-with-github-models) ## Related Articles - [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk) - [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production) - [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story) - [Top-Paying Tech Jobs (with Salaries)](https://www.wearedevelopers.com/magazine/372-top-paying-tech-jobs-with-salaries) - [Data Analyst Salary Germany](https://www.wearedevelopers.com/magazine/277-data-analyst-salary-germany) - [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)