Lead Data Engineer

Experteer Overview In this role, you design and operate large-scale data pipelines for the SimpliFi Data Pool, a central Azure-hosted data repository that harmonizes accounting data and powers reporting.You will own end-to-end data flows across Raw Data Vault, Business Data Vault, and CORE layers, collaborating with data modellers, architects, and IT leads.You drive data quality, governance, and lineage, enabling reliable insights and compliant reporting.This is a hands-on, architecture-influencing role that helps scale finance data capabilities across the organization.Compensaciones / Beneficios- Define and enforce data engineering standards and architecture decisions- Design and oversee end-to-end data pipelines: ingestion, transformation, loading across Raw Data Vault, Business Data Vault, UJT, and CORE layers- Own data quality at the pipeline level: define checks, monitor, resolve data issues- Mentor and review work of junior/mid engineers to raise code quality- Collaborate with Data Modellers, MDM/RDM specialists, Data Architects, and IT leads to align technical solutions with business needs- Ensure pipelines meet data governance requirements (lineage, classification, access controls) and support gate processes- Act as technical proxy in planning sessions to size effort, flag dependencies, and unblock delivery cycles- Contribute to architecture documentation and understand data flow patterns across platformsResponsabilidades- Hands-on experience building and operating large-scale batch and streaming pipelines- Proficiency with Spark, Kafka, Airflow; cloud-native equivalents (ADF, Glue, Dataflow)- Experience with ETL/ELT at enterprise scale and modern lakehouse/warehouse platforms (Databricks, Azure Synapse)- Strong SQL and Python; familiarity with Scala or PySpark for distributed workloads- Experience with data quality tools (e.G., Acceldata, Great Expectations, Monte Carlo)- Knowledge of master/reference data flows; SAP MDG or Informatica MDM is a plus- CI/CD for data pipelines (GitHub Actions, Azure DevOps); IaC (Terraform); containerization (Docker/Kubernetes)- Ability to read/contribute to architecture documents and discuss data governance and metadata (IDMC or Purview)Requisitos principales- hybrid work model- bonus scheme- pension- employee shares program- lifelong learning opportunities- flexible working arrangements and healthcare benefits

Hands-on experience building and operating large-scale batch and streaming pipelines

• Proficiency with Spark, Kafka, Airflow; cloud-native equivalents (ADF, Glue, Dataflow)
• Experience with ETL/ELT at enterprise scale and modern lakehouse/warehouse platforms (Databricks, Azure Synapse)
• Strong SQL and Python; familiarity with Scala or PySpark for distributed workloads
• Experience with data quality tools (e.G., Acceldata, Great Expectations, Monte Carlo)
• Knowledge of master/reference data flows; SAP MDG or Informatica MDM is a plus
• CI/CD for data pipelines (GitHub Actions, Azure DevOps); IaC (Terraform); containerization (Docker/Kubernetes)
• Ability to read/contribute to architecture documents and discuss data governance and metadata (IDMC or Purview)Requisitos principales