Principal Data Engineer

SGS
Municipality of Madrid, Spain
8 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Remote
Municipality of Madrid, Spain

Tech stack

Artificial Intelligence
Airflow
Data analysis
Batch Processing
Google BigQuery
Computer Programming
Information Engineering
ETL
Data Transformation
Data Systems
Python
PostgreSQL
NoSQL
SQL Databases
Data Streaming
Data Processing
Snowflake
Spark
Data Lake
PySpark
Core Data
Apache Flink
Real Time Data
Kafka
Data Management
Data Pipelines
Databricks

Job description

This isnt a standard ETL role. Were looking for a data systems pioneer to build our entire data ecosystem from scratch with the executive backing and autonomy to make it happen. The Vision

Imagine a central data platform that ingests real-time IoT data from industrial inspections to predict equipment failures or unifies decades of lab results to optimise entire supply buildthisanalytics engineand the high-performance real-time data backends for our most critical products. If youre excited by the challenge of turning messy complexreal-worlddata into fast reliable products this is your role.

As a Principal Data Engineer you will own the flow of data across our organization. Your dual mission is to : 1) Engineer a central world-class analytics engine that provides clean trustworthy data for AI and business intelligence. 2) Architect and build the data-intensive backends for our flagship products selecting the right tools to ensure low-latency and high-reliability. What Youll Build and Own

  • Architect Product Data Layers : Design the data models and select the optimal persistence technologies (e.g. PostgreSQL NoSQL Time-Series DBs) for new high-throughput digital products.
  • Build the Core Analytics Engine : Engineer our core data platform using modern tools like dbt Spark and cloud warehouses (Snowflake BigQuery or Databricks) to create a single source of truth.
  • Develop High-Performance Pipelines : Build and operate robust observable data pipelines for both massive batch processing and low-latency real-time streams (e.g. using Kafka Flink).
  • Harvest & Generalize Data Patterns : Identify common data challenges and solutions packaging them into reusable pipelines modules and best practices for other teams to leverage.
  • Champion Data Quality : Implement and promote a strong data quality culture using modern frameworks (e.g. Great Expectations) to ensure our data is always trustworthy.
  • Grow the Foundation : As the first Principal on the team you will play a key role in shaping our technical culture and mentoring future hires as we build out the data engineering function.

Requirements

  • Data Platforms & Warehousing : Deep expertise in modern cloud data platforms like Snowflake BigQuery or Databricks (Delta Lake).
  • Data Processing & Transformation : Expert-level proficiency with Apache Spark (PySpark / Scala) and modern data transformation tools especially dbt.
  • Application Data Architecture : Proven experience designing data models for transactional systems. Hands-on experience with PostgreSQL is essential; experience with NoSQL or Time-Series DBs is a strong plus.
  • Streaming & Orchestration : Hands-on experience with workflow orchestration (Airflow Dagster) and real-time streaming technologies (Kafka Flink).
  • Programming & SQL : Expert-level SQL and strong programming skills in Python or Scala for data engineering.

Who You Are

  • You are a pragmaticdata systems builder with extensive (8 years) of experience.
  • You have a proven track record of turning complex messy data into reliable high-performance products and platforms.
  • You thrive on greenfield challenges and have architected major data systems from the ground up.
  • You are a pragmatist who can balance the needs of large-scale analytics with the low-latency demands of user-facing applications.
  • You are obsessed with data quality and building systems that are both powerful and trustworthy.

Benefits & conditions

  • Top-of-Market Compensation : A highly competitive salary and bonus package for Madrid designed to attract and retain premier talent for this strategic role.
  • Greenfield Ownership & Autonomy : This is not an optimization role. You have a mandate to build from scratch with the freedom to choose the right tools for the job backed by C-level sponsorship.
  • Foundational Impact : You will be the first Principal Data Engineer in our new Digital Hub shaping the technology culture and future of data at a global leader.
  • A Compelling Problem Space : Work on unique tangible data challenges that have a real-world impact on global safety sustainability and supply chains.
  • A Clear Growth Path : This role offers a direct path to technical leadership and the opportunity to build and mentor a team around your architectural vision.

Remote Work : No

Apply for this position