> Markdown version of [/jobs/ext/2957039-mid-data-engineer](https://www.wearedevelopers.com/jobs/ext/2957039-mid-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Mid Data Engineer

- **Company:** Jobtailor
- **Location:** Madrid, Spain
- **Contract:** Permanent contract
- **Skills:** Query Performance, Application Programming Interfaces (APIs), Airflow, Amazon S3, Directed Acyclic Graph (Directed Graphs), Data Transformation, Database Queries, Software Debugging, Apache Hive, Identity and Access Management, Python (Programming Language), Operational Databases, Performance Tuning, Standard Sql, Reverse Engineering, SQL Databases, Scripting, Snowflake, Data Lakes, Pyspark, Data Pipelines, Databricks
- **Published:** September 17, 2026
- **Apply:** https://www.buscojobs.com.es/mid-data-engineer-en-madrid-ID-372297188

## About the Role

3+ years operating production data pipelines
PySpark and SQL - able to read, debug, and modify existing pipelines, Airflow DAGs, operators, scheduling, and dependency management
Airflow retries, backfills, and idempotent task design
Strong SQL, including window functions, complex joins, and reading transformation logic
Python for scripting, automation, and API integration
Incremental loading patterns, idempotency, late-arriving data, and reprocessing
AWS S3 and IAM basics
Basic working knowledge of Redshift and its role in wider architecture
Fluent English
Self-directed and able to progress on an unfamiliar codebase without structured onboarding
Able to explain production incidents to non-technical stakeholders and provide realistic ETAs
Core Competencies Demonstrates expertise in managing production data pipelines using Databricks, including ingestion, transformation, and delivery processes.
Proficient in SQL, PySpark, and dbt for building and testing data models, with a strong understanding of Snowflake and Airflow for orchestration and performance optimization.
Highest-signal resume keywords
Databricks Pipeline Management
SQL Proficiency
PySpark Development
Airflow DAG Development
Dbt Model Building
Hard Skills
SQL
PySpark
Dbt
Airflow
Delta Lake
Unity Catalog
Snowflake
AWS S3
Incremental Loading Patterns
Data Quality Diagnosis
Soft Skills
Self-Directed
Effective Communication
Stakeholder Engagement
Industry Keywords
Data Pipeline
Data Transformation
Data Quality
Data Modeling

## Description

Keep production Databricks pipelines running, including ingestion, transformation, and delivery to downstream consumersDiagnose and resolve pipeline failures and data quality issuesReverse-engineer and document existing transformation logic and business rulesMigrate legacy tables from Hive Metastore to Unity CatalogMaintain Iceberg-enabled table sharing between Databricks and SnowflakeBuild and test dbt models, including incremental materializations and data testsDevelop and maintain Airflow DAGs for orchestrationValidate migrated pipelines against Databricks outputsContribute to Snowflake modeling, performance, and cost decisionsWork directly with client stakeholders on technical topics alongside the team leadRequirements3+ years operating production data pipelinesPySpark and SQL - able to read, debug, and modify existing pipelinesDelta Lake: MERGE/upsert patterns, table properties, OPTIMIZE, partitioningDatabricks Workflows, cluster configuration, job troubleshootingUnity Catalog: catalogs, schemas, grants, lineage, and the metastore modelSnowflake warehouses, roles and grants, and general operating modelSnowflake query performance and awareness of compute cost behaviordbt models, sources, tests, and incremental materializationsdbt project structure and deployment workflowAirflow DAGs, operators, scheduling, and dependency managementAirflow retries, backfills, and idempotent task designStrong SQL, including window functions, complex joins, and reading transformation logicPython for scripting, automation, and API integrationIncremental loading patterns, idempotency, late-arriving data, and reprocessingAWS S3 and IAM basicsBasic working knowledge of Redshift and its role in wider architectureFluent EnglishSelf-directed and able to progress on an unfamiliar codebase without structured onboardingAble to explain production incidents to non-technical stakeholders and provide realistic ETAsCore Competencies Demonstrates expertise in managing production data pipelines using Databricks, including ingestion, transformation, and delivery processes.Proficient in SQL, PySpark, and dbt for building and testing data models, with a strong understanding of Snowflake and Airflow for orchestration and performance optimization.Highest-signal resume keywordsDatabricks Pipeline ManagementSQL ProficiencyPySpark DevelopmentAirflow DAG DevelopmentDbt Model BuildingHard SkillsSQLPySparkDbtAirflowDelta LakeUnity CatalogSnowflakeAWS S3Incremental Loading PatternsData Quality DiagnosisSoft SkillsSelf-DirectedEffective CommunicationStakeholder EngagementIndustry KeywordsData PipelineData TransformationData QualityData ModelingOrchestrationTools & TechnologiesDatabricksSnowflakeAirflowHive MetastoreRedshift#J-*****-Ljbffr


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [PySpark - Combining Machine Learning & Big Data](https://www.wearedevelopers.com/videos/44-pyspark-combining-machine-learning-big-data)
- [Data Science in Retail](https://www.wearedevelopers.com/videos/586-data-science-in-retail)
- [WeAreDevelopers LIVE - CSS is DOOMed](https://www.wearedevelopers.com/videos/1838-wearedevelopers-live-css-is-doomed)
- [Empowering Retail Through Applied Machine Learning](https://www.wearedevelopers.com/videos/976-empowering-retail-through-applied-machine-learning)

## Related Articles

- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Highest Paying Tech Companies for Developers](https://www.wearedevelopers.com/magazine/220-highest-paying-tech-companies-for-developers)
- [How We Built a Worry-Free System That Runs for 10+ Years – And What We’d Do Again](https://www.wearedevelopers.com/magazine/751-how-we-built-a-worry-free-system-that-runs-for-10-years-and-what-we-d-do-again)
- [Why Upskilling And Reskilling is Important For Developers](https://www.wearedevelopers.com/magazine/428-why-upskilling-and-reskilling-is-important-for-developers)