> Markdown version of [/jobs/ext/1397643-data-engineer](https://www.wearedevelopers.com/jobs/ext/1397643-data-engineer). Every page supports `.md` or `Accept: text/markdown`. Links point to the HTML versions so they work for humans too. Agent guide: [/agents.md](https://www.wearedevelopers.com/agents.md).

---

# Data Engineer

- **Company:** Búsqueda Avanzada
- **Location:** Madrid, Spain
- **Contract:** Permanent contract
- **Skills:** Agile Methodology, Artificial Intelligence, Airflow, Unit Testing, Cloud Computing, Data Transformation, Elasticsearch, Apache Hadoop, Apache Oozie, Systems Development Life Cycle, Shell Script, Software Engineering, SQL Databases, Data Streaming, Data Ingestion, Apache Spark, Kubernetes, Deployment Automation, Apache Kafka, Kibana, Data Pipelines, Programming Languages, Data Generation
- **Published:** July 23, 2026
- **Apply:** https://www.adzuna.es/contact-us.html

## About the Role

+ Oozie
 + Shell script
 + Software Development Lifecycle (SDLC) awareness
 + Agile principles and ceremonies
 + Design effective prompts to leverage Gen AI tools across IT domains (e.g., development, testing, data generation, documentation) during the development stage.
 + SOME KNOWLEDGE OF:
 + Kubernetes containerization
 + Dremio as tool to virtualize data
OPTIONALLY/ AS A PLUS:

 + Elasticsearch and Kibana
 + Streaming process (Kafka, event steam…).
BUSINESS AND TRANSVERSAL SKILLS

 + Knowledge of banking industry and processes
 + Problem-Solving and Decision-Making person
 + Business / IT relationship (including IT OPS)
 + Ability to understand, explain and support change
 + Ability to Deliver / Results driven
 + Ability to collaborate / Teamwork with data squads and business teams

## Description

+ Migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a service, and Airflow as a service.
 + Implement data transformation and quality to ensure data consistency and accuracy. Utilize programming languages such as Scala and SQL and tools like Spark for data transformation and enrichment operations.
 + Set up CI/CD pipelines to automate deployments, unit testing and development management.
 + Write and conduct unit and validation tests to ensure accuracy and integrity of code developed.
 + Automate data pipelines and streamline data ingestion through the implementation of different orchestrators and scheduling processes (Airflow as a Service mainly).
 + Writing technical documentation (specifications, operational documents) to ensure knowledge capitalization.
 + Collaborate with cross-functional teams to understand data requirements and deliver solutions.
 + Foster a culture of continuous learning and improvement within the team.


## Related Videos

- [From Messy Queries to Scalable Systems - How Data Engineering actually works](https://www.wearedevelopers.com/videos/100203-from-messy-queries-to-scalable-systems-how-data-engineering-actually-works)
- [Debug a Kubernetes Operator](https://www.wearedevelopers.com/videos/487-debug-a-kubernetes-operator)
- [Understanding Kubernetes in a visual way](https://www.wearedevelopers.com/videos/100085-understanding-kubernetes-in-a-visual-way)
- [Enjoying SQL data pipelines with dbt](https://www.wearedevelopers.com/videos/823-enjoying-sql-data-pipelines-with-dbt)
- [Add Location-based Searching to Site with ElasticSearch](https://www.wearedevelopers.com/videos/77-add-location-based-searching-to-site-with-elasticsearch)
- [AI Model Management Life Circles: ML Ops For Generative AI Models From Research to Deployment](https://www.wearedevelopers.com/videos/1152-ai-model-management-life-circles-ml-ops-for-generative-ai-models-from-research-to-deployment)

## Related Articles

- [Top Big Data Technologies That You Need to Know](https://www.wearedevelopers.com/magazine/108-top-big-data-technologies-that-you-need-to-know)
- [Making Data Warehouses Fast: A Developer’s Story](https://www.wearedevelopers.com/magazine/107-making-data-warehouses-fast-a-developer-s-story)
- [Data Engineer Salary UK](https://www.wearedevelopers.com/magazine/253-data-engineer-salary-uk)
- [Coffee with Developers - Maria Apazoglou - Making AI understandable for all in production](https://www.wearedevelopers.com/magazine/475-coffee-with-developers-maria-apazoglou-making-ai-understandable-for-all-in-production)
- [Navigating the AI Shift](https://www.wearedevelopers.com/magazine/629-navigating-the-ai-shift)
- [MLops – Deploying, Maintaining And Evolving Machine Learning Models in Production](https://www.wearedevelopers.com/magazine/115-mlops-deploying-maintaining-and-evolving-machine-learning-models-in-production)