Data Engineer

Búsqueda Avanzada
Municipality of Madrid, Spain
yesterday

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Shift work
Languages
English

Job location

Municipality of Madrid, Spain

Tech stack

Agile Methodologies
Artificial Intelligence
Airflow
Unit Testing
Cloud Computing
Data Transformation
Elasticsearch
Hadoop
Apache Oozie
Systems Development Life Cycle
Shell Script
Software Engineering
SQL Databases
Data Streaming
Data Ingestion
Spark
Kubernetes
Deployment Automation
Kafka
Kibana
Data Pipelines
Programming Languages
Data Generation

Job description

  • Migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a service, and Airflow as a service.
  • Implement data transformation and quality to ensure data consistency and accuracy. Utilize programming languages such as Scala and SQL and tools like Spark for data transformation and enrichment operations.
  • Set up CI/CD pipelines to automate deployments, unit testing and development management.
  • Write and conduct unit and validation tests to ensure accuracy and integrity of code developed.
  • Automate data pipelines and streamline data ingestion through the implementation of different orchestrators and scheduling processes (Airflow as a Service mainly).
  • Writing technical documentation (specifications, operational documents) to ensure knowledge capitalization.
  • Collaborate with cross-functional teams to understand data requirements and deliver solutions.
  • Foster a culture of continuous learning and improvement within the team.

Requirements

  • Oozie

  • Shell script

  • Software Development Lifecycle (SDLC) awareness

  • Agile principles and ceremonies

  • Design effective prompts to leverage Gen AI tools across IT domains (e.g., development, testing, data generation, documentation) during the development stage.

  • SOME KNOWLEDGE OF:

  • Kubernetes containerization

  • Dremio as tool to virtualize data OPTIONALLY/ AS A PLUS:

  • Elasticsearch and Kibana

  • Streaming process (Kafka, event steam…). BUSINESS AND TRANSVERSAL SKILLS

  • Knowledge of banking industry and processes

  • Problem-Solving and Decision-Making person

  • Business / IT relationship (including IT OPS)

  • Ability to understand, explain and support change

  • Ability to Deliver / Results driven

  • Ability to collaborate / Teamwork with data squads and business teams

Benefits & conditions

  • Contrato indefinido con IRIUM
  • Retribución flexible
  • Banda salarial: Según valía y experiencia (30- 38K)
  • 23 días de vacaciones ️
  • Buen clima laboral ️ ️
  • Acceso ilimitado a formación tecnológica puntera en modalidad barra libre.
  • Club de beneficios para empleados con descuentos directos y miles de ofertas en marcas, hoteles, agencias de viaje, cines, ropa… Pasarás a formar parte de un gran equipo de personas que estarán siempre dispuestas a ayudarte. IRIUM es una empresa formada por profesionales con inquietudes, dinámicos y resolutivos. Nuestros valores son la responsabilidad y el compromiso con el trabajo bien hecho, este es el espíritu que buscamos en IRIUM, sea cual sea tu edad, si te reconoces ¡esta es tu empresa! Podemos construir juntos el futuro. ¿Hablamos? Inscribirse en esta oferta Recibir ofertas similares por correo electrónico Al crear una alerta, aceptas nuestros Términos y condiciones y Política de privacidad, y el uso de cookies.

Apply for this position