Cloud Data Engineer

SIX
Municipality of Madrid, Spain
2 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Working hours
Regular working hours
Languages
English
Experience level
Senior

Job location

Municipality of Madrid, Spain

Tech stack

Artificial Intelligence
Amazon Web Services (AWS)
Azure
Bash
Cloud Computing
Continuous Integration
Information Engineering
ETL
Identity and Access Management
Python
Operational Databases
Cloud Services
DataOps
Software Engineering
SQL Databases
Pulumi
Google Cloud Platform
Spark
Infrastructure as Code (IaC)
PySpark
Machine Learning Operations
Terraform
Databricks

Job description

  • Be part of an international team that designs and maintains a mature cloud native data science platform, currently serving more than 14 internal analytics teams and 600+ users.
  • Closely collaborate with key stakeholders from business data science teams and IT data engineering teams to enable them on the platform. Identify new requirements, analyze platform suitability, propose best practices and existing solutions through demos...
  • Act as a hands-on data engineer to onboard new use cases and optimize existing ones, build production ready ETL pipelines, ML/AI use cases and reporting solutions.
  • Implement new infrastructure components with Infrastructure as code (IaC), as well as creating templates to ensure best practices and standardization among analytics teams.

Requirements

  • 4+ years of practical experience building production data pipelines, data modeling and BI tools in the cloud.
  • Experience with Lakehouse architecture, preferably Databricks, with a strong knowledge of Spark and Open Table formats (Delta and/or Iceberg).
  • Proficiency with Python (PySpark) and SQL with a solid understanding of software development and automation concepts like CI/CD to build production data and ML pipelines (DataOps/MLOps).
  • Basic knowledge of cloud foundation services (IAM, networking, storage...), cloud costs management (FinOps) as well as tools to manage cloud infrastructure with IaC (Terraform/OpenTofu, Pulumi, bash scripting...) on Microsoft Azure (our primary cloud service provider) and/or Amazon Web Services, Google Cloud Platform.
  • Growth mindset and a very proactive attitude for continuous learning and development.
  • Good presentation and communication skills. Fluency in written and spoken English. If you have any questions, check out our or call Yuliya Stoyko at +34 917095993. For this vacancy we only accept direct applications. Diversity is important to us. Therefore, we are looking to receiving applications regardless of any personal background.

Benefits & conditions

We trust our employees and offer a work environment that is well-balanced, productive and fosters success. Personal Development You will benefit from a culture of continuous learning and feedback. Your personal growth is supported through an extensive learning offering. Agile Working Methods Whether through scrum or design thinking, we solve exciting tasks together in teams.

Apply for this position