Senior Data Engineer
Ultra Tendency
Municipality of Seville, Spain
2 days ago
Role details
Contract type
Permanent contract Employment type
Full-time (> 32 hours) Working hours
Regular working hours Languages
English, SpanishJob location
Municipality of Seville, Spain
Tech stack
Unity
Airflow
Amazon Web Services (AWS)
Azure
Computer Programming
Data Systems
Distributed Systems
Github
Python
Machine Learning
Performance Tuning
Spark
Data Lake
PySpark
Machine Learning Operations
Terraform
Data Pipelines
Databricks
Job description
- Design, implement and maintain scalable data pipelines using Databricks Lakehouse Platform, focusing on Apache Spark, Delta Lake and Unity Catalog
- Lead batch and streaming workflows that power analytics, machine-learning and BI use cases
- Collaborate with data scientists, architects and business stakeholders to translate complex requirements into production-grade solutions
- Optimize performance and cost-efficiency of Databricks clusters and jobs using Photon, Auto Loader and Job Workflows
- Establish and enforce best practices for data quality, governance and security within the Databricks environment
- Mentor junior engineers and advance team Databricks expertise
Requirements
- Deep hands-on experience with Databricks on Azure, AWS or GCP (Spark - PySpark/Scala), Delta Lake, MLflow
- Strong programming skills in Python or Scala, and experience with CI/CD pipelines (GitHub Actions, Azure DevOps)
- Solid understanding of distributed computing, data modelling, performance tuning in cloud-native environments
- Familiarity with orchestration tools (Databricks Workflows, Airflow) and infrastructure-as-code (Terraform)
- Proactive mindset, strong communication skills and passion for building scalable, reliable data systems
- Professional Spanish & English communication skills (C1 level, written and spoken)
Benefits & conditions
Design and optimize data processing algorithms as part of a talented, cross-functional team. Contribute to the advancement of data engineering using the Apache open-source suite. What We Offer
- Flexible work options - fully remote or hybrid (must be located in Spain)
- Career acceleration with outstanding colleagues across 3 continents
- Opportunity to shape projects and see impact from ideation to reality
- Work-life balance with personalized workflow organization
- Annual performance review and regular feedback cycles
- Individual development plan and professional development opportunities
- Paid certifications and unlimited access to Udemy Business
- Local, virtual and global team events