Azure Databricks Data Engineer (Castilla Y León)

ResponsibilitiesDevelop and optimize data pipelines in Scala, Python or PySpark by building and executing Spark processes or on Databricks, ensuring performance, quality and scalability.Design and construct Data Lake and Data Warehouse architectures primarily on Azure, with possible use of AWS or GCP.Transform large-scale data using Spark, applying best practices for distributed processing.Optimize queries and storage in Databricks to improve data consumption efficiency and reduce costs.Integrate multiple data sources, ensuring data quality, consistency and traceability.Collaborate with multidisciplinary teams to drive continuous improvement in ingestion, transformation and exploitation processes.QualificationsMinimum of 4-5 years of consolidated experience.Solid experience developing with Scala, Python or PySpark on Spark for distributed processing.Experience with Databricks and knowledge of Data Lakehouse architectures.Advanced SQL skills, query optimization and scalable data modeling.Experience with deployments and automation tools such as Terraform, Airflow or similar.Familiarity with cloud environments.Analytical mindset with a focus on efficiency, optimization and scalability of data processes.Collaborative attitude and passion for innovation, solving complex challenges in a team.DesiredExperience with Apache Iceberg or Apache Flink.English proficiency at level B2.Consideramos que la diversidad es enriquecedora y queremos velar por la inclusión e igualdad de oportunidades, por lo que contamos con un Plan de Igualdad y un Código Ético que recoge estos principios para garantizar la no discriminación de nuestras colaboradoras y colaboradores por cuestión de raza, color, nacionalidad, origen social, edad, sexo, estado civil, orientación sexual, ideología, opiniones políticas, religión o cualquier otra condición personal, física o social.#J-*****-Ljbffr

Solid experience developing with Scala, Python or PySpark on Spark for distributed processing. Experience with Databricks and knowledge of Data Lakehouse architectures. Advanced SQL skills, query optimization and scalable data modeling. Experience with deployments and automation tools such as Terraform, Airflow or similar. Familiarity with cloud environments. Analytical mindset with a focus on efficiency, optimization and scalability of data processes. Collaborative attitude and passion for innovation, solving complex challenges in a team. Desired Experience with Apache Iceberg or Apache Flink. English proficiency at level B2.