Lead Data Engineer
Role details
Job location
Tech stack
Job description
We are seeking an experienced Lead Data Engineer to drive the development and optimization of our data infrastructure within the Vector team. As a key technical leader, you will play a pivotal role in enabling seamless data operations to support machine learning, analytics, and business needs. This is an exciting opportunity to shape the data engineering landscape and collaborate with cross-functional teams to achieve impactful outcomes., * Lead the management and evolution of our in-house ML data platform, overseeing tasks such as data collection, storage, transformation, and accessibility to empower machine learning models and analytics.
- Architect, design, and optimize scalable data workflows, ensuring the seamless orchestration of pipelines.
- Establish and uphold best practices for data handling, storage, and processing within the ML and analytics ecosystem.
- Drive the development of internal tools to automate and enhance data processes, fostering efficiency and scalability across business workflows.
- Collaborate with product, MLOps, and data science teams to deliver an exceptional data experience for all ML practitioners and data consumers.
- Mentor and guide a team of data engineers, fostering professional growth and knowledge-sharing.
Requirements
Do you have experience in Spark?, * 5+ years of professional experience in data engineering, with at least 2 years in a lead or senior role.
- Expertise in Python 3 and data frameworks (Pandas, PySpark).
- Hands-on experience with modern data storage solutions: AWS S3, data lakes, and data warehouses.
- Proficiency in data orchestration tools (Dagster, Airflow).
- Experience with streaming technologies (Kafka, Spark).
- Solid understanding of machine learning and data analytics principles.
- Basic knowledge of data visualization and BI tools.
- Based in Barcelona or ready to relocate to Barcelona., * Experience with Databricks.
- Familiarity with Druid or other OLAP systems.
- Background in building ML data platforms.
With an outstanding product and a mission that excites and inspires, Appodeal offers a unique opportunity to make an impact while being part of an amazing team.