Lead Data Engineer (Remote, United Kingdom)
Role details
Job location
Tech stack
Job description
You will be joining the Data Science and Engineering team to help get the most value out of our real-time data streams with a focus on elevating business decisions and ensuring genuine fans have the best opportunity to buy tickets., * Architect and implement scalable data infrastructure leveraging Databricks, Spark, and cloud platforms (AWS/GCP).
- Design and maintain robust data pipelines and ETL processes for structured and unstructured data.
- Develop and operationalize ML pipelines using MLflow, Git, and CI/CD best practices for model deployment and monitoring.
- Collaborate with Data Scientists, Product, and Engineering teams to define requirements and deliver high-quality data solutions.
- Ensure data quality, governance, and compliance with security and privacy regulations.
- Mentor and guide engineers, fostering a culture of technical excellence and continuous learning.
- Drive innovation in data and ML engineering, identifying opportunities for optimization and automation.
- Oversee mission-critical data systems ensuring reliability, scalability, and performance.
- Develop and maintain a deep understanding of the company's data landscape, identifying opportunities for improvement and innovation
- Oversee the delivery of complex data-related projects, ensuring timely completion, quality, and adherence to budget and resource constraints
Requirements
Do you have experience in Spark?, Do you have a Master's degree?, We are seeking a skilled Lead Data Engineer with ML Engineering expertise to design, build, and optimize large-scale data infrastructure and machine learning pipelines. You will play a hands-on technical leadership role, driving the architecture and implementation of data platforms that support advanced analytics and real-time decision-making.
The ideal candidate will have a deep understanding of data platforms and tools, as well as experience designing and implementing complex ETL processes., * Bachelor's or Master's degree in Computer Science, Engineering, or related field.
- Deep expertise in data engineering: Databricks, Spark, SQL, Kafka, and distributed computing frameworks.
- ML Engineering experience: MLflow, model versioning, deployment, and monitoring in production environments.
- Proficiency with cloud platforms (AWS or GCP) and containerization (Docker, Kubernetes).
- Good understanding of MLOps principles and CI/CD pipelines for ML workflows.
- Experience with data security and privacy compliance in ML systems.
- Proven track record of leading large scale data infrastructure in production
- Bonus: Experience in e-commerce or real-time data environments.
YOU (BEHAVIOURAL SKILLS)
- Excellent problem-solving and analytical skills with a product-focused mindset.
- Strong collaboration and communication skills, able to work across technical and business teams.
- Strategic thinker with leadership capabilities and a passion for innovation in data and ML engineering.
- Curious and proactive, eager to explore new technologies and approaches.