engineer python sql machine learning architecture

Spotify
UK
15 days ago

Role details

Contract type
Permanent contract
Employment type
Full-time (> 32 hours)
Experience level
Starter
Experience required
3 years minimum
Working hours
Regular working hours

Tech stack

Data Analysis BigQuery Information Engineering Dataspaces Data Systems Distributed Data Store Data Flow Control Python (Programming Language) Machine Learning Meta-Data Management Cloud Services SQL Databases
+2 more
Data Streaming Technical Debt

Job description

We’re looking for a Staff Data Engineer who combines deep technical expertise with strategic thinking. You’ll help define the long-term direction of our data ecosystem, influence engineering across multiple teams, and build the foundations that will scale with Spotify’s future.

What You’ll Do

  • Define and drive the long-term data engineering strategy for the Subscriptions User Understanding domain, identifying opportunities that improve scalability, reliability, and business impact.
  • Design scalable batch and streaming data platforms that power analytics, experimentation, machine learning, and subscriber experiences.
  • Partner with Product, Engineering, Data Science, Analytics, and Platform teams to turn complex business challenges into durable, well-designed data solutions.
  • Lead architectural decisions and establish engineering best practices for data quality, governance, observability, reliability, and operational excellence across multiple squads.
  • Design data models and platform architecture that support sustainable growth toward one billion users while balancing performance, cost, and maintainability.
  • Simplify complex systems, reduce technical debt, and improve the developer experience across the broader data ecosystem.
  • Mentor senior engineers, influence technical direction through collaboration, and help foster a culture of thoughtful engineering, knowledge sharing, and continuous improvement.

Requirements

  • You enjoy solving broad, cross-cutting engineering challenges and bringing teams together around a shared technical vision.
  • You have extensive experience designing, building, and operating large-scale cloud-native data platforms.
  • You have deep experience building reliable distributed data pipelines using technologies such as Scio, BigQuery, Dataflow, GCS, SQL, Python, or similar cloud-native tools.
  • You have a strong understanding of modern data modeling, metadata management, governance, and data quality practices.
  • You know how to balance immediate business needs with long-term architectural sustainability.
  • You influence technical direction through strong communication, sound judgment, and collaboration rather than formal authority alone.
  • You communicate effectively with engineers, product managers, data scientists, analysts, and senior leadership.
  • You enjoy mentoring engineers and helping technical teams grow through coaching, technical guidance, and thoughtful feedback.
  • You are motivated by building platforms that directly influence Spotify’s long-term subscriber growth and business success.

About the company

Spotify is on an ambitious path to reach one billion users and $100 billion in revenue, and the Subscriptions User Understanding team plays a key role in making that possible.

Our team builds the foundational data platform that helps Spotify understand conversion, retention, engagement, and subscriber health for more than 200 million Premium subscribers around the world. We enable better decisions across analytics, experimentation, machine learning, and product development., Spotify is an equal opportunity employer. You are welcome at Spotify for who you are, no matter where you come from, what you look like, or what’s playing in your headphones. Our platform is for everyone, and so is our workplace. The more voices we have represented and amplified in our business, the more we will all thrive, contribute, and be forward-thinking! So bring us your personal experience, your perspectives, and your background. It’s in our differences that we will find the power to keep revolutionizing the way the world listens.

Apply for this position

This job is hosted externally. Click below to view the full posting and apply.

Apply on www.workingnomads.com

Good distractions

Talks and stories from around this role — technically off-topic, practically not.

5:00 min

Exploring the specific workplace responsibilities of staff software engineers

Jan Giacomelli · LIVE

2:37 min

Classifying and anonymizing data during system design

Reto Kaeser · LIVE

2:30 min

Leveraging BigQuery ML for scalable SQL-based segmentation experiments

Julian Joseph · LIVE

1:20 min

Identifying multi-disciplinary talent for developer experience engineering roles

Hazal Mestci +1 · Coffee With Developers

4:15 min

Bridging operational and analytical systems using formal data contracts

Matthias Niehoff Matthias Niehoff · WWC 2024

3:27 min

Explaining query execution overhead and caching limitations in BigQuery

Adnan Rahic · JS Congress

Videos

See all

Related articles

See all