This is part one of a series. In this part, I detail how Spotify's recommendation system works and the real-world impact it has (both advertently and inadvertently). In the next part, I will go over how to build a simple recommendation system similar to Spotify's.
I'm certain you've heard the phrase: "Music is terrible these days." This was likely from someone who grew up in the 1980s or earlier remarking about the style of music the 'youngins' listen to and what's been playing on the radio recently. Most of us roll our eyes because every generation seems to think the next generation's music is garbage, but the truth is that music has changed drastically over the past decade.
Generative AI has caused more people to be conscious of how AI impacts everyday life. This is easy to notice when a person frequently has to determine if images and videos are real or fake. This is much more difficult to notice when AI is being used to feed you recommendations instead of generating the content itself. I would argue this can be even more impactful because of how difficult it is to notice.
To understand this, we're going to look at Spotify's music recommendation algorithm. We'll walk through it from the problem statement (what Spotify is trying to accomplish) through to the algorithms they use to accomplish that goal and all the way to the impact their methodology has on their users and the industry.
Spotify has made much more music available to many more people. The purpose of sharing this is to walk through the considerations that go into making a machine learning system, many of which go beyond choosing a model and building software.
All machine learning engineers need to understand the tradeoffs that come with approaching problems using machine learning. Machine learning is fundamentally an optimization problem, and optimizing for specific metrics always has trade-offs.
In this case study, we're going to:
Start from Spotify's problem. What are they trying to solve?
Identify how Spotify is solving that problem.
Understand the side effects their approach has.
Realize the impact those side effects have on users and the music industry as a whole.
My goal is to help you not only understand Spotify's systems, but also have a better understanding of why case studies like this are important to understanding impact.
