I'm not sure how the Spotify recommendation algorithm works at all, but for some reason I imagined them doing fancier things than looking at my liked songs and finding similar ones. I would've thought they'd build a profile of you, and then find similar user profiles and show you songs those folks liked that you hadn't found yet.
That's gotta be how they do it, right? I'm probably wrong.
I don't work at Spotify anymore and I didn't work on the tech I'm describing, but I picked up a bit about what was going on while there.
First, there is/was no single algorithm, but the core ideas driving a lot of recommendations is:
1. Create user taste vectors
2. Match those vectors to other users or collections of tracks
3. Use that information and combinations of other things to find recommendations.
Each step of the process is constantly being experimented with. Different custom playlists might be using a different combination of tech doing those basic steps.
"Other things" including intentional commercial biases presumably?
No matter what I do in Spotify, under several different rounds of accounts, it always seems to gravitate towards the tastes of the general public, i.e. some form of mass-market pop.
Their recent "ai" assistant was a slight improvement because you can ask it for less popular music which is typically better for music discovery.
Collaborative filtering is similar but for huge recommender systems they’re not going to create a huge MxN matrix where M is users and N is items. I think what they’re referring to would be called a “two tower” model where you have a learned vector for the user, a learned vector for the song, and the cosine similarity is their affinity. It’s pretty performant because you can cache the song vectors.
I feel those are how Pandora and Last.fm (used to?) work respectively. Nowadays everything seems to just put a bunch of tags on a track and suggest you things with the same tags to the tracks you liked. Doesn't even need to match the same combination of tags, just some number of them. The problem is, you probably care about the small, specific tags, and the system cares about wide "popular" tags. If you like a couple niche genre covers of songs that happen to be featured in TV openings/OSTs, you are not getting more songs in that genre - you are getting a bunch of covers and OSTs.
I wish I had a music recommendation service built on Pandora's immense dataset of music tags that could build me a playlist that I could link back to whichever music service I happen to be using at the time. I could have it do things like require at least 3 tags in common between adjacent tracks such that it could jump around between 2 dozen genres but the transaction between any 2 given tracks isn't too jarring. It'd also be nice if I could tell it to make a playlist where every song shares one particular tag in common.
Maybe I'll build that. Sure would be nice to have.
The primary advantage of Pandora's algorithm is the human-labelled Music Genome database. I haven't seen any other company do music discovery as well as Pandora, and don't expect that to change any time soon.
I have no specific insider knowledge, but a decade+ ago they bought a company called the Echo Nest that was developing some of the best audio signal analysis algorithms around, I assume much of that influenced their recommender system.
Nowadays, they have a quite busy research department so I would imagine that recommendation is quite fancy indeed: https://research.atspotify.com
That's gotta be how they do it, right? I'm probably wrong.