Has any streaming service trained a model to actually understand the music itself to work out what other songs would be of a similar vibe/genre?
My favorite band (vulfpeck, and more recently jack's solo stuff) often branch out into different genres, and it's a bit of whiplash when it goes to another song just because the artists are similar / appear together in other places.
> Plex Media Server uses a sophisticated neural network to analyze each track in the music library, cataloging a wide variety of characteristics of the track. Think of it as things like female vs male, vocals vs not, sad, happy, rock, rap, etc. All these various characteristic constitute a “Musical Universe” and the server is determining where that particular track exists within it.
> For the math-savvy, the Musical Universe consists of points in N-dimensional space. But what’s important is that this allows us to see how “close” anything in your library is from anything else, where distance is based on a large number of sonic elements in the audio.
I haven't tried it so can't speak to its effectiveness.
There is a whole field of music classification, if anybody (except copyright holders) were interested in using that, I'd expect it to be the likes of Spotify.
The problem with classification is that what makes a genre is not uniform. Some genres are defined by the way people sing, other genres are defined by the singers language, other are purely about the instrumentation or rhythms used, yet others mostly about the sounds and notes used etc.
But there are things like tempograms, tonnetz (tonal centroid features), chromagrams, spectral flatness/contrast/roloff, laplacian segmentation etc. And I guess feeding these into some neural net might give you interesting results.
What someone likes also doesn’t correlate solely with genres. While I like certain genres more than others, I only really like a small fraction of pieces in each genre, so the statistical correlation between what I like and genre affiliation is probably not very high.
ChatGPT is pretty good with this, you can try it yourself. I created a playlist generator for YouTube a while ago. It is powered by GPT-3.5 Turbo and can create playlists based on text descriptions: https://playlists.at/youtube/generate/
I used gpt-4 to generate ideas what to listen to. I just say "I like X, Y and Z" and it gives me interesting, motivated choices. No special recommender transformer, just the plain text one.
AFAIK all efforts in that direction were way too costly a few years back and degraded models considerably. Spotify, for the longest time, only trained on an equivalent of manually curated playlists by experts and users to understand similarity.
My favorite band (vulfpeck, and more recently jack's solo stuff) often branch out into different genres, and it's a bit of whiplash when it goes to another song just because the artists are similar / appear together in other places.