I sometimes wonder if we're looking in the wrong place with music recommendation systems. I've tried both Apple Music and Spotify; it's rare that I hear a song come through on the linear recommendation stream and think to myself "oh my gosh! that's exactly what I wanted!" For me, discovering new music is a branching experience, where I'm constantly listening to little bits of different things, figuring out what I like, and then looking online on forums and blogs to see what's similar to that. It's surprising that the company that owns YouTube, a platform driven by user choice and 'rabbit-hole discovery', would be looking for a new way to feed users linear song recommendations. I would much rather be able to see several 'similar songs' while listening to something, similar to YouTube's recommendation tab. Alas, no streaming service seems to have implemented this (not even YouTube music, afaik).
https://cosine.club/ is the closest I've seen to the ideal branching system. My understanding is that it uses vector embeddings to search for songs that are similar in sound, and it works shockingly well for that purpose. However, it has a limited song database. Also see https://everynoise.com/, which is no longer updated. These use vector embeddings in similar ways, but the exploration experience is controlled by the user, not by a list-generating ranking model. I definitely think that AI-tech is the future of music recommendation, but I would prefer to see more research by large companies in to these user-driven systems, instead of the 'similar autosuggested list', which is, by its very nature, only ever 'good enough.'
I’ve been surprised by how poor Spotify’s recommendations were - they bought the Echo Nest, and seem to have people who are quite smart working on it but when I tried it after Rdio closed no matter what I started with it’d be top 40 after a couple of tracks, enough so that I wondered if there was a background deal with the record labels.
Apple Music is notably better – and has the benefit of not funneling your money to the likes of Rogan – and the recommendations will be fairly good within a genre but it does overweight your library a bit (I wish it had a “I’m looking for something new” / “familiar” toggle).
I am curious what Rdio did differently as I had a very good success rate with their suggestions and it seems unlikely that there was some secret sauce nobody else has been able to figure out.
If you’ve ever had the pleasure of using the DJ X feature in Spotify, it does a decent job mixing in some new things I like, but you’re definitely on to something when it comes to popular record labels. I don’t like any of the new pop, but every other “set” that DJ X provides has Chappel Roan or Taylor Swift or Sabrina Carpenter or some other flavor of generic pop I never am interested in. Play counts for some of these popular artists are probably inflated due to that kind of thing.
The best recommendations I get on Spotify are usually via their users that listened to artist X, also listened to artist Y type recommendations. That combined with their list of most popular tracks per artist gives me a rich source of new things to listen to. Their regular recommendations aren't great; it falls into the same "more of the same shit" trap that most other recommendation systems fall into.
The reason this simple mechanism works so well is that it gets rid of personal biases and instead taps into a community of listeners listening to the same stuff. Confirmation bias is the core issue here. I don't want confirmation bias. I want my biases challenged with new things. Not randomly new but based on what others are listening to that listen to similar things. And not just randomly based on everything I listen to but on specific things that I'm playing.
Vector similarity of artists could be an interesting angle. But it would probably risk pulling out a lot of cover bands and imitators. You want stuff that is close but not too close.
most recs now are based on "users also listened to.." and (very rarely) audio embeddings/features.
however much of my personal discovery is based on trying to understand the history of groups that i piece together from wikipedia and reading about who the artists were.
I want is recommendations based of some sort of in-depth knowledge graph that traces personnel hopping between bands, which other artists worked in the same scene, who they public acknowledge as an influence, etc.
it would be great to uncover things like "hey, did you know that all these songs you like had the same producer? maybe you should dig into other things that this guy produced" or "this artist you loved was really into a performer from a completely different genre -- maybe you should check it to see the influences that they had"
The trouble with "people who listened to X also listened to Y" is it can't ever recommend music that nobody has listened to yet, and is unlikely to recommend anything that doesn't have a reasonable quantity of listeners already, hence likely some level of promotion behind it.
If you select an obscure artist in spotify, the group of people that listen to those might have a few more obscure artists in common. That has worked for me a few times where I go down a rabbit hole of some pretty obscure stuff that is all connected somehow. I have a few things I discovered this way that didn't have more than a few hundred listens.
But you are right that none of this stuff is perfect.
In the old days, hipsters flocked to music with small fanbases of 10,000 or so. Current technology permits us to target down to those in the size of hundreds. And yet, post-hipsters now demand single digit numbers. Scientists hypothesize we may achieve sub-fan levels of popularity at some point, but at what cost?
Don't worry I can be pretentious enough without having to invoke artists you've never heard of; it's not about that!
I have a broader concern for how new artists are supposed to get discovered without a promotion engine behind them. Yes it's always been hard to get started, but the distribution of attention has really become much more top heavy in recent years. I know one guy who played Wembley stadium and still couldn't give up his day job which he was sure he would have been able to do following a gig of that size in the 90s. Yeah so he had a good number of monthly listeners, but it illustrates how the distribution has changed.
Plenty of people on the long tail deserve to be discovered, and use of AI to recommend music - in place of collaborative filtering - really has the potential to fix that.
PS. We were talking monthly listeners weren't we, so you'll be excited to know that fractional fans exist already ;-)
I struggle to imagine what you're saying. In the old days if you had 100 monthly listeners that meant you were likely getting on your local radio station at great effort. You had no shelf space at the record store. You were not searchable on the web. The long tail seems irrefutably better served by modern methods.
Artists struggling to make a living on the back of a single success is, if anything, a product of the longer tails of music being a catered to. The gains are much more spread out now.
imo the one hit wonders of yesterday were fairly significant hits. The one hit wonders of the 2010s are vastly more ephemeral in my personal opinion. Probably mostly driven by the fact that they used to be conveyed by pop radio and now I don't hear pop radio EVER. But I also have some doubts that most of these 2010s songs will be able to carry a band forward like the one hits of the 90s.
Yes I've been doing the same in bandcamp. If I find something I like, I click on interesting user thumbnails (in the "have it in their collection" section) and listen to some of their collection or wishlist. If this resonates with me I follow them, check out more music and then can jump right to the next user.
While their general recommendations don’t work so well for me, following people I regularly saw writing mini reviews on stuff I bought has worked pretty well to discover older stuff or releases I simply missed (I listen to most new stuff that is up my alley every release Friday anyway). The mini-reviews also help narrow down if it’s even something I want to check out, which works better for me than people who buy without those.
> Vector similarity of artists could be an interesting angle.
Any song has many facets: melody, key, rhythm, dynamicity, voice of the artist, lyrics.
I can see how a music browser of the future (rather than an automatic recommender) would be equipped with many different knobs to turn and tweak each of these dimension's weight (as they are going into a similarity calculation) separately, to give the user control.
Same. Back before she blew up in the US I discovered Lorde from a Spotify user generated playlist (i've no idea how i found it, but glad i did) and I played it 100x (it was at the top of the list) and was given the reputation of having good music taste from the person I was dating at the time.
Algorithmic playlists I've not found useful. The Apple Music "create a station from this song" feature is more or less broken imo -- i get so much of the same same same stuff
Similar, Spotify recommended Alice Merton to me before she went viral not entirely 100% sure but I think it was a user playlist too. There’s been a few other artists I’ve discovered very early in their careers and it’s great seeing them get some fame.
Apples recommendations are so bad. It just goes right back to the same top 100 songs.
I feel like all recommendation systems already do similarity well -- and it's not what I want. True, similarity matters to some extent, but my dream is something that can accurately predict what I'd like. Often I'll only like a song or two from a given artist, so finding artists similar to this artist are often useless.
Related question: I wonder if identical twins are good at recommending each other music
One missing factor here is that the recommendation algo is a prime spot for advertising new music, so all for profit services are very incentivised to introduce tweaks that boost the songs of clients.
YT Music user here. I've found many new artists - across genres - through recommendations. There's also a "related" tab that you can bring up for each song.
I don't know how the youtube recommendation system worked in 2014, but I've definitely had way more interesting and novel things shown then than today, where half the time I'm recommended stuff I've already watched.
I notice that too. youtube recommendation system sometimes recommend already watched videos to me. I haven't tried it yet but I was told that clearing your watch history or using incognito mode can help reset recommendations.
"Understanding the music" alone isn't helpful unless you know what features are relevant for recommendations, and these must be learned from meta- and usage-data.
Genre, tempo, key, vocalist sound, instruments, and so on. These might all be relevant in different recommendations, at different times, in some particular order depending on the user. The music-content in effect only serves to align tracks along lines in the embedding space.
Apple is rather interesting. On their new music Fridays the playlists alternate predictably between garbage and reasonable stuff. There are perhaps 1-2 reasonable songs on the reasonable stuff list and maybe 1 gets kept.
Considering the pool of music on the radio when I was a kid, that's a reasonable hit rate in my mind. I'm not sure I would cope with an influx of music larger than that.
I don't know what people expect really. Discovering music that resonates with you is not easy. I find spotify gives me about one band I really like every two months and I think that's actually really good. I dislike the vast majority of what it recommends, but I don't think that's a problem.
One resource I use a lot that I didn't mention in my comment is https://rateyourmusic.com/. I find it's most helpful for finding the "canonical" albums & artists in a genre you're unfamiliar with. You can search by genre, influences, and year range, and its listings are generally very accurate. It also just has a culture of having more in depth and well written reviews, so if I'm looking at an album I've never heard of, I often get to read a review by someone who's been listening to it their entire lives. Much more helpful thoughts & opinions than someone whose job it is to review music (though I do enjoy reading some music critics blogs).
I might be a boomer, but I find Youtube Music automatic suggestions superior to Spotify's. It doesn't "branch out" as much as Spotify, but the next song it puts on is always spot-on, "exactly what I wanted"-vibe.
Seconding this, YouTube Music is uncannily good at making radios from songs. It's always what I'm looking for, and when it does branch out, it's usually introducing me to a new jam.
It doesn't seem that this approach "knows" the actual music. The article doesn't seem to explain how track embedding vectors are produced, but it mentions that user-action signals are of the same length, which makes me doubt track embeddings have any content-derived (rather than metadata-derived) information. Maybe I'm wrong, of course.
I doubt that any recommendation system is capable of providing meaningful results in absence of the "awareness" about the actual content (be it music, books, movies or anything else) of what it's meant to recommend.
It's like a deaf DJ that uses the charts data to decide what to play, guessing and incorporating listeners' profiles/wishes. It's better than a deaf DJ who just picks whatever's popular without any context (or going by genre only), but it's not exactly what one looks forward to when looking for a recommendation.
My experience is that the best music is found randomly. I like so much, I don't even know what I really like. Even what I like is always changing. I need to listen to ton of random things I don't like and I will find a small amount of gems. The absolute gold though is finding songs I didn't even know I would like.
The algorithmic version of sifting through records at a record store for a music lover is random. Random with an easy way to play the next song.
All these recommendation systems are just Satie's musique d'ameublement generators for non-music lovers. Furniture music generators, music to play during a dinner to create a background atmosphere for that activity.
This is a shortcoming of every music recommendation algorithm except Spotify and Pandora's. Spotify holds holds a pretty hefty patent portfolio of music classification algorithms and Pandora employs hundreds of music experts that spend an hour tagging each song.
Spotify's Discover Weekly seems to be a healthy mix of close enough guesses and random but not too random suggestions. Song radio is okay. The pop up recommendation for specific new songs/ albums feels so unrelated to my likes that it must be a sponsored recommendation. """Smart""" shuffle exclusively sends me whatever was popular on the radio in the last 5-30 years despite my listening habits being the opposite.
Pandora was much smarter, but seemed to run out of songs instantly.
Nearly 10 years ago, I was at a Spotify recruiting event and they told us how they did embeddings at the time.
They took all user generated playlists and projected the songs into vectors where songs that appear together on playlists are closer and songs that appear less often are farther.
It’s likely changed a lot since then, but it seemed like a pretty straightforward clustering system at the time.
co-occurrence. It's the real backbone of almost all recommender systems.
This is the same way YT/TikTok does it btw. Co-occurrence is king in recommender systems in production. It's extremely cheap to calculate and by far the most effective method.
That's just bais collaborative filtering. Drdaeman is talking about using the actual content of the songs in your vector embeddings.
This is not really important if you have a lot of user behavior data and/or playlists for each song. But if you have a niche song that few people of listened to, collaborative filtering based recommendations aren't going to be good.
Real semantic embeddings (which can then be part of the input to the recommendation model) can be trained using self-supervision, e.g. an auto encoder or a seperate "next audio token" predicting transformer.
A recommendation from a person you know takes into account not just their knowledge of your preferences, but also how much and in what way they like/care about you, and conversely, your taking of the recommendation is colored by your rapport with the recommender. All that is something a recommender system has no access to.
Or, more bluntly: you aren't going to mate with a For You page, so it doesn't have the same evolutionary cheat code to your preferences as other people have.
Complicated, or worryingly straightforward and effective? It really does seem that over time, this would compress the space of peoples' preferences - and since listening stats also feed into production and promotion - the space of music produced.
> I doubt that any recommendation system is capable of providing meaningful results in absence of the "awareness" about the actual content (be it music, books, movies or anything else) of what it's meant to recommend.
Most of the reasons people like music, or fictional movies and books, is personal, emotional, subjective, and difficult to articulate. You wouldn't know what data to collect. You're better off just asking them to rate song, movies, or novels out of ten. You can then compare their ratings with other people's, and what you'll find is there are clusters of people who rate things similarly (and others who rate things differently), and that the ratings they give overall somehow capture their feelings about whatever they listened to, watched, or read. (Source: I developed a movie recommendation system which predicted ratings reasonably accurately.)
Of course, if you just have sequences of user actions, like in the article, your recommendations won't be anywhere near as accurate.
> I doubt that any recommendation system is capable of providing meaningful results in absence of the "awareness" about the actual content (be it music, books, movies or anything else) of what it's meant to recommend.
Years of experience have proven that you can get quite far with pure collaborative filtering—no user features, no content features. It's a very hard baseline to beat. A similar principle applies to language modeling: from word2vec to transformers, language models never rely on any additional information about what a token "means," only how the tokens relate to each other.
It uses data about overlap in listenership between different artists to determine which artists are related to which others and how. The artists serve the same role as words in sentences.
It's all very nice but if they end up "altering" the results heavily to play you the music they want you to listen for X or Y reason then it's pointless.
I would like to be able to run this model myself and have a pristine and unbiased output of suggestions
It may just be my perception, but I seem to have noticed this steering becoming a lot more heavy handed on Spotify.
If I try to play any music from a historical genre, it's only about 3 or 4 autoplays before it's queued exclusively contemporary artists, usually performing a cheap pastiche of the original style. It's honestly made the algorithm unusable, to the point that I built a CLI tool that lets me get recommendations from Claude conversationally, and adds them to my queue via api. It's limited by Claude's relatively shallow ability to retrieve from the vast library on these streaming services, but it's still better than the alternative.
Hoping someone makes a model specifically for conversational music DJing, it's really pretty magical when it's working well.
No, but it's also biased toward their commercial partners. From this page [0], detailing their recommendation process:
> How do commercial considerations impact recommendations?
> [...] In some cases, commercial considerations, such as the cost of content or whether we can monetize it, may influence our recommendations. For example, Discovery Mode gives artists and labels the opportunity to identify songs that are a priority for them, and our system will add that signal to the algorithms that determine the content of personalized listening sessions. When an artist or label turns on Discovery Mode for a song, Spotify charges a commission on streams of that song in areas of the platform where Discovery Mode is active.
So Spotify's incentivized to coerce listening behavior towards contemporary artists that vaguely match your tastes, so they can collect the commission. This explains why it's essentially impossible to keep the algorithm in a historical era or genre -- even if well defined, and seeded with a playlist full of songs that fit the definition. It also explains why the "shuffle" button now defaults to "smart shuffle" so they can insert "recommended" (read: commission-generating) songs into your playlist.
The link they provided is the disclosure. You'd be surprised to find out this is the business model of the radio for years and why most radio stations that need profits only play recent songs, and usually the same songs over and over until new ones that are pushed by labels come out.
Is there a site that has hand-curated playlists I would love that let's say if I want to listen to Korean pop from the 90s or Minimal Techno from the 00s.
Other than stating there was one, they didn’t show a benefit of this over something like a Wide and Deep model, DCNv2 model, or even a vanilla NN. Transformers make sense if you need to use something N items ago as context (as in text) where N is large. But in their example, any model which takes the last ~5 or so interactions should be able to quickly understand contextual user preferences.
A transformer may also be larger than their baseline, but you still need to justify how those parameters are allocated.
Empirically, I have found that user action sequences are a good way to model user behavior since it can look at several different scales, and specific behaviors. Interest tracking can see what a user generally likes, and the last few actions can help the model see what the user is listening to right now. But with a full sequence, you can start to model things like what the user is listening to right now, what they've been listening to recently, what they tend to listen to at this time of day, how much of a change in genre they could enjoy, etc.
Has any streaming service trained a model to actually understand the music itself to work out what other songs would be of a similar vibe/genre?
My favorite band (vulfpeck, and more recently jack's solo stuff) often branch out into different genres, and it's a bit of whiplash when it goes to another song just because the artists are similar / appear together in other places.
> Plex Media Server uses a sophisticated neural network to analyze each track in the music library, cataloging a wide variety of characteristics of the track. Think of it as things like female vs male, vocals vs not, sad, happy, rock, rap, etc. All these various characteristic constitute a “Musical Universe” and the server is determining where that particular track exists within it.
> For the math-savvy, the Musical Universe consists of points in N-dimensional space. But what’s important is that this allows us to see how “close” anything in your library is from anything else, where distance is based on a large number of sonic elements in the audio.
I haven't tried it so can't speak to its effectiveness.
There is a whole field of music classification, if anybody (except copyright holders) were interested in using that, I'd expect it to be the likes of Spotify.
The problem with classification is that what makes a genre is not uniform. Some genres are defined by the way people sing, other genres are defined by the singers language, other are purely about the instrumentation or rhythms used, yet others mostly about the sounds and notes used etc.
But there are things like tempograms, tonnetz (tonal centroid features), chromagrams, spectral flatness/contrast/roloff, laplacian segmentation etc. And I guess feeding these into some neural net might give you interesting results.
What someone likes also doesn’t correlate solely with genres. While I like certain genres more than others, I only really like a small fraction of pieces in each genre, so the statistical correlation between what I like and genre affiliation is probably not very high.
ChatGPT is pretty good with this, you can try it yourself. I created a playlist generator for YouTube a while ago. It is powered by GPT-3.5 Turbo and can create playlists based on text descriptions: https://playlists.at/youtube/generate/
I used gpt-4 to generate ideas what to listen to. I just say "I like X, Y and Z" and it gives me interesting, motivated choices. No special recommender transformer, just the plain text one.
AFAIK all efforts in that direction were way too costly a few years back and degraded models considerably. Spotify, for the longest time, only trained on an equivalent of manually curated playlists by experts and users to understand similarity.
All this research to create an apparently awesome recommendation system only for the sales department forces the recommendation of what they want to promote.
I look at my own music tastes as roughly two levels. One level is whether I objectively like the music. The other level is what I'm in the mood to listen to. I can definitely not be in the mood to listen to some music I really enjoy. But there is also music I will never enjoy no matter the mood. I don't think a recommendation system will work very well if it conflates those two levels.
Wouldn't it be nice to have recommendations that effectively address both levels and understanding not just your general preferences but also your current emotional state
I tried to make embeddings from my personal listening history a few years ago (https://github.com/ruuda/deepnote) with the idea that listens of music that matches well would be played in the same session, basically word2vec applied to listening history. It didn't work very well, it mostly found that tracks on albums are similar because I tend to listen to full albums. Maybe I didn't stir the parameters enough, or maybe it needs far more data. (All of Listenbrainz?) I also still want to experiment with generating embeddings for tracks just from the time of the day, week, and year. (I tend to listen to different things on a Friday night vs. Sunday morning, and I listen to very different music in summer vs. winter.) But that's for locally recommending relevant tracks that you already have in your library (for https://github.com/ruuda/musium), not for discovering new music. I already implemented a sorting mode to "rediscover" albums (albums that have a lot of listens in the past but few recently), and it works reasonably well. I expect that adding time of day/week/year will improve this a lot, but I haven't implemented it yet. I wonder how much of an improvement a transformer like in the article adds on top of that.
IMO it’s a mistake to try to draw any conclusions from skip actions or from what a user is listening to, as opposed to from explicit like/dislike actions. There are just too many reasons why someone may be skipping a track or (appear to) be listening to tracks. As a user, I don’t want to be in the position of having to fear that the algorithm will misinterpret my skipping or my letting some playlist play.
All megacorp recommendation algos are shit just because they always binded to some media library. Get yourself on any torrent website with lots of music releases and you will unleash what is the meaning of "musical taste". I recommend to have at least 10Tb disk drive to have no reasons to delete any releases.
Youtube Music have no lossless, its "tracks" have no metadata and are stored in pesky format with a lot of stupid limitations about receiving them. Seems you just do not know what is a good music and how it even looks on your filesystem. And good luck about acknowledging (not even receiving) every existing recording of your favorite artist; discography.
It's perfectly fine for streaming and getting music recommendations, which is the topic of this thread. For the local music library I then purchase lossless versions of the tracks I like.
This research isnt very novel either. Transformers for sequential event handling and recommendations is a known thing. Pretty sure this is what spotify uses on their autocomplete shuffle playlist (fill in songs between your songs, similar to how autocompleting text would be)
We are living through the best ever time to be a music lover. With so much good music being made and added to the existing body of recordings, together with the great (albeit commercially compromised) recommendation systems, it's just fantastic.
Algos are useful for finding and playing music but they are just a tool. The human touch is needed to create the best musical experiences. That's why we need DJs now more than ever.
To experience the best music, for you, you're going to have to do a bit of work to search out new sounds and old sounds. So go digging and create awesome playlists, for you, and share them: they'll probably be better than algo-created playlists.
Algorithmic playlist systems are great but I believe we should counteract their prevalence by also supporting human efforts: by also listening to online radio stations and DJ mixes, supporting music journalism and publications (as well as buying physical media and merch, going to gigs, of course!)
I don't understand exactly what this Google paper is describing but it sounds like it's going to anticipate whether I want to listen to up beat or down beat music. What a very googleish thing to do! - this doesn't help me at all. Also I'm turned off by it's opening statement:
>We present a music recommendation ranking system that uses Transformer models to better understand the sequential nature of user actions based on the current user context.
This comes across as nonsensical gobbledygook, but also a bit dystopian.
It’s interesting the amount of research listed in the article and IMHO the recommendation engine/ algorithm used by Rdio in the late aughts and early 2010s eclipses anything I’ve encountered to date.
Seems like folks are reinventing the wheel, and trying to deduce what folks want to engage in with data and “AI”, rather than providing sufficient tools to allow the user to drive the narrative.
any time there's a music recommender thread, there will be comments lamenting old algos like Rdio or play.fm
it's interesting how this continues a trend in across music/audio tech, such as hipsters insisting "i liked the earlier work better" or audiophiles obsessing over amps from the 1970s.
It would be good if they started using this. I moved to spotify because youtube music kept adding songs I thumbs-downed to my playlists. They even added songs I hated to my seasonal mixtapes. If I didn't pay youtube for no ads and creators I'd never give these idiots a cent.
I think the only thing that works for me properly is listenbrainz.org for recommendations. They give similar users, good playlists. Plus they have bunch of other interesting projects like acousticbrainz.org which do even low-level analysis and high-level classification.
We had an informal college group “music appreciation” that would meet every Wednesday to listen to 2 full albums—-each related to the other in some manner. The only rule was “no talking while the music played.” It lasted four years and I miss it!
Virtual crate digging on Discogs.com is still by far the best way to discover music.
Artists, record labels, producers, technicians (mastering, vinyl cutting), distribution channels, etc. are all there just a click away.
Who did/does you favourite artist share a label with? Who else used the same recording studio? What other aliases does an artist use? What other bands/groups have they been in?
Combined with the embedded YouTube player on the release page it's a gold mine.
Every week I'll spend hours down the Discogs rabbit hole - often adding missing YouTube videos to contribute something for future visitors.
Music recommendations algorithms are a fools errand. If you look at Spotify, they do a good job initially but it gets really boring after a while. All algorithms or AI tries and usually fails to guess how you are feeling at that moment, maybe there was a trigger for me to hear a specific type of music, or maybe I suddenly feel nostalgic and want to hear 90s music instead of my favourite modern electronic which I skipped a whole tone if it comes up.
You could still use collaborative filtering, except with users giving different ratings to the same songs depending on their mood. This approach was used in a movie recommendation engine (I forget which one).
I haven't found any music recommendations services that really find what I'm looking for. What I'm looking for is usually something without high pitched noises, rythmic and a bit of novelty based on old patterns. I've tried my playlist on both Spotify, apple and YouTube music but they only find a new song I like 1 in 10 Times.
I suspect there is no available dataset that can teach the models to learn this. The enjoyment of music is entirely internalized. Skipping a track is an incredibly low fidelity data point.
I've often wondered if Spotify could capture volume control data as part of a track to see if that produces better training data. But again its still too low fidelity.
YouTube just "infers the context" meaning it uses some sort of tracking to realize you're at the gym versus "just chilling." But what's really chilling is that they don't mention location data or how they get it anywhere in this post.
Spotify's music recommendation is great not because it's doing smart AI things, but because it gives you options for discovery. You have:
- Playlists provided by spotify and tagged with various terms.
- User-made playlists (harder to find, but they exist, and many of them are fun!)
- Discover Weekly, Release Radar, Daily mixes - generated playlists for your account with 6-7 variations on how they are biased.
- Social features (see what your friends are listening to)
- Radios for songs/albums/artists
- Artist bios and "Fans also like" sections for each artist
- Smart shuffle on playlists (every 3rd song is a recommendation)
- A very permissive search box that lets you make mistakes (I hate competitors that punish me for writing things like "and" instead of "&" for an artist's name... I'm looking at you Tidal)
- Configurable search APIs to build your own funky queries or extensions
... and probably a few more tools to use when discovering music. An AI recommendation system will NEVER beat flexibility and giving users agency. I don't care that an AI takes into consideration that I went to the gym automatically. I care about having lots of options for discovery, each that is decent and biased towards a different style of exploration.
> Using transformers to incorporate different user actions based on the current user context helps steer music recommendations directly towards the user’s current need
What if the user's current need is to not play music? To not consume yet more content? To not make them addicted to the content application?
How can we optimize for user wellbeing, and still make money? That's the question we should be pouring resources into
> What if the user's current need is to not play music?
Then don't play music. You are asking for something that only works in a dystopia.
Either the machine tries to understand "what I need now" from partial information and will not play me music when I want to listen because it thinks it "knows what I need".
Or the machine actually is hooked up to my real time health data and possibly brain activity to actually know what I need.
I definitely don't want to have a personal computing machine thinking it knows what I want and deciding for me in such a way, or to have such access to my internal state.
You missed my point a little. We keep optimizing for more addictive services. It's just as dystopian for a service to always present exactly what will keep me using the service and giving thumbs up.
Every single entertainment service of product in the history of entertainment has been designed so the user has the most entertainment as possible, and its up to the user to moderate consumption. Nobody creates entertainment services that are boring on purpose just because they believe in temperance or whatever, and if they do, nobody uses them because by definition they are more boring than they could've been.
Move away from the songs strawman you're arguing against. My point was about content in general. As we get better and better at recommending the next content item based on engagement and likes, we make things more addictive. We don't have to do this. Not everything needs to be a slot machine that keeps you there as long as possible.
I love listening to one podcast episode each evening while I clean my kitchen. I do this each day. I would pay for this. I don't want this to hijack my brain and keep me up all night with content that requires an inordinate amount of willpower to put down.
There are plenty ways to provide content and make money off it without optimising for the best possible next item that keeps a person engaged until they fall asleep.
What might that look like in this situation? A user goes to play Spotify and it responds with “No” and shuts itself down? I generally agree with you that endless content consumption is a bad thing, but I also can’t envision a system where this is possible. It requires enough friction for the user to decide against continuing, which either comes in the form of a service providing less appealing content, making content more costly to consume, such as literally paying per song played, or services simply refusing to serve more content after a certain point. All of which are complete non-starters.
Those are non-starters, but there is a lot that can be done in the space. Not autoplaying the next item is a good example. Look at how services handle things in kids mode. That's just one way to think about it.
I often make the same criticism about services like this, but personally, discovering good new music greatly enriches my life.
Sometimes I'll bounce off of a given artist several times, over years or decades, until the right track catches me when I'm "ready" for it, and then I'll enjoy discovering their whole catalog. At that point, the affinity is durable. I would love to find whoever is my next (e.g.) Steely Dan.
https://cosine.club/ is the closest I've seen to the ideal branching system. My understanding is that it uses vector embeddings to search for songs that are similar in sound, and it works shockingly well for that purpose. However, it has a limited song database. Also see https://everynoise.com/, which is no longer updated. These use vector embeddings in similar ways, but the exploration experience is controlled by the user, not by a list-generating ranking model. I definitely think that AI-tech is the future of music recommendation, but I would prefer to see more research by large companies in to these user-driven systems, instead of the 'similar autosuggested list', which is, by its very nature, only ever 'good enough.'