[DISCUSSION]: How do we make transitions better (AutoMix, Crossfade)

[gabcaua]

I'm creating this discussions because I've been trying to make AutoMix on a private use case of NodeLink (you may know Listen bot, I tried doing transitions better there, but computational cost are getting crazy and output is not close to what I think it's possible).

Recently, the green music app and the pink/purple one introduced AutoMix, a feature the green app has been testing since 2021. And there's a lot of data unused out there in this matter.

Also, NodeLink was one of the first Discord node projects to use Spotify's Mobile Endpoints/API for needed source tasks.

Enough of that, basically the idea is.

1. We start supporting Spotify Mixed Playlists with their Protobuf API (reverse engineering that I can't do alone)
2. We train a model taking "Custom" transitions (UGC, user generated content, user-made, community, human-made) as the ground-truth (what we're aiming for as output given an input of data), which are far better and DJ-like than "Auto" transitions.
3. For data, we use Musical Information Retrievers (MIRs), NodeLink has plenty of those, and they operate in real-time, which could enable us to make an inference/prediction with the model every n seconds.

Basically, I'm proposing we use Machine Learning to enhance AutoMix/Crossfade decisions fast on CPU.

For those who don't know, inference/execution of already trained machine learning models is cheap and extremely fast, what slows it down usually is cold boot from loading the model weights, the heaviest models are large language models [LLMs] and vision models, but I'm proposing normal machine learning, which is very fast.

Spotify recently added Looping (you can loop a bar, it turns yellow and loops), clicking a bar on one of the decks (A or B) makes it loop til the end of the deck's presence/volume value/number.

And it added beatmatching, which I noticed months ago in their protobuf APIs of Spotify Desktop (macOS).

But they're implemented first on Android/Mobile, which NodeLink already uses a bit on spotify.ts Source Manager. So I was thinking you guys, with your experience, could help build that.

Additionally, I think assuring the quality of the MIR data ("input data") we give to the model is of extreme importance, so the model can make sense of a output numerically close to the ground-truth, approximate.

ScreenRecording_08-28-2026.19-41-46_1.mov

[gabcaua]

Basically we need to teach something to pretend to be a human creating a Spotify Mixed Playlist.

The tough part it's getting Spotify ground truth (good CUSTOM transitions of Mixed Playlists)

Another tough piece in this is getting the model to have enough input data that it can "understand" the music
(have a sort of "canvas" to work with and only so it can "paint" what it thinks is the right/closest answer to an estimated ground-truth/DJ/custom transition)

Spotify custom transitions playlists get so many traction, saves, likes, they are getting popular and are a big corpus of data for this. The wet dream of data scientists, not gonna lie.

Summarizing: predict a compact transition plan from MIR + track-pair context, supervised by human-authored transition parameters