Load all tracks

[Mees-Molenaar]

Hi,

First off, thanks for creating this!
Unfortunately, the current implementation doesn't work for playlists with more than 100 tracks, because the endpoint that is currently used, has a limit of responding with 100 tracks.
I have adjusted the implementation, but was not able to push my branch to this repository.
So if you want to take over the implementation, here is the code:

import { SPOTIFY_API_BASE, SPOTIFY_TOKEN_SOURCE } from "../data/config.ts" ;

export async function getSpotifyToken(): Promise<string> {
  const response = await fetch(`${SPOTIFY_TOKEN_SOURCE}`, {
    mode: 'cors',
    headers: {
      'Accept': 'application/json'
    }
  });
  const data = await response.json();
  console.log(data)
  return data['token'];
}

export function extractPlaylistId(url: string): string {
  const patterns = [
    /spotify:playlist:([a-zA-Z0-9]+)/, // Spotify URI
    /playlist\/([a-zA-Z0-9]+)/, // Web URL
    /^([a-zA-Z0-9]+)$/, // Direct ID
  ];

  for (const pattern of patterns) {
    const match = url.match(pattern);
    if (match) {
      return match[1];
    }
  }

  throw new Error('Invalid Spotify playlist URL or ID');
}

export async function getTotalTrackNumber(playlistId: string, token: string): promise<number> {
  const response = await fetch(
    `${SPOTIFY_API_BASE}/playlists/${playlistId}`,
    {
      headers: {
        Authorization: `Bearer ${token}`,
      },
    }
  );

  if (!response.ok) {
    throw new Error('Failed to fetch playlist data');
  }

  const data = await response.json();

  const total_songs = data.tracks.total;

  return total_songs;
}

export async function fetchPlaylistTracks(playlistId: string): Promise<Song[]> {
  const token = await getSpotifyToken();

  const total_songs = await getTotalTrackNumber(playlistId, token);
  const step = 100; // NOTE: You can fetch 100 tracks in one request

  var all_tracks = [];

  for (let i = 0; i <= total_songs; i += step) {
    const tracks_response = await fetch(
      `${SPOTIFY_API_BASE}/playlists/${playlistId}/tracks?fields=items(track(uri,name,artists(name),album(release_date)))&offset=${i}`,
      {
        headers: {
          Authorization: `Bearer ${token}`,
        },
      }
    );

    if (!tracks_response.ok) {
      throw new Error('Failed to fetch playlist data');
    }

    const track_data = await tracks_response.json();

    const tracks = track_data.items
    .filter((item: { track: SpotifyTrack | null }) => item.track)
    .map((item: { track: SpotifyTrack }) => ({
      title: item.track.name,
      artist: item.track.artists[0].name,
      year: parseInt(item.track.album.release_date.substring(0, 4), 10),
      spotifyUri: item.track.uri,
    }));

    all_tracks = [...all_tracks, ...tracks];

  }

  return all_tracks;
}

Basically, in the above code, first you fetch the total of numbers of tracks and then you request the tracks (100 at a time) until you got everything.

Kind regards,
Mees

[Theys96]

Hi Mees, thanks for contributing! This was kept like this for now sort of intentionally, because I feared playlists of thousands of songs may crash the browser. Do you have ideas to mitigate that? I'm thinking one of two options:

1. A hard limit of the number of imported tracks. For example, 960 (80 pages of 12).
2. A soft limit with a warning to proceed at own risk.

[Mees-Molenaar]

That’s a valid concern! However, in most cases, premature optimisation for this kind of scenario isn’t necessary.

I tested a request for 100 tracks, and the returned data is only about 5.14 KB. While not all of it is stored, JavaScript likely has some overhead. Even if we assume that 100 tracks take up around 5.14 ( I have no idea if this assumption is correct though :P) KB in memory, that means 1 million tracks would only take about 5 MB. I don’t think that would be a problem for a browser.

That said, rendering all of them at once could be an issue. A couple of potential solutions:
• Pagination: Display a maximum of 100 tracks per page.
• Virtualized Rendering: A library like react-window (which I haven’t personally used, but it looks promising) could help by rendering only the visible list items instead of all at once.

Hope this helps! 😊

[Theys96]

With #2, I've made a first step here by storing the track data in local storage. This is a prerequisite because when handling larger sets of tracks, losing progress would become increasingly painful.

[DickenSerm]

Hey, first of all, I'd like to say thank you for this great project.
I'd really like to import a playlist with more than 100 songs.

@Mees-Molenaar `s code is supposed to be able to do this.

I'm just wondering how I can host the project myself with the modified code to make this possible.

I would really appreciate it if you could briefly explain this to me.
Hosting options and a dedicated server with Proxmox are available.

[DickenSerm]

Hey, first of all, I'd like to say thank you for this great project. I'd really like to import a playlist with more than 100 songs.

@Mees-Molenaar `s code is supposed to be able to do this.

I'm just wondering how I can host the project myself with the modified code to make this possible.

I would really appreciate it if you could briefly explain this to me. Hosting options and a dedicated server with Proxmox are available.

It's done :) I actually managed it :D