(Feature Request): Request for ability to download the RYD database.

[eclipsek20]

Extension or Userscript?

Extension

Request or suggest a new feature!

As the title suggests this would allow other people to work on a similar project by themselves, by not doing this RYD is basically going to become a monopoly. If they don't give this ability, I guess the saying "You either die a hero or live long enough to see yourself become the villain" makes a lot of sense.

Ways to implement this!

No response

Can you work on this?

• Yes
• No

[SirLich]

by not doing this RYD is basically going to become a monopoly

Isn't that the point of a collective replacement for the dislike counter? Fracturing the data is the worst thing you could do.

[Mohamed3on]

I'd love this to happen if only for data analysis reasons, to be able to sort videos by like/dislike counts, etc.

[eclipsek20]

by not doing this RYD is basically going to become a monopoly

Isn't that the point of a collective replacement for the dislike counter? Fracturing the data is the worst thing you could do.

What if the database gets deleted or worse the author decides to make a paywall?

[dvingerh]

I believe it's relevant to mention that the scraped data of videos prior to the removal of the dislike counter can be found publicly here:
https://wiki.archiveteam.org/index.php/YouTube#Removal_of_public_video_dislikes_.28December_2021.29
https://archive.org/details/archiveteam_youtubedislikes

[severtheskyline]

Because this extension has similar features to sponsorblock (IE. An public API) Would make sense to be able to download the database.

[eclipsek20]

I guess we will not receive any answers for this...

[sy-b]

The owner had previously replied in positive to these suggestions.

e.g. here
#344 (comment)
#224 (comment)

https://discord.com/channels/909435648170160229/916645010529726464/920395414753149009
https://discord.com/channels/909435648170160229/916645010529726464/922378431356887071

[eclipsek20]

It seems that in spite of "positivity" towards this suggestion, no action to date has been taken apart from lip service.

[sy-b]

Userbase fragmentation might be a concern.

^{Originally posted by @Anarios in /issues/45#issuecomment-997230662_ December 18, 2022 UTC}

By the way - a bunch of copycat-extensons died today once I enabled IP rate limiting.


They were just calling my api in their backend - no own DB, no caching - nothing. Just pretending to provide a service while in reality they didn't. Now imagine they had a DB dump and server code - what good would it make - more userbase fragmentation, less reliable votes? And all while using my work for free.

Posted here: #45 (comment)

[eclipsek20]

That last statement is that of great concern, "And all while using my work for free.", whilst do believe that he has some right to be compensated, it also greatly increases the probability of the scenario where the author going rouge, e.g. a paywall. The statement "more userbase fragmentation, less reliable votes?", is just being used as a way to justify not giving the db to the people, also if fragmentation is such a scary thing then why does SponsorBlock not worry about it? (This is a rhetorical question). I do not want to discredit the author, but there are signs that the following may be happening: he might think he is entitled to the data and therefore everyone should bow to him for allowing access to the API. If I am wrong then please forgive me otherwise well I would be speechless.

[sy-b]

That last statement is that of great concern, "And all while using my work for free.",

I don't think so. That statement (acc. to my view) was probably towards those freeloaders pretending to provide dislike data. That is just my guess cause I am not the owner of this repo.

whilst do believe that he has some right to be compensated, it also greatly increases the probability of the scenario where the author going rouge, e.g. a paywall.

The paywall will technically destroy the project. This project is dependent on it users. More users == better data/service. Of course this also has server costs which are hopefully covered by Patreon supporters.

The statement "more userbase fragmentation, less reliable votes?", is just being used as a way to justify not giving the db to the people,

idk but uploading TBs of data when only a few actually have the capacity to storage download it, doesn't seem very useful to me. I do want the data but how many others can afford it (storage)? My personal opinion is that the data dumps must be provided but unlike like sponsorblock's live dumps, these should be biweekly or monthly or whatever suits the bests.
Also, hosting & serving such large dumps isn't going to be cheap.

I do not want to discredit the author, but there are signs that the following may be happening: he might think he is entitled to the data and therefore everyone should bow to him for allowing access to the API

😆 . Well I feel its going in some other direction. From what I notice around me I can say that people hardly care about dislikes (Just tested this out). Most of them have acclimatized to not having dislikes. But that's just about people around me. So, getting rogue will most probably lead to downfall of this extension. (Backend can be fiddle with without any publicly visible effects).

---

Note

I am not @Anarios & these are my own perspectives.

---

@Anarios can you update us on your plans about database dumps?

[sy-b]

I think the database should be segmented into small downloadable chunks (10MB) & should be hosted on torrent. Let the community handle its hosting 😄.

[sy-b]

Another idea -
This database can be live hosed if RYD server uploads the changes every few minutes/hours. This might be problematic when used with torrents, but I think gun.eco might help here.

Some people were requesting decentralized database, this can be it.

[RuboGubo]

To be honest (and i know this will be unpopular) this sounds like what a block-chain would fix. After all, the block chain does not have to have money, and can simply be there to allow people to have a copy of the database that is automatically synced with the rest of the databases.

It solves fragmentation and centralization without necessarily money.