Discussion Thread for Youtube Connector

[staru09]

This is a discussion thread for the issue #799.

[staru09]

So when we are talking about the yt connector, what all functionalities are expected from the tool.
The issue mention using transcription to answer content of the video along with relevant metadata. So can we start with this scope only or is it something else that needs to be done?

[wenxi-onyx]

Hey @staru09, thanks for creating the thread! I'd like to hear your proposal for this connector. These are the things I'm thinking about:

• Are there any legal implications of indexing this data?
• How would we retrieve the transcripts?
  -- Is there an official API or Python module?
  --If there isn't an official one (or if it isn't viable), is the best 3rd-party option regularly updated and supported long-term?
• How would you use this connector? Is there a real need for this? (we don't necessarily want to add more connectors if they aren't genuinely used by the community)

[staru09]

• Are there any legal implications of indexing this data?

Regarding this, I am assuming that the google allows it's users to scrape data from youtube as long as it comes under fair play but the definition of fair play is ambiguous and vague so we need to figure out what could be the potential use-cases of this connector.
We can refer to these.

--If there isn't an official one (or if it isn't viable), is the best 3rd-party option regularly updated and supported long-term?

There is an official support to fetch captions from videos but based on my experience with this library, there are some issues -

1. This method reaches it's rate limit quite often, so we will need a bunch of residential proxy servers to get it working.
2. I haven't been able to generate very accurate transcripts from this method.

• How would we retrieve the transcripts?

I think as far as the accuracy of generated transcripts in concerned we need to use an ASR model after extracting the audio file from the videos, how we were doing it at bitcoin dev project was to use this to download and process the audio file and then use an ASR model like deepgram and whisper to generate transcripts, we can do something similar here as well but we need to figure out the use-cases before diving deeper.

• How would you use this connector? Is there a real need for this? (we don't necessarily want to add more connectors if they aren't genuinely used by the community)

I think this could be really useful as an AI that can summarize topics, extract insights from long videos, and enable Q&A with the content something that wouldn’t have been possible otherwise. Many people don’t have the time to watch videos longer than 20–25 minutes, so this connector can help bridge that gap.

[wenxi-onyx]

A strong use case I can imagine is indexing a series of lectures, educational content, or history videos. In my mind, this could justify the connector.

I'm quite wary of downloading and processing the audio locally. This would add more dependancies and overall weight to the project and we're already quite heavy given the Vespa container and local embedding model. Given this, I would push back on implementing this connector unless there is a high-quality, permissive-licensed library we could use.

[staru09]

I liked the use case but yes processing audio files locally will be an extra load on the system, in order to remove this load we can go ahead with this library, the only issue would be of rate limits with this.

[wenxi-onyx]

@staru09 This is in that library's README

Unfortunately, YouTube has started blocking most IPs that are known to belong to cloud providers (like AWS, Google Cloud Platform, Azure, etc.), which means you will most likely run into RequestBlocked or IpBlocked exceptions when deploying your code to any cloud solutions. Same can happen to the IP of your self-hosted solution, if you are doing too many requests. You can work around these IP bans using proxies. However, since YouTube will ban static proxies after extended use, going for rotating residential proxies provide is the most reliable option.

Given this, I am inclined to say there is no good way to implement this connector at this time.

[staru09]

Yes it's better to wait till a library that is backed by yt comes for this usecase or the proxy ban should be removed. Anyways thanks for the feedback. Meanwhile, I'll look into other issues to see if I can contribute there ;)