Goal: Enable the scraper to collect video URLs from TikTok hashtag pages.
Tasks:
- Research TikTok's hashtag page structure and identify video elements.
- Develop a method to paginate through hashtag pages to capture a large number of videos.
- Implement code to extract video URLs from hashtag pages.
- Add a command-line option or configuration setting to switch between channel and hashtag scraping modes.
- Enhance URL Extraction
Goal: Provide more options for organizing downloaded video files by hashtag.
Tasks:
- Add options to organize videos by date, uploader, or hashtag.
- Implement a configurable naming scheme for downloaded files.
- Update Documentation
Goal: Ensure the project is easy to use and understand.
Tasks:
- Update the README file with instructions on how to use the new hashtag scraping feature.
- Provide examples of how to use the scraper with both channel and hashtag URLs.
- Document any new configuration options.
- Testing and Bug Fixing
Goal: Improve user experience by providing flexibility in configuration
Tasks:
- Allow users to specify the number of videos to scrape.
- Implement options for setting download directory, number of threads, and other parameters via command line.
- Consider a configuration file for persistent settings.
- Implement Rate Limiting Handling
[ anon request ]
"For example, I want to crawl all video links from the hashtag #sontungmtp. Currently, this hashtag has 592.2K posts, but when scrolling manually on the website, I can only get around 300 videos.
If your application can actually retrieve information from all 592.2K posts, I would be very happy
I have a few questions:
1.) If I input the hashtag sontungmtp, is it really possible to get data from all 592.2K posts?
2.) Is it free or paid?"
Goal: Enable the scraper to collect video URLs from TikTok hashtag pages.
Tasks:
Goal: Provide more options for organizing downloaded video files by hashtag.
Tasks:
Goal: Ensure the project is easy to use and understand.
Tasks:
Goal: Improve user experience by providing flexibility in configuration
Tasks:
[ anon request ]
"For example, I want to crawl all video links from the hashtag #sontungmtp. Currently, this hashtag has 592.2K posts, but when scrolling manually on the website, I can only get around 300 videos.
If your application can actually retrieve information from all 592.2K posts, I would be very happy
I have a few questions:
1.) If I input the hashtag sontungmtp, is it really possible to get data from all 592.2K posts?
2.) Is it free or paid?"