快手限制下载视频的数量

[MYF12]

当爬取数量达到几十个视频之后，就会出现下面的报错，等了很久之后才能继续下载。而且尝试过使用其他ip来继续请求也没用

| kuaishou-crawler (v0.5.0 20-08-06)
| 本程序由oGsLP提供, www.github.com/oGsLP/kuaishou-crawler, 喜欢的话可以给个star >_<

准备开始爬取，共有1个用户...

{"data":{"privateFeeds":{"pcursor":"","list":[],"__typename":"PCProfileFeeds"}}}

[]
Traceback (most recent call last):
File "D:/python_project/test1/crawlers/main.py", line 26, in
main()
File "D:/python_project/test1/crawlers/main.py", line 20, in main
kuaishou.crawler_kuaishou.main()
File "D:\python_project\test1\crawlers\kuaishou\crawler_kuaishou.py", line 30, in main
crawl(param_did,data_dir)
File "D:\python_project\test1\crawlers\kuaishou\crawler_kuaishou.py", line 15, in crawl
crawler.crawl()
File "D:\python_project\test1\crawlers\kuaishou\lib\crawler.py", line 81, in crawl
self.__crawl_user(uid)
File "D:\python_project\test1\crawlers\kuaishou\lib\crawler.py", line 106, in __crawl_user
if works[0]['id'] is None:
IndexError: list index out of range

[MYF12]

请问这是什么原因呢？

[oGsLP]

现在很容易被检测出来限制了

[MYF12]

请问有解决方案吗？比如通过ip池去请求之类的。

[oGsLP]

理论上可行，或者可以先保存json下来，或者保存到数据库，然后一个个爬，被限制了，接着之前的爬就行

[MYF12]

你有试过通过代理爬取吗？

[oGsLP]

没有