This repository was archived by the owner on Oct 11, 2025. It is now read-only.

This repository was archived by the owner on Oct 11, 2025. It is now read-only.

抓取网页出现HTTP ERROR处理问题 #52

Open

Labels

opened

on Oct 16, 2015

1.http error 404 没有丢弃url
2.其他错误在爬虫执行完后，继续尝试，但不能无限次尝试（有的时候会出现爬虫任务根本停不下来）

to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

Labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests