Open
Description
The MIME policy drops requests before they finish receiving but doesn't store any metadata about them. This UX can be confusing because if the crawler is dropping lots of responses it will just look like the crawl isn't doing anything. Should they be stored as exceptions? Or in another category?