Skip to content

DNS Instrument only records the final request in a redirect chain #1020

@englehardt

Description

@englehardt

Originally reported in #989, but I noticed it myself while playing around with some crawl data recently. The problem stems from the listener we're using to record DNS entries.

/*
* Attach handlers to event listeners
*/
this.onCompleteListener = (details: WebRequestOnCompletedEventDetails) => {
// Ignore requests made by extensions
if (requestStemsFromExtension(details)) {
return;
}
const pendingResponse = this.getPendingResponse(details.requestId);
pendingResponse.resolveOnCompletedEventDetails(details);
this.onCompleteDnsHandler(details, crawlID);
};
browser.webRequest.onCompleted.addListener(this.onCompleteListener, filter);
}

The onCompleted event is only fired after redirects are resolved. Instead, we want to use an event that is part of the redirect loop. e.g., onHeadersReceived. I tested this and it works well, but I still need to add some tests.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions