-
Notifications
You must be signed in to change notification settings - Fork 327
Open
Labels
Description
Originally reported in #989, but I noticed it myself while playing around with some crawl data recently. The problem stems from the listener we're using to record DNS entries.
OpenWPM/Extension/src/background/dns-instrument.ts
Lines 29 to 44 in ab5fac2
| /* | |
| * Attach handlers to event listeners | |
| */ | |
| this.onCompleteListener = (details: WebRequestOnCompletedEventDetails) => { | |
| // Ignore requests made by extensions | |
| if (requestStemsFromExtension(details)) { | |
| return; | |
| } | |
| const pendingResponse = this.getPendingResponse(details.requestId); | |
| pendingResponse.resolveOnCompletedEventDetails(details); | |
| this.onCompleteDnsHandler(details, crawlID); | |
| }; | |
| browser.webRequest.onCompleted.addListener(this.onCompleteListener, filter); | |
| } |
The onCompleted event is only fired after redirects are resolved. Instead, we want to use an event that is part of the redirect loop. e.g., onHeadersReceived. I tested this and it works well, but I still need to add some tests.