-
Notifications
You must be signed in to change notification settings - Fork 5
Open
Description
In #76 it was established that a significant amount of time in a check is consumed just reading filters. Most of this work is unnecessary: here is a typical IO pattern for a check that confirms non-revocation:
"20260118-0-default.filter": NotCovered (6337902 bytes)
"20260118-1-default.filter.delta": NotCovered (144518 bytes)
"20260119-0-default.filter.delta": NotCovered (168917 bytes)
"20260119-1-default.filter.delta": NotCovered (160074 bytes)
"20260120-0-default.filter.delta": NotCovered (150195 bytes)
"20260120-1-default.filter.delta": NotCovered (151210 bytes)
"20260121-0-default.filter.delta": NotCovered (131186 bytes)
"20260121-1-default.filter.delta": NotCovered (143546 bytes)
"20260122-0-default.filter.delta": NotCovered (141617 bytes)
"20260122-1-default.filter.delta": NotCovered (136753 bytes)
"20260123-0-default.filter.delta": NotCovered (131369 bytes)
"20260123-1-default.filter.delta": NotCovered (146417 bytes)
"20260124-0-default.filter.delta": NotCovered (148553 bytes)
"20260124-1-default.filter.delta": NotCovered (135185 bytes)
"20260125-0-default.filter.delta": NotCovered (137457 bytes)
"20260125-1-default.filter.delta": NotCovered (136473 bytes)
"20260126-0-default.filter.delta": NotCovered (131625 bytes)
"20260126-1-default.filter.delta": NotCovered (141386 bytes)
"20260127-0-default.filter.delta": NotCovered (181225 bytes)
"20260127-1-default.filter.delta": Good (146882 bytes)
In an ideal world, we could determine more quickly that "20260127-1-default.filter.delta" is relevant to the operation and reach straight for it.
If we maintained an index (computed locally on each fetch) that duplicates the data required for the existing in_universe check in each filter, we could determine (from data in one ~small file) what the relevant filters are.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels