Skip to content

waitForIpniProviderResults: surface per-CID outcomes for diagnostic visibility #417

Description

@SgtPooki

Problem

waitForIpniProviderResults returns Promise<boolean> and on failure throws a single Error whose message references one CID (the last one checked). Consumers cannot tell, after the fact, which CIDs in [rootCid, ...childBlocks] were verified vs. timed out vs. missing-expected-provider vs. fetch-errored.

This matters most on the abort/timeout path: when an outer AbortSignal.timeout(...) fires mid-walk, the in-flight fetch throws AbortError and we lose which CID was current and which CIDs had already been verified successfully.

Concrete example from the dealbot consumer (FilOzone/dealbot):

  • Deals are uploaded with rootCID + ~12 childBlocks. Outer timeout 60s, delayMs 2500ms, so each CID gets ~25 attempts.
  • 7 of 8 SPs verify cleanly at p50 ~2.4s. SP6 (Hangzhou CN) verifies at p50 ~31s with 39% slow_verification rate; ~22 deals/24h time out at the 60s budget.
  • All timeouts surface as a single generic message. We currently cannot distinguish "rootCID never indexed" from "rootCID indexed but childBlock N never indexed."
  • Investigation: IPNI xxx root CID not verified FilOzone/dealbot#473 (comment)

Why this is currently low priority

  1. In observed dealbot data, all timeouts are wall-clock SLO breaches, not selective "missing expected provider" failures. Per-CID granularity would not have shifted root-cause attribution (path quality between filecoinpin.contact / Reston VA and SP6 / Hangzhou CN, AS4134 ChinaNet, ICMP packet loss 24% from a peer test vantage).
  2. At dealbot's current SLO budget (60s with 2.5s polls and 12 CIDs), only the first 4-5 CIDs get any attempts before the outer abort fires anyway — so child-CID granularity is not measurable end-to-end without also rebudgeting.
  3. PR fix(ipni): preserve failure reason in deal-flow rethrow FilOzone/dealbot#490 + feat(ipni): expose failureReason as top-level field on ipni_tracking_failed FilOzone/dealbot#491 close the immediate visibility gap that prompted dealbot#473 by carrying the existing single failureReason via Error.cause.

Proposal (when prioritized)

Change return type from Promise<boolean> to something like:

```ts
type IpniValidationOutcome = {
verified: Array<{ cid: CID; matchedAt: number; attempts: number }>;
failed: Array<{
cid: CID;
reason:
| { type: 'timeout'; attempts: number }
| { type: 'missingProviders'; missingServiceUrls: string[]; actualMultiaddrs: string[] }
| { type: 'fetch'; message: string }
| { type: 'parse'; message: string }
| { type: 'http'; status: number; statusText?: string };
}>;
ipniIndexerUrl: string;
};
```

Existing single-error throw can be preserved as a thin wrapper for callers that prefer the boolean shape, or replaced with always returning the outcome and letting callers decide whether to throw.

Acceptance criteria

  • waitForIpniProviderResults returns or exposes per-CID verified/failed lists.
  • Abort during setTimeout(delayMs) (currently signal-unaware at https://github.com/filecoin-project/filecoin-pin/blob/main/src/core/utils/validate-ipni-advertisement.ts#L387) is preserved or fixed in the same change. Optional related fix: make the inter-attempt sleep signal-aware so abort overshoot is bounded by the in-flight fetch rather than by delayMs.
  • Existing consumers that only need pass/fail can keep that behavior via a wrapper or a boolean accessor on the new return type.
  • Tests cover: timeout-mid-walk, partial verification (some CIDs verified, some failed), all-CIDs-missing, single-CID success.

Consumer signal

Open this when:

  • A real dealbot incident appears where timeout reason "some childBlock never indexed" needs to be distinguished from "rootCID never indexed."
  • Or filecoin-pin gains another consumer that needs per-CID outcomes for its own product semantics.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestready-for-workTriaged: scope, plan, and DoD are clear; contributor can pick upteam/filecoin-pin"Filecoin Pin" project is a stakeholder for this work.team/fs-wgFOC working group is a stakeholder for this work, and thus wants to track it on their project board.

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions