Skip to content

Epic: Easy PDP: Simple Deployment, Reliable Operations, and Safe Dataset Lifecycle #1352

Closed
Task
Closed
Epic: Easy PDP: Simple Deployment, Reliable Operations, and Safe Dataset Lifecycle#1352
Task
Assignees
snadrusLexLuthr
Labels
team/fs-wgItems being worked on or tracked by the "FS Working Group". See FilOzone/github-mgmt #10
Milestone
MX: Priority and sequencing TBD

Description

@LexLuthr
LexLuthr
opened on Jul 10, 2026

Summary

Easy PDP turns Curio's PDP support into an operator-ready product covering the complete storage-provider journey: installation, onboarding, production operation, diagnosis, payment visibility, and safe dataset lifecycle management.

This epic combines two previously separate workstreams:

  1. The PDP-only / Skiff product path, focused on reducing installation complexity, dependencies, hardware requirements, and Filecoin-specific operational knowledge.
  2. The PDP production-hardening work, including task lifecycle management, chain processing, message submission, payment settlement, proving, retention, deletion, alerting, and operator controls.

“Easy” does not only mean that the binary is easy to install. An SP should be able to deploy PDP from a new machine, understand whether it is healthy, identify and recover from failures, assess the economics of affected services, and make deliberate decisions before Curio performs destructive actions.

Product outcomes

An SP should be able to:

  • Install a supported PDP-focused Curio distribution without assembling a full sealing-oriented Curio stack.
  • Complete onboarding through a guided workflow using clear service and datacenter terminology.
  • Verify chain connectivity, wallet funding, storage, network reachability, and PDP service readiness.
  • Operate PDP with actionable health, task, proving, payment, and storage information in the console.
  • Depend on predictable retries, backoff, priorities, fault handling, and terminal task states.
  • Review datasets affected by retention or proving problems before Curio terminates them.
  • See revenue already earned, projected future revenue, and the broader value of the customer relationship when deciding whether to continue or terminate a service.
  • Manually terminate an unprofitable or unrecoverable service during a 30-day review period.
  • Rely on Curio to terminate automatically only after that review period expires.

Completed items below are retained as foundations of the product. Open items and placeholders represent the remaining work.

1. PDP distribution, installation, and runtime footprint

The PDP-only path should have an explicit, supported installation mechanism with appropriate defaults and substantially fewer dependencies than a traditional Curio deployment. It should remain possible to upgrade the deployment to full Curio later.

2. Guided onboarding, console, and documentation

The console should be the primary product interface for setup and ongoing operation. It should guide the SP toward a ready-to-serve state, explain incomplete or unhealthy configuration, and expose relevant system information without requiring routine database or log inspection.

The first iteration should prioritize clear status and actionable next steps over exposing every internal Curio concept. Advanced diagnostic information may remain available, but should not be required for routine PDP operation.

3. Operational reliability and task lifecycle management

Every persistent PDP workflow should have an explicitly reviewed lifecycle: priority, idempotency where required, retry policy, backoff, fault detection, terminal failure behavior, alerting, and rules preventing an already terminal task from being rerun accidentally. This includes tasks triggered by chain events as well as scheduled background work.

Remaining work

Completed reliability foundations

4. Payments and service economics

Payment handling should continue to make progress during large backlogs, select valid settlement epochs, and clearly distinguish between temporarily retryable conditions and operator-actionable failures. The same payment data should provide the financial context needed when an SP reviews an affected dataset. ([GitHub]5)

  • Placeholder — Dataset and customer service economics
    • Identify the customer associated with each dataset or service.
    • Define the customer identity used for aggregation, including how datasets are grouped when payer, client, contract, or account identities differ.
    • Calculate and expose revenue already earned by each dataset or service.
    • Include settled payments and one-time payments.
    • Calculate projected future revenue for each dataset or service using the applicable service rate.
    • Aggregate the SP's complete relationship with the customer, including:
      • Total number of active datasets or services belonging to the customer.
      • Total revenue earned across those datasets or services.
      • Total projected future revenue across those datasets or services.
      • Aggregate recurring revenue or annualized revenue, where the service terms support that calculation.
    • Clearly distinguish the economics of the dataset under review from the economics of the customer's complete portfolio with the SP.
    • Define how dataset-level and customer-level projections behave when rate, duration, settlement, customer identity, or service-state information is incomplete.
    • Expose both dataset-level and customer-level calculations through an API suitable for the console review workflow.
    • Reuse the financial presentation foundation introduced in PR #1351.

These values are decision-support information. Curio should not automatically decide that an individual dataset or an entire customer relationship is unprofitable in the first version.

5. Retention, proving, deletion, and safe termination

Retention or proving failures must become persistent, actionable operator conditions rather than remaining only in logs or leading directly to an irreversible action. Curio now has an ongoing-alert foundation that can activate and resolve persistent conditions and avoid treating every transient task retry as an operator-facing incident. ([GitHub]6)

Existing foundation and related work

The existing lockup grace period and the new 30-day SP review period must be reconciled explicitly. They should not be treated as the same timer without defining which condition starts each period and how the two mechanisms interact.

New work

  • Placeholder — Retention and proving review state

    • Detect when a dataset is affected by a terminal or sustained retention/proving condition that can lead to termination.
    • Move the dataset into an explicit review required state.
    • Record the reason, first detection time, current state, and review deadline.
    • Activate a persistent operator alert through the alert manager.
    • Resolve the alert and cancel pending termination if the underlying condition recovers.
    • Preserve enough failure and service context for the SP to investigate the condition, including through external systems such as the MSP.
    • Maintain an audit trail of state transitions and operator actions.

  • Placeholder — 30-day graceful termination workflow

    • Start a 30-day operator-review period when the dataset enters the agreed review-required state.
    • Do not automatically terminate the dataset before the deadline.
    • Allow the SP to terminate the service manually at any point during the review period.
    • Automatically terminate only after the full review period expires without recovery or operator intervention.
    • Make manual and automatic termination idempotent.
    • Prevent duplicate scheduled termination work.
    • Record who or what initiated termination and when.
    • Define interaction with the existing 60-day lockup grace period.
  • Placeholder — Review to terminate page in the console
    • Show every dataset currently requiring review.
    • Do not apply a profitability or alert-volume threshold in the first version.
    • Show the reason the dataset requires review.
    • Show when the condition began and how much review time remains.
    • Show revenue earned so far by the dataset, including settled and one-time payments.
    • Show projected future revenue for the dataset based on the applicable service rate.
    • Allow the SP to sort the review queue by:
      • Revenue earned so far.
      • Projected future revenue.
    • Identify the customer associated with the dataset.
    • Show the broader value of the customer's relationship with the SP, including:
      • The customer's total number of active datasets or services with the SP.
      • Total revenue earned across the customer's datasets or services.
      • Total projected future revenue across the customer's datasets or services.
      • Aggregate recurring or annualized revenue, where applicable.
    • Clearly distinguish the value of the dataset under review from the aggregate value of the customer's portfolio.
    • Allow the SP to inspect the customer's other datasets or services before terminating the affected dataset.
    • Link to relevant proving, retention, task, payment, customer, and service information.
    • Provide a deliberate, confirmed Terminate service action.
    • Present the customer-level context as part of the termination confirmation so that the dataset is not evaluated in isolation.
    • Clearly distinguish manual termination, scheduled automatic termination, recovery, and completed termination.

Filtering, prioritization, customer grouping, and configurable thresholds may be expanded later if showing all review-required datasets becomes too noisy.

Filtering, prioritization, and configurable thresholds may be added later if showing all review-required datasets becomes too noisy.

Definition of done

  • A supported Easy PDP installation can take a new machine to a ready-to-serve PDP deployment through documented steps.
  • The PDP deployment does not require sealing-specific dependencies or workflows.
  • The deployment has a documented upgrade path to full Curio.
  • Required Synapse integration tests pass as part of the Easy PDP release process.
  • The console guides the SP through setup and reports readiness without requiring routine log or database inspection.
  • The PDP task lifecycle audit is complete, and discovered gaps are fixed or tracked by explicit follow-up issues.
  • Critical chain, message, proving, payment, deletion, and removal failures produce actionable state and alerts.
  • Affected datasets enter a review-required state before automatic termination.
  • SPs receive the complete 30-day manual review period.
  • The console shows all review-required datasets without an initial threshold.
  • Earned revenue and projected future income are available for each reviewed service.
  • SPs can terminate manually after reviewing the service.
  • Curio terminates automatically only after the deadline and only when the condition remains applicable.
  • Recovery, manual termination, and automatic termination are idempotent and auditable.
  • Curio contains the canonical installation and operating documentation for Easy PDP.
  • The Review to terminate page can be sorted by revenue earned so far and projected future revenue.
  • Each reviewed dataset identifies the customer associated with it.
  • The console shows the number and aggregate economic value of the customer's other active datasets or services with the SP.
  • Dataset-level economics are clearly separated from customer-level portfolio economics.
  • The SP can inspect the customer's broader relationship before confirming manual termination.

Initial non-goals

  • Automatically deciding whether a service is profitable enough to retain.
  • Applying profitability, revenue, or alert-volume thresholds to the initial review queue.
  • Automatically terminating an affected dataset before the 30-day review period expires.
  • Absorbing unrelated Curio or PoRep maintenance solely because it was part of the same milestone.
  • Including one-off customer migration tooling, such as [PR #1264](Storacha migration tool #1264), unless it is generalized into a supported Easy PDP migration capability.

Metadata

Metadata

Assignees

Labels

team/fs-wgItems being worked on or tracked by the "FS Working Group". See FilOzone/github-mgmt #10

Type

Task

Projects

Easy PDP
Status
Done

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions