Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,045 changes: 1,045 additions & 0 deletions .agents/skills/incur/SKILL.md

Large diffs are not rendered by default.

3 changes: 2 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,5 +32,6 @@
"[mdx]": {
"editor.defaultFormatter": "DavidAnson.vscode-markdownlint"
},
"markdown.validate.enabled": false
"markdown.validate.enabled": false,
"typescript.tsdk": "node_modules/typescript/lib"
}
2 changes: 1 addition & 1 deletion apps/ponder/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
"devDependencies": {
"@biomejs/biome": "catalog:",
"@types/node": "catalog:",
"drizzle-orm": "catalog:",
"drizzle-orm": "^0.45.2",
"typescript": "catalog:"
}
}
10 changes: 0 additions & 10 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@
"lint:fix": "biome check --fix .",
"test": "pnpm -r --filter='./packages/*' --if-present run test",
"check:repo": "pnpx sherif@latest -r root-package-manager-field",
"update:msw": "pnpm -r --filter='./packages/*' --if-present run update:msw",
"clean": "rm -rf node_modules pnpm-lock.yaml package-lock.json packages/*/{.wireit,pnpm-lock.yaml,package-lock.json,dist,node_modules} apps/*/{.wireit,pnpm-lock.yaml,package-lock.json,dist,node_modules}",
"clean:cache": "rm -rf packages/*/{.wireit,dist} apps/*/{.wireit,dist}"
},
Expand All @@ -20,15 +19,6 @@
"typescript": "catalog:",
"wireit": "^0.14.12"
},
"pnpm": {
"overrides": {
"@hono/node-server": "^1.19.13",
"drizzle-orm": "^0.45.2",
"esbuild": "^0.25.0",
"kysely": "^0.28.14",
"vite": "^6.4.2"
}
},
"packageManager": "pnpm@10.33.0",
"devEngines": {
"runtime": {
Expand Down
22 changes: 22 additions & 0 deletions packages/repair-cli/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Early repair CLI (`@filoz/repair-cli`)

CLI for preparing and running early repair jobs that move pieces away from a faulty Filecoin service provider and into a target PDP provider.

The package is built with [incur](https://www.npmjs.com/package/incur), Drizzle ORM, and `@filoz/synapse-core`. It uses the indexer Postgres database as a read-only catalog of providers, datasets, and pieces, plus a local SQLite database for repair jobs and per-piece operation state.

## Conventions

- Prefer existing command, middleware, and DB helper patterns over new abstractions.
- Extract reusable indexer queries and local database mutations under `src/db/`.
- Keep command files focused on CLI arguments, context wiring, and response shaping.
- Add JSDoc on exported functions/types; use inline comments only for non-obvious logic such as dedupe, pagination, or on-chain state sync.
- Repairs use one IPFS-enabled target dataset with CDN disabled. Do not add per-operation dataset grouping.
- Use `contextMiddleware` for commands that need config, wallet client, indexer DB, or local DB access.
- Do not document or preserve compatibility with unshipped in-progress behavior; update docs to match the current implementation.

## Build And Lint

```bash
pnpm --filter @filoz/repair-cli build
pnpm --filter @filoz/repair-cli lint
```
32 changes: 20 additions & 12 deletions packages/repair-cli/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -102,26 +102,34 @@
]
}
},
"dependencies": {},
"dependencies": {
"@clack/prompts": "^1.5.1",
"@filoz/repair-db": "workspace:*",
"@filoz/synapse-core": "^0.6.0",
"@libsql/client": "^0.17.3",
"conf": "^15.1.0",
"drizzle-kit": "^0.31.10",
"drizzle-orm": "catalog:",
"incur": "^0.4.6",
"iso-base": "^4.4.0",
"iso-web": "^3.1.2",
"p-all": "^5.0.1",
"p-locate": "^7.0.0",
"p-map": "^7.0.4",
"p-queue": "^9.3.0",
"pg": "^8.21.0",
"terminal-link": "^5.0.0"
},
"devDependencies": {
"@biomejs/biome": "catalog:",
"@types/assert": "^1.5.11",
"@types/mocha": "catalog:",
"@types/node": "catalog:",
"assert": "^2.1.0",
"mocha": "catalog:",
"msw": "catalog:",
"@types/pg": "^8.20.0",
"playwright-test": "^14.1.12",
"type-fest": "^5.4.3",
"type-fest": "^5.7.0",
"typescript": "catalog:",
"viem": "catalog:"
},
"publishConfig": {
"access": "public"
},
"msw": {
"workerDirectory": [
"test/mocks"
]
}
}
232 changes: 230 additions & 2 deletions packages/repair-cli/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,243 @@

> Early repair for faulty service providers and datasets

The `repair` CLI helps prepare and run repair jobs that move pieces away from a faulty Filecoin service provider and into a target PDP provider. It uses:

- an indexer Postgres database as the read-only source of providers, datasets, and pieces
- a local SQLite database to track repair jobs and per-piece operations
- a configured Filecoin wallet to create datasets and submit on-chain add-piece transactions

## Installation

```bash
pnpm install @filoz/repair-cli
pnpm add -g @filoz/repair-cli
```

The package exposes the `repair` binary.

```bash
repair --help
```

## Setup

Run setup before using any command that talks to the indexer, local database, or wallet.

```bash
repair setup
```

Setup prompts for:

- private key for the repair wallet
- mainnet indexer Postgres URL
- calibration indexer Postgres URL
- chain, either Filecoin Mainnet `314` or Filecoin Calibration `314159`
- local SQLite database path

The command stores these values in the CLI config and runs the local SQLite schema migration. It returns the configured wallet address.

Most commands also accept:

```bash
--debug
```

Use `--debug` when you want extra error output from wallet operations.

## Command Reference

### `repair setup`

Interactive configuration and local database setup.

```bash
repair setup
```

Use this whenever you need to initialize the CLI, change the active chain, update indexer URLs, or move the local SQLite database.

### `repair wallet fund`

Funds the configured wallet from the Filecoin Calibration faucet.

```bash
repair wallet fund
```

This command only works on Calibration. It claims faucet tokens, waits for the transaction to be mined, and returns the wallet address and FIL balance.

### `repair wallet balance`

Shows wallet and payment account balances.

```bash
repair wallet balance
```

The output includes the wallet address, FIL balance, USDFC balance, and Filecoin Pay account summary fields such as funds, available funds, debt, lockup rates, lockup totals, runway, and current epoch.

### `repair wallet deposit <amount>`

Deposits USDFC from the configured wallet into the wallet's Filecoin Pay account.

```bash
repair wallet deposit 100
```

`amount` is a positive USDFC amount. The command submits the deposit and approval transaction, then waits for it to be mined.

### `repair wallet withdraw <amount>`

Withdraws USDFC from the wallet's Filecoin Pay account.

```bash
repair wallet withdraw 25
```

`amount` is a positive USDFC amount. The command submits the withdraw transaction and waits for it to be mined.

### `repair providers list`

Lists PDP providers from the configured indexer.

```bash
repair providers list
```

By default, the command returns active PDP providers that are approved or endorsed. Each provider includes:

- `id`
- `name`
- `serviceUrl`
- `approved`
- `endorsed`
- `pieceCount`, the number of active indexed pieces for that provider
- `totalSize`, the sum of active piece raw sizes formatted in decimal GB

Use `--all` to include every active PDP provider, even if it is not approved or endorsed.

```bash
repair providers list --all
```

### `repair datasets list`

Lists datasets owned by the configured repair wallet.

```bash
repair datasets list
```

Each dataset includes its ID, CDN/IPFS indexing flags, source, provider URL, PDP end epoch, and piece count.

Filter by provider ID:

```bash
repair datasets list --provider-id 123
```

### `repair repair create`

Creates a local repair plan for a source provider and a target provider.

```bash
repair repair create --provider-id 101 --target-provider-id 202
```

`--provider-id` is the faulty provider whose pieces should be repaired.

`--target-provider-id` is the provider that should receive the repaired pieces. It must be different from `--provider-id`.

The command snapshots the current chain block number, creates a local repair row, scans active pieces for the source provider, deduplicates them by CID, and creates local `add_piece` operations. Pieces with no alternate provider are marked `skipped`. The command returns a `repairId`.

### `repair repair list`

Lists local repair jobs.

```bash
repair repair list
```

Each repair includes:

- repair ID and status
- source provider ID
- target provider ID and target provider URL
- target dataset ID, when one has been created or found
- block number used when the repair was created
- total operations and counts by `pending`, `failed`, `completed`, and `skipped`

### `repair repair run <repairId>`

Runs a pending repair.

```bash
repair repair run 1
```

The command first ensures the target repair dataset exists for the configured wallet and target provider. If no matching dataset exists, it creates one with IPFS indexing enabled and CDN disabled. Then it processes pending `add_piece` operations by pulling pieces from alternate providers into the target provider and committing them on-chain.

Options:

- `--concurrency <number>` controls how many pull batches run at once. Defaults to `4`.
- `--batch-size <number>` controls the maximum number of `add_piece` operations per batch. Defaults to `40`.
- `--reset` retries failed `add_piece` operations as well as pending operations.

Example:

```bash
repair repair run 1 --concurrency 8 --batch-size 100 --reset
```

### `repair repair delete <repairId>`

Deletes a local repair and its operations.

```bash
repair repair delete 1
```

This only deletes local SQLite state. It does not delete on-chain datasets or remove pieces from a provider.

## Typical Workflow

1. Configure the CLI.

```bash
repair setup
```

1. On Calibration, fund the wallet if needed.

```bash
repair wallet fund
```

1. Check balances and deposit USDFC into the payment account.

```bash
repair wallet balance
repair wallet deposit 100
```

1. Pick source and target providers.

```bash
repair providers list
```

1. Create, inspect, and run the repair.

```bash
repair repair create --provider-id 101 --target-provider-id 202
repair repair list
repair repair run 1
```

## Contributing

Read contributing [guidelines](../../.github/CONTRIBUTING.md).
Read contributing [guidelines](../../.github/CONTRIBUTING.md).

[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/FilOzone/early-repair)

Expand Down
21 changes: 20 additions & 1 deletion packages/repair-cli/src/cli.ts
100644 → 100755
Original file line number Diff line number Diff line change
@@ -1 +1,20 @@
// TODO
#!/usr/bin/env node
import { Cli } from 'incur'
import { datasets } from './commands/datasets.ts'
import { providers } from './commands/providers.ts'
import { repair } from './commands/repair.ts'
import { setup } from './commands/setup.ts'
import { wallet } from './commands/wallet.ts'
import { version } from './utils.ts'

const cli = Cli.create('repair', {
version,
description: 'Early repair for faulty service providers and datasets',
})
Comment thread
hugomrdias marked this conversation as resolved.

cli.command(setup)
cli.command(wallet)
cli.command(repair)
cli.command(datasets)
cli.command(providers)
cli.serve()
Loading
Loading