Skip to content

Commit 60272fa

Browse files
authored
Update polkadot-agent-kit_2_diogo-w3f.md
1 parent 5200c23 commit 60272fa

File tree

1 file changed

+5
-10
lines changed

1 file changed

+5
-10
lines changed

evaluations/polkadot-agent-kit_2_diogo-w3f.md

Lines changed: 5 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
| ------ | ----------- | -------- | ---- | ---------------- |
99
| 0a. | License | <ul><li>[x] </li></ul> | `polkadot-agent-kit/LICENSE` | MIT license present at repo root and referenced by package READMEs. |
1010
| 0b. | Documentation | <ul><li>[ ] </li></ul> | `https://cocdap.github.io/agent-docs/` | Unable to verify because testing blockers prevent validating the documented flows. |
11-
| 0c. | Testing & Guide | <ul><li>[ ] </li></ul> | `packages/sdk/tests` | Unit/e2e suites build, but `test:integration:testnet` and `test:integration:mainnet` both fail; coverage run reports unusable 0 % line/statement metrics (see “Testing & coverage issues”). |
11+
| 0c. | Testing & Guide | <ul><li>[ ] </li></ul> | `packages/sdk/tests` | Unit/e2e suites build, and `test:integration:testnet` passes; coverage run reports unusable 0 % line/statement metrics (see “Testing & coverage issues”). |
1212
| 1a. | Hydration swap tooling | <ul><li>[ ] </li></ul> | `packages/core/src/defi/swap.ts` | Implementation exists, but we could not confirm behavior on testnet/mainnet due to failing integration suites. |
1313
| 1b. | Bifrost vDOT tooling | <ul><li>[ ] </li></ul> | `packages/core/src/bifrost/mintVdot.ts` | Implementation exists, but minting flow could not be exercised (tests blocked). |
1414
| 1c. | Nomination pool tooling | <ul><li>[ ] </li></ul> | `packages/core/src/pallets/nomination-pools` | Only part of the staking suite succeeds; withdraw path currently fails. |
@@ -20,18 +20,13 @@
2020
- `pnpm build:packages`
2121
- `pnpm --filter @polkadot-agent-kit/sdk test`
2222
- `pnpm --filter @polkadot-agent-kit/sdk test:cov -- --coverage.reports=html,text`
23-
- `pnpm --filter @polkadot-agent-kit/sdk test:integration:testnet`
23+
- `pnpm --filter @polkadot-agent-kit/sdk test:integration:testnet` (latest rerun on Node 22 with local Ollama `qwen3:latest`)
2424
- `pnpm --filter @polkadot-agent-kit/sdk test:integration:mainnet`
2525

2626
**Testing & coverage issues**
27-
- `test:integration:testnet` (vitest.config.testnet.ts) currently fails 11/19 specs:
28-
- Every XCM transfer scenario (relay→parachain, parachain→relay, parachain→parachain) throws `Error: Missing WebSocket class` from `@paraspell/sdk` because looks like that no WS implementation is provided under Node 20. Please specify better which dependencies versions are required to run the software.
29-
- The transfer-native scenario fails with `TypeError: Cannot read properties of undefined (reading 'getEstimatedFees')` in `tests/integration-tests/utils.ts:27`.
30-
- The withdraw-unbond staking test asserts `slashingSpans`, but the tool now emits `numSlashingSpans`, so the expectation no longer matches actual input.
31-
- These failures occur even with `AGENT_PRIVATE_KEY` configured in `.env`.
32-
33-
- Please provide a way to measure testing coverage, since coverage command (`vitest run --coverage -- --coverage.reports=html,text`) report shows **0 % statements/lines** and **80 % branches/functions**. This indicates the current Vitest config instruments compiled bundles instead of the TypeScript sources; until coverage is reconfigured, these numbers can’t be trusted as evidence.
34-
- Because of the above, deliverable **0c** is not satisfied and the milestone cannot be approved until the integration suites run cleanly (with a Node-compatible WebSocket provider and corrected assertions) and coverage reflects real line/statement execution.
27+
- `test:integration:testnet`: with Node 22 and a running Ollama server the suite now passes only three scenarios (transfer_native on Paseo Asset Hub and two XCM transfers Asset Hub Paseo → People Paseo; example hash `0xb1f3ae9a9db79120ff37b533e1373e5fb1b573991ad11c75c37c7750896b5aef`). The milestone-2 surface is still largely untested: no Hydration swaps, no Bifrost vDOT mint, no nomination pool flows, and no relay↔parachain cases were exercised in this passing subset. Earlier failures (`getEstimatedFees` undefined, `slashingSpans` vs `numSlashingSpans`) were not re-verified.
28+
- Coverage command (`vitest run --coverage -- --coverage.reports=html,text`) writes HTML under `packages/sdk/coverage/`, but the report shows 0 % statements/lines and 80 % branches/functions. This indicates the current Vitest config instruments compiled bundles instead of the TypeScript sources; until coverage is reconfigured, these numbers can’t be trusted as evidence.
29+
- Because of the above, deliverable 0c is not satisfied and the milestone cannot be approved until the integration suites run cleanly (with a Node-compatible WebSocket provider and corrected assertions) and coverage reflects real line/statement execution.
3530

3631
**Additional remarks**
3732
- Running SDK tests on a clean checkout still requires `pnpm build:packages` first, otherwise Vitest cannot resolve `@polkadot-agent-kit/core`.

0 commit comments

Comments
 (0)