diff --git a/README.md b/README.md index 7c81317..bcb0485 100644 --- a/README.md +++ b/README.md @@ -83,10 +83,20 @@ gates close. The Calibration Worker demo in [`wrangler.jsonc`](./wrangler.jsonc) and [`src/worker/calibration-demo.mjs`](./src/worker/calibration-demo.mjs) exposes a -read-only public evidence surface for issue #15. It is configured with live -Calibration evidence for registry object `1`, provider/dataset/piece -`4`/`12524`/`34`, and a committed registry finalization. The deployed demo is -available at `https://foc-platform-calibration-demo.snissn.workers.dev`. Run +read-only public admin dashboard and evidence surface. `/` and `/admin` render +the dashboard; `/api/admin/overview`, `/api/admin/files`, +`/api/admin/accounts`, `/api/admin/datasets`, `/api/admin/coordinators`, and +`/api/admin/reconciliation` expose JSON rows backed by direct registry +count/list/detail reads. Overview uses bounded contract count reads; table +routes expose page metadata with `cursor` or `offset`, and filters are +page-scoped to keep Worker requests bounded. The committed demo config still points at live Calibration +evidence for registry object `1`, provider/dataset/piece `4`/`12524`/`34`, and +a committed registry finalization; issue #33 owns publishing updated public +evidence for the direct pagination ABI. Until that registry/runtime evidence +matches the current artifact hash, the dashboard defaults to skipped read-only +API responses instead of live dashboard reads; use `?live=true` only against an +upgraded pagination-capable registry. The deployed demo is available at +`https://foc-platform-calibration-demo.snissn.workers.dev`. Run `pnpm worker:dev` for a local Worker and `pnpm worker:dry-run` to validate the deploy bundle. The Worker must not receive private keys or session keys; privileged FOC upload and registry transaction submission stay in the local diff --git a/docs/calibration-worker-demo.md b/docs/calibration-worker-demo.md index e856bd8..9289352 100644 --- a/docs/calibration-worker-demo.md +++ b/docs/calibration-worker-demo.md @@ -1,9 +1,10 @@ # Calibration Worker Demo -Issue #15 exposes the Calibration demo through a Cloudflare Worker. The Worker -is intentionally read-only: it serves public evidence, links to generated -Token Host wrapper metadata, and can read the deployed registry through a -public Calibration RPC. It must not upload files, pay FOC, withdraw funds, or +Issue #15 exposes the Calibration demo through a Cloudflare Worker. Issue #32 +turns the Worker first screen into a read-only admin dashboard for the +configured `FocPlatformRegistry`: it serves public evidence, reads dashboard +rows through direct registry list/detail views, and links to generated Token +Host wrapper metadata. It must not upload files, pay FOC, withdraw funds, or submit registry transactions. ## Worker Commands @@ -20,6 +21,14 @@ Check the public endpoints: curl http://127.0.0.1:8787/api/health curl http://127.0.0.1:8787/api/demo/evidence curl http://127.0.0.1:8787/api/demo/registry +curl http://127.0.0.1:8787/api/admin/overview +curl http://127.0.0.1:8787/api/admin/files?limit=10 +curl 'http://127.0.0.1:8787/api/admin/files?limit=10&cursor=1' +curl http://127.0.0.1:8787/api/admin/accounts?limit=10 +curl 'http://127.0.0.1:8787/api/admin/accounts?limit=10&offset=10' +curl http://127.0.0.1:8787/api/admin/datasets?limit=10 +curl http://127.0.0.1:8787/api/admin/coordinators?limit=10 +curl http://127.0.0.1:8787/api/admin/reconciliation?limit=10 ``` Current deployed Worker: @@ -47,21 +56,41 @@ documented in [`docs/production-hardening-runbook.md`](./production-hardening-runbook.md). The current deployed Worker and registry evidence predate the direct pagination -ABI. They prove the read-only Worker demo and one configured object against the -then-deployed registry, but they do not prove `listStorageObjectIds`, -`listAccountIds`, `listDatasetKeys`, `readBatch`, or the direct-onchain admin -dashboard path. Issue #33 must publish updated evidence from a registry build -that includes the pagination ABI before the dashboard stack can claim -end-to-end direct-read proof. +ABI. The Worker code now has direct-onchain dashboard routes, but the deployed +public evidence still points at the earlier registry. For that configuration, +the dashboard defaults to skipped read-only API responses instead of attempting +live dashboard reads against missing count/list methods. `?live=true` should be +used only with a registry whose runtime hash matches the current pagination ABI. +Issue #33 must publish updated evidence from a registry build that includes the +pagination ABI before the dashboard stack can claim end-to-end public +Calibration direct-read proof. ## Public Endpoints | Route | Purpose | | --- | --- | -| `/` | Operator-facing HTML demo surface. | +| `/` | Operator-facing admin dashboard HTML surface. | +| `/admin` | Explicit admin dashboard alias. | | `/api/health` | Worker health and authority boundary. | | `/api/demo/evidence` | Static public demo configuration assembled from Worker vars. | | `/api/demo/registry` | Public registry reads for owner, next object id, and configured object/usage/receipt state. | +| `/api/admin/overview` | Bounded dashboard metrics and source metadata from direct registry count reads. | +| `/api/admin/files` | Paginated object/file rows with status, account, provider, dataset, coordinator, and text filters. Uses `cursor` for next-page reads; cross-surface reconciliation remains in `/api/admin/reconciliation`. | +| `/api/admin/accounts` | Paginated account usage rows from registry account list/detail reads. | +| `/api/admin/datasets` | Paginated dataset/provider rows from registry dataset key/detail reads. | +| `/api/admin/coordinators` | Coordinator policy and relayer rows from registry list/detail reads. | +| `/api/admin/reconciliation` | Page-scoped reconciliation warnings and evidence boundaries for the current object cursor page. Cross-surface account, dataset, and coordinator-policy checks are declared as omitted instead of scanning the whole registry from one Worker request. | + +The table endpoints accept `limit` up to the registry max list limit. Files use +the object-id cursor returned as `pagination.nextCursorIdExclusive`; +reconciliation uses the same object cursor for page-scoped checks. Accounts, +datasets, and coordinators use the returned `pagination.nextOffset`. Filters +and text search apply to the returned page so the Worker keeps each request +bounded instead of scanning the full registry for a global search. + +Append `?live=false` to any dashboard or registry endpoint when you need a +route-level smoke check without making public RPC calls. Unknown dashboard +routes still return `404`. ## Local Evidence Generation Boundary diff --git a/src/registry/read-model.mjs b/src/registry/read-model.mjs index 721f451..13d3941 100644 --- a/src/registry/read-model.mjs +++ b/src/registry/read-model.mjs @@ -1,9 +1,7 @@ -import { readFileSync } from "node:fs"; import { decodeEventLog, encodeFunctionData } from "viem"; +import registryArtifactJson from "../../artifacts/contracts/FocPlatformRegistry.json" with { type: "json" }; -const artifactUrl = new URL("../../artifacts/contracts/FocPlatformRegistry.json", import.meta.url); - -export const registryArtifact = JSON.parse(readFileSync(artifactUrl, "utf8")); +export const registryArtifact = registryArtifactJson; export const registryAbi = registryArtifact.abi; const FINALIZATION_STATUS = ["Committed", "Partial", "Failed"]; diff --git a/src/worker/calibration-demo.mjs b/src/worker/calibration-demo.mjs index 2a155bb..ae1514c 100644 --- a/src/worker/calibration-demo.mjs +++ b/src/worker/calibration-demo.mjs @@ -1,5 +1,17 @@ import { createPublicClient, getAddress, http, isAddress } from "viem"; import { filecoinCalibration } from "viem/chains"; +import { buildAdminSurfaces } from "../admin/reconciliation.mjs"; +import { createTokenHostRegistryDirectReadAdapter } from "../demo/tokenhost-wrapper.mjs"; +import { + createRegistryReadModel, + registryAccountCountRead, + registryCoordinatorCountRead, + registryDatasetRecordCountRead, + registryDirectReadDefaults, + registryObjectCountRead, + registryArtifact, + registryRelayerCountRead, +} from "../registry/read-model.mjs"; const DEFAULT_REGISTRY_ADDRESS = "0x7771d916a9d742B1D60597a332C7ABBd5796609c"; const DEFAULT_REGISTRY_DEPLOY_TX = @@ -7,6 +19,22 @@ const DEFAULT_REGISTRY_DEPLOY_TX = const DEFAULT_REGISTRY_RUNTIME_SHA256 = "0xed478a27e255a1b27989ffa4f2fcbf38f1a9ec61a84c8d3e20aceb4e26f72040"; const DEFAULT_RPC_URL = "https://api.calibration.node.glif.io/rpc/v1"; +const DEFAULT_DASHBOARD_PAGE_LIMIT = 20; +const PAGE_SCOPED_RECONCILIATION_OMITTED_FAMILIES = Object.freeze([ + "account_usage", + "dataset_records", + "coordinator_policies", +]); +const PAGE_SCOPED_RECONCILIATION_OMITTED_CODES = new Set([ + "missing_dataset_record", + "usage_active_bytes_mismatch", + "usage_pending_bytes_mismatch", + "usage_reserved_cost_mismatch", + "account_over_quota", + "uploading_object_missing_coordinator", + "uploading_object_disallowed_coordinator", + "uploading_object_expired_coordinator", +]); const UPLOAD_STATUS_LABELS = Object.freeze([ "None", "Requested", @@ -18,6 +46,14 @@ const UPLOAD_STATUS_LABELS = Object.freeze([ "Expired", "Deleted", ]); +const DASHBOARD_API_ENDPOINTS = Object.freeze({ + overview: "/api/admin/overview", + files: "/api/admin/files", + accounts: "/api/admin/accounts", + datasets: "/api/admin/datasets", + coordinators: "/api/admin/coordinators", + reconciliation: "/api/admin/reconciliation", +}); // Kept local to make the deployed Worker bundle independent of Node-oriented // artifact generation modules. @@ -161,10 +197,11 @@ export async function handleCalibrationDemoRequest(request, env = {}, options = const url = new URL(request.url); const evidence = buildDemoEvidence(env); + const shouldReadDashboard = dashboardLiveReadsEnabled(url, evidence); const shouldReadRegistry = url.searchParams.get("live") !== "false"; - if (url.pathname === "/" || url.pathname === "/demo") { - return htmlResponse(renderDemoHtml(evidence)); + if (url.pathname === "/" || url.pathname === "/demo" || url.pathname === "/admin") { + return htmlResponse(renderAdminDashboardHtml(evidence, { live: shouldReadDashboard })); } if (url.pathname === "/api/health") { @@ -181,6 +218,47 @@ export async function handleCalibrationDemoRequest(request, env = {}, options = return jsonResponse(withLinks(evidence, url)); } + if (url.pathname.startsWith("/api/admin/")) { + const dashboardRoute = dashboardApiRoute(url.pathname); + if (!dashboardRoute) { + return jsonResponse({ error: { code: "not_found" } }, { status: 404 }); + } + + if (!shouldReadDashboard) { + return jsonResponse({ + source: "skipped", + route: dashboardRoute, + metadata: dashboardMetadata(evidence, env), + evidence: withLinks(evidence, url), + }); + } + + try { + return jsonResponse( + await readDashboardApi({ + route: dashboardRoute, + query: url.searchParams, + evidence, + env, + options, + }), + ); + } catch (error) { + const status = error instanceof DashboardApiError ? error.status : 502; + return jsonResponse( + { + error: { + code: error instanceof DashboardApiError ? error.code : "dashboard_read_failed", + message: error?.message ?? "dashboard read failed", + }, + metadata: dashboardMetadata(evidence, env), + evidence: withLinks(evidence, url), + }, + { status }, + ); + } + } + if (url.pathname === "/api/demo/registry") { if (!shouldReadRegistry) { return jsonResponse({ @@ -253,7 +331,14 @@ export function buildDemoEvidence(env = {}) { mode: "read_only_public_evidence", privilegedActions: false, servesPrivateKeys: false, - endpoints: ["/", "/api/health", "/api/demo/evidence", "/api/demo/registry"], + endpoints: [ + "/", + "/admin", + "/api/health", + "/api/demo/evidence", + "/api/demo/registry", + ...Object.values(DASHBOARD_API_ENDPOINTS), + ], }, boundaries: [ "The Worker serves public evidence and performs public registry reads only.", @@ -341,30 +426,526 @@ export async function readPublicRegistrySnapshot(evidence, env = {}) { return snapshot; } -function renderDemoHtml(evidence) { - const status = evidence.demo.status; +async function readDashboardApi({ route, query, evidence, env, options }) { + const metadata = dashboardMetadata(evidence, env); + const adapter = createDashboardReadAdapter({ evidence, env, options, metadata }); + const limit = dashboardPageLimit(query, env); + const includeTerminal = query.get("includeTerminal") !== "false"; + + switch (route) { + case "overview": { + const summary = await readDashboardOverviewSummary({ + adapter, + evidence, + env, + options, + metadata, + }); + return { + metadata, + summary, + sourceOfTruth: registryDirectReadDefaults, + endpoints: DASHBOARD_API_ENDPOINTS, + }; + } + case "files": { + const page = await adapter.readObjectPage({ + cursorIdExclusive: dashboardCursor(query), + limit, + includeTerminal, + }); + return { + metadata, + pagination: dashboardPagination(page.pagination, page.ids.length, limit), + ids: page.ids, + files: filterObjectRows(fileRowsFromObjectPage(page), query), + }; + } + case "accounts": { + const page = await adapter.readAccountPage({ + offset: dashboardOffset(query), + limit, + includeTerminal, + }); + return { + metadata, + pagination: dashboardPagination(page.pagination, page.accounts.length, limit), + accounts: filterAccountRows( + page.accounts.map((row) => ({ + accountId: row.accountId, + objectIds: row.objectIds, + objectPagination: row.objectPagination, + ...row.usage, + })), + query, + ), + }; + } + case "datasets": { + const page = await adapter.readDatasetPage({ + offset: dashboardOffset(query), + limit, + }); + return { + metadata, + pagination: dashboardPagination(page.pagination, page.datasets.length, limit), + datasets: filterDatasetRows( + page.datasets.map((row) => ({ + key: row.key, + ...row.dataset, + })), + query, + ), + }; + } + case "coordinators": { + const [coordinatorPage, relayerPage] = await Promise.all([ + adapter.readCoordinatorPage({ + offset: dashboardOffset(query), + limit, + }), + adapter.readRelayerPage({ + offset: dashboardOffset(query), + limit, + }), + ]); + return { + metadata, + pagination: { + coordinators: dashboardPagination( + coordinatorPage.pagination, + coordinatorPage.coordinators.length, + limit, + ), + relayers: dashboardPagination( + relayerPage.pagination, + relayerPage.relayers.length, + limit, + ), + }, + coordinators: filterCoordinatorRows( + coordinatorPage.coordinators.map((row) => ({ + coordinator: row.coordinator, + ...row.policy, + sessionStatus: coordinatorSessionStatus(row.policy, metadata.readUnixTime), + })), + query, + ), + relayers: filterRelayerRows(relayerPage.relayers, query), + }; + } + case "reconciliation": { + const objectPage = await adapter.readObjectPage({ + cursorIdExclusive: dashboardCursor(query), + limit, + includeTerminal, + }); + const reconciliation = buildPageScopedReconciliation(objectPage, { + now: metadata.readUnixTime, + }); + return { + metadata, + pagination: dashboardPagination( + objectPage.pagination, + objectPage.objects.length, + limit, + ), + ids: objectPage.ids, + reconciliation: { + ...reconciliation, + checks: filterReconciliationRows(reconciliation.checks, query), + }, + sourceOfTruth: objectPage.sourceOfTruth, + }; + } + default: + throw new DashboardApiError(404, "not_found", "dashboard API route not found"); + } +} + +function createDashboardReadAdapter({ evidence, env, options, metadata }) { + if (options.dashboardAdapter) return options.dashboardAdapter; + if (options.createDashboardAdapter) { + return options.createDashboardAdapter({ evidence, env, metadata }); + } + + const registryAddress = evidence.registry.address; + if (!isAddress(registryAddress)) { + throw new Error("FOC_PLATFORM_REGISTRY_ADDRESS must be an EVM address"); + } + + const publicClient = + options.publicClient ?? + createPublicClient({ + chain: filecoinCalibration, + transport: http(metadata.rpcUrl), + }); + + return createTokenHostRegistryDirectReadAdapter({ + publicClient, + registryAddress, + maxPageSize: metadata.maxPageSize, + detailConcurrency: parsePositiveInteger( + env.FOC_PLATFORM_DASHBOARD_DETAIL_CONCURRENCY, + 4, + ), + maxPagesPerSurface: parsePositiveInteger( + env.FOC_PLATFORM_DASHBOARD_MAX_PAGES_PER_SURFACE, + 100, + ), + includeTerminal: true, + now: metadata.readUnixTime, + }); +} + +async function readDashboardOverviewSummary({ adapter, evidence, env, options, metadata }) { + if (adapter.readOverviewCounts) { + return overviewSummaryFromCounts( + await adapter.readOverviewCounts({ evidence, env, metadata }), + ); + } + + if (options.dashboardAdapter || options.createDashboardAdapter) { + const surfaces = await adapter.readAdminSurfaces({ + route: { name: "dashboard" }, + limit: BigInt(metadata.defaultPageLimit), + includeTerminal: true, + now: metadata.readUnixTime, + }); + return surfaces.summary; + } + + const registryAddress = evidence.registry.address; + if (!isAddress(registryAddress)) { + throw new Error("FOC_PLATFORM_REGISTRY_ADDRESS must be an EVM address"); + } + + const publicClient = + options.publicClient ?? + createPublicClient({ + chain: filecoinCalibration, + transport: http(metadata.rpcUrl), + }); + + const [ + objectCount, + accountCount, + datasetCount, + coordinatorCount, + relayerCount, + ] = await Promise.all( + [ + registryObjectCountRead(registryAddress), + registryAccountCountRead(registryAddress), + registryDatasetRecordCountRead(registryAddress), + registryCoordinatorCountRead(registryAddress), + registryRelayerCountRead(registryAddress), + ].map((read) => publicClient.readContract(read)), + ); + + return overviewSummaryFromCounts({ + objectCount, + accountCount, + datasetCount, + coordinatorCount, + relayerCount, + }); +} + +function overviewSummaryFromCounts(counts = {}) { + return { + mode: "contractCounts", + objectCount: jsonCount(counts.objectCount), + accountCount: jsonCount(counts.accountCount), + datasetCount: jsonCount(counts.datasetCount), + providerCount: null, + coordinatorCount: jsonCount(counts.coordinatorCount), + relayerCount: jsonCount(counts.relayerCount), + objectStatuses: null, + mismatchCount: null, + warningCount: null, + pendingEvidenceCount: null, + }; +} + +function jsonCount(value) { + const count = BigInt(value ?? 0); + return count <= BigInt(Number.MAX_SAFE_INTEGER) ? Number(count) : count.toString(); +} + +function dashboardApiRoute(pathname) { + return Object.entries(DASHBOARD_API_ENDPOINTS).find(([, path]) => path === pathname)?.[0] ?? null; +} + +function dashboardLiveReadsEnabled(url, evidence) { + const live = url.searchParams.get("live"); + if (live === "true") return true; + if (live === "false") return false; + return dashboardDirectReadAbiMatches(evidence); +} + +function dashboardDirectReadAbiMatches(evidence) { + return ( + normalizeHash(evidence.registry.runtimeSha256) === + normalizeHash(registryArtifact.deployedBytecodeSha256) + ); +} + +function normalizeHash(value) { + return optionalString(value)?.toLowerCase().replace(/^0x/, "") ?? ""; +} + +function dashboardMetadata(evidence, env = {}) { + const now = new Date(); + return { + schemaVersion: 1, + sourceOfTruth: registryDirectReadDefaults.sourceOfTruth, + network: evidence.network, + chainId: evidence.chainId, + registryAddress: evidence.registry.address, + registryDeployTx: evidence.registry.deployTxHash, + registryRuntimeSha256: evidence.registry.runtimeSha256, + expectedRuntimeSha256: registryArtifact.deployedBytecodeSha256, + dashboardLiveDefault: dashboardDirectReadAbiMatches(evidence), + rpcUrl: optionalString(env.FILECOIN_CALIBRATION_RPC_URL) ?? DEFAULT_RPC_URL, + readAt: now.toISOString(), + readUnixTime: Math.floor(now.getTime() / 1000), + maxPageSize: parsePositiveInteger( + env.FOC_PLATFORM_DASHBOARD_MAX_PAGE_SIZE, + registryDirectReadDefaults.maxPageSize, + ), + defaultPageLimit: parsePositiveInteger( + env.FOC_PLATFORM_DASHBOARD_DEFAULT_PAGE_LIMIT, + DEFAULT_DASHBOARD_PAGE_LIMIT, + ), + workerMode: evidence.worker.mode, + privilegedActions: false, + caveats: [ + "Dashboard endpoints perform public read-only contract calls.", + "FOC payment/provider evidence is shown only when public evidence exists.", + "Session-key coordinator and production payment readiness remain tracked outside this public dashboard.", + ], + }; +} + +function dashboardPageLimit(query, env = {}) { + const fallback = parsePositiveInteger( + env.FOC_PLATFORM_DASHBOARD_DEFAULT_PAGE_LIMIT, + DEFAULT_DASHBOARD_PAGE_LIMIT, + ); + const requested = parsePositiveInteger(query.get("limit"), fallback); + const max = parsePositiveInteger( + env.FOC_PLATFORM_DASHBOARD_MAX_PAGE_SIZE, + registryDirectReadDefaults.maxPageSize, + ); + return BigInt(Math.min(requested, max)); +} + +function dashboardCursor(query) { + return parseBigIntString(query.get("cursor"), 0n); +} + +function dashboardOffset(query) { + return parseBigIntString(query.get("offset"), 0n); +} + +function dashboardPagination(pagination, rowCount, limit) { + const pageLimit = Number(limit); + return { + ...pagination, + rowCount, + hasNextPage: pageLimit > 0 && rowCount >= pageLimit, + }; +} + +function parseBigIntString(value, fallback) { + const raw = optionalString(value); + if (!raw || !/^\d+$/.test(raw)) return fallback; + return BigInt(raw); +} + +function fileRowsFromObjectPage(page) { + return (page.objects ?? []).map((row) => ({ + objectId: row.objectId, + accountId: row.object.accountId, + user: row.object.user, + status: row.object.status, + size: row.object.size, + requestedCopies: row.object.requestedCopies, + completedCopies: row.object.completedCopies, + activeBytes: row.object.activeBytes, + reservedCost: row.object.reservedCost, + actualCost: row.object.actualCost, + receiptHash: row.object.receiptHash, + receiptPayer: row.receiptPayer, + coordinator: row.object.coordinator, + providerIds: (row.copyReceipts ?? []).map((receipt) => receipt.providerId), + datasetIds: (row.copyReceipts ?? []).map((receipt) => receipt.datasetId), + copyReceipts: row.copyReceipts ?? [], + })); +} + +function buildPageScopedReconciliation(objectPage, { now } = {}) { + const model = createRegistryReadModel(); + for (const row of objectPage.objects ?? []) { + model.objects[row.objectId] = row.object; + model.copyReceipts[row.objectId] = row.copyReceipts ?? []; + model.receiptPayers[row.objectId] = row.receiptPayer; + } + + const surfaces = buildAdminSurfaces({ model }, { now }); + const checks = surfaces.reconciliation.checks.filter( + (check) => !PAGE_SCOPED_RECONCILIATION_OMITTED_CODES.has(check.code), + ); + + return { + ...reconciliationSummaryFromChecks(checks), + scope: "object_page", + objectCount: objectPage.objects?.length ?? 0, + objectIds: objectPage.ids ?? [], + omittedCheckFamilies: PAGE_SCOPED_RECONCILIATION_OMITTED_FAMILIES, + omittedCheckCodes: Array.from(PAGE_SCOPED_RECONCILIATION_OMITTED_CODES), + checks, + }; +} + +function reconciliationSummaryFromChecks(checks) { + const mismatchCount = checks.filter((check) => check.severity === "error").length; + const warningCount = checks.filter((check) => check.severity === "warning").length; + const pendingEvidenceCount = checks.filter((check) => check.code === "foc_evidence_not_checked").length; + return { + status: + mismatchCount > 0 + ? "mismatch" + : warningCount > 0 + ? "warning" + : pendingEvidenceCount > 0 + ? "pending_external_evidence" + : "matched", + mismatchCount, + warningCount, + pendingEvidenceCount, + }; +} + +function filterObjectRows(rows, query) { + const status = optionalString(query.get("status")); + const account = optionalString(query.get("account")); + const provider = optionalString(query.get("provider")); + const dataset = optionalString(query.get("dataset")); + const coordinator = optionalString(query.get("coordinator"))?.toLowerCase(); + return textFilter( + rows.filter((row) => { + if (status && row.status !== status) return false; + if (account && row.accountId !== account) return false; + if (provider && !row.providerIds.includes(provider)) return false; + if (dataset && !row.datasetIds.includes(dataset)) return false; + if (coordinator && String(row.coordinator ?? "").toLowerCase() !== coordinator) return false; + return true; + }), + query, + ["objectId", "accountId", "user", "receiptHash", "receiptPayer", "coordinator"], + ); +} + +function filterAccountRows(rows, query) { + return textFilter(rows, query, ["accountId", "objectIds"]); +} + +function filterDatasetRows(rows, query) { + const provider = optionalString(query.get("provider")); + const dataset = optionalString(query.get("dataset")); + return textFilter( + rows.filter((row) => { + if (provider && row.providerId !== provider) return false; + if (dataset && row.datasetId !== dataset) return false; + return true; + }), + query, + ["key", "accountId", "providerId", "datasetId", "payer", "storageClass"], + ); +} + +function filterCoordinatorRows(rows, query) { + const coordinator = optionalString(query.get("coordinator"))?.toLowerCase(); + return textFilter( + rows.filter((row) => !coordinator || String(row.coordinator ?? "").toLowerCase() === coordinator), + query, + ["coordinator", "permissionsHash", "sessionStatus"], + ); +} + +function filterRelayerRows(rows, query) { + return textFilter(rows, query, ["relayer", "allowed"]); +} + +function filterReconciliationRows(rows, query) { + const severity = optionalString(query.get("severity")); + const code = optionalString(query.get("code")); + return textFilter( + rows.filter((row) => { + if (severity && row.severity !== severity) return false; + if (code && row.code !== code) return false; + return true; + }), + query, + ["code", "severity", "objectId", "accountId", "providerId", "datasetId", "coordinator"], + ); +} + +function textFilter(rows, query, fields) { + const raw = optionalString(query.get("q")); + if (!raw) return rows; + const needle = raw.toLowerCase(); + return rows.filter((row) => + fields.some((fieldName) => { + const value = row[fieldName]; + if (Array.isArray(value)) return value.some((item) => String(item).toLowerCase().includes(needle)); + return String(value ?? "").toLowerCase().includes(needle); + }), + ); +} + +function coordinatorSessionStatus(policy, now) { + if (!policy.allowed) return "disabled"; + const expiresAt = BigInt(policy.sessionKeyExpiresAt ?? 0); + if (expiresAt === 0n) return "active"; + return BigInt(now) > expiresAt ? "expired" : "active"; +} + +class DashboardApiError extends Error { + constructor(status, code, message) { + super(message); + this.status = status; + this.code = code; + } +} + +function renderAdminDashboardHtml(evidence, { live = true } = {}) { const registry = evidence.registry.address; - const objectId = evidence.demo.objectId ?? "not configured"; - const pieceCid = evidence.demo.pieceCid ?? "pending"; - const retrievalUrl = evidence.demo.retrievalUrl; + const network = evidence.network; return ` - FOC Platform Calibration Demo + FOC Platform Admin
-
-
-

FOC Platform Calibration Demo

-

Read-only Worker surface for the Filecoin Calibration registry, Token Host wrapper metadata, and local FOC upload evidence.

+
+
+

FOC Platform Admin

+
${escapeHtml(network)} / ${escapeHtml(registry)}
-
- -
-
-

Lifecycle Evidence

-

${escapeHtml(status)}

-
-
1
Registry${escapeHtml(registry)}
-
2
Object${escapeHtml(objectId)}
-
3
FOC piece${escapeHtml(pieceCid)}
-
4
Retrieval${retrievalUrl ? `${escapeHtml(retrievalUrl)}` : "pending"}
-
-
-
-

Public Configuration

-
-
Network
${escapeHtml(evidence.network)}
-
Chain ID
${escapeHtml(String(evidence.chainId))}
-
Mode
${escapeHtml(evidence.mode)}
-
Deploy tx
${escapeHtml(evidence.registry.deployTxHash)}
-
Runtime SHA
${escapeHtml(evidence.registry.runtimeSha256)}
-
Worker authority
read only
-
+
+ Read only + Direct registry reads + Chain ${escapeHtml(String(evidence.chainId))} +
+ +
+ + + + +
+
+ +
+
+
+
+

Files

+ Evidence JSON +
+
+ +
+ `; } @@ -513,9 +1585,13 @@ function withLinks(evidence, url) { ...evidence, links: { html: `${origin}/`, + admin: `${origin}/admin`, health: `${origin}/api/health`, evidence: `${origin}/api/demo/evidence`, registry: `${origin}/api/demo/registry`, + dashboard: Object.fromEntries( + Object.entries(DASHBOARD_API_ENDPOINTS).map(([key, path]) => [key, `${origin}${path}`]), + ), }, }; } diff --git a/test/calibration-worker.test.mjs b/test/calibration-worker.test.mjs index 6aea0ce..f6dc30f 100644 --- a/test/calibration-worker.test.mjs +++ b/test/calibration-worker.test.mjs @@ -2,13 +2,20 @@ import assert from "node:assert/strict"; import { readFile } from "node:fs/promises"; import test from "node:test"; +import { buildAdminSurfaces } from "../src/admin/reconciliation.mjs"; import { buildDemoEvidence, handleCalibrationDemoRequest, } from "../src/worker/calibration-demo.mjs"; +import { registryArtifact } from "../src/registry/read-model.mjs"; const REGISTRY = "0x7771d916a9d742B1D60597a332C7ABBd5796609c"; const ACCOUNT_ID = `0x${"12".repeat(32)}`; +const ACCOUNT_B = `0x${"34".repeat(32)}`; +const USER = "0x0000000000000000000000000000000000001000"; +const PAYER = "0x0000000000000000000000000000000000002000"; +const COORDINATOR = "0x000000000000000000000000000000000000abcd"; +const RELAYER = "0x0000000000000000000000000000000000004000"; test("Worker evidence builder keeps privileged credentials out of public state", () => { const evidence = buildDemoEvidence({ @@ -50,14 +57,45 @@ test("Worker serves HTML and public evidence endpoints", async () => { new Request("https://demo.example/api/health"), { FOC_PLATFORM_REGISTRY_ADDRESS: REGISTRY }, ); + const offlineHtml = await handleCalibrationDemoRequest( + new Request("https://demo.example/admin?live=false"), + { FOC_PLATFORM_REGISTRY_ADDRESS: REGISTRY }, + ); + const liveHtml = await handleCalibrationDemoRequest( + new Request("https://demo.example/admin?live=true"), + { FOC_PLATFORM_REGISTRY_ADDRESS: REGISTRY }, + ); assert.equal(html.status, 200); - assert.match(await html.text(), /FOC Platform Calibration Demo/); + const htmlBody = await html.text(); + assert.match(htmlBody, /FOC Platform Admin/); + assert.match(htmlBody, /\/api\/admin\/files/); + assert.match(htmlBody, /data-page-action="next"/); + assert.match(htmlBody, /function renderCoordinatorView/); + assert.match(htmlBody, /function combinedOffsetPagination/); + assert.match(htmlBody, /function renderSkippedView/); + assert.match(htmlBody, /body\.source === "skipped"/); + assert.match(htmlBody, /summary\.warningCount === undefined/); + assert.match(htmlBody, /const requestId = \+\+state\.requestSeq/); + assert.match(htmlBody, /requestId !== state\.requestSeq/); + assert.match(htmlBody, /function renderView\(body, view = state\.view\)/); + assert.match(htmlBody, /function resetAllPages\(\)/); + assert.match(htmlBody, /Object\.keys\(state\.pages\)\.forEach\(\(view\) => resetPage\(view\)\);/); + assert.match(htmlBody, /\["status", "provider", "limit"\]\.forEach[\s\S]*resetAllPages\(\);/); + assert.match(htmlBody, /\$\("q"\)\.addEventListener\("input"[\s\S]*resetAllPages\(\);/); + assert.match(htmlBody, /Dashboard reads unavailable/); + assert.match(htmlBody, /const relayerRows = body\.relayers \|\| \[\];/); + assert.match(htmlBody, /const cursorViews = new Set\(\["files", "reconciliation"\]\);/); + assert.match(htmlBody, /Relayers/); + assert.match(htmlBody, /const liveReads = false;/); + assert.match(await offlineHtml.text(), /const liveReads = false;/); + assert.match(await liveHtml.text(), /const liveReads = true;/); assert.equal(evidence.status, 200); const evidenceBody = await evidence.json(); assert.equal(evidenceBody.demo.objectId, "7"); assert.equal(evidenceBody.links.registry, "https://demo.example/api/demo/registry"); + assert.equal(evidenceBody.links.dashboard.files, "https://demo.example/api/admin/files"); assert.equal(health.status, 200); assert.equal((await health.json()).privilegedActions, false); @@ -90,16 +128,194 @@ test("Worker registry endpoint accepts injected public read snapshot", async () assert.equal(body.registry.object.statusLabel, "Committed"); }); +test("Worker dashboard APIs expose injected direct-read admin pages", async () => { + let readAdminSurfacesCalls = 0; + const dashboardAdapter = { + ...createDashboardFixtureAdapter(), + async readAdminSurfaces() { + readAdminSurfacesCalls += 1; + throw new Error("dashboard reconciliation should stay page-bounded"); + }, + }; + const env = { + FOC_PLATFORM_REGISTRY_ADDRESS: REGISTRY, + FOC_PLATFORM_DASHBOARD_DEFAULT_PAGE_LIMIT: "2", + }; + + const skippedOverview = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/overview"), + env, + { dashboardAdapter }, + ); + const skippedFiles = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/files"), + env, + { dashboardAdapter }, + ); + const overview = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/overview?live=true"), + env, + { dashboardAdapter }, + ); + const upgradedOverview = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/overview"), + { + ...env, + FOC_PLATFORM_REGISTRY_RUNTIME_SHA256: registryArtifact.deployedBytecodeSha256, + }, + { dashboardAdapter }, + ); + const upgradedBareHexOverview = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/overview"), + { + ...env, + FOC_PLATFORM_REGISTRY_RUNTIME_SHA256: registryArtifact.deployedBytecodeSha256.replace( + /^0x/i, + "", + ), + }, + { dashboardAdapter }, + ); + const files = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/files?live=true&status=Committed&q=0000"), + env, + { dashboardAdapter }, + ); + const accounts = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/accounts?live=true"), + env, + { dashboardAdapter }, + ); + const datasets = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/datasets?live=true&provider=111"), + env, + { dashboardAdapter }, + ); + const coordinators = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/coordinators?live=true"), + env, + { dashboardAdapter }, + ); + const relayerSearch = await handleCalibrationDemoRequest( + new Request(`https://demo.example/api/admin/coordinators?live=true&q=${RELAYER}`), + env, + { dashboardAdapter }, + ); + const uppercaseCoordinatorFilter = await handleCalibrationDemoRequest( + new Request( + "https://demo.example/api/admin/coordinators?live=true&coordinator=0x000000000000000000000000000000000000ABCD", + ), + env, + { dashboardAdapter }, + ); + const reconciliation = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/reconciliation?live=true"), + env, + { dashboardAdapter }, + ); + + assert.equal(skippedOverview.status, 200); + const skippedOverviewBody = await skippedOverview.json(); + assert.equal(skippedOverviewBody.source, "skipped"); + assert.equal(skippedOverviewBody.metadata.dashboardLiveDefault, false); + assert.equal(skippedFiles.status, 200); + const skippedFilesBody = await skippedFiles.json(); + assert.equal(skippedFilesBody.source, "skipped"); + assert.equal("files" in skippedFilesBody, false); + + assert.equal(overview.status, 200); + const overviewBody = await overview.json(); + assert.equal(overviewBody.summary.mode, "contractCounts"); + assert.equal(overviewBody.summary.objectCount, 2); + assert.equal(overviewBody.summary.providerCount, null); + assert.equal(overviewBody.metadata.dashboardLiveDefault, false); + + assert.equal(upgradedOverview.status, 200); + const upgradedOverviewBody = await upgradedOverview.json(); + assert.equal(upgradedOverviewBody.summary.objectCount, 2); + assert.equal(upgradedOverviewBody.metadata.dashboardLiveDefault, true); + assert.equal(upgradedBareHexOverview.status, 200); + const upgradedBareHexOverviewBody = await upgradedBareHexOverview.json(); + assert.equal(upgradedBareHexOverviewBody.summary.objectCount, 2); + assert.equal(upgradedBareHexOverviewBody.metadata.dashboardLiveDefault, true); + + assert.equal(files.status, 200); + const filesBody = await files.json(); + assert.equal(filesBody.metadata.sourceOfTruth, "FocPlatformRegistryDirectReads"); + assert.deepEqual(filesBody.ids, ["2", "1"]); + assert.deepEqual(filesBody.files.map((row) => row.objectId), ["1"]); + assert.equal("issues" in filesBody.files[0], false); + assert.equal("reconciliationStatus" in filesBody.files[0], false); + assert.equal(filesBody.pagination.mode, "objectIdCursor"); + assert.equal(filesBody.pagination.hasNextPage, true); + assert.equal(filesBody.pagination.nextCursorIdExclusive, "1"); + assert.doesNotThrow(() => JSON.stringify(filesBody)); + + assert.equal(accounts.status, 200); + const accountsBody = await accounts.json(); + assert.equal(accountsBody.accounts[0].accountId, ACCOUNT_ID); + assert.deepEqual(accountsBody.accounts[0].objectIds, ["1"]); + + assert.equal(datasets.status, 200); + const datasetsBody = await datasets.json(); + assert.equal(datasetsBody.datasets[0].providerId, "111"); + + assert.equal(coordinators.status, 200); + const coordinatorBody = await coordinators.json(); + assert.equal(coordinatorBody.coordinators[0].coordinator, COORDINATOR); + assert.equal(coordinatorBody.relayers[0].relayer, RELAYER); + assert.equal(relayerSearch.status, 200); + const relayerSearchBody = await relayerSearch.json(); + assert.deepEqual(relayerSearchBody.coordinators, []); + assert.deepEqual( + relayerSearchBody.relayers.map((row) => row.relayer), + [RELAYER], + ); + assert.equal(uppercaseCoordinatorFilter.status, 200); + assert.deepEqual((await uppercaseCoordinatorFilter.json()).coordinators.map((row) => row.coordinator), [ + COORDINATOR, + ]); + + assert.equal(reconciliation.status, 200); + const reconciliationBody = await reconciliation.json(); + assert.equal(reconciliationBody.reconciliation.status, "pending_external_evidence"); + assert.equal(reconciliationBody.reconciliation.scope, "object_page"); + assert.deepEqual(reconciliationBody.reconciliation.objectIds, ["2", "1"]); + assert.deepEqual(reconciliationBody.ids, ["2", "1"]); + assert.equal(reconciliationBody.pagination.mode, "objectIdCursor"); + assert.equal(reconciliationBody.pagination.hasNextPage, true); + assert.ok(reconciliationBody.reconciliation.omittedCheckFamilies.includes("account_usage")); + assert.equal( + reconciliationBody.reconciliation.checks.some((check) => + reconciliationBody.reconciliation.omittedCheckCodes.includes(check.code), + ), + false, + ); + assert.equal(readAdminSurfacesCalls, 0); +}); + test("Worker rejects unsupported methods and unknown routes", async () => { const post = await handleCalibrationDemoRequest( new Request("https://demo.example/api/demo/evidence", { method: "POST" }), ); const missing = await handleCalibrationDemoRequest(new Request("https://demo.example/nope")); + const missingAdmin = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/nope"), + {}, + { dashboardAdapter: createDashboardFixtureAdapter() }, + ); + const missingOfflineAdmin = await handleCalibrationDemoRequest( + new Request("https://demo.example/api/admin/nope?live=false"), + ); assert.equal(post.status, 405); assert.equal((await post.json()).error.code, "method_not_allowed"); assert.equal(missing.status, 404); assert.equal((await missing.json()).error.code, "not_found"); + assert.equal(missingAdmin.status, 404); + assert.equal((await missingAdmin.json()).error.code, "not_found"); + assert.equal(missingOfflineAdmin.status, 404); + assert.equal((await missingOfflineAdmin.json()).error.code, "not_found"); }); test("Committed Worker config and evidence artifact do not contain private keys", async () => { @@ -500,3 +716,259 @@ test("Calibration registry runner rejects stale request config before mutation", /refusing to mutate registry: .*object\.metadataHash/, ); }); + +function createDashboardFixtureAdapter() { + const model = dashboardFixtureModel(); + return { + async readOverviewCounts() { + return { + objectCount: Object.keys(model.objects).length, + accountCount: Object.keys(model.usage).length, + datasetCount: Object.keys(model.datasets).length, + coordinatorCount: Object.keys(model.coordinators).length, + relayerCount: Object.keys(model.relayers).length, + }; + }, + async readAdminSurfaces(options = {}) { + return buildAdminSurfaces({ model }, { now: options.now ?? 1_000 }); + }, + async readObjectPage({ cursorIdExclusive = 0n, limit = 2n } = {}) { + const ids = cursorIds(["2", "1"], cursorIdExclusive, limit); + return { + sourceOfTruth: "FocPlatformRegistryDirectReads", + pagination: { + mode: "objectIdCursor", + cursorIdExclusive: String(cursorIdExclusive), + nextCursorIdExclusive: ids.at(-1) ?? String(cursorIdExclusive), + limit: String(limit), + includeTerminal: true, + }, + ids, + objects: ids.map((objectId) => ({ + objectId, + object: model.objects[objectId], + copyReceipts: model.copyReceipts[objectId] ?? [], + receiptPayer: model.receiptPayers[objectId], + })), + }; + }, + async readAccountPage({ offset = 0n, limit = 2n } = {}) { + const accountIds = offsetRows([ACCOUNT_ID, ACCOUNT_B], offset, limit); + return { + sourceOfTruth: "FocPlatformRegistryDirectReads", + pagination: { + mode: "offset", + offset: String(offset), + nextOffset: String(BigInt(offset) + BigInt(accountIds.length)), + limit: String(limit), + }, + accountIds, + accounts: accountIds.map((accountId) => ({ + accountId, + usage: model.usage[accountId], + objectIds: Object.values(model.objects) + .filter((object) => object.accountId === accountId) + .map((object) => object.objectId), + objectPagination: { + mode: "objectIdCursor", + cursorIdExclusive: "0", + nextCursorIdExclusive: "0", + limit: String(limit), + includeTerminal: true, + }, + })), + }; + }, + async readDatasetPage({ offset = 0n, limit = 2n } = {}) { + const datasets = offsetRows( + Object.entries(model.datasets).map(([key, dataset]) => ({ key, dataset })), + offset, + limit, + ); + return { + sourceOfTruth: "FocPlatformRegistryDirectReads", + pagination: { + mode: "offset", + offset: String(offset), + nextOffset: String(BigInt(offset) + BigInt(datasets.length)), + limit: String(limit), + }, + keys: datasets.map((row) => row.key), + datasets, + }; + }, + async readCoordinatorPage({ offset = 0n, limit = 2n } = {}) { + const coordinators = offsetRows( + Object.entries(model.coordinators).map(([coordinator, policy]) => ({ + coordinator, + policy, + })), + offset, + limit, + ); + return { + sourceOfTruth: "FocPlatformRegistryDirectReads", + pagination: { + mode: "offset", + offset: String(offset), + nextOffset: String(BigInt(offset) + BigInt(coordinators.length)), + limit: String(limit), + }, + addresses: coordinators.map((row) => row.coordinator), + coordinators, + }; + }, + async readRelayerPage({ offset = 0n, limit = 2n } = {}) { + const relayers = offsetRows( + Object.entries(model.relayers).map(([relayer, allowed]) => ({ + relayer, + allowed, + })), + offset, + limit, + ); + return { + sourceOfTruth: "FocPlatformRegistryDirectReads", + pagination: { + mode: "offset", + offset: String(offset), + nextOffset: String(BigInt(offset) + BigInt(relayers.length)), + limit: String(limit), + }, + addresses: relayers.map((row) => row.relayer), + relayers, + }; + }, + }; +} + +function cursorIds(ids, cursorIdExclusive, limit) { + const cursor = BigInt(cursorIdExclusive); + return ids + .filter((id) => cursor === 0n || BigInt(id) < cursor) + .slice(0, Number(limit)); +} + +function offsetRows(rows, offset, limit) { + return rows.slice(Number(offset), Number(offset) + Number(limit)); +} + +function dashboardFixtureModel() { + return { + objects: { + 1: { + objectId: "1", + accountId: ACCOUNT_ID, + user: USER, + idempotencyKey: hex32("01"), + contentHash: hex32("02"), + metadataHash: hex32("03"), + pieceCidHash: hex32("04"), + size: 1024n, + requestedCopies: 1, + completedCopies: 1, + withCDN: true, + maxCost: "10", + reservedCost: "0", + actualCost: "7", + status: "Committed", + coordinator: COORDINATOR, + requestExpiresAt: "2000", + createdAt: 100n, + updatedAt: 120n, + receiptHash: hex32("05"), + }, + 2: { + objectId: "2", + accountId: ACCOUNT_B, + user: USER, + idempotencyKey: hex32("06"), + contentHash: hex32("07"), + metadataHash: hex32("08"), + pieceCidHash: hex32("09"), + size: 512n, + requestedCopies: 2, + completedCopies: 0, + withCDN: false, + maxCost: "20", + reservedCost: "20", + actualCost: "0", + status: "Uploading", + coordinator: COORDINATOR, + requestExpiresAt: "2000", + createdAt: 110n, + updatedAt: 115n, + receiptHash: hex32("00"), + }, + }, + usage: { + [ACCOUNT_ID]: { + activeBytes: "1024", + activeObjects: "1", + pendingBytes: "0", + reservedCost: "0", + totalActualCost: "7", + totalUploadedBytes: "1024", + totalRequestedUploads: "1", + totalFinalizedUploads: "1", + totalFailedUploads: "0", + }, + [ACCOUNT_B]: { + activeBytes: "0", + activeObjects: "0", + pendingBytes: "1024", + reservedCost: "20", + totalActualCost: "0", + totalUploadedBytes: "0", + totalRequestedUploads: "1", + totalFinalizedUploads: "0", + totalFailedUploads: "0", + }, + }, + copyReceipts: { + 1: [ + { + providerId: "111", + datasetId: "222", + pieceId: "333", + addPieceTxHash: hex32("0a"), + retrievalUrlHash: hex32("0b"), + isNewDataSet: true, + }, + ], + 2: [], + }, + receiptPayers: { + 1: PAYER, + 2: PAYER, + }, + datasets: { + [`${ACCOUNT_ID}:111:222`]: { + accountId: ACCOUNT_ID, + payer: PAYER, + providerId: "111", + datasetId: "222", + storageClass: hex32("0c"), + withCDN: true, + createdAt: "100", + updatedAt: "120", + }, + }, + coordinators: { + [COORDINATOR]: { + allowed: true, + maxFinalizeDelay: "3600", + sessionKeyExpiresAt: "9999999999", + permissionsHash: hex32("0d"), + }, + }, + relayers: { + [RELAYER]: true, + }, + idempotency: {}, + }; +} + +function hex32(suffix) { + return `0x${String(suffix).padStart(64, "0")}`; +}