diff --git a/.planning/claude-migration/00-overview.md b/.planning/claude-migration/00-overview.md new file mode 100644 index 0000000..e993f7e --- /dev/null +++ b/.planning/claude-migration/00-overview.md @@ -0,0 +1,145 @@ +# 00 — Overview: What Changed and What It Touches + +## 1. The two upstream changes + +### 1a. Contracts: `createAtomsWithUris` + `AtomContextRegistered` (additive) + +Commit `a402fad` in `intuition-contracts-v2` ("Feat: Add URIs to Atom Create (#155)"). Crucial facts for the backend — **this is an additive ABI change, nothing existing was renamed or removed**: + +- New entrypoint on MultiVault: + ```solidity + createAtomsWithUris(address creator, bytes[] atomDatas, uint256[] assets, bytes[][] uris) + ``` + `uris` is **per-atom** (`uris[i]` for atom `i`, inner list may be empty). Limits: default max **5 URIs/atom, 700 bytes each** (`getAtomUriConfig()`, timelock-settable, `AtomUriConfigUpdated` event). +- New event — **the only place URIs exist; they are not stored on-chain**: + ```solidity + event AtomContextRegistered(bytes32 indexed termId, address indexed registrant, bytes[] uris) + ``` + Emitted at most **once per atom, only at creation, only when the URI list is non-empty**, in the same tx *after* the standard `AtomCreated` + `Deposited` pair. If the indexer misses it, the data is gone short of re-reading logs. +- `AtomCreated(creator, termId, atomData, atomWallet)` is **byte-identical** to before. So are `TripleCreated`, `Deposited`, `Redeemed`, and triple creation generally. +- **Atom ID computation is unchanged and URIs do not affect it**: `keccak256(ATOM_SALT ‖ keccak256(atomData))`. +- **URIs are write-once.** No setter, no append, no edit event. Model them as immutable creation-time context. +- FeeProxy gained a matching `createAtomsWithUrisVia` pass-through (emits the same `CreatedAtomsVia`; URI context still surfaces only via `AtomContextRegistered`). + +> ⚠️ The `.planning/intuition-id` docs predate this contract change ("None of it requires a protocol or contract change"). The URIs field is the on-chain realization of what the docs model as `sameAs` evidence / additional context. See "Decisions to lock" below — we must pin down what goes in `uris` before the sprint. + +### 1b. Data model: atoms become Intuition Identifiers + +From the ratified IID spec (`intuition-v2/.planning/intuition-id/spec.md` and friends): + +- **Grammar:** `int::` — ASCII, ≤256 bytes, parse on first two colons only, byte-exact comparison after per-scheme canonicalization. +- **Scheme registry (25 schemes):** Class A registered authorities (`isbn`, `isrc`, `iswc`, `isni`, `orcid`, `lei`, `gtin`, `doi`, `eidr`, `wd`, `mbid`, `olid`, `imdb`, `tmdb`, `podcastguid`), Class B intrinsic keys (`url`, `caip10`, `caip19`, `hash`, `appid`, `purl`, `geo`, `acct`, `rssitem`, `termset`), Class C derived (`gen1::r:`). +- **Representation profiles (D29–D31):** + - **P0 anchor** — atom data **is** the bare IID string (`int:isrc:USQX91300108`, ~21 bytes). Only for schemes with unambiguous typing. `atomId = calculateAtomId(iid)` ⇒ protocol-level zero-coordination dedupe. + - **P1 identity context** — JSON with `@type` + `identifier` + recipe fields. Required floor for Class C (`gen1` preimage must be verifiable) and polymorphic schemes (`wd`, `url`, `doi`, `hash`, `geo`, `caip10`, `imdb`, `tmdb`, `isni`, `orcid`). + - **P2 enriched** — today's full JSON-LD, now opt-in. +- **Scheme ⇒ classification** is deterministic for unambiguous schemes (`isrc` → MusicRecording, `isbn` → Book edition, `gtin` → Product, `lei` → Organization, `podcastguid` → PodcastSeries, `rssitem` → PodcastEpisode, `caip19` → EthereumERC20/asset, `appid` → MobileApplication, `purl` → Software, `acct` → SocialMediaAccount, …). `mbid` and `gen1` carry the type **in-value** (`mbid:artist:…`, `gen1:music-recording:…`). Polymorphic schemes get `@type` from the P1 payload. +- **Profile detection needs no marker (PE-Q6):** P0 iff `validateIntuitionId(rawPayload)` succeeds (the `int:` prefix distinguishes it from legacy bare-URL atoms); otherwise JSON-parse and look for `identifier`. +- **Enrichment direction inverts:** instead of *parsing* stored JSON to find a `sameAs` URL and scraping it, we *resolve* the identifier against the scheme's home registry/API (ISRC → Spotify/MusicBrainz, ISBN → OpenLibrary, QID → Wikidata, DOI → Crossref, CAIP → on-chain, …). Display metadata (name, image) comes from enrichment/claims, not from atom bytes. +- **Reference implementation already shipped:** `@0xintuition/iid` (in `intuition-v2/intuition/iid`) — `norm1()`, all per-scheme canonicalizers/validators with checksums, `gen1` derivation, `parseIntuitionId`/`validateIntuitionId`/`isIntuitionId`, `SCHEME_TYPING` + `isAnchorEligible`, 92 golden tests. Ladders live as `identity` blocks on all 37 `ClassificationSpec`s in `intuition/classifications`. **We are wiring, not inventing.** + +## 2. Current pipeline and where it changes + +``` +Chain (MultiVault) + │ AtomCreated (unchanged) + AtomContextRegistered (NEW) + ▼ +crates/rindexer-ingestion ─ Rust, ABI-generated typings ← Track 1 + │ event_store + typed tables (Timescale) + ▼ +crates/projections ─ term rows, kg.nodes, kg.events, Surreal ← Track 2 + │ kg.nodes (raw_type, data, data_resolved…) + kg.node_urls + ▼ +services/workers (TS) + ├─ parse packages/atom-parser ← Track 3 + ├─ classification packages/atom-classification ← Track 3 + └─ enrichment packages/atom-enrichment / atom-services ← Track 4 + ▼ +services/api (Hono REST) → apps/explorer ← Track 6 + +intuition-v2 seed pipeline (DB-first, no IPFS/tx today) ← Track 5 +``` + +Per-layer impact, in one line each: + +| Layer | Today | After | +|---|---|---| +| Ingestion | Decodes `AtomCreated` only | Also decodes `AtomContextRegistered`; stores URIs in a typed table | +| Projections | Materializes `atom_data` into `term`/`kg.nodes` with `raw_type ∈ {string,json,http_uri,ipfs_uri}` | Adds `intuition_identifier` raw_type; projects on-chain URIs into `kg.node_urls` | +| Parse | Detect cascade: ipfs → eth addr → ens → json → url → isbn → plain_string (an `int:` atom falls to `plain_string` today) | New `intuition_identifier` kind (checked **first**), scheme/value extracted and validated via `@0xintuition/iid`; no remote fetch needed | +| Classification | `@type` read out of the JSON-LD document; URL-domain plugins | Scheme→type lookup for unambiguous schemes (no fetch, `recognized` instantly); payload `@type` for P1/P2; legacy path retained | +| Enrichment | Keyed by `hints.url` / `sameAs` targets; plugins match on URL shape | Keyed by `hints.identifiers` (`{isrc: …}`, `{isbn: …}`); providers gain lookup-by-identifier paths; on-chain URIs feed the URL path as before | +| Database | `kg.nodes` has no identifier columns; `node_urls` has no on-chain source | `identifier_scheme`/`identifier_value` (or equivalent), `raw_type` CHECK extended, `node_urls.source='onchain'`, Timescale `atom_context_registered_events` | +| API | `detectRawType` knows 4 kinds; serialization has no identifier fields | Identifier detection + new response fields; OpenAPI/docs updated | +| Seed pipeline (v2) | `deriveAtomData()` emits JSON-LD; ad-hoc identifier projection (`isrc:{…}` strings) | Ladder-driven `deriveIntuitionId()`; P0 strings / P1 JSON per class; URIs sourced from `sameAs` harvest | + +Detailed file-level inventories live in each track doc. + +## 3. Dependency graph & sequencing + +``` + ┌──────────────────────────┐ + │ PRE-SPRINT: decisions + │ + │ contracts npm pin exists │ + └───────────┬──────────────┘ + ┌──────────────┼───────────────────────┬─────────────────┐ + ▼ ▼ ▼ ▼ + Track 1 Track 3 (parser/class) Track 4 (enrich) Track 5 (seed, other repo) + ingestion — independent of 1/2 — depends on 3's — independent; only shares + │ once contracts pin parse-result the iid package + decisions + ▼ is bumped shape (agree on + Track 2 interface at + projections/DB ◄── raw_type/columns ──── standup, then + │ agreed with 3 parallel) + └────────────┬────────────────────────┘ + ▼ + Track 6 — API/explorer/devnet/QA (integrates everything; starts + on independent pieces day 1, integration day 2) +``` + +Hard dependencies (everything else is parallel): + +1. **The bumped `@0xintuition/contracts-v2` npm package must exist before the sprint** (Track 1's first step is `bun run abis:sync`). If it isn't published, vendoring the ABI JSON manually is the day-1 fallback. +2. **`@0xintuition/iid` must be consumable from intuition-core** (Tracks 3, 4, 5, 6). Decide publish vs. vendor before the sprint — see decisions below. +3. Track 2 needs Track 1's typed-table shape (agree in the morning standup; it's one table definition). +4. Tracks 3/4 share the `CompactParseResult`/hints interface — agree on the `{ scheme, value, canonical, identifiers }` shape at standup, then work in parallel against fixtures. +5. Track 6's end-to-end verification is the last thing to run (day 2 afternoon): devnet → `createAtomsWithUris` with `int:` atoms → indexer → parse → classify → enrich → API → explorer. + +### Suggested schedule + +**Day 0 (before the sprint, lead):** lock the decisions in §4; confirm contracts npm pin + deployed contract address for devnet/testnet; publish or vendor `@0xintuition/iid`; share this plan. + +**Day 1 morning:** 30-min standup — walk the interface agreements (typed table shape, raw_type value, parse-result shape, node_urls source enum). Then all tracks start. + +**Day 1 target:** Tracks 1–2 code-complete (Rust compiles, migrations run, devnet event lands in Timescale + kg). Track 3 parser done, classification mapping in review. Track 4 first two providers (Spotify-by-ISRC, MusicBrainz) working against fixtures. Track 5 derivation swapped, dry-run diff produced. Track 6 devnet acceptance updated + smoke-test scaffold. + +**Day 2:** integration. Wire workers end-to-end on devnet, fix interface drift, remaining enrichment providers, API/explorer surface, docs, run full smoke. End with the checklist in `06-track-api-explorer-qa.md` §5 green. + +## 4. Decisions to lock BEFORE the sprint + +These are the things that will cause mid-sprint thrash if left open. Recommended defaults included so the lead can just ratify. + +| # | Decision | Recommendation | +|---|---|---| +| D-1 | **What goes in the on-chain `uris` field?** The planning docs predate it. | Additional-context URLs (the old `sameAs` payload: Spotify links, Wikipedia URLs…) and/or secondary `int:` identifiers. Indexer treats each entry as an opaque string: if it validates as an IID → record as secondary identifier; if it parses as a URL → `kg.node_urls`; else keep raw. Don't over-model on day 1. | +| D-2 | **`raw_type` value name** for identifier atoms (kg.nodes CHECK + API + explorer). | `'intuition_identifier'`. | +| D-3 | **How does intuition-core consume `@0xintuition/iid`?** It lives in intuition-v2. | Publish to npm (it's already a clean package with golden tests). Fallback: vendor into `packages/` as a temporary copy with a tracking issue to swap to the npm dep. Do **not** hand-port canonicalizers. | +| D-4 | **Seed pipeline default profile (PE-Q3):** P0-only, P0+P1, or P2+anchor? | P0 bare anchors for Class A/B unambiguous schemes; P1 JSON for Class C and polymorphic schemes (spec-required floor). No P2 for new seeds — display data goes to enrichment artifacts, which the DB-first seed path already writes. | +| D-5 | **Rehydration/search floor (PE-Q4):** a P0 atom is unrenderable until enriched. | `search_text` and display label come from enrichment artifacts (`data_resolved`), falling back to the raw IID string. Explorer shows the IID + scheme badge while unenriched. Accept the loading state for the sprint. | +| D-6 | **Legacy atom backfill (Q5)** — indexer-computed IIDs for existing atoms, `sameAs` clustering, equivalence layer. | **Out of scope for the sprint.** The additive design means legacy atoms keep working through the old parse path. Schedule backfill (checklist Phase 5) + equivalence layer (Phase 4) as the follow-up project. | +| D-7 | **Do secondary identifiers from `uris` trigger their own enrichment?** | Not in the sprint. Store them; enrich off the primary identifier only. | +| D-8 | **Which schemes must be enrichable by end of sprint?** | Match live seed lanes: `isrc` (Spotify/MusicBrainz), `isbn` (OpenLibrary/ISBN provider), `wd` (Wikipedia/Wikidata), `doi` (Crossref — already keys off identifiers today), `caip10`/`caip19` (existing ethereum providers), `url` (existing path). Everything else: parse + classify correctly, enrichment `skipped` status, add providers later. | +| D-9 | **Processing-scope semantics** (`music`/`podcast` domains) for identifier atoms. | Map scheme → domain (`isrc`,`mbid`,`iswc` → music; `podcastguid`,`rssitem` → podcast) in the same place classification maps scheme → type. | + +Open questions from the spec that do **not** block the sprint (park them): Q1 formal ratification, Q2 equivalence thresholds, AL-Q1/Q2 type-as-a-claim predicates, Q10 pipeline signer identity, cross-chain Q9. + +## 5. Risks / gotchas surfaced during research + +- **The `AtomContextRegistered` event is fire-and-forget.** URIs exist only in logs. The indexer handler must be in place before real `createAtomsWithUris` traffic, or we'll need a log-backfill job. (Backfill-from-logs is possible — note it as the recovery path, don't rely on it.) +- **Atom IDs change for seeded data.** Same entity, new atom data string ⇒ new `calculateAtomId`. The DB-first seed corpus will mint *different* node IDs than the JSON-LD run. Decide whether to wipe-and-reseed the target environment (recommended for the sprint) or run both corpora side by side pending the equivalence layer. +- **`int:` atoms currently fall through to `plain_string`** in `packages/atom-parser/src/detect.ts` — the detection cascade must check `int:` **before** other branches (cheap prefix check, then validate). +- **Case sensitivity:** IID comparison is byte-exact; canonicalization is per-scheme (e.g. ISRC uppercase, DOI lowercase). Never lowercase an IID wholesale anywhere in the pipeline — always go through `@0xintuition/iid` canonicalizers. +- **Multi-valued selection (D28):** anywhere we pick one value from a set (e.g. choosing which `sameAs` URL becomes a `url`-scheme IID), selection must be a pure function of the set (lexicographically smallest canonical value), never first-seen order. A real bug shipped from getting this wrong once already (audit F1). +- **`mbid` requires the entity-type segment** (`int:mbid:artist:`), and `tmdb`/`appid`/`acct` are similarly compound — parse values per-scheme, don't assume `value` is atomic. +- **Deploy acceptance uses `parseEventLogs` with the full MultiVault ABI** — the event addition is additive so it won't break, but the acceptance script should start exercising `createAtomsWithUris` so the whole new path is covered on every deploy. +- **Do not gut the legacy path.** IPFS/JSON-LD parsing, `structured.ts`, URL-domain classification plugins all stay — the chain has existing atoms and D29 makes profiles additive. This is an *add-a-path* migration, not a replace. diff --git a/.planning/claude-migration/01-track-ingestion.md b/.planning/claude-migration/01-track-ingestion.md new file mode 100644 index 0000000..b07473d --- /dev/null +++ b/.planning/claude-migration/01-track-ingestion.md @@ -0,0 +1,64 @@ +# Track 1 — Chain Ingestion (Rust): ABI + `AtomContextRegistered` + +**Owner:** 1 engineer (Rust, comfortable with rindexer codegen) +**Repo:** `intuition-core` +**Mission:** the indexer decodes the new `AtomContextRegistered` event and lands URIs in Timescale, with the updated MultiVault ABI flowing through the whole provenance chain. `AtomCreated` handling is untouched (the event did not change — only its payload *content* shifts to `int:` strings, which is opaque bytes at this layer). + +## 1. Contract facts you're building against + +- New event (the only URI carrier; not stored on-chain): + `AtomContextRegistered(bytes32 indexed termId, address indexed registrant, bytes[] uris)` + topic0 `0x006dfca493b1686f1cc639fa9675dbd6f4a694a9d3230c346f3879d925382097` +- Emitted at most once per atom, only from `createAtomsWithUris`, only when the atom's URI list is non-empty, same tx **after** that batch's `AtomCreated`/`Deposited` events. `registrant` == `AtomCreated.creator`. +- Limits: ≤5 URIs/atom, ≤700 bytes each (defaults; `AtomUriConfigUpdated(uint32,uint32)` exists but doesn't need indexing for the sprint). +- URIs are write-once — no update event will ever arrive for an existing atom. +- Nothing else changed: `AtomCreated`, `TripleCreated`, `Deposited`, `Redeemed` are byte-identical; atom ID formula unchanged. + +## 2. Work items (in order) + +### 2.1 ABI provenance chain + +The ABI flows: `@0xintuition/contracts-v2` npm → `packages/contracts/src/abis.ts` (`MultiVaultAbi`) → `scripts/sync-abis.ts` → `crates/rindexer-ingestion/abi/MultiVault.json` (CI-gated by `abis:check`). + +1. Bump the `@0xintuition/contracts-v2` pin in `packages/contracts/package.json` to the version containing commit `a402fad` (upstream regenerated `abis/MultiVault.ts` in that commit). *Fallback if unpublished:* hand-patch `crates/rindexer-ingestion/abi/MultiVault.json` with the `AtomContextRegistered` event + `createAtomsWithUris` function and leave a TODO to re-sync. +2. `bun run abis:sync`, verify `abis:check` passes. +3. Confirm `AtomContextRegistered` is present in `crates/rindexer-ingestion/abi/MultiVault.json`. + +### 2.2 rindexer typings + handler + +- Regenerate rindexer bindings → `crates/rindexer-ingestion/src/rindexer_lib/typings/be_v_3_indexer/events/multi_vault_abi_gen.rs` and `.../events/multi_vault.rs` gain `AtomContextRegistered{Data,Result,Event}` alongside the existing `AtomCreated*` types (enum around line 1006). +- `crates/rindexer-ingestion/src/handlers.rs` — add `register_atom_context_registered_handler` next to `register_atom_created_handler` (line 154). Payload: `{ term_id, registrant, uris }` with each URI hex-encoded the same way `atomData` is (line 182) — URIs are `bytes`, not guaranteed UTF-8; decode to string at projection time. +- Register the handler wherever the existing handlers are wired (same module). +- `crates/rindexer-ingestion/rindexer.yaml` — confirm/update contract address + start block env vars for the redeployed contract. + +### 2.3 Storage (Timescale) + +- New migration in `migrations/timescale/` (append after 039): `atom_context_registered_events` table mirroring `atom_created_events` (`migrations/timescale/002_create_typed_event_tables.sql:10`) — `term_id NUMERIC`, `registrant TEXT`, `uris TEXT[]` (hex-encoded entries), plus the standard block/tx/log metadata columns of the other typed tables. + - *Why a new table, not a column on `atom_created_events`:* it's a distinct event with its own log index; joining by `term_id` at projection time is trivial and keeps ingestion append-only. +- `crates/rindexer-ingestion/src/storage.rs` — add `insert_atom_context_registered_events` modeled on `insert_atom_created_events` (line 301): write both to `event_store` (JSON payload) and the typed table (like line 341). + +### 2.4 Shared event model (`crates/shared`) + +- `crates/shared/src/models.rs:139` — new `AtomContextRegisteredRecord { term_id, registrant, uris: Vec, … }` next to `AtomCreatedRecord`. Leave `AtomCreatedRecord` alone. +- `crates/shared/src/parsed_event.rs:111` — new `ParsedEvent::AtomContextRegistered` variant + reserialization arm (line 361) + fixtures. Existing fixtures with `"atom_data": "ipfs://QmFoo"` stay valid; add sibling fixtures with `atom_data` = hex of `int:isrc:USSM10007459` and a matching context event. +- `crates/shared/src/types.rs:33` — `EventType::AtomContextRegistered`. +- `crates/shared/src/test_utils.rs:150` + `crates/shared/src/proptest_invariants.rs:197` — fixture + proptest strategy for the new event. + +## 3. Interfaces to agree at standup + +- **Typed table shape** (`atom_context_registered_events` columns) — Track 2 reads it in `typed_reader.rs`. +- **Hex vs UTF-8 for stored URIs** — recommendation: store hex in Timescale (faithful to bytes), decode in projections (Track 2), same division of labor as `atom_data`/`decode_atom_data_hex`. + +## 4. Definition of done + +- [ ] `abis:check` green; `AtomContextRegistered` in the synced ABI JSON. +- [ ] Rust workspace compiles; shared-crate tests + proptests pass with new fixtures. +- [ ] Timescale migration applies cleanly on a fresh datastore (`docker-compose` migrate services). +- [ ] On devnet: a `createAtomsWithUris` tx (coordinate with Track 6, who is updating the acceptance script) produces rows in both `atom_created_events` and `atom_context_registered_events` with matching `term_id`. +- [ ] An atom created via plain `createAtoms` (no URIs) still indexes exactly as before — no regression. + +## 5. Explicitly out of scope + +- Interpreting URI contents (IID vs URL) — Track 2/3. +- `AtomUriConfigUpdated` indexing, FeeProxy `createAtomsWithUrisVia` call-level indexing (log-level capture already covers the URIs). +- Historical log backfill tooling (only needed if real `createAtomsWithUris` traffic predates handler deploy — flag to lead if so). diff --git a/.planning/claude-migration/02-track-projections-db.md b/.planning/claude-migration/02-track-projections-db.md new file mode 100644 index 0000000..673ed99 --- /dev/null +++ b/.planning/claude-migration/02-track-projections-db.md @@ -0,0 +1,52 @@ +# Track 2 — Projections & Database Schema + +**Owner:** 1 engineer (Rust + Drizzle/SQL) +**Repo:** `intuition-core` +**Mission:** projections materialize identifier atoms and on-chain URIs into `term`, `kg.nodes`, and `kg.node_urls`; the kg schema gains identifier columns and the extended `raw_type`; all Drizzle/Timescale schema mirrors and manifests are regenerated. + +**Depends on:** Track 1's `atom_context_registered_events` table shape (agree at standup — don't block, the shape is one table). Coordinate `raw_type` value and identifier column names with Track 3. + +## 1. KG Postgres schema (`packages/database-kg`) + +- `src/schemas/kg/nodes.ts:16` — + - Extend the `raw_type` CHECK (line 89: `IN ('string','json','http_uri','ipfs_uri')`) with `'intuition_identifier'` (per decision D-2). + - Add nullable columns `identifier_scheme TEXT` and `identifier_value TEXT` (promoted by the parse worker; also directly settable by projections for bare `int:` payloads). Index on `(identifier_scheme, identifier_value)` — this is the future `iid → [atomId]` equivalence key (spec D7), so make it a plain (non-unique) btree index now. +- `src/schemas/kg/node_urls.ts` — add `'onchain'` to the `source` enum/values. On-chain URIs land here (one row per URI, `is_primary=false`). +- New Drizzle migration (after `drizzle/0000_kg_core_init.sql` + 2 successors) via `src/migrate.ts` — remember the `raw_type` CHECK lives in generated SQL too. +- `src/actions/nodes.ts` / `src/actions/processing.ts` — `completeNodeProcessingStage` promoted-fields path must accept `identifierScheme`/`identifierValue` (Track 3's parse worker writes them). +- Predicate seeds (`src/seeds/predicates.ts`): no change required for the sprint (equivalence predicates like `sameAs`/`differentFrom` are the follow-up project — park unless trivially cheap). + +## 2. Timescale schema (`packages/database-timescale` + `migrations/timescale`) + +- Track 1 owns the new `atom_context_registered_events` migration; you own its Drizzle mirror: add the table to `src/schemas/timescale/`, regenerate `schemas/timescale/manifest.json` + `compat-inventory.json` (generation code `src/timescale-generation/`), fix `packages/database-timescale/tests/`. +- `term` table (`migrations/timescale/006_create_term_table.sql`, mirror `src/schemas/timescale/terms.ts:20-23`): **recommendation — no new columns.** `term` keeps `atom_data`/`atom_data_hex` as-is (the identifier *is* the atom data); URIs live in kg (`node_urls`) and the typed event table. Only add term columns if the API/explorer team (Track 6) makes a case at standup. + +## 3. Projections (`crates/projections`) + +- `src/event/typed_reader.rs:41-48` — extend the union SQL to also read `atom_context_registered_events` (new event kind flowing to the projection loop). +- `src/projection/dual/core_entities.rs` — the central atom materializer: + - On `AtomCreated` (existing path, `build_atom_ops_typed` line 181): after `decode_atom_data_hex` (line 111), detect the `int:` prefix on the decoded UTF-8 string. If it validates shape-wise (cheap check: `^int:[a-z0-9-]{1,32}:.{1,220}$` — full canonical validation stays in TS workers), set `raw_type='intuition_identifier'` and populate `identifier_scheme`/`identifier_value` on the `kg.nodes` insert (lines 360–410) instead of the default `'string'`. Doc comment at line 34 (raw_type refinement contract with the parse worker) needs updating. + - New handler for `AtomContextRegistered`: decode each hex URI to UTF-8 (lossy-tolerant; skip+log undecodable entries) and insert into `kg.node_urls` (`node_id=termId`, `source='onchain'`) — idempotent on conflict (event replay). Also record in `kg.events` (line 439 pattern) if other event kinds do. + - Ordering note: the context event arrives after `AtomCreated` in the same tx, so the node row exists by the time you project it within a block batch — but make the insert order-independent anyway (upsert semantics), since batch boundaries aren't guaranteed. +- `src/projection/surreal/atom.rs` — mirror the raw_type/identifier handling in the Surreal projection (same decode path, lines 30–130). + +## 4. Interfaces to agree at standup + +- `raw_type` value string (D-2: `'intuition_identifier'`) — shared with Track 3 (parser, worker, API `detectRawType`) and Track 6 (API serialization, explorer). +- Identifier column names (`identifier_scheme`/`identifier_value`) — Track 3 writes them from the parse stage; Track 6 serializes them. +- Division of validation: projections do **shape** detection only; canonical validation/quarantine is the parse worker (Track 3). Rationale: don't port 25 canonicalizers to Rust for the sprint. +- What happens to a `uris` entry that is itself an IID (D-1): sprint answer — it still lands in `node_urls` as an opaque string with `source='onchain'`; Track 3+ may later promote secondary identifiers. Keep projections dumb. + +## 5. Definition of done + +- [ ] Fresh-stack migration run green (Timescale + kg Drizzle), plus migration on a datastore with existing rows. +- [ ] Devnet atom with data `int:isrc:USSM10007459` + 2 URIs ⇒ `kg.nodes` row with `raw_type='intuition_identifier'`, scheme/value populated, 2 `kg.node_urls` rows with `source='onchain'`; `term` row unchanged in shape. +- [ ] Legacy JSON-LD and `ipfs://` atoms project exactly as before (regression: rerun `scripts/smoke-index.sh` block window). +- [ ] Event replay (re-run projections over the same range) is idempotent — no duplicate node_urls. +- [ ] Manifest/compat-inventory regenerated; `database-timescale` and `database-kg` tests pass. + +## 6. Out of scope + +- The `iid → [atomId]` unique-index clustering/equivalence layer (follow-up project; you're only laying the index). +- Rust-side canonical IID validation (TS workers own it). +- Backfilling `identifier_*` for legacy atoms (Q5 backfill project). diff --git a/.planning/claude-migration/03-track-parsing-classification.md b/.planning/claude-migration/03-track-parsing-classification.md new file mode 100644 index 0000000..b740516 --- /dev/null +++ b/.planning/claude-migration/03-track-parsing-classification.md @@ -0,0 +1,78 @@ +# Track 3 — Atom Parser & Classification Packages + +**Owner:** 1 engineer (TypeScript; owns `packages/atom-parser`, `packages/atom-classification`, the parse/classification worker stages, and the shared `@0xintuition/iid` adoption) +**Repo:** `intuition-core` (+ coordination on publishing `@0xintuition/iid` from intuition-v2) +**Mission:** an `int:` atom is detected as a first-class `intuition_identifier` kind, validated/canonicalized via `@0xintuition/iid`, and classified instantly from its scheme with no network fetch. Legacy JSON-LD/URL/IPFS paths keep working untouched. + +## 1. Foundation: `@0xintuition/iid` + +Everything in this track consumes the already-shipped package from `intuition-v2/intuition/iid` (`parseIntuitionId`, `validateIntuitionId`, `isIntuitionId`, per-scheme canonicalizers, `SCHEME_TYPING`, `isAnchorEligible`, 92 golden tests). Per decision D-3: publish it to npm (preferred) or vendor it into `packages/`. **Do this first — Tracks 4, 5, 6 also depend on it.** Do not re-implement canonicalizers. + +Key semantics to respect (from the spec): +- Parse on the **first two colons only**; values may contain `:` and `/` (`mbid:artist:`, `caip10:eip155:1:0x…`, `doi:10.1000/182`). +- Byte-exact comparison after per-scheme canonicalization; never case-fold an IID wholesale. +- Malformed IIDs (bad checksum, unknown scheme, bad casing) → **quarantine as invalid, never merge/classify on them** (spec D7). Parse status `failed` with a reason, not silent `plain_string` fallback. + +## 2. `packages/atom-parser` + +- `src/detect.ts:6` `detectLocal` — add `intuition_identifier` detection **first** in the cascade (before ipfs/ethereum/ens/json/url/isbn/plain_string; cheap `int:` prefix guard, then `validateIntuitionId`). An `int:`-prefixed string that fails validation → explicit invalid-identifier parse failure (see quarantine above), not fall-through. +- `src/types.ts:1` — extend `ParsedKind` union; add + ```ts + interface IntuitionIdentifierParseResult { + kind: 'intuition_identifier' + scheme: string // e.g. 'isrc' + value: string // canonical value, e.g. 'USSM10007459' + canonical: string // full canonical IID, e.g. 'int:isrc:USSM10007459' + inValueType?: string // for mbid/tmdb/appid/acct/gen1 compound values, e.g. 'artist' + } + ``` + (exact shape = the standup interface agreement with Track 4). +- `src/remote.ts` — identifier atoms need **no remote fetch**; short-circuit before fetch-policy logic. +- **P1/P2 JSON payloads:** `src/structured.ts` already extracts from JSON-LD; extend it to surface a top-level `identifier` field when present (per spec, positioned after `@type`) and validate it — a JSON atom with a valid `identifier` gets `identifiers`/scheme hints populated too, while keeping `kind: 'json'`. +- Fixtures: `__tests__/fixtures/atom-parser-contract-fixtures.json` + local-detection/parity/integration suites — add P0 anchors for a spread of schemes (`isrc`, `isbn`, `wd`, `caip10`, `mbid:artist`, `gen1:…:r4:…`), a P1 payload, invalid cases (bad ISRC checksum-free format, unknown scheme, uppercase scheme), and the legacy-bare-URL-vs-IID disambiguation case. + +## 3. Worker parse stage (`services/workers`) + +- `src/core/parse.ts:3` `CompactParseResult` — add the identifier branch in `toCompactParseResult`: `canonicalId` = canonical IID; `identifiers` = `{ [scheme]: value }` (this is what Track 4's enrichment keys off — e.g. `{ isrc: 'USSM10007459' }`); `hints` carries scheme/inValueType. +- `src/kg/atom-parsing/index.ts` — promote `identifier_scheme`/`identifier_value` to `kg.nodes` on parse completion (column names from Track 2); `resolveSearchText` for identifier atoms: fall back to the raw IID string (per decision D-5, enrichment later overwrites the display label via `data_resolved`). +- Also fold in **on-chain URIs**: node_urls rows with `source='onchain'` (written by Track 2) should be visible to downstream stages the same way `sameAs`-derived URL candidates are today — check `src/core/structured-targets.ts` and thread them into the candidate/hints flow so Track 4 can use them as a fallback enrichment path. + +## 4. Classification + +### `packages/atom-classification` + +- **Scheme→type mapping — the heart of this track.** Add a plugin (or registry-level short-circuit) that maps identifier schemes to `TYPE_DEFINITIONS` entries in `src/plugins/type-profiles/index.ts`, sourced from `SCHEME_TYPING` in `@0xintuition/iid` rather than a hand-written table: + - Unambiguous: `isrc`→MusicRecording, `isbn`→Book, `iswc`→(composition — nearest existing type or add), `gtin`→Product, `lei`→Organization, `eidr`→Movie/TVSeries, `podcastguid`→PodcastSeries, `rssitem`→PodcastEpisode, `caip19`→EthereumERC20, `appid`→SoftwareApplication, `purl`→SoftwareSourceCode, `acct`→SocialMediaAccount, `olid`→Book, `termset`→DefinedTerm. + - In-value typing: `mbid:artist`→MusicGroup, `mbid:recording`→MusicRecording, `mbid:release-group`→MusicAlbum, `tmdb:movie`→Movie, `tmdb:tv`→TVSeries, `gen1:`→that classification. + - Polymorphic (`wd`, `url`, `doi`, `hash`, `geo`, `caip10`, `imdb` bare, `isni`, `orcid`): type comes from the P1 payload `@type` when present; otherwise classify as the scheme's broadest sensible type or `Thing`, status `ambiguous` — do **not** guess. + - Gap check: some ladder classifications have no existing `TYPE_DEFINITION` (e.g. composition/iswc). Add minimal definitions rather than mis-mapping. +- Existing URL-domain plugins (`spotify`, `imdb`, `isbn`, …) untouched — legacy path. +- Cache keys (`src/cache.ts`): canonical IID string is the natural stable key. +- Update `packages/atom-classification-example-plugin` + `docs/writing-a-classification-plugin.md` if the plugin interface gains an identifier input shape. + +### Worker classification stage + +- `src/core/classification.ts:39` `deriveClassificationPlan` — new branch: `kind === 'intuition_identifier'` → scheme lookup → status `recognized`, zero runtime fetch. P1 JSON atoms flow the existing `structuredDocument`/`schemaType` path (line 180) but cross-check payload `@type` against the scheme's typing when the scheme is unambiguous (mismatch ⇒ scheme wins, flag in metadata). +- `resolveClassificationType` (line 124) unchanged mechanically — writes `kg.nodes.classification_type`. +- **Processing scope (D-9):** `src/shared/processing-scope.ts` — map schemes to `ProcessingDomain` (`isrc`/`mbid`/`iswc`→`music`, `podcastguid`/`rssitem`→`podcast`) so scoped deployments pick up identifier atoms. Mirror anything needed in `packages/graph-flags` / `crates/shared/src/graph_flags.rs` only if scope filtering happens there too (check with Track 2). +- `src/kg/atom-classification/index.ts` — `engine.classify({input})` currently takes a URL/string `runtimeInput`; thread the parsed identifier through (interface change coordinated with `@0xintuition/atom-services/runtime` — Track 4 co-owns runtime wiring). +- Shared types: `packages/types/src/classification/index.ts` re-exports. + +## 5. API-side detection duplicate + +- `services/api/src/app.ts:91` `detectRawType` + `services/api/tests/detect-raw-type.test.ts` — add `intuition_identifier` (used by `POST /api/atoms` offchain create, app.ts:237/259). Small; do it here since it must match the parser's semantics exactly. Track 6 owns the rest of the API surface. + +## 6. Definition of done + +- [ ] `@0xintuition/iid` consumable in intuition-core (published or vendored), goldens passing in CI. +- [ ] Parser: all new fixtures green; `int:isrc:USSM10007459` ⇒ kind `intuition_identifier` with scheme/value/canonical; invalid IIDs ⇒ explicit failure; every legacy fixture unchanged. +- [ ] Worker parse on devnet promotes scheme/value + search_text to `kg.nodes`. +- [ ] Classification: `int:isrc:…` ⇒ `MusicRecording`/`recognized` with no network call; `int:mbid:artist:…` ⇒ MusicGroup; `int:wd:Q42` without payload ⇒ ambiguous-but-classified path per the mapping rules; JSON-LD fixture atoms classify exactly as before. +- [ ] `detectRawType` parity test between API and parser. +- [ ] Interface handoff to Track 4 honored: `identifiers` map populated in `CompactParseResult` for enrichment keying. + +## 7. Out of scope + +- Enrichment providers (Track 4). +- Equivalence clustering on the identifier index (follow-up project). +- Backfilling classifications for legacy atoms. diff --git a/.planning/claude-migration/04-track-enrichment.md b/.planning/claude-migration/04-track-enrichment.md new file mode 100644 index 0000000..9a2794c --- /dev/null +++ b/.planning/claude-migration/04-track-enrichment.md @@ -0,0 +1,77 @@ +# Track 4 — Enrichment: Identifier-Keyed Providers + +**Owner:** 1 engineer (TypeScript; owns `packages/atom-enrichment`, the enrichment worker stage, and `services/atom-services`) +**Repo:** `intuition-core` +**Mission:** enrichment is triggered and keyed by the parsed identifier (`{isrc: 'USSM10007459'}`) instead of by a `sameAs` URL scraped from atom JSON. Providers gain lookup-by-identifier paths; on-chain URIs serve as the secondary URL-shaped path; unenrichable schemes skip cleanly. + +**Depends on:** Track 3's `CompactParseResult.identifiers` shape (agree at standup, then develop against hand-built fixtures — don't wait for the parser to merge). + +## 1. The inversion in one paragraph + +Today: JSON atom → `structured-targets` picks a `sameAs` URL → plugin `supports(request)` matches URL shape → provider scrapes/queries by URL. After: identifier atom → `hints.identifiers = { isrc: '…' }` → plugin `supports()` matches on identifier presence → provider queries its API's *lookup-by-identifier* endpoint (Spotify search `isrc:` filter, MusicBrainz `/recording?query=isrc:…`, OpenLibrary by ISBN, Wikidata by QID, Crossref by DOI). The plumbing hook **already exists**: `packages/atom-enrichment/src/plugins/providers/__shared__/request.ts:65` `getIdentifier(request, ...keys)` reads `request.input.hints.identifiers`, and the **crossref provider already keys off `doi` this way (line 103)** — it is the reference pattern for every provider you touch. + +## 2. Work items + +### 2.1 Types & request plumbing (`packages/atom-enrichment`) + +- `src/types.ts` — extend `enrichmentRequestSchema` (line 213) / hints schema so `identifiers: Record` plus `identifierScheme`/`canonical` flow through validation; add an identifier-shaped variant to `classifiedAtomTargetsSchema` where targets are currently URL-only. +- `src/plugin-registry.ts` / presets — register updated providers; ensure `supports()` short-circuit order prefers identifier lookups over URL scraping when both are present. +- Cache keys: canonical IID (stable, byte-exact) — check enrichment cache tests. + +### 2.2 Providers (priority order per decision D-8 — match live seed lanes) + +| Priority | Provider dir (`src/plugins/providers/`) | Identifier | Lookup path | +|---|---|---|---| +| 1 | `spotify` | `isrc` | `resolveSpotifyTarget` (`index.ts:34`) currently URL-shaped; add ISRC branch → Spotify search `q=isrc:USSM10007459&type=track` | +| 1 | `musicbrainz` | `isrc`, `mbid` (typed) | ISRC lookup + direct MBID fetch per entity type | +| 2 | `isbn` | `isbn` | already ISBN-centric; accept identifier input directly (no URL) | +| 2 | `wikipedia` | `wd` | QID → Wikidata entity + sitelink → Wikipedia summary | +| 2 | `crossref` | `doi` | **already works via `getIdentifier` — verify + fixture, likely zero code** | +| 2 | `ens` / `nft-metadata` / existing ethereum path | `caip10`, `caip19` | address/chain extracted from CAIP value | +| 3 | `apple-music` | `isrc` | iTunes lookup by ISRC (the v2 extraction layer already chains Spotify→iTunes ISRC) | +| 3 | `podcast-index` | `podcastguid` | Podcast Index lookup by feed GUID | +| 3 | `tmdb` | `tmdb:movie/tv`, `imdb` | direct ID fetch / find-by-external-id | +| — | everything else (opengraph, favicon, screenshot, oembed…) | — | unchanged; they run off URL targets — including **on-chain URIs** (see 2.4) | + +Schemes with no provider this sprint (`gtin`, `lei`, `geo`, `purl`, `appid`, `acct`, `gen1`, …): enrichment status `skipped` with reason `no-provider-for-scheme` — must be a clean skip, not an error loop. + +### 2.3 Worker enrichment stage (`services/workers`) + +- `src/core/enrichment.ts` — + - `deriveEnrichmentPlan` (line 55): currently keyed on `targetUrl` + classification; add identifier-keyed planning (scheme+classification → provider set). Keep URL planning as the fallback chain. + - `buildClassifiedInputFromPlan` (line 73): **stops synthesizing a JSON-LD envelope from atom data for identifier atoms** — build the engine input from `{identifiers, classificationType, urls}` instead. This is the deepest assumption-break in the file; budget time here. + - Artifact-type allowlists (lines 11–35) and `isSpotifyTrackUrl` gating (135–138): add identifier-equivalents (an ISRC atom is in scope for MUSIC/SPOTIFY_TRACK artifact kinds without any URL). +- `src/kg/atom-enrichment/index.ts` — runtime input to `engine.enrich(...)` (line 150-152) carries the identifier fields; artifacts written as today (`kg.artifacts.source_uri` = canonical IID for identifier-sourced artifacts). +- Post-enrichment promotion: display name from artifacts must land in `search_text`/`data_resolved` so P0 anchors become renderable (decision D-5) — verify the existing promotion path does this once artifacts exist; fix if it only triggers off structured documents. + +### 2.4 On-chain URIs as enrichment inputs + +`kg.node_urls` rows with `source='onchain'` (Track 2) + threaded candidates from Track 3: treat them exactly like `sameAs` URL candidates today — they feed opengraph/oembed/provider-URL paths when identifier lookup yields nothing or as supplements. Per decision D-7, `uris` entries that are themselves IIDs are **stored but not enriched** this sprint. + +### 2.5 `services/atom-services` + +- HTTP schemas for `POST /v1/classify | /v1/enrich | /v1/process | /v1/process/batch` (`src/app.ts:111-185`) accept identifier-shaped input (a bare `int:…` string should be a valid `process` input). +- Runtime wiring `src/service/{runtime,processing,dependencies,persistence,batch-store}.ts` — co-owned with Track 3 where the shared `@0xintuition/atom-services/runtime` interface changes. +- Cache provider config unchanged; confirm keys. +- Shared artifact types: `packages/types/src/enrichment/artifacts.ts` if any provider adds new artifact fields. + +## 3. Interfaces to agree at standup + +- `identifiers` map shape from Track 3 (scheme-keyed record; compound schemes like `mbid` pre-split into `{ mbid: 'artist:' }` vs `{ mbid_artist: '' }` — pick one, recommend keeping raw value + separate `inValueType`). +- Engine runtime input schema change (with Track 3 — both stages call `@0xintuition/atom-services/runtime`). +- Which artifact kinds Track 6's smoke test asserts (pick Spotify track fields). + +## 4. Definition of done + +- [ ] Fixture-level: `{isrc: 'USQX91300108'}` ⇒ Spotify + MusicBrainz artifacts with track metadata, no URL involved anywhere in the request. +- [ ] `int:isbn:9780684832722`, `int:wd:Q42`, `int:doi:10.1000/182` each produce their provider's artifact via identifier lookup. +- [ ] Unenrichable scheme (`int:geo:9q8yyk8y`) ⇒ clean `skipped`, worker lease completes, no retry loop. +- [ ] Legacy JSON-LD atom with Spotify `sameAs` URL enriches exactly as before (regression). +- [ ] End-to-end on devnet with Tracks 1–3: `int:isrc:…` atom reaches `enrichment_status=completed` with artifacts, and `search_text` shows the track name afterward. +- [ ] atom-services `POST /v1/process` with body input `int:isrc:…` returns classification + enrichment. + +## 5. Out of scope + +- Providers beyond the priority table (add stubs/skips only). +- Enriching secondary identifiers from `uris` (D-7). +- `atom-rules-engine` identifier rules (no in-repo consumer; ticket it for the app teams). diff --git a/.planning/claude-migration/05-track-seed-pipeline.md b/.planning/claude-migration/05-track-seed-pipeline.md new file mode 100644 index 0000000..d9c5fe2 --- /dev/null +++ b/.planning/claude-migration/05-track-seed-pipeline.md @@ -0,0 +1,84 @@ +# Track 5 — Seed Data Pipeline (intuition-v2) + +**Owner:** 1 engineer (TypeScript; knows the seed control plane / import-data scripts) +**Repo:** `intuition-v2` (this is the only track outside intuition-core) +**Mission:** the seed pipeline emits atoms in the new shape — P0 bare IID strings (or P1 JSON where required) derived through the classification ladders via `@0xintuition/iid` — with URI context carried alongside, before any upload/write. + +**Reality check from research:** the current seed path is **DB-first** — no IPFS pinning (`packages/ipfs-pinata` has zero consumers), no transactions, no wallets. It computes `calculateAtomId(atomData)` locally and writes straight into KG Postgres. Nothing in `packages/`, `scripts/`, `backend/`, or `apps/` imports `@0xintuition/iid` yet — the spec shipped but the pipeline never adopted it (checklist Phase 3, unstarted). That's this track. + +## 1. Current shape (what you're changing) + +- **Atom assembly:** `scripts/import-data/prepare-data-seed-enrichment.ts` — `deriveAtomData()` at **line 1945** builds the JSON-LD string (`{"@context":"https://schema.org/","@type":…,"name":…,"url":…}`); node ID = `calculateAtomId(derivedAtomData)` (line 1596); node rows written `rawType:'json'`, `is_onchain:false` (line 2385). +- **Control plane:** `packages/database-seed-control-plane/src/` (`cli.ts` stages `import-csv` → `claim-source-tasks` → `process-claimed-jobs` → `write-ready-jobs`, `staged-enrichment.ts`, `identifier-projection.ts`, `relations.ts`). +- **Identifier extraction already exists** and computes the right raw values in the wrong format: `identifier-projection.ts` emits ad-hoc `spotify_track → isrc + spotify:track`, `openlibrary_book → isbn`, `wikipedia_article → wd:{QID}`, `coingecko_coin → eip155:…`, `google_place → gplace:…`; plus `packages/atom-enrichment/src/extraction/isrc.ts` (incl. Spotify→iTunes chaining) and `wikidata-claims.ts`. +- **Writers:** `packages/database-kg/scripts/write-ledger.ts` (plan-first, `--execute`, batching, non-local guard) and control-plane `write-ready-jobs`; stack/perspective writers alongside. + +## 2. Work items + +### 2.1 Derivation swap (the core change) + +In `prepare-data-seed-enrichment.ts` and the control-plane equivalent (`staged-enrichment.ts` job processing): + +1. After classification+enrichment, assemble the field set and call `deriveIntuitionId(ladder, values)` from `@0xintuition/iid` with the classification's `identity` ladder (from `intuition/classifications`). The ladder walks rungs highest-first; you get `{iid, scheme, class, tag?}`. +2. Replace `deriveAtomData()` output per decision D-4: + - **Class A/B unambiguous scheme ⇒ P0**: atom data = the bare IID string; node `rawType:'string'` → **coordinate: intuition-core is introducing `'intuition_identifier'`** — write that value if the shared kg schema (Track 2) lands first; also populate the new `identifier_scheme`/`identifier_value` columns. + - **Class C (`gen1`) or polymorphic scheme ⇒ P1**: JSON `{ "@context", "@type", "identifier": "", …recipe fields…, "sameAs": […] }` — recipe fields are the hash preimage evidence and are **required** (all-or-nothing rungs, D5); `rawType:'json'`. + - No P2 for new seeds; display metadata stays in enrichment artifacts / `data_resolved` (already written by this path). +3. Node ID remains `calculateAtomId(newAtomData)` — **IDs change for the whole corpus** (see §4). +4. D28 discipline: wherever one value is selected from a set (multiple `sameAs`, multiple ISRCs), select the lexicographically smallest canonical value — pure function of the set, never first-seen. + +### 2.2 Converge `identifier-projection.ts` onto the canonical registry + +Replace the ad-hoc scheme strings with `@0xintuition/iid` canonicalizers: `isrc:{v}` → `int:isrc:{canonical}`, `wd:{QID}` → `int:wd:Q…`, `eip155:{chain}:{addr}` → `int:caip10`/`caip19` (lowercased address), `openlibrary → int:isbn`/`int:olid`. **Drop `gplace:{placeId}` as a primary identifier** — Google Place IDs are explicitly rejected from the registry (licensing); places floor at `gen1{name, geo7}` per the local-business ladder (keep the place ID in enrichment artifacts only). Spotify/Steam provider IDs are not registry schemes: they become **URI context** (§2.3) and enrichment keys, not primary IIDs. + +### 2.3 URI context (the future `createAtomsWithUris` payload) + +Source per atom, capped at the contract limits (**5 URIs × 700 bytes**, deterministic priority order so truncation is stable): +1. canonical provider URL (`canonical_url` / Spotify track URL), +2. manifest `Same As` column entries, +3. enrichment `sameAs` harvest (Wikidata claims etc.). + +DB-first today: persist as `kg.node_urls` rows (`source` value TBD with core Track 2 — suggest `'seed'` vs `'onchain'`) and/or `data_resolved.sameAs` as currently. Also emit the URI list into the ledger/control-plane row so the future on-chain mint (`createAtomsWithUris`) can replay it without recomputation. **No on-chain seed path exists yet — do not build one this sprint**; just make the data shape mint-ready. + +### 2.4 Writers & validation + +- `write-ledger.ts` / `write-ready-jobs`: write the new `rawType`, identifier columns, URI rows; keep plan-mode diffs meaningful (they'll show 100% ID churn — add an explicit summary line "N nodes re-keyed by IID migration"). +- Update co-located tests (`prepare-data-seed-enrichment.*.test.ts`, control-plane tests), and `buildNodeSearchText` — `name` no longer lives in atom data for P0; search text comes from enrichment (mirror core's D-5 rule). +- Add a validation gate before write: every P0 atom data must pass `validateIntuitionId`; every P1 must contain a valid `identifier` consistent with its recipe fields (re-derive and compare — catches recipe/normalization drift). + +### 2.5 Dry run (day-1 deliverable, feeds PE-Q3 evidence) + +Run the music lane (`spotify-music-seed-candidates`, strongest identifier coverage) through the new derivation **in plan mode** and produce a report: % reaching Class A (`isrc`), % falling to `mbid`/`gen1`, byte sizes, dedupe collisions (same ISRC from multiple candidate rows now collapsing to one atom — that's the feature working), URI-cap truncations. This is the go/no-go artifact for flipping `--execute`. + +## 3. Identifier feasibility by lane (from research — sets expectations) + +| Lane | Primary IID | Strength | +|---|---|---| +| Spotify music | `int:isrc:` (from spotify/apple/musicbrainz artifacts) | Strong (Class A) | +| Crypto (CoinGecko) | `int:caip19:` / `int:caip10:` | Strong | +| Books (OpenLibrary) | `int:isbn:` / `int:olid:` | Strong | +| Wikipedia | `int:wd:` | Strong (polymorphic ⇒ P1) | +| GitHub / arXiv | `int:url:` / `int:purl:` / `int:doi:` | Good | +| Podcasts | `int:podcastguid:` / `int:rssitem:`; Spotify show IDs only as URI context | Medium — needs feed-GUID resolution; UUIDv5-from-feed-URL fallback is offline-derivable | +| Places | `int:gen1:local-business:…{name, geo7}` (Place ID rejected) | Weak by design (Class C) — expect heavy reliance on the future equivalence layer; geohash-7 boundary forking is a known 23.6% issue, don't fight it this sprint | +| Movies/TV (TMDB, planned lane) | `int:wd:` → `int:tmdb:movie:` per ladder | Good when lane activates | + +## 4. The ID-churn decision (raise at standup, lead decides) + +New atom data ⇒ new `calculateAtomId` for every seeded node. Options: **(a) wipe-and-reseed** the target environment with the new corpus (recommended for the sprint — clean, exercises the whole new pipeline); (b) side-by-side corpora reconciled later by the equivalence layer (spec Layer 2.5/3, unbuilt). Old JSON-LD seed atoms and new IID atoms will NOT auto-cluster until that layer exists — don't promise dedupe across the migration boundary. + +## 5. Definition of done + +- [ ] `@0xintuition/iid` + `intuition/classifications` identity ladders wired into both seed paths (legacy script + control plane); no ad-hoc identifier formats remain in `identifier-projection.ts`. +- [ ] Music-lane dry-run report produced (§2.5) and reviewed. +- [ ] P0/P1 selection matches D-4 exactly; validation gate green over the full music lane. +- [ ] One lane written `--execute` into a local stack running Tracks 1–4's code: seeded `int:isrc:` nodes parse/classify/enrich in intuition-core without special-casing (proves seed output == on-chain-equivalent shape). +- [ ] URI context persisted and mint-ready (≤5 × ≤700 bytes, deterministic order). +- [ ] Tests updated; write-ledger plan mode reports ID churn explicitly. + +## 6. Out of scope + +- On-chain minting of seeds (`createAtomsWithUris` batch tooling) — future project once the seed→chain path is prioritized; the data shape from this sprint feeds it directly. +- IPFS pinning (still unconsumed — the P0 model makes it less relevant, not more). +- Backfill/equivalence for the old JSON-LD corpus (Q5 / Phase 4–5 follow-up). +- Held lanes (OpenLibrary bulk hydrate, TMDB, Steam) — migrate the *code paths*, don't activate new lanes. diff --git a/.planning/claude-migration/06-track-api-explorer-qa.md b/.planning/claude-migration/06-track-api-explorer-qa.md new file mode 100644 index 0000000..2ebb74f --- /dev/null +++ b/.planning/claude-migration/06-track-api-explorer-qa.md @@ -0,0 +1,63 @@ +# Track 6 — API Surface, Explorer, Devnet & End-to-End QA + +**Owner:** 1 engineer (full-stack; ideally the lead — this track integrates everyone else's work and owns the final green checklist) +**Repo:** `intuition-core` +**Mission:** identifier atoms are visible and correct through the API and explorer; devnet/acceptance/smoke tooling exercises the new `createAtomsWithUris` path end-to-end; docs reflect the new model. + +## 1. API (`services/api`) + +- Serialization: atom responses expose `identifierScheme`, `identifierValue`, and on-chain URIs (join `kg.node_urls where source='onchain'`, or via the existing node serialization if node_urls are already included). Routes: `GET /api/atoms`, `/api/atoms/:id`, `/api/atoms/:id/artifacts`. +- `GET /api/schema` introspects live kg schema (`src/schema.ts`) — auto-reflects Track 2's new columns; verify, don't assume. +- `detectRawType` update itself is Track 3 (must match parser semantics); you own the `POST /api/atoms` offchain-create flow around it (app.ts:237/259) accepting a bare `int:` string. +- Filtering: add `identifier_scheme` as a query filter on `/api/atoms` if cheap (nice-to-have, enables "all ISRC atoms" queries in QA). +- Docs: `docs/openapi.yaml`, `docs/api-reference.md`, `docs/example-queries.md`. Tests: `services/api/tests/{schema,pipeline-stats}.test.ts` + new serialization cases. + +## 2. Explorer (`apps/explorer`) + +- `src/routes/{atoms.index,atoms.$atomId,triples.$tripleId,index}.tsx`, `src/components/term-chip.tsx`, `src/lib/api.ts`: + - Render `intuition_identifier` raw_type: scheme badge (e.g. `isrc`) + canonical IID, monospace. + - Unenriched P0 anchor (decision D-5): show IID + "resolving…" state; post-enrichment, show the promoted display name from `data_resolved`. + - Atom detail: list on-chain URIs (linkified when they parse as URLs). +- Keep it modest — this is operator tooling, not product UI. + +## 3. Devnet & deploy acceptance (`packages/contracts`) + +- `src/deploy/acceptance.ts:50` — extend beyond the current `createAtoms([toHex('devnet-atom-…')])`: + 1. keep the legacy plain-string atom (regression), + 2. add `createAtomsWithUris` with atom data `toHex('int:isrc:USSM10007459')` (use a real ISRC so enrichment QA can reuse it) and 2 URIs (e.g. a Spotify track URL + a Wikipedia URL), + 3. assert both `AtomCreated` and `AtomContextRegistered` logs via `parseEventLogs` (additive ABI — existing assertions won't break, but pin the new event explicitly). +- `src/multivault.ts` helper + `__tests__/multivault.test.ts`: add a `createAtomsWithUris` wrapper (Track 5's future mint tooling and QA scripts both want it). +- CLI (`src/deploy/cli.ts:156`) — flag or default to include the identifier atom. +- Confirm devnet contract build/address includes commit `a402fad` (`devnet/deployments-devnet.json`, `docker/Dockerfile.devnet`, compose `devnet-deploy`); if the devnet image pins an older contracts build, updating it is a **day-1 blocker to escalate immediately**. + +## 4. Smoke & scripts + +- `scripts/smoke-index.sh` — currently replays public-testnet blocks 9030416–9030916 (50 legacy atoms) and asserts counts via the API. Keep as the **legacy regression**. Add a devnet-based smoke (or extend `scripts/smoke-test.sh`) asserting the new path: create identifier atom with URIs → poll API until `parse=completed`, `classification_type=MusicRecording`, `enrichment_status=completed` (or `skipped` if the environment lacks Spotify creds — assert either, gated on env), URIs present. +- `scripts/scope-dry-run.ts` — verify scheme→domain scoping (Track 3's D-9 mapping) picks up identifier atoms in `music`/`podcast` scopes. +- `scripts/explore-data.sh` — spot-check output includes identifier columns. + +## 5. The final integration checklist (day 2 afternoon — the sprint's exit criteria) + +Run on a fresh local stack (`docker-compose` full pipeline, Spotify creds set): + +- [ ] `devnet-deploy` acceptance green, incl. `createAtomsWithUris` + both events asserted. +- [ ] Timescale: rows in `atom_created_events` + `atom_context_registered_events` (Track 1). +- [ ] kg: node `raw_type='intuition_identifier'`, scheme/value populated, 2 `node_urls` rows `source='onchain'` (Track 2). +- [ ] Parse worker: `parse_status=completed`, `canonicalId` = the IID, search_text fallback set (Track 3). +- [ ] Classification: `classification_type='MusicRecording'`, status `recognized`, no network call in logs (Track 3). +- [ ] Enrichment: Spotify/MusicBrainz artifacts for the real ISRC; display name promoted to search_text/data_resolved (Track 4). +- [ ] API: atom serialized with scheme/value/URIs; OpenAPI updated; `POST /api/atoms` accepts a bare IID (Track 6). +- [ ] Explorer: renders scheme badge, then enriched name after pipeline completes (Track 6). +- [ ] Regressions: `smoke-index.sh` legacy window still green; a JSON-LD atom and an `ipfs://` atom created on devnet still flow the old path unchanged. +- [ ] Seed proof (Track 5, in intuition-v2 against this stack): one music-lane batch written; seeded nodes indistinguishable in shape from the devnet-minted identifier atom. +- [ ] Invalid-IID case: `int:bogus:xyz` atom lands quarantined (`parse failed`, no classification), doesn't wedge any worker. + +## 6. Docs sweep (parallelizable filler between integration runs) + +`docs/architecture.md`, `docs/data-model.md`, `docs/classification-taxonomy.md`, `docs/enrichment-providers.md`, `docs/contracts.md` (new event + function), `docs/indexing-scope.md`, `docs/local-devnet.md` (new acceptance atom), crate READMEs touched by Tracks 1–2, `docs/writing-a-classification-plugin.md` / `writing-an-enrichment-plugin.md` (identifier input shape, with Tracks 3–4). + +## 7. Out of scope + +- Product frontend work outside `apps/explorer`. +- `atom-rules-engine` identifier rules (external consumers — ticket for app teams). +- Performance/load validation of enrichment API quotas at seed scale (Track 5 rate configs cover the sprint). diff --git a/.planning/claude-migration/README.md b/.planning/claude-migration/README.md new file mode 100644 index 0000000..059ad82 --- /dev/null +++ b/.planning/claude-migration/README.md @@ -0,0 +1,25 @@ +# Intuition Identifier Migration — Divide & Conquer Plan + +**Goal:** switch the entire stack from "atom data = JSON-LD object on IPFS, parsed and classified from its contents" to "atom data = Intuition Identifier (`int:isrc:USSM10007459`), classification inferred from the scheme, enrichment queried from external APIs keyed by the identifier" — plus indexing the new contract-level `URIs` context field. + +**Format:** 1–2 day all-hands sprint, one engineer per track, big-bang switch instead of piecemeal. + +## Documents + +| Doc | What it covers | +|---|---| +| [00-overview.md](00-overview.md) | What changed (contracts + data model), how it impacts each layer, dependency graph, day-by-day sequencing, decisions to lock **before** the sprint | +| [01-track-ingestion.md](01-track-ingestion.md) | **Track 1 — Chain ingestion (Rust):** ABI bump, new `AtomContextRegistered` event, rindexer typings, event storage, Timescale migrations | +| [02-track-projections-db.md](02-track-projections-db.md) | **Track 2 — Projections & database:** term/kg.nodes materialization, `node_urls` from on-chain URIs, `raw_type` extension, Drizzle + Timescale schema | +| [03-track-parsing-classification.md](03-track-parsing-classification.md) | **Track 3 — Parser & classification:** `intuition_identifier` parse kind, `@0xintuition/iid` adoption, scheme→classification mapping, worker parse/classify stages | +| [04-track-enrichment.md](04-track-enrichment.md) | **Track 4 — Enrichment:** identifier-keyed provider lookups (ISRC→Spotify/MusicBrainz, ISBN, DOI, QID, CAIP…), worker enrichment stage, atom-services | +| [05-track-seed-pipeline.md](05-track-seed-pipeline.md) | **Track 5 — Seed data pipeline (intuition-v2):** ladder-driven IID derivation replacing JSON-LD assembly, identifier-projection convergence, write paths, URIs sourcing | +| [06-track-api-explorer-qa.md](06-track-api-explorer-qa.md) | **Track 6 — API, explorer, devnet & QA:** API serialization/OpenAPI, explorer rendering, devnet acceptance atoms, smoke tests, docs, end-to-end verification | + +## Research base + +Plan synthesized from: + +- `intuition-v2/.planning/intuition-id/` — the IID spec, scheme registry (25 schemes), classification ladders (37 types), P0/P1/P2 representation profiles, equivalence/dedupe design, decisions D1–D31 (+ proposed D32–D35) +- `intuition-contracts-v2` commit `a402fad` "Feat: Add URIs to Atom Create (#155)" — `createAtomsWithUris` + `AtomContextRegistered` event +- Full code walk of `intuition-core` (crates, packages, services, migrations) and the `intuition-v2` seed pipeline (`lab/data-seed`, `scripts/import-data`, seed control plane) diff --git a/.planning/codex-migration/00-executive-overview.md b/.planning/codex-migration/00-executive-overview.md new file mode 100644 index 0000000..91d8fa0 --- /dev/null +++ b/.planning/codex-migration/00-executive-overview.md @@ -0,0 +1,119 @@ +# Executive overview + +> Program note (2026-08-10): this document is the original vertical-slice baseline. The authoritative full-program sequencing, public package release lane, and ownership model are in [09-program-roadmap.md](./09-program-roadmap.md). The one-to-two-day scope below is an integration spike, not the production migration schedule. + +## The change in one sentence + +Atom data is moving from being the descriptive record to being a deterministic identity anchor; description and display data become a versioned, provenance-bearing projection produced by resolvers and claims around that anchor. + +The new flow is: + +```text +on-chain atom bytes + AtomContextRegistered URIs + | + v + profile-aware IID parser + | + +----------+----------+ + | | + deterministic type resolver plan + and identity facts (networked, retryable) + | | + +----------+----------+ + | + normalized artifacts + provenance + | + API/search/display projections and clusters +``` + +For an ISRC-backed recording, the canonical anchor is a valid value such as: + +```text +int:isrc:USRC17607839 +``` + +The example `int:src:132456798` should not enter code or fixtures: `src` is not a registered scheme and the sample value is not a canonical 12-character ISRC. + +## What changed at the protocol boundary + +The URI-enabled contract introduces: + +- `createAtomsWithUris(address creator, bytes[] atomDatas, uint256[] assets, bytes[][] uris)` +- `AtomContextRegistered(bytes32 indexed termId, address indexed registrant, bytes[] uris)` +- URI configuration exposed through `getAtomUriConfig()`; current defaults are five URIs per atom and 700 bytes per URI +- a fee-proxy path, `createAtomsWithUrisVia` + +URIs are opaque event bytes. They are not stored in contract state and are excluded from atom-ID calculation. The event is only emitted when a non-empty URI list is supplied. In a batch, all atom/deposit events are emitted before context events, so log adjacency is not a valid correlation strategy. + +## What changes in the data model + +The backend currently treats atom bytes as the descriptive source of truth: + +1. parse JSON/IPFS/URL data; +2. infer `@type` or classify a URL; +3. choose enrichment plugins from the type/URL; +4. store the parsed/enriched result on the node. + +That fails for a P0 IID. Today a bare `int:isrc:...` is a plain string, has no URL, cannot select the Spotify/music resolver path, and is normally skipped by enrichment. + +The new backend must represent four distinct concepts: + +| Concept | Meaning | Mutability | +| --- | --- | --- | +| Atom payload | Exact on-chain bytes and detected profile | Immutable | +| Identity | Canonical IID, scheme, class, and cluster membership | Derived deterministically; cluster edges are reversible | +| Context | URI event evidence supplied at creation | Immutable event, mutable processing status | +| Enrichment | Provider responses and normalized display/search data | Refreshable and versioned | + +`data_resolved` can remain as a compatibility projection, but it must stop pretending to be the atom's identity or the only source of provenance. + +## The work that must land + +1. **Freeze the shared interpretation contract.** Every consumer needs the same profile detection, IID validation, scheme typing, classification result, and resolver target structure. +2. **Sync the contract surface.** Update ABI/package artifacts, Rindexer event declarations, Rust event types/handlers/readers, and URI projections. +3. **Add IID parsing before generic strings.** Support P0, P1, P2, and legacy payloads, with a quarantine path for malformed or unknown IIDs. +4. **Split classification from resolution.** Classification must remain deterministic; network calls produce enrichment artifacts through scheme-specific resolvers. +5. **Add identity/context persistence.** Keep exact bytes, normalized identifiers, context-event provenance, resolver state, and identity-cluster membership separately. +6. **Move seed output to identity ladders and profiles.** Derive the highest usable canonical IID, choose P0/P1/P2, compute the atom ID from the final bytes, and submit aligned URI arrays. +7. **Update writes and reads together.** SDK/app creation, API ingestion, search, and explorer display all need dual-format support. +8. **Backfill without rewriting history.** Compute identifiers for legacy atoms off-chain, cluster related atoms, and only mint new anchors under an explicit policy. + +## Critical path + +The parallel work starts only after a 60–90 minute interface freeze covering: + +- spec/package version and registry hash; +- the shared `AtomInterpretation` and resolver contracts; +- P0/P1/P2 selection policy; +- URI allowlist, ordering, decoding, and retention policy; +- database table/column names and API compatibility shape; +- contract address, deployment block, ABI version, and supported write route. + +After that freeze, Tracks A–D can work in parallel against the same golden fixtures. Track E integrates in this order: + +```text +additive DB migration + -> ABI and context-event ingestion + -> IID parsing/classification/resolution + -> API/read compatibility + -> seed and mint writers + -> replay/backfill + -> IID-default flag +``` + +## What fits in one to two days + +A focused switch can deliver the minimum safe vertical slice: + +- valid P0/P1 detection and IID persistence; +- scheme-driven classification and an ISRC resolver path; +- URI event indexing and API exposure; +- profile-aware seed output; +- dual-read compatibility and end-to-end fixtures; +- feature-flagged rollout with additive schema and rollback. + +It should not pretend to finish the entire equivalence product. Candidate generation, attestation thresholds, union-find dispute behavior, broad resolver coverage for all 26 schemes, and opportunistic re-minting are follow-up phases. The minimum slice should create stable extension points for them rather than implementing partial heuristics in the hot path. + +## Success criteria + +The switch is successful when one golden P0 ISRC atom can be minted with context URIs, indexed after replay, classified as `MusicRecording` without reading JSON-LD, enriched through the selected resolver(s), searched and rendered through the API, and joined to any legacy atom carrying the same derived IID—with repeat processing producing no duplicate state. diff --git a/.planning/codex-migration/01-target-architecture.md b/.planning/codex-migration/01-target-architecture.md new file mode 100644 index 0000000..9160e0d --- /dev/null +++ b/.planning/codex-migration/01-target-architecture.md @@ -0,0 +1,211 @@ +# Target architecture and shared contracts + +## Architectural boundary + +Canonicalization and identity derivation are pure functions. Resolution is a networked, cached, retryable operation. Persistence retains both the immutable evidence and the mutable projection. + +```text + deterministic / offline networked / retryable + --------------------------------------------------+----------------------------------- + bytes -> profile -> IID validate -> scheme/type | resolver target -> provider APIs + | | | + v | v + identity row + cluster key | raw + normalized artifacts + | | + +----------v + display/search projection +``` + +No resolver may participate in IID canonicalization, atom-byte construction, or atom-ID calculation. + +## Representation profiles + +Consumers must accept all profiles plus legacy data. + +| Profile | Atom bytes | Required use | Identity behavior | +| --- | --- | --- | --- | +| P0 anchor | Bare IID string | Class A/B plus unambiguous scheme | Atom ID is a direct function of IID bytes | +| P1 identity context | Stable JSON with `@type`, `identifier`, recipe fields/evidence | Class C or polymorphic schemes | IID joins the identity cluster; payload supplies required type/evidence | +| P2 enriched | P1 plus descriptive fields | Explicit opt-in only | Same cluster as any profile with the same IID | +| Legacy | Existing JSON, IPFS, URL, or string forms | Read/backfill compatibility | May gain a computed IID without changing on-chain bytes | + +The parser must not infer P0 eligibility merely because the payload starts with `int:`. It must validate the registered scheme, canonical value, identity class, and typing rule. + +## Shared interpretation contract + +Freeze a package-level contract before engineers split up. Names are illustrative; the semantics are required. + +```ts +type AtomProfile = "p0" | "p1" | "p2" | "legacy"; + +type ParsedIdentifier = { + iid: string; + scheme: string; + value: string; + identityClass: "A" | "B" | "C"; + schemeTyping: "unambiguous" | "polymorphic"; + canonical: boolean; + valid: boolean; + registryVersion: string; +}; + +type AtomInterpretation = { + profile: AtomProfile; + identifier?: ParsedIdentifier; + classificationSlug?: string; + schemaType?: string; + classificationSource: + | "iid-scheme" + | "iid-value" + | "payload-type" + | "legacy-classifier" + | "unresolved"; + confidence: "exact" | "payload-asserted" | "provider-derived" | "unknown"; + contextUris: ContextUri[]; + resolutionTargets: ResolutionTarget[]; + errors: InterpretationError[]; +}; + +type ResolutionTarget = { + resolver: string; + key: string; + source: "iid" | "context-uri" | "payload" | "legacy"; + priority: number; +}; +``` + +Required behavior: + +- Generic IID parsing splits only on the first two colons; scheme code owns remaining value structure. +- Unknown schemes and non-canonical values are invalid for new writes, but historical bytes remain indexable and visible. +- P1/P2 `identifier` is validated by the same code as P0. +- The classifier may use deterministic scheme/value structure and a P1/P2 `@type`; it may not call a provider. +- Resolvers accept typed targets rather than scraping meaning back out of an arbitrary URL. +- Context URLs supplement resolution. They are not trusted as identity and must preserve their event provenance. +- The interpretation result is serializable and versioned so workers, seed jobs, and APIs can replay deterministically. + +## Resolver interface + +Move enrichment plugins from URL-first matching to target-first resolution while retaining URL adapters for legacy atoms. + +```ts +interface IdentifierResolver { + name: string; + version: string; + supports(target: ResolutionTarget, atom: AtomInterpretation): boolean; + resolve(target: ResolutionTarget, context: ResolveContext): Promise; +} + +type ResolverArtifact = { + provider: string; + sourceUri?: string; + fetchedAt: string; + resolverVersion: string; + rawPayloadRef?: string; + normalized: Record; + identifiers: string[]; + classification?: { slug: string; schemaType: string }; +}; +``` + +The resolution cache key should include canonical IID or target key, resolver name/version, and material request parameters. Provider failure changes resolution status, not identity status. Artifacts remain independently refreshable. + +For the initial ISRC path: + +```text +int:isrc:USRC17607839 + -> exact classification: MusicRecording + -> target: { resolver: "music-recording/isrc", key: "USRC17607839" } + -> MusicBrainz lookup and configured Spotify/Apple matching + -> normalized recording artifact + provider identifiers + provenance +``` + +Spotify is an enrichment source, not the identity namespace. A Spotify track URL in `AtomContextRegistered` can improve matching but does not replace the ISRC or alter the atom ID. + +## Persistence model + +Use additive structures. Exact names can be adjusted during the interface freeze. + +### Chain event storage + +Add a typed event record for `AtomContextRegistered` containing at least: + +- chain/network, contract address, block number/hash, transaction hash, log index; +- `term_id`, registrant; +- raw `bytes[]` values without lossy decoding; +- canonical/reorg status and ingestion timestamp. + +Normalize each URI into a KG projection such as `kg.node_context_uris` keyed by `(node_id, transaction_hash, log_index, ordinal)`. Store raw hex/bytes, a best-effort decoded string, URI kind, normalized URL when applicable, processing state, and validation error. Duplicate values may be semantically deduplicated for resolution while the original event entries remain intact. + +### Identity storage + +Prefer a normalized `kg.node_identifiers` table rather than overloading `kg.nodes`: + +- `node_id`, `iid`, `scheme`, `value`, identity class, profile; +- canonical/valid flags, registry and interpreter versions; +- source (`payload`, `backfill`, `artifact`, `claim`) and primary flag; +- derivation/evidence reference for Class C or backfilled identities; +- timestamps and processing status. + +Use a non-unique index on `iid`. Multiple atoms carrying the same IID are expected and must join a cluster; a database-wide unique `iid -> node` constraint would make P0/P1/P2 coexistence impossible. Enforce idempotency with a key scoped to node, IID, source, and role. + +Add identity cluster and member projections when Layer 2 ships. Cluster membership is reversible projection state, never a destructive node merge. Elect identity-canonical and display-canonical nodes separately. + +### Compatibility fields + +- Preserve `kg.nodes.data`, `data_hex`, and current raw types. +- Add `iid` to the raw-type discriminator or derive profile separately without rewriting old rows. +- Keep `data_resolved` as a materialized compatibility/display projection. +- Keep normalized enrichment in `kg.artifacts` with source URI, content hash, provider, and resolver version. +- Update `search_text` from selected normalized artifact fields while retaining the IID as a searchable token. + +## Contract event semantics + +The indexer must follow these rules: + +1. Sync the ABI from the actual deployed contract artifact. +2. Subscribe to `AtomContextRegistered` from the correct upgrade/deployment block. +3. Join the event to an atom by `termId`; never rely on `AtomCreated` being the previous log. +4. Treat no event as an empty context list, not as ingestion failure. +5. Decode URI bytes only after preserving the raw value. +6. Project idempotently and honor canonical/reorg state during replay. +7. Read `getAtomUriConfig()` in write clients or configuration sync so client limits do not drift from the contract. + +## Write path + +The canonical mint pipeline is: + +1. Enrich source records only enough to find the strongest usable identifier and required P1 evidence. +2. Evaluate the classification identity ladder. +3. Canonicalize and validate with the pinned `@0xintuition/iid` implementation. +4. Select P0/P1/P2 from identity class and scheme typing. +5. Serialize atom bytes with an explicitly versioned deterministic serializer. +6. Calculate the atom ID from those final bytes. +7. Build a bounded, ordered list of context URI bytes. +8. Submit `atomDatas[i]`, `assets[i]`, and `uris[i]` with exact index alignment through `createAtomsWithUris` or its approved proxy route. +9. Record the intended bytes, expected term IDs, URIs, package versions, and transaction result in the seed/mint ledger. + +## Read path + +API clients should receive a backward-compatible atom plus an optional identity envelope: + +```json +{ + "id": "0x...", + "data": "int:isrc:USRC17607839", + "dataResolved": { "name": "...", "@type": "MusicRecording" }, + "identity": { + "iid": "int:isrc:USRC17607839", + "scheme": "isrc", + "class": "A", + "profile": "p0", + "valid": true, + "clusterId": "..." + }, + "contextUris": [], + "resolution": { "status": "complete", "artifacts": [] } +} +``` + +Old clients continue to use `dataResolved`; new clients can distinguish identity, context, and refreshable enrichment explicitly. + diff --git a/.planning/codex-migration/02-impact-inventory.md b/.planning/codex-migration/02-impact-inventory.md new file mode 100644 index 0000000..7500d9d --- /dev/null +++ b/.planning/codex-migration/02-impact-inventory.md @@ -0,0 +1,212 @@ +# Impact inventory + +This inventory records the current behavior, why it fails for IID-first atoms, and the required change. Paths are relative to `intuition-core` unless explicitly identified as another repository. + +## Contract packages and deployment artifacts + +### Current state + +- The current Rindexer ABI at `crates/rindexer-ingestion/abi/MultiVault.json` has no `AtomContextRegistered`, `createAtomsWithUris`, or `getAtomUriConfig`. +- `crates/rindexer-ingestion/rindexer.yaml` subscribes to the existing atom, triple, vault, and fee events only. +- Core's generated Rust bindings therefore cannot represent the new event. +- The URI-enabled source exists in `intuition-contracts-v2`, while checked-in copies and older documentation elsewhere may still describe the pre-URI interface. + +### Required change + +- Publish/sync one authoritative contract artifact and version across Core, SDK, app, and seed tooling. +- Add the new call routes and event to generated clients and bindings. +- Pin the deployed address and start block per network. +- Verify approval semantics when `creator != msg.sender` and decide whether clients use `createAtomsWithUris` directly or `createAtomsWithUrisVia`. +- Add ABI drift checks to CI so a future contract change cannot silently bypass ingestion. + +## Rust/Rindexer ingestion + +### Current state + +- `crates/rindexer-ingestion/src/handlers.rs`, generated event handlers under `src/rindexer_lib`, `crates/shared/src/parsed_event.rs`, storage, and typed readers only cover existing event variants. +- `migrations/timescale/002_create_typed_event_tables.sql` and related Timescale schema code have no atom-context event table. +- Atom creation stores atom data bytes, but there is no durable path for creation-time context. + +### Required change + +- Regenerate Rindexer output after ABI/config changes; do not hand-maintain generated code. +- Add `AtomContextRegistered` to shared event enums, parsed records, storage, typed readers, and metrics. +- Preserve `bytes[]` losslessly and store transaction/log provenance. +- Join to terms by `term_id`, tolerate context arriving after node projection, and make the projection idempotent. +- Exercise replay, canonical/reorg behavior, empty URI lists, binary/non-UTF-8 bytes, duplicate URIs, and multi-atom batches. + +## Atom parser + +### Current state + +- `packages/atom-parser/src/types.ts` recognizes IPFS, Ethereum address, ENS, JSON, URL, ISBN, and plain strings. +- `packages/atom-parser/src/parse.ts`, `detect.ts`, and `structured.ts` favor JSON-LD and URL discovery. +- A bare IID currently falls through to `plain_string`. +- Structured payload handling knows `@context`, `@type`, and `sameAs`, but not representation profiles. + +### Required change + +- Add IID detection before generic string detection. +- Parse P0 bare IIDs and P1/P2 JSON `identifier` values with the same validator. +- Return scheme, value, class, typing, profile, canonicality, registry version, and validation errors. +- Keep the exact input bytes/string; do not silently rewrite an invalid on-chain IID. +- Reject unknown/non-canonical IIDs on new API writes while quarantining rather than dropping historical data. +- Add shared golden fixtures for values containing additional colons, maximum-length values, unknown schemes, non-ASCII, and profile eligibility. + +## Classification packages + +### Current state + +- `packages/atom-classification` classifies URL/raw input and exposes URL-oriented plugins. +- Core consumes `@0xintuition/classifications@0.1.0-alpha.0` in `packages/atom-enrichment`, while the IID-enabled implementation is a workspace package in `intuition-v2`. +- Worker classification in `services/workers/src/core/classification.ts` derives directly from structured `@type` or calls the current classification runtime. +- Music routing is based on classification category/schema type and recognized provider URLs. + +### Required change + +- Publish/pin compatible `@0xintuition/iid` and `@0xintuition/classifications` versions or vendor them through one controlled workspace boundary. +- Make identity ladders and scheme typing the only write-side identifier selection policy. +- Add deterministic `scheme/value -> classification` routing where the mapping is exact. +- Require P1/P2 type evidence for polymorphic schemes; provider-derived typing may enrich or flag a conflict but must not retroactively make an invalid P0 valid. +- Extend classification results with IID source, confidence, and resolution targets. +- Preserve legacy URL/raw classification as a compatibility adapter. +- Add missing taxonomy decisions for schemes such as ISWC and polymorphic provider entities before claiming full registry coverage. + +## Enrichment packages and atom services + +### Current state + +- `packages/atom-enrichment/src/engine.ts`, `plugins.ts`, extraction helpers, and classification registry select work from classified input and URL candidates. +- `services/workers/src/core/enrichment.ts` and `structured-targets.ts` expect a target URL or structured object; a P0 IID provides neither and is skipped. +- `services/atom-services/src/service/processing.ts` follows classify-then-enrich behavior designed for current raw inputs. +- `ClassifiedAtomInput` has limited atom-type hints and does not carry the full IID interpretation/resolver plan. + +### Required change + +- Introduce identifier resolver targets as first-class inputs. +- Add scheme resolver adapters with explicit provider/version/cache/provenance behavior. +- Implement ISRC as the first required vertical slice: exact `MusicRecording` classification, registry lookup, provider matching, and normalized artifacts. +- Use context URIs as ranked hints. Validate/allowlist them and retain their origin; never treat presence of a Spotify URL as stronger identity than the IID. +- Store resolver outcomes independently so retry, refresh, and provider outages do not re-run or mutate identity. +- Update atom-services `/process` and batch/cache keys for IID input. +- Preserve existing URL enrichment as a legacy resolver target. + +## Workers and orchestration + +### Current state + +- `services/workers/src/kg/atom-parsing`, `atom-classification`, and `atom-enrichment` implement leased, retryable stages. +- `services/workers/src/kg/processing.ts` and reconciliation code carry current stage state. +- The parser promotes structured data into `data_resolved` and search text; classification/enrichment assume payload-derived targets. + +### Required change + +- Keep the existing lease/retry/reconciliation mechanics. +- Insert profile/IID interpretation into parsing and persist it before classification. +- Make classification deterministic and independently complete even when resolution is pending or failed. +- Queue resolution by stable target key, deduplicate across atoms with the same IID where safe, and fan results back to cluster members. +- Introduce explicit statuses for invalid identity, no resolver, retryable provider failure, permanent provider miss, and completed resolution. +- Ensure search/display projection updates are atomic with artifact selection and are safe to replay. + +## Knowledge-graph database + +### Current state + +- `packages/database-kg/src/schemas/kg/nodes.ts` and initial migration constrain `raw_type` to `string`, `json`, `http_uri`, or `ipfs_uri`. +- Nodes carry `data`, `data_hex`, `data_resolved`, classification/enrichment state, and `search_text`. +- `kg.artifacts` stores enrichment payloads and `kg.node_urls` stores source/provenance/artifact/primary URLs. +- There are no normalized IID, context-event, or identity-cluster structures. + +### Required change + +- Add normalized node identifiers and context URI projections. +- Either add `iid` to `raw_type` or add a separate profile field; keep old values valid. +- Add indexes for IID lookup, scheme/status queues, context `term_id`, and cluster membership without making IID globally unique. +- Add reversible identity clusters with separate identity-canonical and display-canonical members when Layer 2 is enabled. +- Version the interpretation and display projections. +- Update schema exports, actions, generated Drizzle snapshots, migrations, and migration tests. +- Keep artifact payload/provenance separate from `data_resolved`; use the latter as a compatibility materialization. + +## Timescale/event database + +### Current state + +- The raw event store and typed tables retain chain history and projection checkpoints. +- `term` stores atom data/data hex, but no URI context. + +### Required change + +- Add a typed context-event table and schema bindings. +- Preserve raw URI bytes or hex plus array ordering. +- Index `term_id`, block/transaction/log identity, and canonical status. +- Add projection checkpoint/dead-letter observability for the new event. +- Backfill/replay only from the contract upgrade/deployment boundary after validating network configuration. + +## API and explorer + +### Current state + +- `services/api/src/app.ts` and schema accept raw atom data and detect existing raw types. +- Atom responses expose current node/classification/enrichment fields. +- `apps/explorer` presents raw data and classification but has no identity/profile/context model. + +### Required change + +- Validate canonical IIDs on creation endpoints and return actionable errors. +- Add backward-compatible identity, context URI, resolver status, and cluster/canonical fields to reads. +- Define whether APIs return raw event bytes, sanitized decoded URIs, or both; default public output should be safe and size-bounded. +- Update search to index both IID and selected hydrated fields. +- Update explorer cards/details to show the IID/profile and distinguish context sources from enrichment sources. +- Continue rendering legacy atoms with no IID. + +## Seed data control plane + +Location: `/Users/metasudo/workspace/intution/workspace/intuition-v2/packages/database-seed-control-plane` and related import scripts. + +### Current state + +- Candidate, processing, enrichment, and write jobs are already separated and resumable. +- `deriveAtomData` builds enriched JSON-LD, calculates the node ID from that description, and writes it as JSON with completed processing stages. +- `WRITE_PROJECTION_VERSION` is currently `atom-v4`. +- Existing identifier projection includes legacy strings such as `isrc:...`, `spotify:track:...`, and other provider IDs that are not necessarily registered IIDs. + +### Required change + +- Replace ad hoc identifier strings with the registered IID library and classification identity ladders. +- Choose the highest usable rung and record why lower rungs were skipped. +- Emit P0 bare bytes only when anchor-eligible; emit deterministic P1 for Class C/polymorphic cases; keep P2 opt-in. +- Move source/provider URLs to the URI array and/or artifact provenance rather than identity bytes. +- Recompute expected atom/node IDs from final serialized bytes and bump the write projection version. +- Add ledger fields for IID, scheme, identity class, profile, resolver plan/status, context URIs, serializer/library versions, old/new IDs, and dedupe decision. +- Reproject only safe, unwritten/held jobs automatically. Treat already-on-chain atoms as immutable and backfill/cluster them instead. +- Produce a mint-ready manifest even if the current control plane writes only off-chain KG rows; the on-chain submitter must preserve `atomDatas/assets/uris` alignment. + +## SDK and application creation + +Location: primarily `intuition-v2` application/contracts helpers. + +### Current state + +- Current create flows still construct descriptive JSON-LD and use the older atom creation helper. +- `buildAtomDataObject` can inject identifiers only as an opt-in and does not by itself implement profile selection. + +### Required change + +- Introduce a profile builder (`buildAtomAnchor`/equivalent) backed by the pinned IID/classification packages. +- Update simulation, fee estimation, approval, submission, receipt parsing, and batch behavior for the URI-enabled method. +- Validate contract URI limits at runtime and surface per-item errors before wallet confirmation. +- Record exact bytes and expected atom IDs in UI/SDK results. +- Gate IID-default minting independently from dual-read support. + +## Observability and operations + +Add metrics and dashboards for: + +- payload counts by P0/P1/P2/legacy/invalid; +- valid/invalid/canonical IID counts by scheme; +- context events and URIs per atom, decoding failures, and orphan joins; +- resolver queue depth, hit/miss/retry/permanent-failure rate, latency, and provider throttling; +- enrichment freshness and selected display artifact; +- same-IID cluster size and classification conflicts; +- seed profile distribution, deterministic rerun mismatches, and submitted/expected ID mismatches. + diff --git a/.planning/codex-migration/03-engineering-tracks.md b/.planning/codex-migration/03-engineering-tracks.md new file mode 100644 index 0000000..b0f4e79 --- /dev/null +++ b/.planning/codex-migration/03-engineering-tracks.md @@ -0,0 +1,223 @@ +# Engineering tracks + +## Operating model + +Name one integration lead before work starts. During the opening interface freeze, the team commits the shared fixture file and the versioned interpretation contract. Each owner then works in a separate branch/worktree and avoids editing another track's generated files unless coordinated. + +All tracks use the same first fixture: + +```text +payload: int:isrc:USRC17607839 +profile: P0 +classification MusicRecording (exact, from scheme) +context: one Spotify track URL, one MusicBrainz recording URL +expected: stable atom ID, indexed context, normalized artifacts, searchable display +``` + +Add P1, invalid, and legacy fixtures before implementation starts; the full matrix is in [05-decisions-risks-and-tests.md](./05-decisions-risks-and-tests.md). + +## Track A — contract artifacts and chain ingestion + +**Suggested owner:** protocol/indexer engineer +**Can start after:** deployed contract version/address/start block confirmed +**Primary repositories:** `intuition-contracts-v2`, `intuition-core`, shared contract package + +### Deliverables + +- Publish/sync the URI-enabled ABI into all contract consumers. +- Add `AtomContextRegistered` to Rindexer configuration and regenerate bindings. +- Extend `ParsedEvent`, storage records, Timescale typed tables/readers, and metrics. +- Add the KG projection handoff keyed by `term_id` plus tx/log/ordinal. +- Update SDK write bindings for direct and proxy routes, including config reads. +- Supply event fixtures/receipts for single, empty-context, and batch cases. + +### Definition of done + +- A local/devnet batch emits and indexes context for the correct atoms even though context logs are not adjacent to their `AtomCreated` logs. +- Replaying the range produces byte-for-byte equivalent rows and no duplicates. +- Empty URI lists create no false failure. +- Raw non-UTF-8 context survives ingestion without loss. +- ABI drift CI fails when the generated artifact differs from the contract source. + +### Handoffs + +- To Track C: event row/projection schema and sample records. +- To Track D: final function signature, proxy choice, limits, and receipt shape. +- To Track E: deployment boundary and replay command. + +## Track B — IID interpretation, classification, and enrichment + +**Suggested owner:** data-semantics/enrichment engineer +**Can start after:** registry/spec version and shared interpretation contract frozen +**Primary repositories:** `intuition-v2/intuition/iid`, `intuition-v2/intuition/classifications`, `intuition-core/packages`, atom services/workers + +### Deliverables + +- Pin/publish compatible IID and classification package versions. +- Add P0/P1/P2 parsing and validation to `atom-parser`. +- Add scheme/value classification rules and polymorphic-profile validation. +- Define resolver targets and update enrichment engine/plugin interfaces. +- Build the ISRC-to-music vertical slice, including context URL hints and artifact provenance. +- Update worker and atom-services processing contracts/statuses. +- Keep legacy URL/JSON/IPFS behavior as adapters. + +### Definition of done + +- The golden P0 ISRC fixture reaches `MusicRecording` classification with no provider call. +- Provider failure leaves identity/classification complete and resolution retryable. +- A polymorphic IID in bare P0 is flagged as profile-invalid; the same IID in valid P1 is accepted. +- Unknown schemes and non-canonical values are rejected on writes and quarantined on reads. +- Parser, worker, and atom-services parity tests share the same fixtures. +- Resolver output records provider, source, fetch time, version, raw reference/hash, and normalized payload. + +### Handoffs + +- To Track C: interpretation schema, statuses, and artifact fields. +- To Track D: profile-selection API, identity ladder API, and deterministic serializer requirements. +- To Track E: package versions/registry hash and golden expected outputs. + +## Track C — persistence, projections, API, and explorer + +**Suggested owner:** backend/platform engineer +**Can start after:** table/API field names and interpretation contract frozen +**Primary repository:** `intuition-core` + +### Deliverables + +- Add KG identity and context projection migrations/actions/indexes. +- Add the Timescale typed event schema in coordination with Track A. +- Update parsing/classification/enrichment persistence and reconciliation statuses. +- Add compatible API input validation and identity/context/resolution response fields. +- Update search/display materialization and explorer presentation. +- Implement same-IID cluster keying or, for the minimum slice, a deterministic query/view that proves expected cluster membership without destructive merges. +- Add a legacy IID backfill job with dry-run/report mode. + +### Definition of done + +- Migrations are additive, reversible at the application level, and pass schema tests. +- Multiple nodes can share an IID and query as one identity cluster. +- Exact atom bytes, context event bytes, and enrichment artifacts remain independently inspectable. +- Legacy API consumers continue to receive usable `dataResolved` and classification fields. +- Search finds a node by IID immediately and by hydrated name after resolution. +- Reprocessing and context replay are idempotent. + +### Handoffs + +- To Track A: storage actions for event projection. +- To Track B: persistence actions and status enums. +- To Track D: seed write schema and conflict/upsert behavior. +- To Track E: migration order, health queries, and rollback-compatible flags. + +## Track D — seed control plane, SDK, and mint clients + +**Suggested owner:** data pipeline/application engineer +**Can start after:** profile-selection API, serializer version, and contract route frozen +**Primary repositories:** `intuition-v2`, seed/import tooling, shared SDK/contract package + +### Deliverables + +- Replace ad hoc/provider identifiers with registered canonical IIDs. +- Evaluate classification identity ladders and choose the highest usable rung. +- Produce P0/P1/P2 atom bytes and bounded context URI arrays. +- Bump the seed write projection version and extend the job ledger/audit report. +- Recompute expected IDs from final bytes; make repeated preparation deterministic. +- Update app/SDK create flows to URI-enabled calls and receipt handling. +- Create a dry-run diff showing legacy atom bytes/IDs versus proposed profile/bytes/IDs before any submission. + +### Definition of done + +- Two clean runs over the same input yield identical IID, profile, atom bytes, URI order, and expected atom ID. +- P0 is never emitted for Class C or polymorphic schemes. +- Registered IIDs use `int::`; provider-local keys do not masquerade as schemes. +- Batch arrays stay aligned and preflight against live URI limits. +- Already-written/on-chain jobs are not silently reprojected or overwritten. +- The golden seed item can be submitted and matches Track A's indexed result and Track C's API response. + +### Handoffs + +- To Track A: submission transaction and expected event mapping. +- To Track C: write-ready row/manifest shape. +- To Track E: dry-run distribution, rejects, and ID-diff report. + +## Track E — integration, QA, and release + +**Suggested owner:** tech lead/release engineer +**Starts immediately and remains unshared:** merge order, flags, go/no-go, rollback + +### Deliverables + +- Run and record the interface freeze. +- Own the golden fixture corpus and end-to-end harness. +- Track dependency/API/package versions across repositories. +- Prepare deployment order, write freeze, replay/backfill, feature flags, and rollback. +- Verify metrics and query-based acceptance gates. +- Prevent scope creep into full candidate generation/equivalence during the cutover. + +### Definition of done + +- All cross-stack fixtures pass from transaction receipt to public API. +- No unresolved contract/spec/package drift remains. +- The team has an explicit go/no-go checklist and one person with rollback authority. +- Dual reads run before IID-default writes. +- A rollback disables new writers/resolvers without dropping additive data or losing already-minted atoms. + +## Two-day schedule + +### Preparation before the focused window + +- Contract deployment/artifact is available. +- Spec and package versions are selected. +- Provider credentials/dev mocks work. +- Branches/worktrees, test databases, and devnet are ready. +- The integration lead has a list of actual owners and communication channel. + +### Day 1 morning + +- 60–90 minute interface freeze. +- Track C lands additive schema skeleton. +- Tracks A and B commit shared event/interpretation fixtures. +- Track D produces dry-run output format and begins profile conversion. + +### Day 1 afternoon + +- Track A indexes URI events on devnet. +- Track B parses/classifies P0/P1 and resolves ISRC through mocks. +- Track C exposes identity/context through storage and API. +- Track D creates deterministic write manifests and updated SDK simulation. +- Track E runs the first stitched fixture and records gaps. + +### Day 2 morning + +- Fix integration mismatches. +- Exercise real provider sandbox/API where permitted. +- Run legacy regression, replay/idempotency, batch, and invalid-input tests. +- Produce seed and legacy backfill dry-run reports. + +### Day 2 afternoon + +- Deploy dual-read/indexing changes with writers still on legacy behavior. +- Replay context events from the confirmed block and validate metrics. +- Perform a short write freeze, deploy IID writers, run canary mints, then enable IID default gradually. +- Keep candidate generation/full equivalence and broad registry resolver coverage in the follow-up backlog. + +## Three-engineer fallback + +If only three engineers are available: + +| Owner | Combined scope | Guardrail | +| --- | --- | --- | +| 1 | Tracks A + event portions of C | Land migrations/schema contract first; regenerate rather than hand-edit bindings | +| 2 | Track B + interpretation portions of C | Freeze shared types before touching resolver plugins | +| 3 | Track D + API/UI portions of C | Use fixtures/mocks until Tracks A/B land; do not invent a second IID serializer | + +The tech lead role still must be explicit even if one of the three engineers performs it. + +## Follow-up backlog, not cutover scope + +- Resolver coverage for every registered scheme. +- Candidate generation for Class C and cross-scheme entities. +- Accepted `sameAs`/`differentFrom` edge thresholds and union-find recomputation. +- Identity-canonical versus display-canonical election at production scale. +- Automated opportunistic anchor minting for legacy data. +- P2 authoring policy and long-term context URI governance. + diff --git a/.planning/codex-migration/04-cutover-runbook.md b/.planning/codex-migration/04-cutover-runbook.md new file mode 100644 index 0000000..c1c25e0 --- /dev/null +++ b/.planning/codex-migration/04-cutover-runbook.md @@ -0,0 +1,146 @@ +# Cutover runbook + +## Rollout principle + +Ship readers before writers, make schema changes additive, and gate every behavior independently. New IID atoms remain valid raw atom data even if resolution is disabled; this is the basis of the rollback plan. + +Recommended flags: + +| Flag | Effect | +| --- | --- | +| `ATOM_CONTEXT_INGEST_ENABLED` | Project `AtomContextRegistered` into KG context rows | +| `IID_INTERPRETATION_ENABLED` | Detect/validate P0/P1/P2 and persist identity | +| `IID_RESOLUTION_ENABLED` | Queue identifier resolver work | +| `IID_READ_API_ENABLED` | Expose identity/context fields to selected clients | +| `IID_SEED_WRITE_ENABLED` | Make seed manifests IID/profile-first | +| `IID_MINT_DEFAULT_ENABLED` | Make user/app atom creation IID-first | +| `IID_CLUSTERING_ENABLED` | Join equal IIDs in the identity projection | + +Do not use one master flag. Parser, resolver, reads, and writes need independent rollback. + +## Phase 0 — go/no-go inputs + +- [ ] URI-enabled contract artifact, address, chain, and activation block are recorded. +- [ ] Direct versus fee-proxy write route is selected and exercised. +- [ ] Contract URI config is read successfully on the target network. +- [ ] IID spec/package/classification versions and registry hash are pinned. +- [ ] P0/P1/P2 policy is approved; unsupported scheme policy is explicit. +- [ ] Golden fixtures and expected atom IDs are committed. +- [ ] Provider credentials, rate limits, and mocks are ready. +- [ ] Database backup/recovery posture and migration owner are confirmed. +- [ ] Feature flags default off and can be changed without a redeploy. +- [ ] One integration lead holds go/no-go and rollback authority. + +No production switch should start with any of these unresolved. + +## Phase 1 — land contracts and additive storage + +1. Publish/sync the ABI and client package. +2. Apply additive Timescale and KG migrations. +3. Deploy ingestion capable of reading the new event, initially with KG context projection disabled if necessary. +4. Deploy dual-format parser/persistence/API code with new response fields disabled or additive. +5. Validate health queries, migration versions, and old-atom regressions. + +Acceptance gates: + +- Existing event indexing is unchanged. +- Existing atom APIs pass contract tests. +- No rows are rewritten merely by deploying the new schema. +- A synthetic context event can be decoded without projecting it. + +## Phase 2 — enable dual reads and context ingestion + +1. Enable context projection on a canary worker. +2. Replay from the exact contract activation block. +3. Verify event counts against chain receipts and check orphan joins. +4. Enable IID interpretation for a sample of new and historical nodes. +5. Expose identity/context fields to internal clients. +6. Enable resolver processing with mocks or a strict canary/rate limit. + +Health queries should cover: + +- chain context events versus typed rows versus KG URI rows; +- distinct `(tx_hash, log_index)` and `(node_id, tx_hash, log_index, ordinal)` counts; +- contexts with no matching node after the expected projection delay; +- invalid IID/profile counts and examples; +- resolver retries, throttling, permanent misses, and artifact writes; +- legacy stage success/error rates compared with the pre-deploy baseline. + +Stop if context counts diverge, raw bytes cannot round-trip, or legacy processing degrades materially. + +## Phase 3 — dry-run seed and legacy backfill + +### Seed dry run + +Run the entire target dataset without writes and produce: + +- counts by classification, chosen scheme, class, and profile; +- canonicalization rejects and missing identity ladder inputs; +- unsupported P0 scheme/type routes; +- old atom bytes/ID versus new bytes/ID; +- URI count/length rejects and URI source distribution; +- duplicate IIDs and proposed cluster sizes; +- resolver coverage and estimated provider volume; +- deterministic repeat-run diff, which must be empty. + +Any provider-local identifier that appears as an unregistered `int:` is a hard failure. + +### Legacy backfill dry run + +For each existing atom: + +1. Read immutable original bytes and existing artifacts. +2. Derive the strongest available IID using the pinned ladder/version. +3. Persist nothing in dry-run mode; report IID, evidence source, confidence, and proposed cluster. +4. Flag classification conflicts and ambiguous ontological levels. +5. Do not claim that a computed IID changes the historical atom ID. + +Backfill writes, when approved, add `source=backfill` identifier rows and cluster projection state. They never rewrite atom data. Minting a new P0 anchor is a separate, auditable action. + +## Phase 4 — enable IID writes + +1. Start a short write freeze for seed jobs and app atom-create routes. +2. Drain or hold in-flight write jobs at a known projection version. +3. Deploy/enable the IID seed writer and URI-enabled SDK/app route. +4. Submit the golden canary atom and one P1 canary. +5. Verify expected atom IDs, receipts, event rows, KG identity/context, enrichment artifacts, API output, search, and explorer rendering. +6. Resume writes for an allowlisted classification/scheme—ISRC music recordings first. +7. Expand only after canary metrics remain healthy. + +Avoid turning every registered scheme on at once. Launch readiness is a pair of capabilities: valid profile selection and an approved classification/resolver policy for that scheme. + +## Phase 5 — replay, backfill, and expand + +- Replay missed context events from the activation block. +- Run legacy identifier backfill in bounded, checkpointed batches. +- Enable exact-IID cluster projection after dry-run comparison. +- Add schemes in cohorts from [06-scheme-resolution-matrix.md](./06-scheme-resolution-matrix.md). +- Refresh search/display materializations from selected artifacts. +- Keep provider concurrency and quotas isolated by resolver. + +## Rollback + +Rollback is behavioral, not destructive. + +1. Disable `IID_MINT_DEFAULT_ENABLED` and `IID_SEED_WRITE_ENABLED`. +2. Return apps/seeders to the previously supported create route. +3. Disable failing resolvers independently; keep parsing and event ingestion running when healthy. +4. Disable new API fields for incompatible clients if necessary. +5. Leave additive tables and already-ingested events intact. +6. Requeue failed projections after the corrected code deploys. + +Do not drop migrations, delete context rows, rewrite atom bytes, or attempt to remove already-minted IID atoms. A P0 IID is still valid immutable atom data while downstream hydration is unavailable. + +## Post-cutover checks + +- [ ] New writes use only canonical registered IIDs. +- [ ] P0 profile counts match the approved scheme allowlist. +- [ ] Expected and indexed atom IDs match for all canaries. +- [ ] Context event counts reconcile with receipts. +- [ ] Resolver errors do not change identity/classification status. +- [ ] Search finds canaries by IID and hydrated name. +- [ ] Legacy atom processing and reads meet the previous baseline. +- [ ] Same-IID nodes appear in the same reversible cluster projection. +- [ ] Seed ledger records package/serializer/contract versions and exact submitted bytes. +- [ ] A repeat replay/backfill creates no duplicate rows or artifacts. + diff --git a/.planning/codex-migration/05-decisions-risks-and-tests.md b/.planning/codex-migration/05-decisions-risks-and-tests.md new file mode 100644 index 0000000..e82f834 --- /dev/null +++ b/.planning/codex-migration/05-decisions-risks-and-tests.md @@ -0,0 +1,143 @@ +# Decisions, risks, and acceptance tests + +## Decisions to lock in the opening session + +| ID | Decision | Recommended default | Why it blocks parallel work | +| --- | --- | --- | --- | +| D1 | IID specification/package version | Pin an exact version and registry hash; treat canonicalization as frozen | Every writer and reader must produce the same bytes | +| D2 | Formal spec status | Ratify the current grammar/profile rules or explicitly name exceptions before default writes | Planning notes and formal package status are not fully aligned | +| D3 | P0 allowlist | Start with implemented, unambiguous routes such as ISRC; expand per scheme | “Unambiguous” in the IID package does not guarantee Core taxonomy/resolver readiness | +| D4 | Shared interpretation schema | Use one versioned contract across parser, worker, services, API, and seed | Prevents each track inventing incompatible IID state | +| D5 | Serializer | Pin canonical UTF-8/JSON serialization and version it in the seed ledger | Atom IDs are byte-sensitive | +| D6 | URI policy | Preserve raw bytes; define allowlisted schemes, ordering, normalization, privacy, and public API shape | Contract URIs are opaque and can contain unsafe or non-text data | +| D7 | Contract route | Choose direct `createAtomsWithUris` or approved fee-proxy path per client | Changes approvals, simulation, fees, and receipt parsing | +| D8 | Context limits | Read contract config and use lower/equal client limits | Defaults are configurable, not constants to duplicate forever | +| D9 | Legacy handling | Dual-read indefinitely; additive computed identifiers; no automatic remint | Existing on-chain bytes cannot be migrated | +| D10 | Identity indexing | Non-unique IID membership plus a cluster key | P0/P1/P2 atoms with one IID are expected to coexist | +| D11 | Canonical roles | Separate identity-canonical anchor from display-canonical enriched member | Stable claim targets and rich rendering have different goals | +| D12 | Provider policy | Registry-first resolver order, cache TTLs, credentials, quotas, and provenance requirements | Network behavior must not leak into deterministic identity logic | +| D13 | Seed write boundary | Define whether the control plane writes KG only or submits on-chain; require a mint manifest either way | Avoids a gap between prepared bytes and actual contract arrays | +| D14 | Equivalence scope | Exact same-IID clustering only in the cutover; candidate/attested cross-scheme merges later | Keeps a two-day switch achievable and reversible | + +## Major risks and mitigations + +### Contract and ABI drift + +**Risk:** Core, the SDK, and the app compile against different MultiVault interfaces. The current Core ABI is pre-URI, and older transaction guidance is also stale. + +**Mitigation:** publish one artifact, generate all bindings from it, record bytecode/deployment metadata, and enforce ABI drift in CI. Use receipt fixtures from the deployed version. + +### Wrong event correlation + +**Risk:** a consumer associates context with the previous `AtomCreated` log. Batch emission order makes that incorrect. + +**Mitigation:** join exclusively on indexed `termId`, make late projection safe, and include interleaved batch fixtures. + +### Invalid examples becoming production data + +**Risk:** `int:src:132456798` is copied into tests or seed output. + +**Mitigation:** only construct IIDs through `@0xintuition/iid`, keep negative tests for unknown `src`, and use canonical ISRC test vectors. + +### Treating API data as identity + +**Risk:** a provider redirect, outage, mutable name, or local ID changes atom bytes or the selected IID. + +**Mitigation:** hard separation between pure canonicalization/ladder evaluation and resolver execution. Provider output is a versioned artifact/equivalence candidate only. + +### P0 overreach + +**Risk:** a scheme is technically marked unambiguous but Core lacks a precise taxonomy mapping, or a polymorphic IID is minted bare. + +**Mitigation:** use a product P0 allowlist narrower than `isAnchorEligible`, require P1 for polymorphic schemes, and make taxonomy gaps explicit in the scheme matrix. + +### Descriptive regressions + +**Risk:** moving description off-chain yields blank cards/search until enrichment completes. + +**Mitigation:** render the IID and resolver state immediately, hydrate asynchronously, reuse shared artifacts for equal IIDs, retain cached results, and define provider-miss fallbacks. P1 carries minimum display/evidence where P0 is not valid. + +### URI abuse and privacy + +**Risk:** opaque event bytes contain secrets, tracking URLs, malicious schemes, oversized render content, or non-UTF-8 data. + +**Mitigation:** preserve raw data privately, decode safely, allowlist network resolvers, strip tracking only in a derived normalized field, cap public output, and never automatically fetch arbitrary schemes/private network targets. + +### Accidental uniqueness constraint + +**Risk:** a unique database index on `iid` rejects valid P1/P2 or legacy members that should join an identity cluster. + +**Mitigation:** index IID for lookup, make membership idempotent within a node/source role, and represent clusters separately. + +### Seed re-projection of immutable writes + +**Risk:** bumping the seed projection silently changes IDs for jobs already written or minted. + +**Mitigation:** distinguish candidate/prepared/submitted/on-chain states, reproject only safe states, produce an old/new diff, and backfill/cluster immutable results. + +### Resolver cost and rate limiting + +**Risk:** replay/backfill sends one provider request per node and exhausts quotas. + +**Mitigation:** cache by stable resolver target, coalesce equal-IID work, rate-limit per provider, batch where supported, and stage rollout by scheme. + +### Ontological-level merges + +**Risk:** ISBN editions are automatically treated as the same node as a book work, or a recording is merged with its composition. + +**Mitigation:** preserve each classification's `identifies` level, treat cross-level mappings as claims/candidates, and never merge merely because labels/providers are similar. + +## Cross-stack acceptance matrix + +| Fixture | Expected interpretation | Expected downstream behavior | +| --- | --- | --- | +| P0 canonical ISRC | valid A/unambiguous, `MusicRecording` | resolve by ISRC, accept context hints, searchable hydrated card | +| `int:src:132456798` | unknown scheme, invalid | new write rejected; historical atom preserved/quarantined | +| ISRC with separators/lowercase inside an already-minted IID | non-canonical IID | new write rejected; historical bytes not silently rewritten | +| P0 `int:wd:Q42` | syntactically valid IID but profile-invalid because polymorphic | classification unresolved/invalid-profile; no automatic P0 mint | +| P1 Wikidata person | valid A/polymorphic with payload type | `Person` accepted as payload-asserted; Wikidata resolver may corroborate/conflict | +| P1 `gen1` movie with exact recipe fields | valid C/unambiguous, P1 required | recipe re-derives IID; no external resolver required | +| P0 `gen1` | profile-invalid | reject new mint even though type is embedded in value | +| P0 CAIP-19 ERC-20 | valid B/unambiguous | classify token asset; chain/metadata resolution is retryable | +| P1 CAIP-10 contract | valid B/polymorphic with type | resolve chain state; account/contract distinction retained | +| Legacy JSON-LD music recording + Spotify `sameAs` | legacy profile | existing classification/enrichment still works; backfill may derive ISRC if evidence supports it | +| Legacy IPFS JSON | legacy profile | current remote parsing continues; derived IID remains additive | +| Plain string | legacy profile | no false IID detection | +| IID value containing colons | valid when scheme canonicalizer accepts it | parser splits first two colons only | +| Unknown registered-future-looking scheme | invalid under closed registry | no forward-compatible pass-through | +| Non-ASCII or >256-byte IID | invalid | write rejected; historical input visible with error | +| Empty URI list | no context event | atom processing completes normally | +| Five max-length URI byte values under current defaults | contract-valid | raw bytes round-trip and API output is bounded | +| URI count/length over live config | preflight and contract failure | no partially submitted batch; actionable error | +| Duplicate URI values | preserved as event evidence | resolution work may deduplicate safely | +| Binary/non-UTF-8 URI | valid opaque event bytes | lossless storage, no unsafe fetch, decode error recorded | +| Multi-atom URI batch | context events emitted after atom/deposit events | every URI set joins by `termId` correctly | +| Replayed context block | same input twice | no duplicate typed/projection rows | +| Reorged context event | canonicality changes | derived context projection follows established reorg policy | +| Same IID in P0 and P1 | two atom IDs, one identity cluster | P0 identity-canonical; richer/attested member eligible for display-canonical | +| Provider timeout/429 | identity/classification remain complete | resolution retries with provider-specific backoff | +| Provider permanent miss | valid atom with unresolved display | explicit terminal resolver status; no identity mutation | +| Seed repeated twice | identical output | IID/profile/bytes/URI order/expected ID all match | + +## End-to-end go/no-go assertions + +The integration lead should be able to answer “yes” to all of these from automated output: + +1. Did exact submitted bytes produce the expected on-chain atom ID? +2. Did all context URIs join the correct term by `termId` and survive a replay? +3. Did the parser select the expected profile and use the pinned registry version? +4. Did classification complete without a network dependency? +5. Did resolver failure leave identity stable and retryable? +6. Can the API distinguish atom payload, identity, context, and enrichment provenance? +7. Can search/render work after enrichment while old clients still use `dataResolved`? +8. Did legacy fixtures remain unchanged? +9. Did equal IIDs cluster without deleting or overwriting either atom? +10. Can flags restore the old write behavior without a destructive database rollback? + +## Known documentation/package drift to resolve + +- The formal IID specification is labeled `0.1.0 — Draft` while planning material also describes several format decisions as ratified/stable. Record the release decision before enabling default writes. +- The current Core dependency on `@0xintuition/classifications@0.1.0-alpha.0` is not automatically the same as the `intuition-v2` workspace implementation that depends on `@0xintuition/iid`. +- Some planning checklists still mark P0 builder/indexer/backfill phases incomplete. This migration should not assume those implementations exist merely because the specification does. +- Older transaction guidance describes atom creation without URI context. Generate operational docs from the shipped ABI after the upgrade. + diff --git a/.planning/codex-migration/06-scheme-resolution-matrix.md b/.planning/codex-migration/06-scheme-resolution-matrix.md new file mode 100644 index 0000000..39c0575 --- /dev/null +++ b/.planning/codex-migration/06-scheme-resolution-matrix.md @@ -0,0 +1,87 @@ +# Scheme classification and resolution matrix + +This matrix separates three questions that must not be collapsed: + +1. Is the IID valid and canonical? The IID package answers this offline. +2. Does the scheme/value determine a classification? The classification registry answers this deterministically. +3. How is display/enrichment data obtained? A versioned resolver answers this over the network or from cached artifacts. + +Identity classes and scheme typing below come from the current IID implementation. “Classification route” describes the required Core route or a gap to close; it is not permission to mint P0 without tests and an approved allowlist. + +## Registry matrix + +| Scheme | Class | Typing | Classification route | Likely resolver/evidence | Cutover stance | +| --- | --- | --- | --- | --- | --- | +| `isbn` | A | Unambiguous | `Book`; retain edition-vs-work level explicitly | Open Library/ISBN metadata | Implemented taxonomy path; P0 canary after level decision | +| `isrc` | A | Unambiguous | `MusicRecording` | MusicBrainz ISRC lookup; Spotify/Apple matching/context | First P0 vertical slice | +| `iswc` | A | Unambiguous | Musical composition/work; current taxonomy gap | ISWC authority/music metadata | Do not enable P0 until classification exists | +| `isni` | A | Polymorphic | P1 type: person, music group, or organization | ISNI and corroborating registries | P1 only; resolver cohort later | +| `orcid` | A | Polymorphic | P1 `Person` or approved narrower researcher type | ORCID | P1 only | +| `lei` | A | Unambiguous | Legal entity/company; current ladder coverage must be aligned | GLEIF | Close taxonomy/ladder gap before P0 | +| `gtin` | A | Unambiguous | `Product` | GS1/approved product source | P0 after resolver/provider policy | +| `doi` | A | Polymorphic | P1 type: article, dataset, or other DOI object | Crossref/DataCite/provider named by DOI | P1 only | +| `eidr` | A | Unambiguous | Audiovisual work; verify movie/series taxonomy coverage | EIDR and approved media databases | Close taxonomy route before P0 | +| `wd` | A | Polymorphic | P1 `@type` required | Wikidata; Wikipedia links as artifacts | P1 high-value cohort | +| `mbid` | A | Unambiguous | Decode value subtype: artist, recording, release-group, etc.; define unsupported subtypes | MusicBrainz | P0 per supported subtype only | +| `olid` | A | Unambiguous | Decode work/edition/author suffix; keep ontological level | Open Library | P0 per supported subtype only | +| `imdb` | A | Polymorphic | P1 movie/series/person type | IMDb context; TMDB find API where licensed/approved | P1 only | +| `tmdb` | A | Polymorphic | P1 type; value subtype can route but does not remove profile rule | TMDB | P1 only | +| `podcastguid` | A | Unambiguous | `PodcastSeries` | Podcast Index and feed metadata | P0 after feed/GUID policy test | +| `url` | B | Polymorphic | P1 type required | Safe URL fetch, metadata/extraction plugins | Legacy-compatible; P1 only | +| `caip10` | B | Polymorphic | P1 `EthereumAccount` or `EthereumSmartContract` | Chain RPC, explorer, ENS as display evidence | P1 only | +| `caip19` | B | Unambiguous | Current supported asset subtype maps to `EthereumERC20`; future namespaces need routes | Chain RPC/token metadata/CoinGecko | P0 only for supported namespaces | +| `hash` | B | Polymorphic | P1 image/video/content type | Context URI or content-addressed store | P1; never fetch unknown locations blindly | +| `appid` | B | Unambiguous | `MobileApplication` | Apple/Google app stores | P0 after store/resolver policy | +| `purl` | B | Unambiguous | `Software` package | Package registry, deps.dev, source repository | P0 after ecosystem coverage tests | +| `geo` | B | Polymorphic | P1 `Location` or `LocalBusiness` | Geospatial/open place sources and context | P1; candidate matching later | +| `acct` | B | Unambiguous | `SocialMediaAccount` | Platform API/page; immutable ID preferred | P0 only for approved strong value forms | +| `rssitem` | B | Unambiguous | `PodcastEpisode` | Feed item/feed resolver | P0 after container/value tests | +| `termset` | B | Unambiguous | `DefinedTerm` | Owning term-set artifact/context | P0 after term-set lookup exists | +| `gen1` | C | Unambiguous | Decode classification slug; verify exact recipe fields | No required external resolver; P1 evidence is primary | Never P0 | + +## Resolver priority rules + +For any scheme, build targets in this order: + +1. Authoritative/open registry lookup keyed by the canonical IID value. +2. Provider crosswalk discovered from the registry response. +3. Explicit creation-time context URIs. +4. P1/P2 payload evidence. +5. Legacy URLs and previously stored artifacts. + +The order is a routing preference, not a truth ranking. Every result retains provenance, and conflicting facts remain visible. A context URI may be tried before a slow registry for latency, but the stored plan and artifact must still say that it was context-derived. + +## Classification conflicts + +Handle conflicts explicitly: + +- For an unambiguous scheme, a contradictory P1/P2 `@type` is an interpretation error and must not overwrite the deterministic route. +- For a polymorphic scheme, P1/P2 supplies the asserted type. A resolver may corroborate it or return a conflict that requires review. +- Provider output cannot make an invalid P0 payload valid after the fact. +- A scheme subtype not supported by the current taxonomy is `unsupported-classification`, not `Thing` by default. +- Cross-level relations—book work versus edition, recording versus composition, release versus release-group—become equivalence/relationship candidates, not silent merges. + +## Minimum resolver implementation contract + +Each resolver must document: + +- accepted scheme/value or context target; +- classification/subtype coverage; +- authentication and legal/licensing constraints; +- request/cache key, TTL, timeout, retry, and rate-limit behavior; +- raw response retention policy and normalized artifact version; +- identifiers/crosswalks it may emit; +- permanent-miss versus retryable-failure rules; +- safe handling of redirects, private network addresses, and content size; +- deterministic mocks and one opt-in live integration test. + +## Cohort rollout + +Recommended order: + +1. `isrc` / MusicRecording — proves the exact user scenario end to end. +2. `wd`, `isbn`, `olid`, `mbid` — broad open-registry coverage and P1/P0 mix. +3. `caip10`, `caip19`, `purl`, `appid`, `podcastguid`, `rssitem` — domain resolvers already suggested by current enrichment capabilities. +4. Remaining authority schemes after taxonomy and licensing decisions. +5. `url`, `hash`, `geo`, and `gen1` specialized behavior, followed by candidate generation/equivalence. + diff --git a/.planning/codex-migration/07-reference-implementation-analysis.md b/.planning/codex-migration/07-reference-implementation-analysis.md new file mode 100644 index 0000000..e0cf3d0 --- /dev/null +++ b/.planning/codex-migration/07-reference-implementation-analysis.md @@ -0,0 +1,105 @@ +# Reference implementation analysis + +## Why the private implementation matters + +The private application monorepo demonstrates the semantic half of this migration across real application surfaces. Its useful architecture is a sequence of explicit stages: + +```text +DERIVE -> RECOGNIZE -> RESOLVE + INDEX -> RENDER -> EMIT -> OPERATE + PUBLISH and DEPRECATE run alongside the sequence +``` + +Core should preserve these boundaries. They make it possible to turn on readers before writers, retry external resolution without changing identity, and validate each handoff independently. + +## Proven patterns to carry into Core + +### A pure IID grammar package + +The implemented `@0xintuition/iid` module establishes the correct low-level boundary: parsing, canonicalization, profile handling, and deterministic serialization are pure operations. They do not make network calls and do not own provider routing. + +Core should consume this implementation rather than grow another parser. All language implementations must share golden fixtures, especially for values containing colons, mixed-case identifiers, Unicode, and invalid closed-registry schemes. + +### A semantic bridge between identity and classifications + +The implemented `@0xintuition/iid-registry` boundary owns questions such as: + +- Which classification, if any, follows from this IID? +- Which providers can resolve this scheme or typed profile? +- Which identifier hints can be extracted without a network call? +- Is the scheme unambiguous enough for P0? + +This is the most important reusable boundary. Core parser, classification, enrichment, seed, and UI code must not each maintain their own scheme maps. + +The private implementation also proved several decision outcomes that should be ratified for public packages: `eidr` maps to movie, `iswc` remains unclassified, MusicBrainz release maps to album, MusicBrainz label maps to company, and CAIP-19 maps only for the supported ERC-20 subtype. + +### Identifier-first enrichment + +The private backend stopped requiring a legacy JSON-LD document or provider URL as the entry point. It creates a resolution plan from the IID, invokes providers, stores artifacts with provenance, and projects a stable resolved view. This is the target for Core workers. + +Important details to copy: + +- Provider routing means desired semantic capability, not merely a matching hostname. +- Authentication, throttling, and transient provider failures remain retryable. +- A source URI may be derived from the IID when the provider supports it, but it is not the identity. +- Projection populates display/search fields from resolved artifacts and classification hints; it never substitutes raw `int:...` text as the user-facing label. +- Worker handoffs carry an explicit identity object rather than reparsing ad hoc strings. + +### Read-before-write deployment + +The private plan correctly treated search and display as part of the read path. “The backend parses IID” is not enough. Core's read gate includes parsing, database representation, resolution, API shape, explorer rendering, search purity, failure handling, replay, and observability. + +### Golden-fixture integration + +The private implementation has a cross-layer golden fixture. Core should extend that model to include the on-chain URI event and the public package tarballs. One fixture must be executable in each repository and assert the same canonical bytes, classification, provider plan, atom ID, and normalized context. + +## What does not port directly + +### The public classification model is different + +The private IID code uses executable source callbacks. The public `classifications` repository now contains declarative `IdentitySpec` ladders (`ladder`, `IdentityValueSource`, and recipe fields). The public design is preferable as the source of semantic metadata, but the two representations must be reconciled deliberately. Copying the private package wholesale would create two classification models. + +The target is: + +1. `iid` owns grammar and canonicalization types. +2. `classifications` declares identity ladders using those shared types. +3. `iid-registry` interprets those declarations and exposes stable lookup APIs. +4. Builders and Core consume the registry, not declaration internals. + +### URI context was outside the private migration + +The private plan explicitly treated contract URI support as separate future work. Core is the first system that must integrate it end to end. None of the private backend tables, event workers, or frontend seams should be assumed to cover `AtomContextRegistered`. + +### Core is an event-indexed platform, not the same application backend + +Core's chain flow is Rindexer -> event store/typed tables -> projections -> knowledge graph/API. The private backend model cannot be transplanted directly. The semantic contracts port; the persistence and replay implementation must follow Core's event architecture. + +### Core does not currently have atom semantic embeddings + +The private search/embedding work includes application-specific semantic embedding paths. Core currently uses vector infrastructure for other domains, not atom semantic embeddings. This migration should add lexical/search/display guards and a future-safe resolved schema, but must not invent an atom embedding subsystem merely for parity. + +### The private canonical builder is not yet a public contract + +The private plan proposed a single `buildAtomAnchor` seam, but the currently inspected implementation still contains local derivation and identifier injection. Core should not depend on that incomplete seam. The public package track must define, test, and publish the canonical builder first. + +## Reference-to-Core mapping + +| Private concept | Core destination | Adaptation required | +| --- | --- | --- | +| IID parser/canonicalizer | public `iid`; Core atom parser | publish package and share fixtures | +| IID registry | public `iid-registry`; classification/enrichment workers | interpret public declarative ladders | +| Identifier-first enrichment | atom enrichment workers | use Core queues, artifacts, retries, and projections | +| Raw/resolved split | KG identity/context/artifact tables and node projection | preserve event replay and legacy rows | +| Search/display guards | API and explorer | Core-specific queries and UI states | +| Golden fixture | package CI + Core integration suite | add ABI/event/URI assertions | +| Writer migration | seed and atom creation services | use URI-aware contract method and public builder | + +## Review method for future changes + +When the private implementation advances, classify each change as one of: + +- **semantic contract** — consider upstreaming to public packages; +- **application adapter** — port only its interface expectation; +- **private product behavior** — do not copy into Core; +- **bug fixture** — add to the shared conformance corpus. + +This keeps the private monorepo useful as a proving ground without allowing it to become an undeclared second source of truth. diff --git a/.planning/codex-migration/08-public-package-architecture-and-release.md b/.planning/codex-migration/08-public-package-architecture-and-release.md new file mode 100644 index 0000000..5ca37ba --- /dev/null +++ b/.planning/codex-migration/08-public-package-architecture-and-release.md @@ -0,0 +1,170 @@ +# Public package architecture and release plan + +## Outcome + +The public packages repository becomes the shared semantic and transaction boundary between applications and Core. Core should integrate published, exact-pinned packages; it should not copy the private monorepo implementation or maintain local lookup tables. + +Repository: `/Users/metasudo/workspace/intution/workspace/packages` ([GitHub](https://github.com/0xIntuition/packages)) + +## Target package ownership + +| Package | Owns | Must not own | +| --- | --- | --- | +| `@0xintuition/iid` | grammar, parse/serialize, profiles, canonicalization, scheme registry types, conformance vectors | classifications, provider clients, contract calls | +| `@0xintuition/classifications` | schema.org classifications and declarative identity ladders | parser implementations or network resolution | +| `@0xintuition/iid-registry` | IID-to-classification lookup, provider plans, identifier hints, profile selection | provider I/O or persistence | +| `@0xintuition/ids` | on-chain atom/triple term ID calculation | semantic identity interpretation | +| `@0xintuition/primitives` | canonical high-level atom/triple builders | duplicated IID maps or raw contract transports | +| `@0xintuition/protocol` | consumer ABI, encoders, readers/writers, event decoders, URI configuration | deployed bytecode source | +| `@0xintuition/react` | ergonomic URI-aware hooks over protocol | alternate transaction semantics | +| `@0xintuition/deployments` | network addresses and optional activation metadata | Core-local devnet lifecycle | + +`ids` and `iid` are intentionally separate: `iid` identifies an external entity in canonical bytes; `ids` hashes atom/triple bytes into protocol term IDs. + +## Required package work + +### `iid` + +- Port and harden the private grammar implementation. +- Export stable types for scheme, profile, parsed IID, normalization result, and typed errors. +- Make parsing split only the first two colons. +- Publish a closed scheme registry and canonicalization behavior. +- Export machine-readable conformance vectors through a supported package export. +- Test P0/P1/P2, canonical round trips, unsafe Unicode, invalid schemes, and values containing colons. +- Keep the module pure and runtime-neutral. + +### `classifications` + +- Reconcile its current declarative identity ladders with `iid` types. +- Remove duplicate local scheme/class/typing definitions when that can be done without a circular dependency. +- Complete schema.org coverage needed by the registry, including provenance for generated classification data. +- Add reverse lookup tests from classification to candidate identity recipes. +- Decide and document whether identity ladders are public stable API or generated internal data exposed through the registry. + +### `iid-registry` + +- Implement total `classificationForScheme` and value-aware `classificationForIid` behavior. +- Implement `providersForScheme`, `providersForIid`, and `identifierHintsForIid`. +- Express P0 eligibility and P1/P2 profile selection explicitly. +- Preserve provider ordering and desired-capability semantics. +- Return `undefined` classification for genuinely polymorphic/unmapped cases while still returning safe resolver hints. +- Port the private decision fixtures and make every supported scheme explicit in a matrix test. + +### `primitives` + +Add one canonical builder seam, tentatively: + +```ts +type AtomAnchor = { + iid: string + data: `0x${string}` + atomId: `0x${string}` + classification?: string + providerPlan: readonly string[] + contextUris: readonly string[] +} + +buildAtomAnchor(input, options): AtomAnchor +``` + +The final API may differ, but it must satisfy these contracts: + +- use the registry to select the profile; +- emit canonical IID bytes exactly once; +- calculate the matching atom ID through `ids`; +- produce an ordered, deduplicated, contract-valid URI manifest; +- support an explicit legacy JSON mode during migration; +- expose structured validation failures before transactions are submitted. + +Replace or adapt the existing `buildAtom` path, which currently assumes JSON-LD classification output and validates only JSON object atom data. + +### `protocol` + +- Update the ABI to the URI-enabled contract release. +- Expose `createAtomsWithUris` and matching encoders/simulators. +- Expose `AtomContextRegistered` event decoding. +- Expose `AtomUriConfig`/`getAtomUriConfig` reads. +- Assert tuple/array shape parity, ordered URI preservation, and empty URI behavior. +- Keep existing `createAtoms` APIs additive and supported for compatibility. + +### `react` + +- Add an additive URI-aware atom-creation hook or extend the existing hook with a backward-compatible discriminated input. +- Validate lengths and URI limits before wallet interaction. +- Return simulation/revert details suitable for product UI. +- Keep the existing non-URI method usable during the compatibility period. + +### `deployments` + +Update only if the release needs ABI activation metadata or changed addresses. Core-local devnet addresses remain owned by Core. If activation blocks are added, define a network-aware shape and test it against Core's Rindexer start blocks. + +## Core integration boundary + +Core should use packages at these seams: + +| Core area | Public dependency | +| --- | --- | +| atom parser | `iid` | +| classification worker | `iid`, `iid-registry` | +| enrichment worker | `iid`, `iid-registry`, `classifications` as needed for output schemas | +| KG term-ID actions | `ids` through thin Core wrappers | +| seed/preparation code | `primitives`, `iid-registry` | +| contract clients and ABI consumers | `protocol` | +| UI transaction adapters, if present | `react` or `protocol` | + +No Core service may add a private scheme-to-classification or scheme-to-provider table. A lint/architecture test should scan for known scheme lists outside allowed package adapters. + +## Contract ABI source-of-truth rule + +There are currently two relevant layers: + +- `@0xintuition/contracts-v2` owns Solidity source and deployable artifacts. +- `@0xintuition/protocol` owns the public consumer ABI and helpers. +- Core's `packages/contracts` owns local devnet deployment and synchronized artifacts. + +The release gate is byte-for-byte ABI semantic parity between the exact `contracts-v2` version and `protocol`. Core may retain its ABI synchronization workflow for Rindexer, but CI must compare functions, events, inputs, indexed fields, and tuple components against the public protocol ABI. A mismatch blocks publishing or Core adoption. + +## Proposed release train + +Exact versions must be chosen by the package release owner. A coherent prerelease train would be: + +| Order | Package | Proposed version | Reason | +| --- | --- | --- | --- | +| 1 | `iid` | `0.1.0-alpha.0` | new package | +| 2 | `classifications` | `0.1.0-alpha.1` | unreleased identity ladder work plus shared types | +| 3 | `iid-registry` | `0.1.0-alpha.0` | new bridge; depends on 1–2 | +| 4 | `ids` | unchanged unless exports/behavior change | term hashing is conceptually stable | +| 5 | `primitives` | `0.1.0-alpha.1` | canonical builder and IID support | +| 6 | `protocol` | `3.1.0` | additive URI ABI and APIs | +| 7 | `react` | `0.1.0-alpha.1` | URI-aware hook, if included | +| 8 | `deployments` | bump only if data changes | avoid empty releases | + +Update the repository's hard-coded pack/smoke package lists and publish order to include `iid` and `iid-registry` in topological order. Require exact internal dependency pins. + +## Release verification + +For every package: + +1. Run its build, typecheck, tests, schema checks, and lint. +2. Run repository-wide pack dry-run and packed-tarball smoke tests. +3. Inspect `npm pack --json` contents and exports; source-only files are not an API. +4. Install all packed tarballs together in a clean temporary consumer. +5. Run the shared conformance fixture through public exports only. +6. Compare protocol ABI to the exact contracts artifact. +7. Publish in dependency order with provenance and immutable exact versions. +8. Verify registry metadata, tarball integrity, export resolution, and installation from NPM. + +## Core's 14-day release-age policy + +Core enforces a 14-day minimum package age. New IID packages and new versions cannot be adopted in a normal production lockfile on publication day. + +Use a two-lane process: + +- **Integration lane:** test clean packed tarballs from the package repository in temporary worktrees and CI. Do not commit `file:` or Git dependencies. +- **Release lane:** publish prereleases, monitor them for 14 days, then commit exact NPM versions to Core after all gates pass. + +The preferred solution is to let the release age. An emergency exception must be time-bounded, explicitly approved, and removed after the soak. The current package-name-based exclusion mechanism is broader than a single version, so it is not the default migration strategy. + +## Package track exit gate + +The package track is complete when a clean consumer using only packed/public exports can build the golden IID anchor, derive the expected classification and provider plan, calculate the expected atom ID, encode URI-aware atom creation, decode the resulting context event fixture, and reproduce the same outputs in Core. diff --git a/.planning/codex-migration/09-program-roadmap.md b/.planning/codex-migration/09-program-roadmap.md new file mode 100644 index 0000000..c50d127 --- /dev/null +++ b/.planning/codex-migration/09-program-roadmap.md @@ -0,0 +1,170 @@ +# Program roadmap + +## Delivery strategy + +Treat this as a staged platform migration, not a flag day. Semantic packages, URI event ingestion, and Core reader work can run in parallel after the shared contracts are frozen. Production writers are the final dependent track. + +```text +Phase 0: ratify contracts and fixtures + |----------------------| + v v +Phase 1A: public packages Phase 1B: URI event ingestion + | | + v v +Phase 2: Core recognize + persist + resolve + | + v +Phase 3: API + search + explorer reader gate + | + v +Phase 4: shadow seed/mint writers + | + v +Phase 5: controlled default switch + backfill + deprecation + +NPM prerelease publish -> 14-day soak runs alongside Phases 2–3 +``` + +## Phase 0 — Foundation and decision freeze + +Goal: make every parallel team build against the same contract. + +Deliverables: + +- Ratify [12-decision-log.md](./12-decision-log.md). +- Freeze canonical identity and resolution types. +- Freeze the public builder input/output contract. +- Freeze normalized atom-context representation and bounds. +- Establish one golden fixture set: unambiguous P0, typed polymorphic IID, colon-bearing value, invalid IID, URI context, no-context IID, and legacy JSON. +- Record exact contract artifact/version and expected ABI fingerprint. +- Establish dashboards, feature flags, and rollback ownership before rollout work starts. + +Exit gate: fixture schemas and ownership are approved by package, contract/indexer, backend, seed, and product/API owners. + +## Phase 1A — Public semantic and protocol packages + +Goal: create the shared implementation that every repository consumes. + +Deliverables: + +- `iid`, reconciled classification ladders, and `iid-registry`. +- Canonical builder in `primitives`. +- URI-enabled ABI and helpers in `protocol`. +- Optional URI-aware React hook. +- Packed-tarball integration harness and ABI parity test. +- Prereleases published in dependency order. + +Exit gate: package tarballs pass the end-to-end package fixture and are published. The 14-day release-age clock begins. + +## Phase 1B — Contract URI ingestion + +Goal: ensure Core never loses URI context emitted by the protocol. + +Deliverables: + +- Exact URI-enabled contracts artifact in Core's contracts workspace. +- Synchronized Rindexer ABI/config/generated bindings. +- `AtomContextRegistered` in raw/typed event storage, shared event models, and replay readers. +- Knowledge-graph context projection joined by `termId`. +- Raw bytes preserved plus a separately normalized view. +- Event metrics, malformed URI handling, and replay fixture. + +This work can initially compile against local contract artifacts. Public `protocol` parity must pass before production release. + +Exit gate: replaying the URI fixture twice is idempotent, order-preserving, and produces the same durable context. + +## Phase 2 — Recognize, persist, classify, and resolve + +Goal: make Core a complete reader of IID atoms. + +Deliverables: + +- IID raw type and canonical parsing in the atom parser. +- Identity persistence with raw/canonical IID and parsed components. +- Classification through the registry, including safe behavior for polymorphic schemes. +- Identifier-first enrichment plans and provider adapters. +- Artifact provenance, retry taxonomy, resolved projection, and reconciliation jobs. +- Legacy paths preserved. + +Exit gate: backfilled and live IID atoms converge to identical identity, classification, provider plan, and resolved projection. + +## Phase 3 — Data product reader gate + +Goal: make the migration visible and correct to API and explorer consumers. + +Deliverables: + +- Versioned API shape for identity, context, resolution status, classification, artifacts, and display data. +- Search uses display labels/resolved metadata rather than raw IIDs where resolution exists. +- Explorer shows identity and context provenance without replacing the raw on-chain value. +- Loading, unresolved, retryable failure, terminal failure, and legacy states are distinct. +- Query/index performance is measured against production-scale datasets. + +Exit gate: all reader acceptance tests pass in staging; no surface relies on a legacy JSON object being present. + +## Phase 4 — Shadow writer and seed migration + +Goal: validate the future write path without changing production defaults. + +Deliverables: + +- Seed source data maps to canonical IID inputs before upload/mint. +- Single public builder used by seed jobs and any application transaction adapter. +- URI manifest produced deterministically, deduplicated, ordered, and contract-bounded. +- Dry-run output records canonical data bytes, predicted atom ID, provider plan, URI manifest, and validation warnings. +- Shadow comparison against legacy output and duplicate/collision reports. +- Idempotent resume ledger for bulk jobs. + +Exit gate: the seed golden corpus produces deterministic results across reruns and the reader path displays shadow-minted staging atoms correctly. + +## Phase 5 — Cutover, backfill, and compatibility + +Goal: enable IID writes safely and move existing data into the new read model. + +Order: + +1. Deploy all readers with IID writer flags off. +2. Replay/backfill URI context and IID identity projections. +3. Verify API, search, explorer, queue, provider, and database SLOs. +4. Enable URI-aware writer for an internal allowlist. +5. Enable one scheme at a time, beginning with a deterministic unambiguous P0 scheme such as ISRC. +6. Expand by scheme and producer while watching failure budgets. +7. Make IID the default for approved new atoms. +8. Retain legacy reads and an explicit legacy write escape hatch for the agreed compatibility period. +9. Deprecate duplicate derivation code only after telemetry shows no callers. + +Rollback disables writers first. Reader support and durable context storage remain deployed because they are backward compatible and necessary to interpret already-created atoms. + +## Staffing model + +Recommended: one program/integration lead and one owner for each of Tracks 1–6; Track 7 is jointly staffed by the integration lead and platform/reliability. A smaller team can combine work as follows: + +- Package foundation + IID recognition +- URI ingestion + data/API +- Resolution/enrichment + seed/mint +- Integration/operations remains a named owner, not an unowned shared responsibility + +Do not assign both sides of a producer/consumer contract without a second reviewer. In particular, package APIs require review from Core consumers, and URI event projection requires review from the contract owner. + +## Integration cadence + +- Daily contract review during active implementation: changed exports, schemas, fixtures, migrations, and event shapes. +- Merge package work in dependency order; merge Core reader work behind flags. +- Run the cross-repository golden fixture on every boundary change. +- Use one integration branch only for packed-tarball validation; normal feature work remains independently reviewable. +- Cutover authority belongs to one named integration lead with explicit stop/rollback criteria. + +## Schedule reality + +The hands-on Core integration can be compressed into focused parallel days after contracts and packages are ready. The production program cannot honestly complete in one to two days because package publication must satisfy Core's release-age policy and external resolver behavior needs observation. Plan around gates, not calendar optimism: publish early, run Core integration against packed tarballs while versions soak, and use the waiting period for reader/backfill/load validation. + +## Program-level success metrics + +- 100% of recognized IID atoms have canonical parsed identity rows. +- 100% of context events are durably stored; projection lag stays within the existing event SLO. +- Resolution success and retry rates are visible by scheme/provider. +- No user-facing label or search document is only a raw IID when a resolved label exists. +- Zero atom-ID mismatch between public builder, Core, and contract fixture. +- Zero ABI drift between contracts artifact, public protocol package, and Rindexer. +- Legacy atom query success does not regress. +- Writer rollback can be completed without schema rollback or loss of ingested context. diff --git a/.planning/codex-migration/10-core-file-change-map.md b/.planning/codex-migration/10-core-file-change-map.md new file mode 100644 index 0000000..b44012d --- /dev/null +++ b/.planning/codex-migration/10-core-file-change-map.md @@ -0,0 +1,208 @@ +# Intuition Core file and subsystem change map + +This is an implementation map, not a substitute for code-level discovery in each PR. Paths name current ownership seams; generated filenames may change after ABI synchronization. + +## Contract artifacts and Rindexer + +### `packages/contracts` + +Current role: exact `@0xintuition/contracts-v2` dependency, local devnet deployment, vendored ABI/bytecode/address artifacts. + +Changes: + +- Bump the exact contracts artifact to the URI-enabled release. +- Regenerate vendored artifacts; never hand-edit generated ABI. +- Extend devnet deployment/config tests to read `getAtomUriConfig`. +- Add an ABI parity test against the exact public `@0xintuition/protocol` release. +- Document whether activation blocks belong in public deployments or Core-local addresses. + +### `scripts/sync-abis.ts` and Rindexer configuration/generated code + +Changes: + +- Include `AtomContextRegistered` in ABI synchronization and configured events. +- Regenerate handlers/types through the existing workflow. +- Assert event signature, indexed `termId`, creator, and ordered `bytes[]` representation. +- Add fixture decoding against a real encoded log, not only hand-built objects. + +## Event store and shared event types + +### Event type/model/storage layer + +The current event domain models six protocol events. Add `AtomContextRegistered` everywhere the event union is assumed exhaustive: + +- shared `EventType` and `ParsedEvent` variants; +- raw event normalization; +- typed event database model/table; +- handler persistence; +- typed event reader used by projections/replay; +- metrics and dead-letter/error reporting. + +Create the new typed table with `sequence_number` from its first migration. Preserve raw context entries as hex/bytes and store normalized URI interpretation separately. The event is associated by `termId`; log order or transaction adjacency is never a join key. + +## Projections and knowledge graph + +### `crates/projections` + +Changes: + +- Consume the new typed context event. +- Upsert context idempotently using chain/event identity. +- Join to the atom/node through `termId`, tolerating event-before-node projection timing. +- Preserve URI array order and duplicates in raw evidence; expose a normalized deduplicated list separately. +- Requeue/reconcile orphan context rows when the corresponding atom projection arrives. +- Ensure replay from any safe checkpoint converges. + +### Knowledge-graph schema/actions + +Add or evolve storage for: + +- canonical identity (`raw`, `canonical`, `profile`, `scheme`, `value`, parse/version/status); +- atom context (`raw bytes`, normalized URI if valid, ordinal, event provenance); +- resolution artifacts (`provider`, request key, fetched-at, resolver version, raw payload/reference, status); +- resolved node projection (`classification`, label, description, image, links, search text, provenance/version). + +Do not overload the existing raw atom `data` column. Identity, evidence, and materialized presentation have different lifecycles. + +### `packages/database-kg/src/actions/ids.ts` + +Replace duplicate atom/triple hashing logic with thin wrappers over `@0xintuition/ids`. Preserve Core's existing input normalization at the boundary. Add parity tests for IID UTF-8 bytes, hex bytes, legacy JSON bytes, and triples. + +### Node creation/search defaults + +Audit `ensureNodeWithCreation` and all callers that default `searchText` to raw `data`. For IIDs, initialize search state as unresolved or use safe canonical hints; never promote the opaque raw IID as the final display/search document. + +## Atom parser + +### `packages/atom-parser` + +Changes: + +- Add `iid` to the raw type domain and database-compatible enum/check constraint. +- Run IID recognition before generic URL/string handling. +- Delegate validation/canonicalization to `@0xintuition/iid`. +- Return structured identity details and typed parse failures. +- Preserve legacy JSON, HTTP URI, IPFS URI, and string behavior. +- Add fixtures for colon-bearing values and lookalike `int:` strings. + +The parser recognizes syntax and identity. It does not fetch providers or guess semantic types. + +## Classification worker + +### Classification packages/workers + +Current behavior primarily classifies structured `@type` values or raw content. Add an IID-first branch: + +- consume parsed canonical identity; +- call `@0xintuition/iid-registry`; +- persist inferred classification with source `iid-registry`, package/version provenance, and confidence/totality state; +- leave polymorphic/unmapped classification absent rather than guessing; +- retain the legacy classifier for non-IID atoms. + +Classification output should be idempotent and re-projectable when registry versions change. + +## Enrichment worker + +### `packages/atom-enrichment` and worker services + +Current behavior can return no work when there is no structured document/provider URL. Add identifier-first planning: + +- derive providers and identifier hints from the registry; +- invoke providers by desired capability and typed identifier; +- persist raw artifacts before projection; +- separate retryable transport/auth/rate-limit failures from terminal unsupported/invalid identities; +- version provider adapters and projection logic; +- generate resolved node fields and search text without mutating canonical identity; +- schedule reconciliation when new provider coverage is deployed. + +Remove any local scheme/provider mapping once equivalent public registry behavior exists. The published `classifications` version currently used by Core must be upgraded as part of the coordinated release. + +## Database migrations + +Migrations must be additive and deployable before code that writes new states. + +Required changes: + +- extend the atom raw-type constraint/enumeration with `iid`; +- add identity and canonicalization status/version columns or a normalized identity table; +- add atom-context event/evidence storage and indexes on `termId`; +- add artifact/projection provenance and resolution status where absent; +- add partial indexes for IID scheme/value and unresolved/retryable work; +- retain nullability/defaults that let old binaries operate during rolling deployment; +- include forward-only rollback strategy: disable new code, do not drop data. + +Backfill should be chunked, resumable, observable, and use the same domain functions as live processing. + +## API + +### Atom read endpoints + +Return an additive shape similar to: + +```json +{ + "raw": { "type": "iid", "data": "int:isrc:..." }, + "identity": { "canonical": "int:isrc:...", "profile": "p0", "scheme": "isrc", "value": "..." }, + "classification": { "type": "MusicRecording", "source": "iid-registry" }, + "context": [{ "uri": "https://...", "ordinal": 0, "source": "onchain" }], + "resolution": { "status": "resolved", "updatedAt": "..." }, + "display": { "name": "...", "image": "..." } +} +``` + +Keep legacy fields during the compatibility window. Do not silently replace raw data with resolved data. Add filters/search facets for scheme, classification, resolution status, and context presence as justified by consumers. + +### Atom write endpoint + +The current raw-data POST path is not sufficient as a canonical IID writer. If Core owns atom submission, add a structured builder-backed request that accepts identity inputs plus URI context, returns a dry-run/predicted anchor, and calls the URI-aware protocol method. If Core remains read-only and applications own minting, explicitly deprecate or constrain the existing endpoint so it cannot become a second builder. + +## Explorer + +Replace raw-data-only presentation with distinct sections: + +- raw on-chain atom data; +- parsed/canonical identity and profile; +- inferred classification and its source; +- on-chain context URIs in ordinal order; +- resolution status, provider artifacts, and provenance; +- resolved display fields; +- clear unresolved/retry/error states. + +Search results should prefer resolved labels, show a compact IID secondary label, and remain navigable before enrichment succeeds. + +## Seed and bulk creation + +Core seed/bulk jobs must: + +- map source records to typed identity inputs; +- use the public canonical builder; +- validate canonicalization, P0 eligibility, duplicates, URI limits, and atom-ID expectations offline; +- write a reviewable manifest before upload or transaction submission; +- use `createAtomsWithUris` with exactly aligned `atomDatas`, `assets`, and `uris` outer arrays; +- record transaction/batch/atom results in an idempotent resume ledger; +- re-read indexed output as the final acceptance test. + +Legacy source documents may remain in seed provenance storage, but they are not the default atom bytes. + +## Configuration and observability + +Add configuration for: + +- IID reader enablement and per-scheme writer allowlists; +- provider credentials, concurrency, rate limits, and circuit breakers; +- resolver and projection versions; +- backfill cursors/chunk sizes; +- URI normalization policy and contract-limit cache; +- emergency writer disable. + +Metrics/log dimensions: + +- IID parse outcome by scheme/profile/version; +- context events received/projected/orphaned/malformed; +- provider request outcome and latency by provider/scheme; +- resolution and projection status/age; +- search/display fallback use; +- builder validation failure and writer outcome; +- ABI/fixture version deployed. + +Never log provider secrets or full sensitive payloads. Raw external artifacts follow explicit retention and redaction policy. diff --git a/.planning/codex-migration/11-master-execution-checklist.md b/.planning/codex-migration/11-master-execution-checklist.md new file mode 100644 index 0000000..c978fba --- /dev/null +++ b/.planning/codex-migration/11-master-execution-checklist.md @@ -0,0 +1,142 @@ +# Master execution checklist + +> Completion tracking has moved to [17-core-cutover-completion-plan.md](./17-core-cutover-completion-plan.md), which reconciles this original checklist with the implemented Core foundation and the current public package train. Keep this document as the full program gate inventory; use document 17 for current remaining work and PR order. + +Use this as the program board seed. Every item needs an owner, issue/PR link, environment, and evidence link. A checkbox is complete only when its acceptance evidence exists. + +## Gate 0 — Decisions and contracts + +- [ ] Ratify every blocking item in [12-decision-log.md](./12-decision-log.md). +- [ ] Freeze the canonical IID/domain types and errors. +- [ ] Freeze public registry lookup semantics and provider ordering. +- [ ] Freeze the canonical atom builder input/output. +- [ ] Freeze raw and normalized URI-context storage shapes. +- [ ] Record exact URI-enabled contract artifact, addresses, activation blocks, ABI fingerprint, and configured URI limits. +- [ ] Name track owners, cross-reviewers, integration lead, release owner, and rollback authority. +- [ ] Create the shared fixture manifest and document versioning rules. +- [ ] Create feature flags and per-scheme writer allowlist definitions. + +Evidence: approved decision record, fixture package, ABI manifest, and assigned program board. + +## Gate 1 — Public package release candidate + +- [ ] Add and validate public `@0xintuition/iid`. +- [ ] Reconcile declarative classification identity ladders with shared IID types. +- [ ] Add and validate public `@0xintuition/iid-registry`. +- [ ] Add canonical IID atom builder and legacy compatibility mode to `primitives`. +- [ ] Update `protocol` ABI/helpers/events/config readers for URI context. +- [ ] Update `react` if it is part of the supported write surface. +- [ ] Update deployments only if addresses/activation metadata change. +- [ ] Update publish order, pack dry-run list, and tarball smoke list. +- [ ] Enforce exact internal package pins and no undeclared cycles. +- [ ] Add ABI parity test against exact `contracts-v2` artifact. +- [ ] Install packed tarballs together in a clean consumer. +- [ ] Run shared fixture exclusively through public exports. +- [ ] Publish release train and verify NPM integrity/provenance. +- [ ] Record the date each version becomes eligible under Core's 14-day rule. + +Evidence: package CI, tarball manifest, ABI report, NPM versions/integrity, eligibility calendar. + +## Gate 2 — URI context is durable + +- [ ] Bump Core's exact contracts artifact. +- [ ] Synchronize ABI and regenerate Rindexer bindings. +- [ ] Configure and ingest `AtomContextRegistered`. +- [ ] Add shared event type and parsed model. +- [ ] Add typed event table with `sequence_number` and provenance. +- [ ] Add handler storage and typed event reader. +- [ ] Add idempotent projection keyed by event identity and joined by `termId`. +- [ ] Preserve raw bytes and ordinal order. +- [ ] Normalize supported URIs separately; flag malformed/unsupported entries. +- [ ] Reconcile event-before-atom/orphan timing. +- [ ] Replay the same range twice and compare results. +- [ ] Expose context metrics and alerts. + +Evidence: migration output, real-log fixture decode, replay diff, KG query, dashboard. + +## Gate 3 — IID reader and resolver + +- [ ] Add IID raw type to parser and database constraint. +- [ ] Recognize IID before URL/string fallbacks. +- [ ] Persist raw/canonical identity, profile, scheme, value, version, and status. +- [ ] Backfill existing `int:` atoms through the same parser. +- [ ] Classify only through public registry. +- [ ] Preserve unknown/polymorphic state without guesses. +- [ ] Plan enrichment from IID provider/hint lookup. +- [ ] Store raw artifacts with provider/fetch/version provenance. +- [ ] Implement retryable versus terminal failure taxonomy. +- [ ] Project resolved display/search fields idempotently. +- [ ] Add reconciliation for resolver/registry upgrades. +- [ ] Prove live ingestion and replay/backfill converge. +- [ ] Prove legacy parser/classification/enrichment behavior does not regress. + +Evidence: fixture results, migration/backfill report, worker tests, queue metrics, convergence report. + +## Gate 4 — API, search, and explorer readers + +- [ ] Add additive API identity/context/resolution/display fields. +- [ ] Preserve raw and legacy fields for compatibility. +- [ ] Add unresolved/retryable/terminal status semantics. +- [ ] Update queries/loaders to avoid JSON-only assumptions. +- [ ] Add search documents from resolved labels and semantic metadata. +- [ ] Prevent raw IID from becoming the final label/search content when resolution exists. +- [ ] Render raw identity, context provenance, and resolved display distinctly in explorer. +- [ ] Add scheme/classification/status/context filters only where indexed and supported. +- [ ] Run contract tests with known downstream consumers. +- [ ] Load-test identity/context joins and search backfill. +- [ ] Validate accessibility and safe URI rendering. + +Evidence: OpenAPI/GraphQL diff as applicable, consumer tests, screenshots, query plans, load results. + +## Gate 5 — Seed and writer shadow mode + +- [ ] Inventory every production atom writer and seed source. +- [ ] Route each writer through the public canonical builder. +- [ ] Transform source rows into typed identity inputs. +- [ ] Validate P0/P1/P2 eligibility and canonicalization offline. +- [ ] Create deterministic URI manifests within live contract config. +- [ ] Generate reviewable dry-run artifact with predicted atom IDs. +- [ ] Detect intra-batch and on-chain duplicates. +- [ ] Create idempotent batch/resume ledger. +- [ ] Simulate URI-aware calls and report per-record failures. +- [ ] Run golden corpus twice and compare byte-for-byte. +- [ ] Mint to staging/devnet and verify through indexed reader output. +- [ ] Run shadow comparison against legacy seed output. + +Evidence: dry-run manifest, duplicate report, repeatability hash, staging transaction and indexed API proof. + +## Gate 6 — Production cutover + +- [ ] Adopt exact NPM versions after release-age eligibility. +- [ ] Verify one version of each `@0xintuition/*` dependency in lockfile. +- [ ] Deploy migrations and readers with writer flags off. +- [ ] Complete bounded backfills and reconciliation. +- [ ] Record baseline SLOs and error budgets. +- [ ] Enable internal allowlist for one unambiguous scheme. +- [ ] Verify on-chain event, projection, resolution, API, search, and explorer for canary atoms. +- [ ] Expand by scheme/producer only after observation window passes. +- [ ] Make IID the default for approved new atom types. +- [ ] Retain and test emergency writer disable. +- [ ] Publish migration notes for downstream package/API consumers. + +Evidence: deployment manifest, dashboard snapshots, canary ledger, go/no-go sign-off. + +## Gate 7 — Compatibility exit + +- [ ] Measure remaining legacy write callers and legacy-read traffic. +- [ ] Migrate or explicitly exempt every caller. +- [ ] Announce deprecation dates and supported escape hatch. +- [ ] Remove duplicate scheme maps and local builders only after usage is zero. +- [ ] Keep historical raw data and immutable event evidence. +- [ ] Archive backfill/runbook reports and final fixture versions. +- [ ] Conduct post-migration review and convert follow-ups to owned issues. + +Evidence: usage telemetry, deprecation record, code search proof, final program report. + +## Immediate next actions + +1. Hold the decision review using [12-decision-log.md](./12-decision-log.md). +2. Open the public package foundation issues in dependency order. +3. Open the Core URI-ingestion issue independently so it can begin in parallel. +4. Create the golden fixture repository/export and CI contract before feature PRs diverge. +5. Publish prereleases as soon as package gates pass so the 14-day clock overlaps Core reader work. diff --git a/.planning/codex-migration/12-decision-log.md b/.planning/codex-migration/12-decision-log.md new file mode 100644 index 0000000..9d0a110 --- /dev/null +++ b/.planning/codex-migration/12-decision-log.md @@ -0,0 +1,137 @@ +# Migration decision log + +Status values: **proposed**, **ratified**, or **deferred**. All blocking proposed decisions must be ratified before dependent implementation merges. + +## D01 — Public semantic ownership + +- Status: proposed, blocking +- Decision: `iid` owns grammar/canonicalization; `classifications` owns declarative ladders; `iid-registry` owns semantic and resolver interpretation. +- Reason: this reconciles the private proven boundary with the public repository's declarative model and eliminates shadow maps. +- Approvers: packages lead, backend/Core lead + +## D02 — Canonical builder ownership + +- Status: proposed, blocking +- Decision: `@0xintuition/primitives` exports the only supported high-level atom anchor builder; lower-level packages remain composable but application/seed code does not duplicate profile or URI-manifest logic. +- Reason: deterministic bytes and predicted atom IDs must be identical across all writers. +- Approvers: packages lead, seed/application leads + +## D03 — P0 eligibility + +- Status: proposed, blocking +- Decision: P0 is allowed only for schemes whose classification is total and unambiguous. Polymorphic schemes require a typed P1/P2 representation chosen explicitly. +- Reason: type inference must never depend on an external API guess. +- Approvers: semantic/schema owner + +## D04 — Initial scheme outcomes + +- Status: proposed, blocking for corresponding schemes +- Decision: carry forward the implemented outcomes: `eidr` -> movie; `iswc` has no inferred classification; MusicBrainz release -> album; MusicBrainz label -> company; CAIP-19 classifies only the canonical supported ERC-20 subtype. +- Reason: these have implementation fixtures but need public semantic approval. +- Approvers: semantic/schema owner, product/domain owner + +## D05 — URI ordering and normalization + +- Status: proposed, blocking +- Decision: raw contract context preserves exact bytes, ordinal order, and event provenance. A separate normalized view validates supported URI schemes, canonicalizes where safe, and may deduplicate for resolution; raw evidence is never rewritten. +- Reason: ordered on-chain evidence and operational resolver inputs have different requirements. +- Approvers: contract/indexer lead, security lead + +## D06 — URI manifest policy + +- Status: proposed, blocking for writers +- Decision: the canonical builder orders context deterministically by declared role/priority, removes exact duplicates, validates encoding, and enforces live `getAtomUriConfig` limits before simulation. Policy limits may be stricter than contract limits. +- Reason: prevent mismatched batches, excessive cost, and nondeterministic output. +- Approvers: contract lead, packages lead, seed lead + +## D07 — URI security policy + +- Status: proposed, blocking for resolver/UI +- Decision: allow only approved URI schemes for automated fetching; block private/link-local network targets, enforce redirects/size/content-type/timeouts, sanitize rendered links, and retain unsupported raw bytes without fetching. +- Reason: on-chain context is untrusted input and creates SSRF/content risks. +- Approvers: security lead, platform lead + +## D08 — Identity clustering versus atom identity + +- Status: proposed, blocking +- Decision: canonical IID creates an off-chain identity cluster/linkage key but never merges or rewrites distinct on-chain atom IDs automatically. +- Reason: atom IDs are derived from exact bytes; same semantic identity can exist in multiple profiles or legacy encodings. +- Approvers: data model owner, protocol owner + +## D09 — API compatibility + +- Status: proposed, blocking +- Decision: add structured `raw`, `identity`, `context`, `resolution`, and `display` fields while retaining existing raw/resolved fields for a measured compatibility window. +- Reason: downstream consumers need a non-breaking transition and provenance-preserving model. +- Approvers: API owner, consumer representatives + +## D10 — Public ABI source and parity + +- Status: proposed, blocking +- Decision: `contracts-v2` is the Solidity/deployment artifact source; public `protocol` is the consumer ABI/helper source; Core maintains devnet artifacts. CI semantic-ABI parity among exact versions is mandatory. +- Reason: prevent the current three representations from drifting. +- Approvers: contract lead, packages lead, Core indexer lead + +## D11 — Package release-age strategy + +- Status: proposed, blocking for production adoption +- Decision: publish prereleases early, integrate packed tarballs in clean temporary environments, and wait for Core's 14-day eligibility before committing NPM dependencies. Emergency exceptions require explicit approval and expiry. +- Reason: preserve supply-chain policy without stalling implementation. +- Approvers: Core maintainer, security/release owner + +## D12 — Writer ownership + +- Status: proposed, blocking +- Decision: enumerate the supported production atom writers. Every supported writer consumes the canonical builder and URI-aware protocol API; Core's existing raw POST endpoint is either upgraded to that contract or explicitly scoped away from production atom creation. +- Reason: an ambiguous second writer would immediately reintroduce drift. +- Approvers: Core API owner, application owner, seed owner + +## D13 — Resolution artifacts and retention + +- Status: proposed, blocking for database migration +- Decision: store canonical identity separately from provider artifacts and materialized display projection. Define payload/reference retention, redaction, maximum sizes, and refresh policy per provider class. +- Reason: external data changes and may contain licensed or sensitive content; identity must remain stable. +- Approvers: data owner, legal/security as applicable + +## D14 — Retry taxonomy + +- Status: proposed +- Decision: throttling, network errors, provider 5xx, and credential outages are retryable with bounded backoff/circuit breaking; invalid canonical identity and unsupported scheme/provider pairs are terminal until registry/version changes. +- Reason: prevents permanent data loss during external outages and endless retry loops for invalid work. +- Approvers: enrichment owner, platform owner + +## D15 — Backfill scope + +- Status: proposed, blocking for cutover +- Decision: backfill all existing syntactically valid IID atoms and all URI context events from the protocol activation block. Do not automatically rewrite legacy atom bytes into new atoms. +- Reason: readers need complete history; creating new atoms is a separate intentional operation. +- Approvers: data owner, protocol owner + +## D16 — Compatibility exit criteria + +- Status: proposed +- Decision: remove legacy write paths only after telemetry shows zero unapproved callers for the agreed period. Legacy read support remains for immutable historical atoms. +- Reason: on-chain history cannot be migrated away. +- Approvers: program lead, API/product owners + +## D17 — Atom embeddings + +- Status: proposed +- Decision: this program adds resolved lexical/search fields and schema future-proofing but does not create a new atom semantic-embedding pipeline in Core. +- Reason: no such Core subsystem exists today; importing the private application design would expand scope without a proven Core requirement. +- Approvers: search/data owner, program lead + +## D18 — First production canary + +- Status: proposed +- Decision: use one deterministic, unambiguous P0 scheme—provisionally ISRC—with a small internal allowlist and representative context URIs. +- Reason: isolates infrastructure correctness from polymorphic classification decisions. +- Approvers: program lead, domain owner, operations + +## D19 — Forward-compatible event-store type boundary + +- Status: ratified for the Core migration +- Decision: `event_store.event_type` is no longer governed by a closed-world PostgreSQL `CHECK` whitelist. Event admission is governed by the exact contract ABI and generated Rindexer decoder; typed reconstruction is governed by the Rust `EventType`/`ParsedEvent` exhaustiveness checks and per-event typed tables. Unknown raw events remain observable instead of requiring a schema migration for every new protocol event. +- Reason: on a real TimescaleDB instance with compressed chunks, extending the existing table constraint is not an online-safe operation: adding the replacement check failed against columnstore chunks, `ALTER TABLE ONLY` was unsupported, and disabling columnstore required decompression. Dropping the obsolete whitelist succeeded without decompressing or rewriting historical chunks. Keeping the whitelist would make ordinary additive protocol events operationally destructive. +- Guardrails: ABI drift CI, generated decoder review, raw/typed dual-write parity, typed-table constraints, exhaustive Rust conversions, dead-letter handling, and bounded unknown-event metrics replace the brittle database whitelist. This decision does not weaken per-event payload validation or typed-table constraints. +- Approvers: Core data/indexing architecture; security/release owner to review before production rollout diff --git a/.planning/codex-migration/13-open-source-program-interlock.md b/.planning/codex-migration/13-open-source-program-interlock.md new file mode 100644 index 0000000..3373350 --- /dev/null +++ b/.planning/codex-migration/13-open-source-program-interlock.md @@ -0,0 +1,85 @@ +# Open-source program interlock + +## Why this needs an explicit interlock + +The earlier [open-source program](../open-source/index.md) successfully defined and delivered Intuition Core as the public, self-hostable backend. This migration now changes the semantic and protocol contracts that Core exposes. It is therefore both a product migration and maintenance of the open-source promise: an independent operator must reconstruct IID identity and URI context just as faithfully as Intuition's hosted deployment. + +The migration does not reopen the old repository-topology decision. `intuition-core` is the public service monorepo; `0xIntuition/packages` is the public reusable TypeScript package repository. It does update several assumptions made when the OSS plan was written. + +## Assumptions that must change + +| Earlier OSS-plan assumption | New reality | Required update | +| --- | --- | --- | +| Public packages are a completed set of ten and this program need not change them | IID grammar/registry do not exist on NPM; classifications contain unreleased ladder work; protocol 3.0 lacks URI support | Run the coordinated package release in Track 1 | +| Atom intelligence begins from URL/JSON parsing | New default atom data is canonical IID bytes | Make IID recognition the first parser/classification branch | +| Classification plugins infer type from provider URLs/content | Unambiguous IID scheme/profile can classify deterministically | Registry decision precedes plugin/provider resolution | +| Enrichment requires a URL or parsed classification object | Resolver plan can be derived from IID plus context hints | Add identifier-first enrichment and explicit unresolved states | +| Current six events reconstruct the protocol state needed by Core | URI-enabled contracts emit `AtomContextRegistered` | Extend ABI, ingestion, typed storage, replay, and KG projection | +| Explorer/API raw data model is sufficient | Raw IID is meaningful but not presentation-ready; context and resolution need provenance | Add structured identity/context/resolution/display API and UI | +| Public packages and Core are loosely adjacent artifacts | Deterministic identity, builder bytes, IDs, ABI, and event decode cross both repos | Add packed-tarball conformance and ABI parity CI | + +## Workstream mapping + +| OSS program workstream | Migration continuation | +| --- | --- | +| Atom intelligence libraries | Tracks 1, 3, and 4: IID grammar, semantic registry, parser/classifier/enricher changes | +| Node skeleton/data layer | Tracks 2 and 5: URI tables, identity/artifact projections, indexes and migrations | +| Indexing/projections | Track 2: new event and replay; Track 7: reconstruction proof | +| API/services/workers/explorer | Tracks 3–5: new worker contracts and public read model | +| Security/reconciliation | Tracks 1 and 7: supply chain, ABI parity, URI/SSRF hardening, cross-repo releases | +| Documentation/adoption | Track 7 plus the documentation changes below | + +## Public operator contract + +The migration is not done when Intuition's hosted stack understands IIDs. A clean external Core operator must be able to: + +1. Configure the URI-enabled contract address and activation block. +2. Index both atom creation and atom-context events from chain history. +3. Parse and classify canonical IIDs without private services. +4. Run keyless resolution providers in the minimal tier and see explicit skipped states for credentialed providers. +5. Query raw identity, URI evidence, resolution provenance, and resolved presentation through the public API. +6. Replay the chain range and get the same result. + +This becomes the updated independent-reconstruction acceptance test from the OSS program. + +## Documentation updates required in Core + +Track PRs must update public documentation alongside code: + +- `README.md`: explain IID atom data, URI context, and keyless/credentialed resolver behavior. +- `docs/architecture.md`: replace URL/JSON-first semantic flow with recognize -> classify -> resolve -> project. +- `docs/contracts.md`: document URI-enabled version, `createAtomsWithUris`, context event, limits, addresses, and activation blocks. +- API documentation: add identity/context/resolution/display schemas and compatibility notes. +- Explorer/operator docs: explain unresolved/retry states and raw versus normalized context. +- Configuration reference: flags, scheme allowlists, resolver versions, provider controls, and backfill settings. +- Plugin authoring guides: distinguish classification inference from provider resolution; forbid plugins from changing canonical identity. +- Release notes: exact public package versions, database migrations, reindex/backfill instructions, and rollback behavior. + +The minimal stack must continue to start with zero paid provider accounts. IID recognition/classification and URI ingestion are fully functional offline; enrichment providers degrade explicitly when credentials are unavailable. + +## Repository reconciliation policy + +The private `alpha` monorepo remains a proving ground and consumer. It must consume public package outputs once released rather than export code into Core indefinitely. + +For every semantic change: + +1. Land the reusable contract and fixtures in `0xIntuition/packages`. +2. Validate private application behavior with packed tarballs. +3. Validate Core behavior with the same tarballs and fixtures. +4. Publish exact prereleases and complete supply-chain gates. +5. Adopt eligible NPM versions in both repositories. + +Application-specific adapters stay private; shared grammar, mappings, fixtures, and builders move public. CI should detect a private shadow implementation of an exported public function where practical. + +## Open-source release gate + +In addition to the migration's functional gates, a release candidate must pass: + +- a fresh Core bootstrap with documented configuration only; +- full secret/supply-chain checks for newly added packages and provider configuration; +- packed/public dependency installation, with no local workspace assumptions; +- independent reconstruction of the golden transaction and historical event range; +- updated operator documentation reviewed by someone who did not implement the feature; +- exact container/package/contract/fixture versions recorded in release notes. + +This preserves the original program thesis: the graph is credibly neutral only when independent operators can reproduce its identity and context, not merely download the code. diff --git a/.planning/codex-migration/14-24-hour-parallel-execution.md b/.planning/codex-migration/14-24-hour-parallel-execution.md new file mode 100644 index 0000000..b7dd5ee --- /dev/null +++ b/.planning/codex-migration/14-24-hour-parallel-execution.md @@ -0,0 +1,259 @@ +# 24-hour parallel execution plan + +Status: active + +Prepared: 2026-08-10 + +Implementation snapshot: 2026-08-10 13:10 PDT + +| Lane | Core state | +| --- | --- | +| A01–A04 URI chain truth | Implemented and focused-test green, including linked devnet artifacts, raw/typed ingestion, `kg.node_contexts`, and the independent `atom_context:dual` replay cursor. | +| B01–B04 semantic runway | Implemented and dark: HTTP(S)-only remote parsing, additive IID storage, package-neutral worker DTOs, and terminal/retryable/partial enrichment semantics. | +| C01–C03 readers/operations | Implemented and default-off: lossless semantic API/Explorer envelope, safe context display, golden cross-stack fixtures, smoke gates, and rollout runbook. | +| J03a public IDs | Implemented with exact, age-eligible `@0xintuition/ids@0.1.0-alpha.0`; duplicate Core hash bodies removed. | +| J01/J02/J03b/J04 | Package-gated as recorded in the live join board; no unpublished/Git/file package dependency has entered Core. | + +## Objective + +Use the package-development window to make Core ready to consume the new packages as thin adapters. The next 24 hours should not recreate IID semantics locally. They should land the contract event path, additive storage and worker contracts, future-compatible readers, regression fixtures, and rollout controls. + +## What is genuinely independent + +```text +Lane A — URI chain truth +contracts-v2 -> ABI -> Rindexer -> event store -> typed reader -> KG context + +Lane B — Core semantic runway +KG schema -> worker DTOs -> parser safety -> enrichment completion semantics + | + public packages plug in here + +Lane C — Reader and operations runway +API presenter -> Explorer presenter -> flags/metrics -> golden smoke + +External Lane D — public packages +iid-spec -> iid -> classifications -> iid-registry -> primitives +contracts artifact -> protocol -> react + +A + B + C + D join at packed-package conformance and the end-to-end fixture. +``` + +URI ingestion is never feature-flagged after the ABI/migrations deploy. It records chain truth. Flags control semantic interpretation, external resolution, and writers. + +## Active wave: hours 0–4 + +These three PR-sized changes run concurrently and have disjoint ownership. + +### Core PR A01 — URI contract artifact readiness + +Owner: contract/indexer engineer + +Scope: + +- pin `@0xintuition/contracts-v2@1.1.0-alpha.0`; +- add `AtomContextRegistered` to Core's critical event list; +- assert `createAtomsWithUris` and `getAtomUriConfig` in the exact ABI; +- regenerate `crates/rindexer-ingestion/abi/MultiVault.json`; +- update contract documentation and provenance; +- keep `createAtoms` supported. + +Gate: contract tests, typecheck, `abis:check`, generated diff review. + +### Core PR B01 — Parser URL safety + +Owner: parser engineer + +Scope: + +- restrict generic URL recognition and remote inspection to HTTP(S); +- add regression cases for `int:*`, other custom schemes, and colon-bearing strings; +- do not implement IID parsing, canonicalization, or scheme tables. + +Gate: atom-parser tests/typecheck; every legacy fixture unchanged. + +### Core PR C01 — Future-compatible atom presentation + +Owner: API/explorer engineer + +Scope: + +- add a pure API atom-view presenter with additive `raw`, optional `identity`, `classification`, optional `context`, `resolution`, and `display` sections; +- retain every existing response field; +- add an Explorer presentation adapter that prefers resolved display fields and falls back safely; +- allow only approved URI schemes to render as links; opaque, malformed, `javascript:`, and `data:` values remain escaped text; +- never return an empty context list merely because context ingestion is not deployed. + +Gate: API and Explorer unit tests/typechecks; legacy response contract remains compatible. + +## Second wave: hours 3–9 + +Begin as soon as the overlapping first-wave change merges or stabilizes. + +### Core PR A02 — Rindexer and canonical context event storage + +Depends on: A01 + +- include `AtomContextRegistered` in Rindexer configuration and generated types; +- add handler conversion preserving URI bytes as ordered `0x` hex strings; +- add Timescale migration 050 extending the event-type constraint; +- add `atom_context_registered_events` with event provenance, numeric/hex term ID, registrant, ordered JSONB URI array, and `sequence_number` from inception; +- dual-write raw and typed rows atomically and idempotently. + +Do not UTF-8 decode, normalize, fetch, or validate URI schemes in ingestion. Join by `termId`, never by event adjacency. + +### Core PR B02 — IID database runway + +Depends on: none; deploy before IID-aware workers + +- extend KG `raw_type` with `iid` through an additive migration; +- add nullable, non-unique canonical-cluster `iid` column and partial index; +- allow parse completion to promote `rawType` and `iid` atomically; +- expose the optional field through API projections; +- preserve every legacy row and raw type. + +The column permits multiple nodes to share an IID because semantic identity does not merge distinct on-chain atom IDs. + +### Core PR B03 — Dark worker contracts + +Depends on: B02 schema contract; can be coded in parallel + +- define Core-owned persistence/handoff DTOs for normalized identity, classification decision, and provider plan; +- add optional identity/provenance fields to parse, classification, enrichment, and persisted worker payloads; +- add default-off `WORKERS_IID_READ_ENABLED` and `WORKERS_IID_RESOLUTION_ENABLED`; +- maintain backward deserialization compatibility; +- add no grammar, canonicalizer, classification map, or provider routing. + +### Core PR C02 — Rollout controls and metrics + +Depends on: no semantic packages + +- add default-off reader/resolution/writer/scheme flags at the owning service boundaries; +- do not imply Core's current `POST /api/atoms` is an on-chain writer; +- add bounded-cardinality metrics for recognition, registry decision, resolution, display fallback, and context storage/projection; +- never use complete IIDs, URIs, payloads, or secrets as labels; +- document stop/rollback operations. + +## Third wave: hours 7–15 + +### Core PR A03 — Typed event model and reader + +Depends on: A02 + +- add `EventType::AtomContextRegistered` and exhaustive conversions; +- add `AtomContextRegisteredRecord` and `ParsedEvent` variant; +- add typed-reader SQL reconstruction without converting the URI array to a scalar; +- regenerate the TypeScript Timescale layout/manifest; +- prove raw and typed readers return equivalent stored events. + +### Core PR A04 — Dedicated context projection + +Depends on: A03 and KG migration coordination with B02 + +- add `kg.node_contexts` ledger with raw bytes canonical and optional safe UTF-8 text; +- create a dedicated `atom_context:dual` projection and checkpoint; +- insert one row per URI with ordinal and immutable event provenance; +- retry a missing node without advancing the checkpoint; +- add `kg.events` entry without changing node data, data hex, or atom ID. + +Do not add the event to an already-advanced `core_entities` checkpoint: historical context would be skipped. + +### Core PR B04 — Enrichment completion runway + +Depends on: coordinate overlap with B02 in `processing.ts` + +- complete artifacts and promoted resolved/search fields atomically under the same run guard; +- keep all-retryable provider failure retryable; +- allow partial artifacts with explicit error metadata; +- stop unspecified/opaque input from becoming search text by default; +- leave actual IID projection logic unplugged until public APIs arrive. + +### Core PR C03 — Golden fixture and smoke contract + +Depends on: API envelope frozen; package outputs can be added incrementally + +- add package-neutral fixtures for P0 ISRC, typed MBID, polymorphic Wikidata, invalid/lookalike IID, legacy JSON, URI order/duplicates/unsafe bytes, and resolution states; +- consume presentation cases in API/Explorer immediately; +- add opt-in smoke expectations for IID/context without replacing the stable legacy testnet window; +- verify canonical bytes/hashes against packed public packages as soon as they are available. + +## Package join: hours 10–20 + +Start integration from packed tarballs as individual package PRs stabilize; do not wait for the final NPM publish. + +### Join J01 — `iid` into atom-parser + +- install the packed public artifact in a clean integration worktree; +- recognize a valid IID before URL/string fallbacks; +- delegate all grammar/canonicalization to the package; +- promote `raw_type = iid` only after successful public validation; +- map public types into the Core DTO; +- run package and Core conformance fixtures together. + +### Join J02 — registry into classification and enrichment + +- consume `iid-registry` and updated classifications from packed tarballs; +- add deterministic IID-first classification; +- add identifier-first provider plans/hints; +- keep provider clients in Core but all mapping in the public registry; +- assert provider-slug alignment and unknown/polymorphic behavior. + +### Join J03 — public `ids`/`protocol` parity + +- replace duplicate KG atom/triple hash implementations with thin public `ids` wrappers; +- compare Core's exact contracts ABI with public `protocol` URI ABI; +- run calldata/event fixture parity; +- do not commit `file:` or Git dependencies. + +### Join J04 — builder/seed/writer validation + +- consume the public primitive builder and protocol helper in a clean seed/writer harness; +- compare predicted atom ID, aligned arrays, URI manifest, and decoded receipt; +- keep production writer flags off until the reader gate passes. + +## Final gate: hours 18–24 + +1. Rebase/merge lanes in dependency order: migrations -> ABI/ingestion -> typed reader -> projections -> semantic adapters -> readers. +2. Run package, TypeScript workspace, Rust workspace, migration, ABI drift, and clean-tarball conformance gates. +3. Run devnet golden transaction with a canonical, valid ISRC—not the invalid illustrative `int:src:*` string. +4. Confirm atom ID is unchanged when URI context changes. +5. Confirm `AtomCreated` and `AtomContextRegistered` associate by `termId` through raw, typed, KG, API, and Explorer layers. +6. Replay ingestion/projections and compare row counts/content. +7. Confirm legacy JSON, URL, IPFS, string atoms and existing `createAtoms` behavior. +8. Deploy readers dark, writers disabled. + +## Staffing and collision map + +| Engineer | Primary lane | Avoids | +| --- | --- | --- | +| Contract/indexer | A01–A03 | KG schema until A03 handoff | +| Projection/data | A04 + context KG migration | node identity migration files without coordination | +| Parser/worker | B01 + B03 | public semantic implementations | +| KG/backend | B02 + B04 | event Timescale migration | +| API/explorer | C01 + C02 | parser/worker internals | +| Integration/release | C03 + J01–J04 + final gate | feature ownership | + +Only one owner at a time edits: + +- `packages/database-kg/src/actions/processing.ts` (B02/B04); +- Drizzle journal/snapshot files (B02/A04); +- Rindexer generated code (A02 only); +- root lockfile (A01, then package joins in a clean coordinated update); +- shared `ParsedEvent` exhaustiveness (A03). + +## Work that must wait + +- Exact IID parser/canonicalizer until `@0xintuition/iid` is available. +- Scheme-to-classification/provider behavior until `iid-registry` is available. +- Resolved semantic field projection until classification contracts are frozen. +- Production URI/IID writers until indexed reader proof passes. +- NPM dependency commits until versions satisfy Core's release-age policy, unless an explicit separately reviewed exception is approved. + +## Stop conditions + +- Contract ABI differs between `contracts-v2`, public protocol, Core, or Rindexer. +- Canonical bytes or atom ID differ across package/Core fixtures. +- A Core PR introduces its own IID scheme/canonicalization/provider table. +- Context ingestion drops, reorders, or requires UTF-8 URI bytes. +- A migration requires destructive rollback. +- Reader compatibility breaks legacy consumers. diff --git a/.planning/codex-migration/15-reader-exposure-runbook.md b/.planning/codex-migration/15-reader-exposure-runbook.md new file mode 100644 index 0000000..5f616fe --- /dev/null +++ b/.planning/codex-migration/15-reader-exposure-runbook.md @@ -0,0 +1,128 @@ +# Semantic atom reader exposure runbook + +This runbook controls only the additive API and Explorer read model introduced +for IID identity, atom context, resolution state, and resolved display fields. +It does not control chain ingestion, worker behavior, or `POST /api/atoms`, +which remains an off-chain KG insert endpoint. + +## Controls + +| Boundary | Variable | Default | Effect | +| --- | --- | --- | --- | +| Query API | `API_ATOM_SEMANTIC_READS_ENABLED` | `false` | Adds `raw`, optional `identity`, `classification`, optional `context`, optional `resolution`, and optional `display` to atom responses and expanded triple terms. | +| Explorer | `VITE_ATOM_SEMANTIC_READS_ENABLED` | `false` | Prefers resolved labels/images and renders optional identity/context sections. | + +Both controls accept only `true`/`1` as enabled. The API rejects ambiguous +values at startup. The Explorer treats every other value as disabled. + +## Preconditions + +Before enabling either boundary: + +1. Legacy API contract tests and Explorer tests pass. +2. IID parsing evidence is populated only by the authoritative parser. +3. Context reads preserve on-chain order and raw evidence; an unavailable + context reader omits the field instead of returning a fabricated empty list. +4. Resolution/display fields carry the expected source and freshness evidence. +5. Unsafe context values render as text. Only credential-free HTTP(S) values + become external links. + +## Enable sequence + +1. Deploy the API build with `API_ATOM_SEMANTIC_READS_ENABLED=false`. +2. Verify legacy responses for atom list, detail, and expanded triples. +3. Set `API_ATOM_SEMANTIC_READS_ENABLED=true` on one API canary and restart it. +4. Compare the same legacy and IID fixtures through the canary. Confirm all + legacy fields remain byte-for-byte compatible and optional fields appear + only when backed by evidence. +5. Roll the API flag to the remaining replicas. +6. Build/start one Explorer canary with + `VITE_ATOM_SEMANTIC_READS_ENABLED=true`. +7. Verify unresolved IID, resolved IID, legacy JSON, empty-context, unsafe URI, + and expanded-triple states. +8. Promote the Explorer build after consumer sign-off. + +API exposure should precede Explorer exposure. The two independent gates make +it possible to inspect the wire contract before changing visible labels. + +For the local Compose API canary, use: + +```bash +API_ATOM_SEMANTIC_READS_ENABLED=true \ + docker compose up -d --no-deps --force-recreate api +docker compose logs --tail=100 api +``` + +The Explorer control is consumed by Vite. It requires a new dev process or +production build; changing the variable underneath an already-built bundle has +no effect: + +```bash +cd apps/explorer +VITE_ATOM_SEMANTIC_READS_ENABLED=true \ + VITE_API_URL="$API_URL" \ + bun run build +bun run start +``` + +## Stop conditions + +Immediately stop exposure when any of these occurs: + +- a legacy response field disappears or changes meaning; +- context is reported empty when the reader is unavailable; +- URI order/provenance is lost or an unsafe value becomes clickable; +- unresolved IIDs cause a blank or crashing atom page; +- raw IID replaces an available resolved display name; +- expanded triple terms leak non-public atom data; +- API error rate or latency exceeds the existing service budget. + +## Rollback + +1. Set `VITE_ATOM_SEMANTIC_READS_ENABLED=false` and restore/restart the previous + Explorer build configuration. This immediately restores raw-data labels and + hides identity/context sections. +2. Set `API_ATOM_SEMANTIC_READS_ENABLED=false` on API replicas and restart or + roll them. Atom endpoints then return their legacy shapes. +3. Do not roll back additive schema, context ingestion, or stored evidence + unless those components have an independent incident. +4. Keep IID/URI writers disabled and reconcile any already-submitted writes + through their transaction ledger before resuming rollout. + +Local Compose API rollback is explicit and does not touch databases: + +```bash +API_ATOM_SEMANTIC_READS_ENABLED=false \ + docker compose up -d --no-deps --force-recreate api +docker compose logs --tail=100 api +``` + +Rebuild/restart Explorer with `VITE_ATOM_SEMANTIC_READS_ENABLED=false`, or +redeploy the last known-good bundle. No schema rollback or data deletion is +part of this procedure. + +## Verification commands + +```bash +curl -fsS "$API_URL/api/atoms?limit=1" +curl -fsS "$API_URL/api/atoms/$ATOM_ID" +curl -fsS "$API_URL/api/triples/$TRIPLE_ID?expand=terms" +``` + +With API exposure disabled, `raw`, `identity`, `classification`, `context`, +`resolution`, and `display` are absent unless a field with the same name was +already part of the legacy database row. With exposure enabled, raw legacy +fields remain and the semantic envelope is additive. + +## Metrics decision + +Core currently has no Prometheus registry or normalized-route instrumentation +inside `services/api`, and browser-side counters would be incomplete and easy +to distort. This PR therefore does not invent an in-memory metric path solely +for rollout. Use existing API platform request/error/latency telemetry plus +fixture comparisons during the canary. + +When API Prometheus instrumentation is introduced, add bounded-cardinality +counters using only controlled labels such as `surface`, `enabled`, and +`fallback_kind`. Never label by atom ID, IID, URI, query text, provider payload, +or error message. Worker and projection metrics remain owned by their services. diff --git a/.planning/codex-migration/16-live-package-join-board.md b/.planning/codex-migration/16-live-package-join-board.md new file mode 100644 index 0000000..038083c --- /dev/null +++ b/.planning/codex-migration/16-live-package-join-board.md @@ -0,0 +1,314 @@ +# Live package-join board: next 24 hours + +Status: active integration control plane; see the authoritative Core completion stack in [17-core-cutover-completion-plan.md](./17-core-cutover-completion-plan.md) + +Package/Core snapshot: 2026-08-12 19:48 UTC / 12:48 PDT + +Package source inspected read-only: `/Users/metasudo/workspace/intution/workspace/packages` + +Umbrella release PR: [0xIntuition/packages#15](https://github.com/0xIntuition/packages/pull/15) + +This board is the operational companion to [the 24-hour execution plan](./14-24-hour-parallel-execution.md). It records what is verifiably available now, what exists only on an open child branch, and what has not been built. Recheck the package repository, PRs, and NPM immediately before every join because this is a live release train. + +## Executive state + +- **J01 can begin in a draft Core worktree** against a clean `@0xintuition/iid@0.1.0-alpha.0` tarball built from integration commit `501b770cedaae8a40672204fc5f96069bc1426c7`. +- **J02 can begin as a packed-tarball draft.** Classification reconciliation and `@0xintuition/iid-registry@0.1.0-alpha.0` are merged into the clean integration branch at `d1aebd3`; Core's DTOs and switches are ready, but the switches are not wired into runtime behavior. +- **J03a is implemented and green in Core** against the already-published and aged `@0xintuition/ids@0.1.0-alpha.0`; exact known answers cover UTF-8, raw hex, IID-shaped strings, legacy JSON, and triples. **P06-P07 are now implemented and green in the local packages worktree**, so J03b has moved from API-blocked to review/commit and packed-export conformance. +- **J04 can pack both producer APIs now.** `@0xintuition/primitives@0.1.0-alpha.1` is merged at `6a484dc`, while the URI-aware `@0xintuition/protocol@3.1.0` helper is implemented in the local integration worktree. The remaining package-side gate is an immutable reviewed commit and publication, not helper design. +- **No new IID package can enter Core's normal lockfile today.** NPM returned `E404` for `@0xintuition/iid`, `@0xintuition/iid-spec`, `@0xintuition/iid-registry`, `@0xintuition/classifications@0.1.0-alpha.1`, `@0xintuition/primitives@0.1.0-alpha.1`, and `@0xintuition/protocol@3.1.0` at the snapshot time. Packed-tarball work is development evidence, not a publish substitute. + +## Verified release-train snapshot + +The committed package integration base is `update/v1.1.0-alpha` at `6a484dc`; the active working copy contains the uncommitted P06-P07 and release-gate changes described below. Package release evidence must still come from a clean detached worktree at the exact reviewed commit being tested. + +| Item | Verified state | Commit / branch | Operational meaning | +| --- | --- | --- | --- | +| Public `main` | clean reference | `46cd2f0` / `main` | Does not contain the IID release train. | +| Umbrella PR 15 | remote draft CI is green; local committed integration branch plus uncommitted P06-P07 work | `update/v1.1.0-alpha` at `6a484dc` -> `main` | Integration base contains P00-P05; P06-P07 and release-gate changes must be split/reviewed/committed and are not assumed released. | +| P00 / PR 16 | merged into integration branch | child head `5e05256`; integration commit `2de35cc` | Release registry, pack/smoke tooling, and planning are committed. | +| P01 / PR 17 | merged into integration branch | child head `3a7a4ee`; integration commit `be96936` | `iid-spec@0.1.0-alpha.0` source and 145 conformance vectors are committed, but unpublished. | +| P02 / PR 18 | merged into integration branch | child head `338bf79`; integration commit `501b770` | `iid@0.1.0-alpha.0` source and inspection API are committed, but unpublished. | +| P03 / PR 19 | merged into integration branch | integration commit `9f208ef` | Public classification/IID type ownership is now an integration-branch contract; consumer tarball evidence is still required. | +| P04 / PR 20 `iid-registry` | merged into integration branch | integration commit `d1aebd3` | Pure classification, provider-slug, and identifier-hint APIs exist; J02 may begin its thin packed-tarball adapter. | +| P05 IID primitive builder | merged into integration branch | integration commit `6a484dc` | `buildIidAnchor` emits deterministic data/dataHex/id, profile, IID, provider plan, and normalized URI manifest; J04 still requires a clean immutable integration pack containing reviewed P07. | +| P06–P07 protocol URI work | implemented and locally green; not committed/published | local `update/v1.1.0-alpha` worktree on `6a484dc` | Exact contracts-v2 artifact sync/provenance, `createAtomsWithUris`, URI-config read, and URI event/context helpers pass tests, artifact drift checks, build, pack, and clean tarball smoke. Review and split/land the changes before consumer adoption. | +| P08 FeeProxy | conditional | planned branch `feat/protocol-fee-proxy-uris` | Not on the direct MultiVault critical path. | +| P09 React URI flow | planned, not built | planned branch `feat/react-atom-uris` | Not a Core join dependency. | + +P03-P05 are part of the integration branch; P06-P07 are applied but still uncommitted. J02 must consume public packed exports, map registry results into Core-owned DTOs, and persist exact package provenance; it must not copy registry tables into Core. J04 can exercise local P05/P07 tarballs now, but production adoption still requires reviewed immutable producer SHAs and eligible published versions. + +## Verified versus planned public APIs + +| Package/API | State | Safe consumer assumption | +| --- | --- | --- | +| `@0xintuition/iid-spec@0.1.0-alpha.0` | committed on integration branch; unpublished | Normative fixture/source input for conformance; not an installable NPM dependency yet. | +| `@0xintuition/iid@0.1.0-alpha.0` | committed on integration branch; unpublished | `inspectIntuitionId` and the types below are verified at `501b770`; use only a clean tarball until publication. | +| `@0xintuition/classifications@0.1.0-alpha.1` | merged into integration commit `9f208ef`; publication not assumed | Use only a clean integration-branch tarball until publication/age eligibility. | +| `@0xintuition/iid-registry@0.1.0-alpha.0` | merged into integration commit `d1aebd3`; publication not assumed | `classificationForIid`, `providersForIid`, and `identifierHintsForIid` are available as pure public APIs; use a clean tarball for draft integration. | +| `@0xintuition/primitives@0.1.0-alpha.1` IID anchor builder | merged into integration commit `6a484dc`; unpublished | `buildIidAnchor` is ready for clean packed-tarball consumption; full writer adoption waits for reviewed/landed P07 and publication. | +| `@0xintuition/ids@0.1.0-alpha.0` | existing and published 2026-06-15 | `calculateAtomId`, `calculateTripleId`, and `calculateCounterTripleId` are verified public exports and have passed Core's 14-day age window. | +| `@0xintuition/protocol@3.1.0` URI APIs | implemented in the local integration worktree; unpublished | `createAtomsWithUris`, encoding, `getAtomUriConfig`, `AtomContextRegistered`/config parsers, and creation-context joining are available for packed-tarball conformance; do not treat them as released until landed and published. | + +## Exact IID inspection contract + +The following union is verified in `packages/iid/src/types.ts` at integration commit `501b770`: + +```ts +type IidInspection = + | { + readonly valid: true + readonly iid: IntuitionId + readonly scheme: SchemeName + readonly value: string + readonly class: 'A' | 'B' | 'C' + readonly typing: 'unambiguous' | 'polymorphic' + readonly anchorEligible: boolean + readonly anchorIneligibilityReason?: 'class-c' | 'polymorphic-scheme' + } + | { + readonly valid: false + readonly reason: 'malformed' | 'unknown-scheme' | 'noncanonical' + readonly scheme?: string + readonly value?: string + readonly canonical?: IntuitionId + } +``` + +`inspectIntuitionId(input)` is a synchronous, offline export. It recognizes the full IID grammar, requires a registered scheme, checks canonical form, and reports P0 anchor eligibility. The valid `iid` field is the canonical IID. `profile` is intentionally absent: P0/P1/P2 describes an atom representation, not the normalized identity. + +Do not substitute these nearby APIs: + +- `isIntuitionId`/`parseIntuitionId` recognize registered shape but do not prove canonical form. +- `validateIntuitionId` proves canonical validity but discards the inspection metadata Core needs. +- `canonical` on an invalid `noncanonical` result is a read-repair hint, not permission to accept a new write. + +### Exact mapping into Core + +| Public valid inspection / adapter input | Core `NormalizedAtomIdentity` | Rule | +| --- | --- | --- | +| original function input | `raw` | Preserve byte-for-byte string input passed to inspection. | +| `inspection.iid` | `canonical` | Direct mapping; never reconstruct locally. | +| `inspection.scheme` | `scheme` | Direct mapping; no Core prefix/scheme table. | +| `inspection.value` | `value` | Direct mapping; values may contain additional colons. | +| no inspection field | `profile` | Omit. Set only when a later public builder/representation parser supplies P0/P1/P2 context. | +| `inspection.class` | `class` | Direct optional persistence mapping. | +| `inspection.typing` | `typing` | Direct optional persistence mapping. | +| `inspection.anchorEligible` | `anchorEligible` | Direct mapping. Do not derive from class/typing in Core. | +| `inspection.anchorIneligibilityReason` | `anchorIneligibilityReason` | Copy when present; omit otherwise. | +| adapter build metadata | `provenance.producer` | Fixed value `@0xintuition/iid/inspectIntuitionId`. | +| exact consumed package version | `provenance.version` | Fixed value `0.1.0-alpha.0` for this tarball/release; assert against the exported package manifest. | +| exact normative fixture version | `provenance.specificationVersion` | Use `@0xintuition/iid-spec@0.1.0-alpha.0` when the adapter has verified that corpus; otherwise omit. | + +The adapter should be this thin: + +```ts +function toCoreIdentity(raw: string, inspection: IidInspection): NormalizedAtomIdentity | null { + if (!inspection.valid) return null + + return { + raw, + canonical: inspection.iid, + scheme: inspection.scheme, + value: inspection.value, + class: inspection.class, + typing: inspection.typing, + anchorEligible: inspection.anchorEligible, + ...(inspection.anchorIneligibilityReason + ? { anchorIneligibilityReason: inspection.anchorIneligibilityReason } + : {}), + provenance: { + producer: '@0xintuition/iid/inspectIntuitionId', + version: '0.1.0-alpha.0', + specificationVersion: '@0xintuition/iid-spec@0.1.0-alpha.0', + }, + } +} +``` + +Invalid inspections do not become `NormalizedAtomIdentity`: + +| Invalid reason | Core read behavior | Core write behavior | +| --- | --- | --- | +| `malformed` | Preserve legacy parser fallback and bounded reason metrics. | Reject as IID; never promote `raw_type='iid'` or `iid`. | +| `unknown-scheme` | Preserve legacy fallback; do not invent a scheme mapping. | Reject as IID. | +| `noncanonical` without repair | Preserve diagnostic/fallback. | Reject as IID. | +| `noncanonical` with `canonical` repair | A separately designed historical-read path may re-inspect the returned canonical value and record repair provenance. | Reject the original new write; never silently canonicalize it into an anchor. | + +## Tarball and publication gate + +### Current registry fact + +At 2026-08-10 19:41 UTC, `npm view` returned `E404` for all three new package names: `@0xintuition/iid`, `@0xintuition/iid-spec`, and `@0xintuition/iid-registry`. `@0xintuition/ids@0.1.0-alpha.0` is published and reported integrity `sha512-ILFtC1ldVBvQsLGMZ9Tu+rY+VpUYJFqczFbQD/tMTi0vjBjbRoF8LJ/j8iaTJhJOvqBB3yx7sC/dRsAGaEwwwQ==`. + +### Gate T1 — producer commit + +- [ ] Package owner supplies a clean detached worktree at the exact integration commit. +- [ ] `git status --short` is empty; do not pack an unmerged child branch as if it were the integration release. +- [ ] Record commit SHA, package name/version, build tool versions, tarball filename, shasum, integrity, and unpacked file list. +- [ ] Re-run the package's unit, typecheck, Biome, conformance, package-registry, pack-dry-run, and clean-tarball smoke gates. + +For J01 today, the producer SHA is `501b770cedaae8a40672204fc5f96069bc1426c7` and the expected tarball name is `0xintuition-iid-0.1.0-alpha.0.tgz`. + +### Gate T2 — clean Core consumer + +- [ ] Install the supplied tarball in a disposable Core worktree or temporary clean consumer. +- [ ] Import only the package's declared public entrypoint; never import `src/` or workspace aliases. +- [ ] Run the shared valid, malformed, unknown, noncanonical, colon-bearing, Class C, and polymorphic fixtures. +- [ ] Compare outputs against the package conformance corpus and Core DTO expectations. +- [ ] Do not commit a tarball, `file:` dependency, Git dependency, workspace link, or tarball-resolved lockfile. + +### Gate T3 — publish + +- [ ] Merge the completed package train through PR 15 and publish in topological order: `iid-spec`, `iid`, updated classifications, `iid-registry`, updated primitives, protocol, then React. +- [ ] After each publish, verify `npm view @ version dist.integrity dist.tarball time --json` and install it in a clean consumer. +- [ ] Record the immutable integrity and the timestamp when Core's 14-day minimum-release-age window ends. +- [ ] Use exact package versions in Core; update the shared lockfile once in a coordinated package-join PR. +- [ ] No join PR becomes normally mergeable before the version is published and age-eligible. A time-bounded exception requires separate Core maintainer and security/release approval; it is not implied by this 24-hour sprint. + +## PR-ready join lanes + +### J01 — IID parser adapter + +Status: **ready for draft implementation from a clean P02 tarball; publish/age gated for normal merge** + +Owner: parser/worker engineer. Required review: public IID owner plus KG persistence owner. + +Branch/PR: `feat/core-iid-parser-adapter`, one focused PR against the active Core integration branch. + +Steps: + +- [ ] Pass T1/T2 for `@0xintuition/iid@0.1.0-alpha.0` at `501b770`. +- [ ] Add IID inspection before generic URL/plain-string fallback, delegating all recognition and canonicalization to `inspectIntuitionId`. +- [ ] Map only `valid: true` results through the exact adapter above. +- [ ] Keep invalid results backward compatible; persist bounded reason codes without storing raw IIDs in metrics. +- [ ] Populate `CompactParseResult.kind='iid'` and `identity`; promote `rawType='iid'` and canonical `iid` atomically only after valid inspection. +- [ ] Keep remote fetch disabled for IID/custom schemes and preserve legacy JSON, HTTP(S), IPFS, ENS, address, ISBN, and string paths. +- [ ] Keep `WORKERS_IID_READ_ENABLED` default off; prove disabled behavior is byte-compatible. +- [ ] After T3 and release-age eligibility, add the exact NPM dependency and coordinated lockfile update; remove all temporary tarball setup. + +Checks: + +- IID package conformance vectors and public-entrypoint import. +- Atom-parser fixture suite, worker tests/typecheck/Biome, database-KG tests. +- Known valid ISRC, colon-bearing value, `int:src:*` unknown scheme, repairable noncanonical, Class C, and polymorphic cases. +- Assertion that `profile` is absent unless a representation-aware public API supplies it. +- Assertion that atom ID/raw data do not change during recognition. + +Stop conditions: any locally added IID grammar/scheme table, different canonical output, invalid input promoted to IID, raw IID used as search text, or package provenance mismatch. + +### J02 — semantic registry, classification, and identifier-first enrichment + +Status: **ready for packed-tarball draft from integration commit `d1aebd3`; publication/age gated for normal merge** + +Owner: semantic adapter/enrichment engineer. Required review: classifications owner, registry owner, provider-runtime owner. + +Branch/PR: `feat/core-iid-semantic-adapter`; do not open with hand-authored mappings. + +Steps: + +- [ ] Require PR 19 to merge, then require a clean reviewed `classifications@0.1.0-alpha.1` tarball from the new integration head. +- [ ] Require a clean `iid-registry@0.1.0-alpha.0` tarball and freeze its classification/provider/hint result types. +- [ ] Map registry output into Core `IdentityClassificationDecision` and `IdentityProviderPlan`; registry owns every scheme/value decision and provider ordering. +- [ ] Preserve explicit `classified`, `unmapped`, and `ambiguous` outcomes without converting them to `Unknown` guesses. +- [ ] Translate provider capabilities/hints into existing Core clients; add no provider HTTP client to the public registry package. +- [ ] Carry identity, decision, plan, errors, skips, and provenance across persisted worker handoffs. +- [ ] Keep `WORKERS_IID_RESOLUTION_ENABLED` default off and retain partial-artifact/all-retryable completion semantics. +- [ ] Plug in the public resolved-projection API only when it exists; until then leave IID `dataResolved/searchText` projection unplugged. + +Checks: + +- Registry totality over all public schemes and classification-slug existence. +- Golden ISRC classification/provider order; polymorphic WD; value-narrowed MBID/OLID/CAIP-19; explicit unmapped ISWC. +- No network I/O from classification/registry packages. +- Worker deserialization compatibility, retry/partial behavior, bounded metrics, and no raw identifiers in labels. +- Packed public exports only; exact provenance versions persisted. + +Stop conditions: registry package absent, provider/classification maps copied into Core, private `AtomCategory` leaked into public contracts, provider slug mismatch, or resolved projection lacks deterministic provenance. + +### J03 — public IDs and protocol parity + +Status: **J03a implemented and verified; P06/P07 unblock J03b locally, with immutable packed-export parity still required** + +Owner: protocol/KG parity engineer. Required review: IDs owner, Solidity contract owner, Core indexer owner. + +Branch/PRs: + +1. `refactor/core-public-id-wrappers` — independent J03a. +2. `feat/core-public-protocol-uri-parity` — J03b after P07. + +J03a steps: + +- [x] Exact-pin the already-aged `@0xintuition/ids@0.1.0-alpha.0`. +- [x] Replace duplicate KG atom/triple hashing bodies with thin `calculateAtomId`/`calculateTripleId` wrappers while preserving Core boundary normalization and exported names. +- [x] Compare string UTF-8 and hex-byte paths, including canonical IID strings and legacy JSON. +- [x] Update the lockfile once and run the supply-chain guard. + +J03b steps: + +- [ ] Require clean P06 artifact provenance and a packed `protocol@3.1.0` P07 API. +- [ ] Compare exact function/event/error signatures against contracts-v2, Core vendored ABI, and Rindexer ABI. +- [ ] Verify `createAtomsWithUris(address,bytes[],uint256[],bytes[][])`, `getAtomUriConfig`, `AtomContextRegistered`, and legacy `createAtoms` parity. +- [ ] Add calldata/receipt fixtures proving URI order/duplicates/opaque bytes and `termId` association without log adjacency. +- [ ] Do not couple direct MultiVault readiness to conditional FeeProxy P08. + +Checks: + +- Known-answer atom/triple IDs for UTF-8, hex, IID, legacy JSON, and triples. +- Package tests, database-KG tests, ABI drift checks, Rindexer Rust tests, TypeScript/Rust typechecks, supply-chain guard. +- Atom ID identical when URI context changes. + +Stop conditions: any ID mismatch, ABI mismatch, Git/file dependency, URI included in ID material, or event association by log position. + +### J04 — canonical builder, seed, and writer validation + +Status: **P05 integrated at `6a484dc`; P07 is locally implemented, so writer integration is blocked on review/publication rather than API availability** + +Owner: seed/writer integration engineer. Required review: primitives owner, protocol owner, Core reader owner, release owner. + +Branch/PR: `test/core-iid-builder-writer-harness` first; production writer changes remain separate and gated. + +Steps: + +- [ ] Prepare a package-neutral golden harness now: canonical valid ISRC input, expected profile, bytes, atom ID, ordered URI bytes, classification decision, provider plan, and receipt events. +- [ ] Require packed P05 primitive builder and P07 protocol helper; freeze their public result/input shapes before wiring writers. +- [ ] Feed builder output directly into protocol simulation/encoding without reserializing identity or URI arrays in Core. +- [ ] Compare predicted atom ID to public IDs output and decoded receipt term ID. +- [ ] Prove URI manifest changes do not alter atom data or ID and that outer data/assets/URI arrays stay aligned. +- [ ] Exercise empty URI lists, configured limits, duplicates/order, unsafe UTF-8 bytes, simulation failure, and replay. +- [ ] Keep every production writer flag off until URI ingestion, KG/API/Explorer reads, search/display, rollback, and backfill gates pass. +- [ ] Inventory every real writer and seed pipeline; no second ad hoc IID serializer may remain. + +Checks: + +- Clean-tarball Node and Bun harness using public exports only. +- Seed dry run and deterministic manifest snapshot. +- Protocol calldata decode and receipt association by `termId`. +- Core legacy JSON/createAtoms regression suite and end-to-end reader fixture. +- Writer-disabled deployment smoke and rollback exercise. + +Stop conditions: builder API not frozen, P0 emitted for Class C/polymorphic identities, URI context affects ID, a production writer bypasses the builder, or reader gate is incomplete. + +## 24-hour ownership and handoff cadence + +| Time | Package/release owner | J01 owner | J02 owner | J03 owner | J04 owner | +| --- | --- | --- | --- | --- | --- | +| Hour 0–4 | Pack P01-P05 from `6a484dc`; implement P06 | Implement thin inspection adapter behind the real flag | Implement registry DTO mapping | Keep J03a green; review P06 ABI | Freeze package-neutral fixture and pack P05 | +| Hour 4–10 | Complete P06 and implement P07 | Complete parser/KG conformance | Add identifier-hint/provider-plan adapter | Begin J03b against packed P07 | Build protocol/builder harness | +| Hour 10–16 | Run P10 clean-room package conformance | Rebase after adapter review | Complete provider coverage and retry tests | Complete ABI/calldata/event parity | Complete deterministic writer dry run | +| Hour 16–20 | Complete P11 docs and P12 release evidence | Full legacy/IID regression | Full worker/provider regression | Full TypeScript/Rust/ABI regression | End-to-end golden fixture; writers remain off | +| Hour 20–24 | Decide publish readiness; record integrities/timestamps | Join Core completion stack | Join API/context reader evidence | Join protocol parity evidence | Final no-Docker gate and deferred-environment ledger | + +The integration/release owner updates this board at each package merge or publish. Replace snapshot facts rather than appending contradictory status notes. + +## Final join checklist + +- [ ] Every consumed API is linked to a committed package SHA and packed public export. +- [ ] P03/P04/P05/P07 are clearly either landed or still blocking their join. +- [ ] Tarball names, shasums, integrities, and producer SHAs are recorded. +- [ ] NPM publication metadata and 14-day eligibility timestamps are recorded. +- [ ] Core contains no Git, `file:`, workspace, tarball, or source-path dependency. +- [ ] All package versions are exact and appear once in the coordinated lockfile. +- [ ] J01/J02/J03/J04 tests pass independently and as one golden end-to-end fixture. +- [ ] Legacy JSON, URL, IPFS, string, and `createAtoms` behavior remains supported. +- [ ] Reader/resolution/writer switches remain off until their named gates pass. diff --git a/.planning/codex-migration/17-core-cutover-completion-plan.md b/.planning/codex-migration/17-core-cutover-completion-plan.md new file mode 100644 index 0000000..1467b68 --- /dev/null +++ b/.planning/codex-migration/17-core-cutover-completion-plan.md @@ -0,0 +1,377 @@ +# Intuition Core cutover completion plan + +Status: authoritative remaining-work plan; execution update appended 2026-08-12 + +Prepared: 2026-08-11 + +Original constraint (2026-08-11): complete useful local work without Docker. Docker was explicitly authorized and started on 2026-08-12; the execution ledger below supersedes deferred-environment statements where evidence now exists. + +## Execution update — 2026-08-12 + +The implementation is substantially beyond the 2026-08-11 baseline in this document. Do not use the older R1-R7 descriptions as current code-state claims; retain them as the rationale for the PR stack. + +| Cutover capability | Current evidence | Remaining release action | +| --- | --- | --- | +| URI-aware contract API | Local v1.1 deployment passed `getAtomUriConfig`, `createAtomsWithUris`, predicted term ID, and joined creation/context events | Split/review the mixed Core diff and packages P06-P07 diff | +| Canonical fixtures | `int:isrc:USUM71703861` and `int:isbn:9780684832722` were created idempotently, indexed, projected, resolved, and read through the API | Preserve as the release/CI golden path | +| URI ingestion/projection | Raw/typed Rindexer storage and exact-term-ID KG projection passed applied chain replay and checkpoint/dependency-order verification | Serialize and archive the same evidence in CI | +| IID recognition/classification | Package-neutral inspection/registry adapters, persistence, fallback taxonomy, and fail-closed worker behavior are implemented | Publish and age-gate exact public packages, then wire the composition root | +| IID provider resolution | Total provider capability adapter, MusicBrainz ISRC, OpenLibrary ISBN/OLID, identifier-to-plugin bridge, retry taxonomy, and resolved projection are implemented | Live worker activation awaits eligible package dependencies | +| API/Explorer | Ordered context reads and exact IID clusters passed live API/database/query-plan verification; safe Explorer views build and test | Browser smoke and production-sized load evidence | +| Reconciliation | Bounded `kg-reconcile-iid` dry-run/apply CLI is implemented and passed against the live fixture database | Production report-only run, approval, bounded apply, and convergence report | +| Public packages | P00-P07 are integrated locally; frozen install, exception policy tests, artifact parity, all workspace tests/typechecks/checks/builds, generated drift checks, pack dry-run, and Node/Bun clean-room tarball smoke are green | Split/review/commit P06-P07 and release work, restore NPM credentials, publish with approval, record integrity, and qualify the exact Core runtime dependencies | + +The only hard production activation dependency is now the public package release gate. Core intentionally does not commit `file:`, tarball, Git, arbitrary-module-loader, or workspace dependencies to bypass its 14-day supply-chain policy. Local acceptance loads built sibling public exports only inside a test harness; production worker composition remains fail-closed until the exact releases are eligible. + +### Applied end-to-end evidence — 2026-08-12 + +The local acceptance loop has now crossed every Core-owned durable boundary: + +```text +createAtomsWithUris on Anvil + -> AtomCreated + AtomContextRegistered joined by termId + -> Rindexer raw + typed Timescale rows + -> core_entities exact-ID KG node + -> atom_context:dual ordered byte-preserving KG context + -> semantic atom detail + exact-IID cluster API +``` + +| Gate | Result | +| --- | --- | +| Contract deployment | v1.1 MultiVault deployed on chain 31337; URI config read as 5 values x 700 bytes | +| Canonical writer fixture | `int:isrc:USUM71703861` and `int:isbn:9780684832722` simulated, created, event-joined by term ID, then returned `already-created` on replay | +| Identifier invariance | Each term ID was derived from IID bytes only; URI values were independently verified from context events | +| Typed ingestion | Rindexer indexed blocks 26-34 and persisted 3 context events, including the canonical fixtures at typed sequences 291 and 292 | +| Indexer container | The pinned, lockfile-enforced Linux/arm64 image built successfully and replayed blocks 26-34 in Docker, exiting 0; the second replay retained 3/3 unique context events | +| Dependency ordering | `atom_context:dual` pinned at sequence 283 while its exact node was missing; after `core_entities` advanced through sequence 286, context projection advanced through 292 | +| Context projection | KG contains the exact chain transaction hashes/log index 8 and byte-preserving URI hex for both fixtures; no adjacency assumption was used | +| Live resolution | MusicBrainz resolved the ISRC to “Cut to the Feeling”; OpenLibrary resolved the ISBN to “The Sovereign Individual” / `OL7721520M` | +| Failure taxonomy | A MusicBrainz HTTP 503 and an OpenLibrary timeout were retryable; subsequent attempts succeeded | +| API read | Both atom details return separate identity, classification, resolution, display, and ordered on-chain context; exact IID cluster lookup returns only the matching atom | +| Query plan | `idx_nodes_iid` and `idx_node_contexts_node_sequence` were selected; warm executions were 0.038 ms and 0.086 ms respectively in the fixture database | +| Reconciliation | `kg-reconcile-iid` dry-run found the two canonical candidates with bounded pagination and emitted explicit parser/registry/resolver provenance | + +The developer-machine acceptance intentionally ran the already-built native projection binary after Docker Desktop terminated two simultaneous cold Rust image builds under memory pressure. The Dockerfiles now pin the Rust builder and Linux runtime, copy `Cargo.lock`, use `--locked`, and no longer swallow dependency-build failures. A serialized indexer image build/replay passed. The projection binary passed the applied E2E path and 546 unit tests; its final container build should remain a required CI artifact gate. + +### Remaining work to declare production cutover + +These are release and operations gates, not missing Core feature design: + +1. Split the mixed Core worktree into C01-C14 review units and land them in dependency order. +2. Review and land packages P06-P07, finish the public conformance/release PRs, restore NPM organization credentials (current preflight returns `E401`), publish exact versions with provenance/integrity, and start the 14-day eligibility clock. +3. After package eligibility, add exact public IID package dependencies at the worker composition root, freeze the install, and rerun the same acceptance without the test-only sibling-package harness. +4. Run the production-sized historical reconciliation in report-only mode, record counts/error taxonomy, then apply in bounded resumable batches. +5. Execute the reader-first canary: IID recognition -> semantic reads -> ISRC resolution -> ISBN resolution -> external IID-default writers. +6. Record production query/load budgets, provider rate behavior, rollback rehearsal, and release-owner approval in the evidence ledger. + +No production flag should be enabled globally before step 3. The correct interim posture is additive readers deployed, flags off, durable URI evidence retained, and legacy reads/writes available. + +Package release note: leadership approved a narrowly scoped release-age +exception for `@0xintuition/contracts-v2@1.1.0-alpha.0`. The packages worktree +now records its exact version, registry SHA-512 integrity, publication time, +and automatic removal time in `supply-chain-exceptions.json`. Bun can exempt +only by package name, so the repository guard binds the exception back to the +exact manifest version and lockfile integrity, rejects additional exclusions, +and fails after the normal 14-day age is reached on +`2026-08-18T19:56:37.884Z`. Frozen installation and +`protocol:check-artifacts` now pass. This resolves the P06 artifact-generation +gate; it does not authorize release-age exceptions for Core's unpublished IID +runtime packages. + +The package release rehearsal subsequently passed the full workspace and +clean-room tarball matrix. Registry preflight found one versioning defect before +release: React's exact dependency moved to `protocol@3.1.0`, so the candidate +was correctly advanced from the already-published +`@0xintuition/react@0.1.0-alpha.0` to the unpublished `0.1.0-alpha.1`. The exact +IID, classifications, primitives, protocol, and React candidate versions remain +unpublished; no registry mutation was performed. + +After the corrected React version and final package rebuild, Core reran the +sibling-package acceptance boundary: canonical ISRC/ISBN semantic enrichment, +cross-stack semantic identity persistence, and exact public runtime manifest +binding passed 6/6. This is valid pre-publication consumer evidence, while the +production composition root remains correctly blocked on eligible NPM +artifacts. + +### Docker shutdown checkpoint — 2026-08-12 + +The Intuition Core Compose services and the profile-managed Anvil container +were stopped, then Docker Desktop was quit to release developer-machine +resources. Containers, named database volumes, and the persisted Anvil state +were not removed. Docker-free Core tests, typechecks, checks, the supply-chain +guard, and diff validation passed after shutdown. Package policy tests and +frozen installation also remain green. Continue package review, PR separation, +documentation, and static consumer integration with Docker off; restart it +only for the final clean migration/replay/container-image gate. + +## Executive verdict + +Intuition Core is **not fully switched yet**. The current branch contains most of the additive foundation for contract URI context and IID-aware read models, but the identifier-first runtime path is still deliberately dark. + +The remaining critical path is: + +```text +publish/qualify public IID packages + | + v +IID inspection -> registry classification -> identifier/provider enrichment + | | + v v + identity persistence resolved display/search + \ / + +-> API/explorer <-+-- ordered URI context + | + v + backfill, canary, cutover +``` + +The URI event ingestion path and the identifier/enrichment path are independent until the read model. They should continue in parallel. URI values are evidence attached to a term; they do not participate in atom ID calculation and are not an initial source of automated network fetches. + +## What “fully switched” means + +The first production cutover is complete when all of the following are true: + +1. A canonical IID such as `int:isrc:USUM71703861` is recognized before the legacy string fallback and persisted with its exact public-package provenance. +2. Its classification comes from `@0xintuition/iid-registry`, not a duplicate Core scheme map. +3. Its provider targets and identifier hints drive the existing enrichment engine without requiring a source URL or JSON-LD object. +4. Raw provider artifacts remain distinct from the resolved display/search projection. +5. `AtomContextRegistered` values are joined by `termId`, persisted in ordinal order, and exposed safely on the atom detail read path. +6. API and Explorer show raw identity, classification, resolution status, display data, and URI context as distinct concepts. +7. Existing `int:` rows can be reprocessed through the same code path, and replay/reconciliation is idempotent. +8. One golden IID fixture passes builder -> hash -> protocol calldata/event -> ingestion -> resolution -> API/Explorer checks, while legacy JSON, URL, IPFS, and string fixtures remain compatible. +9. Reader and resolver switches can be enabled separately and rolled back without deleting durable evidence. + +This does **not** require removing legacy readers or writers. Legacy compatibility exits only after usage telemetry reaches zero. + +## Reconciled state on 2026-08-11 + +### Foundation present in the current Core worktree + +| Capability | State | Remaining action | +| --- | --- | --- | +| URI-enabled contract artifact and development deployment configuration | Implemented, unlanded | Split and review as C01. | +| Raw and typed `AtomContextRegistered` ingestion | Implemented, unlanded | Split and review as C02-C03; exercise against a database later. | +| Replay-safe `kg.node_contexts` projection joined by `termId` | Implemented, unlanded | Split and review as C04; expose it through API reads in C11. | +| IID database runway (`raw_type='iid'`, nullable `nodes.iid`) | Implemented, unlanded | Use it from C08; no new identity table is required for the first cutover. | +| Package-neutral identity/classification/provider DTOs | Implemented, unlanded | Wire public package adapters in C08-C10. | +| Strict IID read/resolution configuration flags | Implemented, unlanded | The flags are parsed but do not yet alter worker behavior. | +| API/Explorer semantic response envelope | Implemented, default off, unlanded | Supply real context/identity data in C11 and canary the flags in C14. | +| Public `@0xintuition/ids` hash wrappers and known-answer tests | Implemented, unlanded | Land as C07. | +| Parser generic URL safety and enrichment completion semantics | Implemented, unlanded | Land as C05 and retain legacy regression coverage. | + +These changes are currently a large mixed worktree. They are implementation progress, not release evidence, until separated into reviewable PRs and merged. + +### Public package train + +The local `packages` integration branch `update/v1.1.0-alpha` is clean at `6a484dc` and contains P00-P05. In particular, P05 is now integrated and `buildIidAnchor` is available from source. P06-P07, the contract artifact/provenance sync and direct MultiVault URI APIs, remain unimplemented. P10-P12 conformance, documentation, and publication also remain. + +At the audit time, the new IID packages, `@0xintuition/primitives@0.1.0-alpha.1`, and `@0xintuition/protocol@3.1.0` were not published on NPM. Core's normal dependency policy requires a 14-day release age. Draft integration may use packed tarballs in a disposable clean worktree, but committed Core dependencies must not use a tarball, `file:`, Git, or workspace reference. + +Recommendation: publish the package train as early as possible and do not create additional release-age exceptions. If product leadership elects to override that policy, record a separate explicit supply-chain decision; urgency alone is not an implicit exception. + +## Confirmed remaining gaps + +### R1 — IID recognition is not connected + +`WORKERS_IID_READ_ENABLED` is loaded and tested but unused by the parser worker. The worker still sends every atom through the legacy parser, so a valid P0 IID is stored and processed as a plain string. + +Required result: a thin adapter calls `inspectIntuitionId`, maps the public result to Core's `NormalizedAtomIdentity`, persists canonical IID/provenance, and falls back to legacy parsing for malformed, unknown, and noncanonical historical inputs. It must split only the first two colons and must never silently repair a new write. + +### R2 — Classification still follows JSON/URL inference + +Core does not import the public registry. An IID classification therefore cannot be inferred from its registered scheme/profile, and the dark `identityDecision`/`providerPlan` fields are never populated. + +Required result: use public registry results directly, distinguish polymorphic from ratified-but-unmapped outcomes, and preserve unknown state without guessing. Do not copy classification or scheme tables into Core. + +### R3 — Identifier-first enrichment cannot run + +The current enrichment handoff requires a URL or a structured object. For an IID-only atom, `buildClassifiedInputFromPlan` returns `null`. Public provider targets and identifier hints are not translated into the existing plugin engine. + +Required result: adapt the ordered public provider plan into Core plugin requests, pass exact identifier hints, retain existing retryable/terminal semantics, and project resolved fields only from successful artifacts. + +There is one known coverage mismatch: the public registry emits `openlibrary` for ISBN/OLID, while Core has no `openlibrary` plugin. C10 must either add that plugin or explicitly mark those provider plans unsupported. Silent dropping is forbidden. A contract test must compare every `PROVIDER_SLUGS` value with a Core adapter capability. + +### R4 — Stored URI context is not served + +`kg.node_contexts` exists and the presentation type supports context, but API list/detail queries never load that table. As a result, Explorer cannot receive the stored values. + +Required result: add an ordered context reader and include it in atom detail responses. Keep list endpoints bounded: expose a context count or omit context on lists unless a batch query is explicitly requested. Render only allowlisted HTTP(S) links as clickable; preserve opaque bytes/raw values as non-clickable evidence. + +### R5 — Identity cluster reads and reprocessing are absent + +There is no exact-IID query path and no bounded job to reprocess historical `int:` rows after the adapter or registry version changes. + +Required result: use the indexed `nodes.iid` column for exact same-IID cluster reads, add resumable parser/classifier/resolver reconciliation jobs, and persist checkpoint/version evidence. Do not add a separate identity table or indexed scheme column until a concrete query/SLO requires it. + +### R6 — Public protocol and writer conformance is incomplete + +Core's vendored contract surface is URI-aware, but the public `protocol` package is not. Core also does not own a production on-chain atom writer; `POST /api/atoms` inserts an off-chain KG row with a protocol-compatible content ID and does not submit a transaction. + +Required result: after packages P06-P07, prove ABI selector, calldata, receipt event, term ID, and URI ordering parity. Document `POST /api/atoms` as an ingestion endpoint, not a mint endpoint. Production seed/application writers remain owned by their respective repositories and must consume the public builder/protocol APIs. + +### R7 — Operational proof remains + +The repository has unit/static evidence, but no applied migration, real event replay, live provider resolution, devnet mint, or query-load evidence for this change set. Those checks require a database or chain environment and are intentionally deferred while Docker is off. + +## Core PR stack + +The first seven PRs extract the existing mixed worktree into reviewable foundations. C08-C14 finish the behavior. Each PR should target the migration branch in the listed dependency order; use stacked branches where needed and rebase after its parent merges. + +### Foundation PRs already represented in the worktree + +| PR | Recommended title | Scope | Depends on | Docker-free merge gate | +| --- | --- | --- | --- | --- | +| C01 | `chore(contracts): sync URI-enabled protocol artifacts` | Exact contracts dependency, vendored ABI/provenance, config, development deployment acceptance, docs | none | ABI/artifact checks, package tests, shell checks | +| C02 | `feat(indexer): ingest atom context events` | Rindexer ABI/bindings/config, raw event storage, handler metrics | C01 | Rust unit tests, formatting, clippy, schema generation diff | +| C03 | `feat(shared): add typed atom context event reads` | Timescale migration/schema, shared event model, typed reader | C02 | Rust/TypeScript schema tests and typed-reader fixtures | +| C04 | `feat(projections): persist IID and atom context read models` | KG IID runway, `node_contexts`, replay-safe projection by `termId` | C03 | migration snapshot tests, projection unit tests, replay fixture twice in memory | +| C05 | `refactor(workers): prepare identifier-first processing contracts` | URL safety, Core DTOs, strict flags, structured target handling, enrichment completion taxonomy | none | scoped worker/parser tests and typecheck | +| C06 | `feat(readers): add gated semantic atom presentation` | API/Explorer envelope, safe URI presentation helpers, golden presentation fixture | C04-C05 | API/Explorer unit tests, legacy response snapshots, typecheck | +| C07 | `refactor(ids): use public protocol ID helpers` | Thin `@0xintuition/ids` wrappers and exact known answers | none | hash/ID parity tests and dependency policy check | + +Do not fold C01-C07 back into one PR. The current diff crosses contract, Rust ingestion, projection, database, workers, API, and UI trust boundaries and needs separate rollback points. + +### Behavioral completion PRs + +| PR | Recommended title | Scope | Depends on | Docker-free merge gate | Deferred environment gate | +| --- | --- | --- | --- | --- | --- | +| C08 | `feat(workers): recognize canonical Intuition Identifiers` | `inspectIntuitionId` adapter, flag wiring, persistence, invalid/fallback taxonomy | C04-C05; packages P01-P02 | public-entrypoint tarball consumer, full IID conformance corpus, legacy parser regression | applied migration and live queued atom | +| C09 | `feat(workers): classify IIDs through the public registry` | classification adapter, source/version evidence, ambiguous/unmapped handling | C08; packages P03-P04 | registry fixtures and no-local-map code-search test | live classification queue/replay | +| C10 | `feat(enrichment): execute IID provider plans` | identifier hints, ordered provider adapter, OpenLibrary decision/plugin, artifact projection | C09; package P04 | mocked provider tests, provider-slug coverage, retry taxonomy, ISRC golden fixture | credentialed provider canary and retry observation | +| C11 | `feat(api): serve IID clusters and atom context` | ordered context action/query, atom detail join, exact IID cluster route/filter, Explorer detail | C04-C06 | database query-shape tests, presenter/UI tests, safe-link tests | real query plan and load test | +| C12 | `feat(operations): reconcile IID identity and resolution` | resumable reprocessing CLI/job, checkpoints, registry/resolver version selection, bounded metrics | C08-C11 | dry-run fixtures, restart/idempotency tests, metrics label cardinality test | database backfill/replay convergence | +| C13 | `test(conformance): prove IID URI creation parity` | consume public builder/protocol exports, builder/hash/calldata/event golden path, document API ingest semantics | C07-C10; packages P05-P07/P10 | packed-tarball clean consumer under Node/Bun; no source imports | devnet mint/index/read proof | +| C14 | `chore(release): stage the IID reader cutover` | config/runbook, canary allowlist, dashboards/alerts, rollback, consumer notes | C08-C13; eligible published packages | config tests, lockfile/supply checks, runbook review | migrations, replay, load, provider and devnet acceptance | + +## Dependency graph and parallel tracks + +```text +Track A — land foundations +C01 -> C02 -> C03 -> C04 -> C06 + C05 --^ C07 (independent) + +Track B — identifier runtime +P01/P02 -> C08 -> P03/P04 -> C09 -> C10 + +Track C — reader exposure +C04 + C06 -> C11 + +Track D — operations +C08 + C09 + C10 + C11 -> C12 + +Track E — public writer parity +P05 + P06/P07 -> C13 -> C14 +``` + +Recommended ownership for a concentrated sprint: + +| Owner | Primary responsibility | First deliverable | +| --- | --- | --- | +| Core integration owner | Split/sequence C01-C07, guard shared lockfile and migrations | Clean PR dependency graph and green foundation stack | +| Identity owner | C08-C09 | Canonical IID reaches persisted identity and registry classification | +| Enrichment owner | C10 | ISRC resolution plus provider-slug coverage report | +| Reader/data owner | C11-C12 | Real context detail read, IID cluster read, resumable reconciliation | +| Package/protocol owner | packages P06-P12 and C13 handoff | Packed public protocol API and end-to-end conformance evidence | +| Release owner | C14 and external seed/app coordination | Gate ledger, canary manifest, go/no-go packet | + +With four engineers, combine integration/release and reader/operations. Avoid multiple owners editing `bun.lock`, KG migration metadata, or worker orchestration simultaneously. + +## What can be completed now without Docker + +### Immediate wave: hours 0-4 + +1. Freeze the current mixed worktree and split C01-C07 without changing behavior. +2. Update the live package board: P05 is integrated; P06-P07 and publication are the package blockers. +3. Build C08-C10 against clean packed package tarballs in disposable consumers/worktrees. +4. Start C11 immediately; it depends on existing Core context persistence, not on package publication. +5. Add the public-provider/Core-plugin coverage test and settle OpenLibrary support. + +### Integration wave: hours 4-12 + +1. Complete IID inspection and registry adapter fixtures. +2. Complete identifier-hint/provider-plan execution with network mocks. +3. Add atom-detail context and exact-IID cluster reads. +4. Build the reconciliation CLI in dry-run mode with deterministic page/checkpoint fixtures. +5. Repack packages after every package merge and rerun clean-consumer tests. + +### Closure wave: hours 12-24 + +1. Land P06-P07, then complete C13 against their packed public exports. +2. Run all TypeScript/Rust unit, type, formatting, clippy, ABI, schema, supply-chain, and shell gates that do not contact Docker. +3. Produce the exact package/version/integrity manifest and release-age calendar. +4. Finalize C14 with reader-first flag order, canary scheme, rollback conditions, and deferred environment commands. +5. Publish the public package prereleases as soon as P10-P12 pass so the 14-day eligibility clock starts. + +## No-Docker verification matrix + +| Concern | Verify locally now | Verify later in CI/remote environment | +| --- | --- | --- | +| IID grammar/canonicalization | public conformance fixtures, invalid reason mapping, colon-bearing values | live queued atoms | +| Registry/classification | public tarball exports, all known/ambiguous/unmapped fixtures | historical distribution report | +| Provider planning | slug capability contract, mocked HTTP, retry/terminal semantics | credentialed API calls and rate limits | +| URI contract | ABI semantic parity, calldata/event fixtures, ID invariance | devnet transaction and receipts | +| Event ingestion | handler/storage unit tests, raw event fixture | Rindexer against real logs | +| Database | SQL/snapshot/schema tests, query generation | apply migrations, explain plans, constraints, rollback rehearsal | +| Replay/backfill | deterministic in-memory/dry-run twice | real bounded range twice and convergence diff | +| API/Explorer | response contracts, safe URI rendering, legacy snapshots | real database responses, browser smoke, load test | +| Supply chain | clean pack, public entrypoint smoke, exact versions, single lockfile copy | NPM provenance/integrity and release-age eligibility | + +No PR should claim the deferred column as complete based only on a mock. C14 owns one evidence ledger with command, environment, commit, package versions, timestamp, and result for every deferred gate. + +## Cutover order + +Use reader-before-writer rollout: + +1. Deploy migrations and URI/IID readers with semantic exposure and IID flags off. +2. Replay context events and run IID recognition/classification backfill in report-only mode. +3. Enable IID recognition for an internal allowlist; keep resolution off. +4. Enable semantic API/Explorer exposure and verify raw identity/context evidence. +5. Enable IID resolution for one unambiguous canary scheme. Recommended canary: ISRC. +6. Verify provider artifacts, display/search projection, same-IID cluster lookup, latency, retries, and rollback. +7. Expand resolver schemes by explicit capability matrix, not all at once. +8. Enable IID-default writes in external seed/application writers only after the complete reader proof. +9. Keep legacy reads and the legacy `createAtoms` contract path throughout the compatibility window. + +Rollback is flag-first: disable writers, then resolution, then semantic exposure/recognition as needed. Do not delete identity, URI, event, or provider evidence during rollback. + +## Decisions frozen for the first cutover + +These defaults minimize the 24-hour critical path. Change them only through an explicit decision record. + +- Direct MultiVault URI creation is in scope; FeeProxy URI support is deferred and does not block Core readers. +- React writer changes are not a Core dependency because this repository has no supported production mint UI. +- `POST /api/atoms` remains off-chain KG ingestion and must be documented accordingly. +- Context URIs are stored and displayed safely but are not automatically fetched during the initial cutover. +- `nodes.iid` is the first-cutover identity cluster key; no new identity table is required. +- Exact IID lookup and atom-detail context are required; scheme/status filters are deferred until backed by an indexed query need. +- ISRC is the first resolver canary; wider registry coverage is enabled only when Core has a tested provider capability or an explicit unsupported result. +- No new package release-age exceptions are assumed. + +## Final go/no-go checklist + +### Code-complete without Docker + +- [ ] C01-C07 are split, reviewed, and green. +- [ ] C08 recognizes valid canonical IIDs behind the real worker flag. +- [ ] C09 classifies exclusively through public registry exports. +- [ ] C10 resolves an IID from identifier hints and has total provider-slug accounting. +- [ ] C11 exposes ordered URI context and exact same-IID cluster reads. +- [ ] C12 dry-runs and resumes deterministically without duplicate promotion. +- [ ] C13 passes the packed-package golden path without private/source imports. +- [ ] Legacy parser, classification, enrichment, API, and contract paths remain green. +- [ ] Exact dependency, lockfile uniqueness, integrity, and release-age policy pass. + +### Environment acceptance before production + +- [ ] Timescale and KG migrations apply cleanly and constraints/indexes are verified. +- [ ] A real `AtomContextRegistered` range replays twice with identical projections. +- [ ] Historical IID backfill and reconciliation converge with a recorded report. +- [ ] A credentialed ISRC resolves within the agreed latency/error budget. +- [ ] Atom detail returns real ordered context; list/cluster queries meet the query budget. +- [ ] A URI-aware devnet mint produces the predicted term ID and indexed read model. +- [ ] Reader, resolver, semantic exposure, and external writer rollback switches are rehearsed. +- [ ] Release owner signs the evidence ledger before IID-default production writes begin. + +## Immediate next action + +Start C11 and the C08-C10 packed-tarball adapters in parallel while the package owner implements P06-P07. In the same window, split the existing foundation into C01-C07 so none of the behavioral work grows on top of an unreviewable mixed diff. diff --git a/.planning/codex-migration/index.md b/.planning/codex-migration/index.md new file mode 100644 index 0000000..92667ee --- /dev/null +++ b/.planning/codex-migration/index.md @@ -0,0 +1,107 @@ +# Intuition Core IID and atom-context migration + +Status: active comprehensive program; current completion tracking is in [17-core-cutover-completion-plan.md](./17-core-cutover-completion-plan.md) + +Prepared: 2026-08-10 + +Scope: Intuition Core, `0xIntuition/packages`, contract artifacts, indexing, data APIs, explorer, seed preparation, and atom creation + +This packet is the implementation and coordination plan for making Intuition Identifiers (IIDs) the default identity representation for new atoms and for carrying the contract's URI context through the complete Core stack. It supersedes the earlier assumption that the whole migration is a one-to-two-day change. A focused integration sprint is still useful, but it sits inside a stage-gated cross-repository release program. + +## Program at a glance + +The migration has two independent inputs that meet in Core: + +```text +public semantic packages URI-enabled contracts +iid -> classifications -> registry ABI -> event ingestion + | | + +-----------> Core <-------------------+ + | + recognize -> resolve -> project -> serve + | + seed and mint writers +``` + +The governing rule is **read before write**: Core must recognize, store, resolve, search, and display IIDs and URI context before production writers make IID atoms the default. + +## Reading order + +### Architecture baseline + +1. [00-executive-overview.md](./00-executive-overview.md) — problem, target state, and critical path +2. [01-target-architecture.md](./01-target-architecture.md) — shared data contracts and end state +3. [02-impact-inventory.md](./02-impact-inventory.md) — repository and subsystem impact +4. [03-engineering-tracks.md](./03-engineering-tracks.md) — initial ownership model +5. [04-cutover-runbook.md](./04-cutover-runbook.md) — rollout, backfill, and rollback baseline +6. [05-decisions-risks-and-tests.md](./05-decisions-risks-and-tests.md) — original decisions and acceptance matrix +7. [06-scheme-resolution-matrix.md](./06-scheme-resolution-matrix.md) — IID-to-type and resolver routing + +### Comprehensive execution program + +8. [07-reference-implementation-analysis.md](./07-reference-implementation-analysis.md) — what the private implementation proves, and what does not port directly +9. [08-public-package-architecture-and-release.md](./08-public-package-architecture-and-release.md) — package boundaries, Core integration, versions, and NPM release train +10. [09-program-roadmap.md](./09-program-roadmap.md) — phases, dependency graph, staffing, gates, and integration cadence +11. [10-core-file-change-map.md](./10-core-file-change-map.md) — concrete Core seams and expected changes +12. [11-master-execution-checklist.md](./11-master-execution-checklist.md) — owner-ready execution and cutover checklist +13. [12-decision-log.md](./12-decision-log.md) — decisions to ratify before implementation begins +14. [13-open-source-program-interlock.md](./13-open-source-program-interlock.md) — how this migration changes and extends the completed OSS program +15. [14-24-hour-parallel-execution.md](./14-24-hour-parallel-execution.md) — immediate package-independent Core work and the 24-hour merge order +16. [16-live-package-join-board.md](./16-live-package-join-board.md) — verified package state, exact IID adapter mapping, tarball gates, and live J01–J04 ownership +17. [17-core-cutover-completion-plan.md](./17-core-cutover-completion-plan.md) — authoritative remaining gaps, Core PR stack, no-Docker work, and final cutover gates + +Detailed track charters live under [`tracks/`](./tracks/00-program-foundation.md). + +## Recommended ownership + +Seven workstreams can proceed with controlled overlap. With fewer engineers, combine adjacent tracks, but keep the ownership boundaries and acceptance gates intact. + +| Track | Owns | Depends on | +| --- | --- | --- | +| 0. Program foundation | decisions, schemas, fixtures, release coordination | none | +| 1. Public packages | IID grammar, semantic registry, builders, NPM releases | Track 0 | +| 2. Contract URI ingestion | ABI parity, event storage, projection of context | Track 0 | +| 3. IID recognition | parser, classification, normalization, identity persistence | Tracks 0–1 | +| 4. Resolution and enrichment | provider plans, workers, artifacts, projections | Tracks 1 and 3 | +| 5. Data, API, and explorer | schema/read models, search, compatibility, UI | Tracks 2–4 | +| 6. Seed and mint integration | canonical writer, seed projection, URI-aware minting | Tracks 1 and 5 read gate | +| 7. Quality and operations | contract tests, observability, backfill, cutover | all tracks | + +Track charters: + +- [00-program-foundation.md](./tracks/00-program-foundation.md) +- [01-public-packages.md](./tracks/01-public-packages.md) +- [02-contract-uri-ingestion.md](./tracks/02-contract-uri-ingestion.md) +- [03-iid-recognition.md](./tracks/03-iid-recognition.md) +- [04-resolution-enrichment.md](./tracks/04-resolution-enrichment.md) +- [05-data-api-explorer.md](./tracks/05-data-api-explorer.md) +- [06-seed-mint-integration.md](./tracks/06-seed-mint-integration.md) +- [07-quality-release-operations.md](./tracks/07-quality-release-operations.md) + +## Source hierarchy + +1. URI-enabled protocol source: `/Users/metasudo/workspace/intution/workspace/intuition-contracts-v2/src/protocol` +2. Public package source and release policy: `/Users/metasudo/workspace/intution/workspace/packages` ([GitHub](https://github.com/0xIntuition/packages)) +3. Implemented private reference: `/Users/metasudo/workspace/intution/workspace/alpha/.planning/iid-migration` and its corresponding code +4. Current Core runtime: this repository +5. Earlier IID design material: `/Users/metasudo/workspace/intution/workspace/intuition-v2/.planning/intuition-id` + +The private monorepo is evidence that the architecture works, not a package API specification. Public package source and contract source win when interfaces differ. Any disagreement at those boundaries is a release blocker. + +## Non-negotiable invariants + +- Identity bytes are deterministic and offline. External APIs enrich an atom but never change its IID. +- Parse an IID by splitting only the first two colons; the value may contain additional colons. +- New writes use only registered schemes and canonical values. +- P0 (`int::`) is used only when the scheme is unambiguously typed. Polymorphic schemes use an explicit P1/P2 profile. +- URI context is ordered, bounded evidence. It never contributes to the atom ID. +- `AtomContextRegistered` is joined to the atom by `termId`, never by log adjacency. +- Raw identity, resolution artifacts, and resolved projection remain separate. +- Legacy JSON, URL, IPFS, and string atoms remain readable during the compatibility window. +- Same IID means the same off-chain identity cluster; it does not imply the same on-chain atom ID. +- Every published `@0xintuition/*` dependency is exact-pinned, provenance-checked, and represented once in the lockfile. +- Production writers remain disabled until reader, API, search, display, and rollback gates pass. + +## Definition of program completion + +The program is complete when one golden IID fixture can be followed end to end from canonical builder output through URI-aware contract creation, event ingestion, durable identity/context storage, provider resolution, resolved projection, API/search/explorer display, replay/backfill, and operational rollback—and the same test proves that legacy atoms continue to behave correctly. diff --git a/.planning/codex-migration/tracks/00-program-foundation.md b/.planning/codex-migration/tracks/00-program-foundation.md new file mode 100644 index 0000000..2479aec --- /dev/null +++ b/.planning/codex-migration/tracks/00-program-foundation.md @@ -0,0 +1,38 @@ +# Track 0 — Program foundation + +## Mission + +Create the shared contracts, fixtures, ownership, flags, and release controls that allow all implementation tracks to work independently without semantic drift. + +## Owner and reviewers + +- Primary: migration/integration lead +- Required reviewers: packages, contract/indexer, Core data, seed/writer, API/product, security + +## Work packages + +1. Ratify the blocking decisions in [../12-decision-log.md](../12-decision-log.md). +2. Define canonical `ParsedIid`, `ClassificationDecision`, `ResolutionPlan`, `AtomContextEvidence`, `ResolutionArtifact`, `ResolvedAtom`, and `AtomAnchor` schemas. +3. Create versioned golden fixtures covering P0, typed profile, colon-bearing value, invalid input, URI ordering/limits, empty context, and legacy JSON. +4. Record contract artifact/ABI fingerprint, network addresses, activation blocks, and live URI config. +5. Define package, schema, resolver, and projection version stamping. +6. Create feature flags: reader, projection, resolver, API exposure, writer global kill switch, and per-scheme writer allowlist. +7. Define dashboards, go/no-go template, rollback authority, and cross-repository PR dependency labels. + +## Interfaces produced + +- Machine-readable fixture bundle consumed by package and Core CI. +- Schema definitions owned in one agreed repository/package. +- Release manifest mapping exact contract, package, migration, and fixture versions. +- Program board seeded from [../11-master-execution-checklist.md](../11-master-execution-checklist.md). + +## Acceptance + +- Each fixture has canonical bytes and expected outputs reviewed by both a producer and consumer owner. +- No blocking decision is merely implied by code. +- Every production write surface and downstream API consumer has a named owner. +- Rollback disables writers without reverting schema or losing context. + +## Handoff + +Tracks 1 and 2 begin when semantic types, ABI artifact, and fixture formats are frozen. Later compatible fixture additions are allowed; changing existing expected bytes requires a versioned breaking decision. diff --git a/.planning/codex-migration/tracks/01-public-packages.md b/.planning/codex-migration/tracks/01-public-packages.md new file mode 100644 index 0000000..386955c --- /dev/null +++ b/.planning/codex-migration/tracks/01-public-packages.md @@ -0,0 +1,48 @@ +# Track 1 — Public packages and NPM release + +## Mission + +Publish the reusable IID, semantic registry, canonical builder, and URI-aware protocol interfaces that Core and all atom writers consume. + +## Owner and reviewers + +- Primary: public packages maintainer +- Reviewers: Core parser/enrichment owner, seed/writer owner, contract owner, release/security owner +- Repository: `/Users/metasudo/workspace/intution/workspace/packages` + +## Work packages + +1. Implement `iid` grammar/canonicalization and public fixture exports. +2. Reconcile public declarative classification identity ladders with shared IID types. +3. Implement pure `iid-registry` lookup and provider-plan APIs. +4. Add the canonical IID atom anchor/URI-manifest builder in `primitives`. +5. Update `protocol` for URI function, event, and config APIs; preserve old methods. +6. Update `react` only if it is a supported atom writer. +7. Add semantic ABI parity against exact `contracts-v2`. +8. Update publish order, pack dry-run, tarball smoke, and clean-consumer tests. +9. Publish exact prereleases in topological order and verify registry artifacts. +10. Maintain a Core eligibility calendar for the 14-day minimum-age gate. + +## Constraints + +- Pure packages perform no network I/O. +- No circular dependency between classifications and registry. +- Public exports, not source paths, are the integration contract. +- `ids` remains term hashing; `iid` remains semantic identity. +- Internal dependency versions are exact. +- No `file:` or Git dependency is committed to Core. + +## Acceptance + +- A clean consumer installed from packed tarballs passes the golden fixture. +- Canonical builder bytes and atom ID match Core and contract fixtures. +- URI transaction encoding and event decoding match the exact contract ABI. +- Every declared scheme has explicit classification/provider/eligibility test behavior. +- NPM artifacts contain all documented exports and provenance. + +## Handoffs + +- Track 3 consumes `iid` and `iid-registry`. +- Track 4 consumes registry provider plans. +- Track 6 consumes builder and protocol APIs. +- Track 7 owns cross-repo verification and final Core adoption. diff --git a/.planning/codex-migration/tracks/02-contract-uri-ingestion.md b/.planning/codex-migration/tracks/02-contract-uri-ingestion.md new file mode 100644 index 0000000..d4520d2 --- /dev/null +++ b/.planning/codex-migration/tracks/02-contract-uri-ingestion.md @@ -0,0 +1,45 @@ +# Track 2 — Contract URI event ingestion + +## Mission + +Carry URI context from the authoritative contract ABI through Rindexer, event storage, replay, and the knowledge graph without loss or accidental reinterpretation. + +## Owner and reviewers + +- Primary: contract/indexer engineer +- Reviewers: Solidity owner, projection/data owner, security owner + +## Work packages + +1. Bump Core's exact contracts artifact and regenerate all ABI-derived files. +2. Add `AtomContextRegistered` to Rindexer configuration and generated handler types. +3. Add the event to shared unions/models, typed storage, handler persistence, and typed readers. +4. Create additive database migrations with `sequence_number`, chain identity, `termId`, creator, raw ordered URI bytes, and timestamps/block provenance. +5. Add idempotent KG projection joined by `termId`. +6. Preserve raw evidence; create separate safe normalization and validation results. +7. Reconcile event-before-atom/orphan projection ordering. +8. Add activation-block backfill, restart/resume, reorg, and double-replay tests. +9. Add URI event lag, malformed entry, orphan, and projection error metrics. +10. Prove ABI parity with public `protocol` before production. + +## Edge cases + +- Zero context entries. +- Maximum entry count and maximum bytes per entry. +- Non-UTF-8 bytes and unsupported schemes. +- Duplicate URI values and order preservation. +- Multiple context events for a term if the protocol permits/produces them. +- Reorg/removal semantics and projection idempotency. +- Context event observed before the atom event projection. + +## Acceptance + +- Real encoded event log decodes identically in contracts package, public protocol package, and Rindexer. +- Raw bytes and ordinal order survive storage/replay exactly. +- Normalization cannot overwrite evidence. +- Replaying the same range twice creates no duplicates or drift. +- Orphan context converges after the corresponding atom is available. + +## Handoff + +Track 5 consumes normalized context and evidence provenance. Track 6 validates writer-produced context against the indexed result. diff --git a/.planning/codex-migration/tracks/03-iid-recognition.md b/.planning/codex-migration/tracks/03-iid-recognition.md new file mode 100644 index 0000000..d8b2154 --- /dev/null +++ b/.planning/codex-migration/tracks/03-iid-recognition.md @@ -0,0 +1,44 @@ +# Track 3 — IID recognition and classification + +## Mission + +Make canonical IID a first-class atom raw type and derive safe semantic classification without network calls or duplicated scheme tables. + +## Owner and reviewers + +- Primary: Core parser/classification engineer +- Reviewers: packages semantic owner, database owner, enrichment owner + +## Work packages + +1. Add IID to parser/domain/database raw-type definitions. +2. Recognize IID before generic URL and string branches using public `iid`. +3. Persist raw input, canonical IID, profile, scheme, value, parser version, and validity/status. +4. Add typed failures while preserving the raw on-chain value for invalid/lookalike inputs. +5. Add classification branch through `iid-registry` with explicit source/version. +6. Represent unknown and polymorphic classification honestly. +7. Keep legacy structured/raw classifier branches unchanged for non-IIDs. +8. Backfill existing candidate `int:` atoms in resumable chunks. +9. Add parity tests using public fixture exports and database round trips. +10. Delete local scheme/classification tables only after all callers migrate. + +## Ordering contract + +```text +raw bytes -> UTF-8 candidate -> IID recognition/canonicalization + -> otherwise existing JSON/HTTP/IPFS/string detection +``` + +Recognition is not resolution. This track must never call Spotify, MusicBrainz, chain RPCs, or other providers. + +## Acceptance + +- All valid golden IIDs round-trip canonically and values containing colons remain intact. +- Invalid or unregistered schemes do not become trusted IID identities. +- Classification equals registry behavior for all schemes and is absent where ambiguous. +- Existing non-IID parser/classifier fixtures are unchanged. +- Live and backfill processing produce the same persisted representation. + +## Handoff + +Track 4 receives a versioned canonical identity record, not an arbitrary string. Track 5 can expose recognition state before resolution completes. diff --git a/.planning/codex-migration/tracks/04-resolution-enrichment.md b/.planning/codex-migration/tracks/04-resolution-enrichment.md new file mode 100644 index 0000000..c63ea23 --- /dev/null +++ b/.planning/codex-migration/tracks/04-resolution-enrichment.md @@ -0,0 +1,43 @@ +# Track 4 — Resolution and enrichment + +## Mission + +Resolve canonical identity through registered provider capabilities, preserve artifacts and provenance, and project stable display/search data without changing identity. + +## Owner and reviewers + +- Primary: enrichment engineer +- Reviewers: IID registry owner, database owner, search/API owner, security owner + +## Work packages + +1. Accept the Track 3 identity contract in worker messages/jobs. +2. Produce provider plans and identifier hints exclusively through `iid-registry`. +3. Implement/adapt provider clients by desired capability. +4. Store raw artifacts or durable references before materialization, with provider, request key, fetch time, resolver version, license/retention metadata, and status. +5. Define retryable, terminal, partial, stale, and unsupported outcomes. +6. Project classification-compatible label, description, image, links, and search fields. +7. Merge multiple provider results using deterministic precedence and field-level provenance. +8. Add refresh TTLs, circuit breakers, concurrency/rate limits, and credential outage handling. +9. Add version-change reconciliation and targeted re-resolution. +10. Preserve the existing URL/JSON enrichment path for legacy atoms. + +## Security and reliability + +- Treat context URIs and provider responses as untrusted. +- Enforce network egress/SSRF controls, redirect limits, timeouts, response-size limits, and content-type validation. +- Avoid placing full payloads or credentials in logs/dead-letter queues. +- A provider outage never invalidates the canonical identity. +- Projection failure is retryable independently of refetching where possible. + +## Acceptance + +- A canonical IID without a legacy document schedules the correct provider plan. +- Each artifact and each projected field has traceable provenance/version. +- Rate-limit/auth/network failures remain recoverable; invalid/unsupported identities do not loop forever. +- Replaying the same artifacts is idempotent. +- Registry/provider version changes can be reconciled without rewriting identity. + +## Handoff + +Track 5 consumes explicit resolution status and resolved projection. Track 7 monitors provider and projection SLOs during canary/cutover. diff --git a/.planning/codex-migration/tracks/05-data-api-explorer.md b/.planning/codex-migration/tracks/05-data-api-explorer.md new file mode 100644 index 0000000..5870c72 --- /dev/null +++ b/.planning/codex-migration/tracks/05-data-api-explorer.md @@ -0,0 +1,54 @@ +# Track 5 — Data model, API, search, and explorer + +## Mission + +Make IID identity, URI evidence, resolution state, and resolved presentation available across Core while preserving legacy consumers and raw on-chain truth. + +## Owner and reviewers + +- Primary: Core data/API engineer +- Reviewers: parser, context projection, enrichment, explorer/product, downstream consumer owners + +## Work packages + +1. Deliver additive KG schema/indexes for identity, context, artifacts, and resolved projection. +2. Replace duplicated term-ID hashing with thin `@0xintuition/ids` wrappers and parity tests. +3. Stop raw IID from becoming the default final `searchText`/label. +4. Extend atom read APIs with structured raw/identity/classification/context/resolution/display sections. +5. Preserve current legacy fields and define deprecation telemetry. +6. Add supported filters/sorts and query indexes. +7. Build/rebuild search documents from resolved data plus canonical safe hints. +8. Update explorer detail and result cards for raw identity, context, provenance, resolution state, and display data. +9. Render external URIs safely and distinguish evidence from verified links. +10. Add consumer contracts, query-plan/load tests, and cache invalidation behavior. + +## Required UI states + +- Recognized, awaiting resolution. +- Partially resolved. +- Resolved. +- Retryable provider failure. +- Terminal unsupported/invalid identity. +- Legacy atom with existing structured enrichment. +- Raw unrecognized atom. + +## API compatibility rules + +- Additive first; do not silently change the meaning of an existing field. +- Raw on-chain bytes/data remain queryable. +- Resolved values carry source/version/freshness. +- Context preserves on-chain order and provenance. +- Missing classification or resolution is a modeled state, not a fabricated fallback. +- Search result labels may fall back gracefully, but the full raw IID remains visible as secondary identity. + +## Acceptance + +- Golden IID and legacy fixtures render correctly before and after resolution. +- Search uses resolved semantic fields when available and never depends on legacy JSON-LD. +- Context links are safe against injection and unsupported URI schemes. +- Production-scale query plans stay inside agreed SLOs. +- Known consumers pass contract tests against the compatibility shape. + +## Handoff + +The passing reader gate authorizes Track 6 to begin staging/shadow writes. It does not by itself authorize production IID defaults. diff --git a/.planning/codex-migration/tracks/06-seed-mint-integration.md b/.planning/codex-migration/tracks/06-seed-mint-integration.md new file mode 100644 index 0000000..e182e34 --- /dev/null +++ b/.planning/codex-migration/tracks/06-seed-mint-integration.md @@ -0,0 +1,57 @@ +# Track 6 — Seed pipeline and atom creation + +## Mission + +Move every supported atom producer from classification JSON blobs to canonical IID anchors with aligned URI context, deterministic dry runs, and resumable transaction execution. + +## Owner and reviewers + +- Primary: seed/data pipeline engineer +- Reviewers: public builder owner, contract owner, Core reader owner, operations owner + +## Work packages + +1. Inventory every seed source, CLI, API, script, application, and job that can create atoms. +2. Define source-to-identity mappings per dataset, including scheme, typed profile where required, canonical value derivation, and evidence URI policy. +3. Replace local serializers with the public canonical builder. +4. Produce a preflight manifest containing source key, IID/profile, canonical data bytes, predicted atom ID, classification decision, provider plan, ordered URI context, warnings, and source lineage. +5. Validate against registry rules and live contract URI config. +6. Detect duplicate source rows, duplicate canonical IIDs, pre-existing atom IDs, and cross-profile identity clusters. +7. Align `atomDatas[i]`, `assets[i]`, and `uris[i]`; simulate before broadcasting. +8. Create an idempotent batch ledger with planned/simulated/submitted/confirmed/indexed/failed states and retry lineage. +9. Shadow-run against current seed output; review semantic deltas and rejected records. +10. Mint to devnet/staging and accept only after Core re-reads the expected identity/context/resolution. +11. Roll out production by producer and scheme allowlist. + +## Dataset contract + +Each dataset mapping must document: + +- source authority and immutable source key; +- canonical scheme and profile-selection rule; +- normalization and rejection rules; +- primary and optional context URI roles; +- how licensed metadata is retained outside atom bytes; +- duplicate and update policy; +- expected resolver/provider coverage; +- accountable domain reviewer. + +## Failure policy + +- Validation failure: do not submit; return actionable row-level error. +- Simulation/revert: do not mark submitted; preserve batch plan. +- Partial transaction failure: resume from ledger, never rebuild prior successful identities differently. +- Indexing timeout: keep transaction confirmed state and reconcile; do not blindly remint. +- Reader mismatch: stop the producer allowlist and invoke integration rollback. + +## Acceptance + +- Running a seed corpus twice yields identical data bytes, atom IDs, and context manifests. +- No unsupported polymorphic scheme is emitted as P0. +- Batch arrays always align and comply with current contract limits. +- Staging atoms are visible end to end in Core and match the manifest. +- Emergency writer disable stops new submissions without affecting readers. + +## Handoff + +Track 7 controls production canary and expansion. Dataset-specific mapping work can continue after the initial scheme, but every new scheme repeats the same acceptance gate. diff --git a/.planning/codex-migration/tracks/07-quality-release-operations.md b/.planning/codex-migration/tracks/07-quality-release-operations.md new file mode 100644 index 0000000..9eb7058 --- /dev/null +++ b/.planning/codex-migration/tracks/07-quality-release-operations.md @@ -0,0 +1,92 @@ +# Track 7 — Quality, release, backfill, and operations + +## Mission + +Prove cross-repository compatibility, run safe backfills and canaries, measure the system, and retain an immediate writer rollback path. + +## Owner and reviewers + +- Primary: integration/release lead with platform reliability partner +- Reviewers: every track owner; security for URI/provider controls + +## Test portfolio + +### Contract and package contracts + +- Golden IID/profile/canonicalization vectors across public packages and Core. +- Atom ID parity across `primitives`, `ids`, Core wrappers, and contract fixture. +- ABI semantic parity across `contracts-v2`, public `protocol`, Core artifacts, and Rindexer. +- Packed-tarball clean-consumer tests using public exports only. +- Lockfile assertion for one exact version per `@0xintuition/*` package. + +### Core integration + +- Real-log URI event decode, storage, projection, replay, and reorg behavior. +- Live versus backfill convergence for identity/classification/resolution. +- Provider retry/timeout/auth/rate-limit/invalid response tests. +- API/search/explorer tests for every modeled state and legacy atoms. +- Seed manifest -> simulation -> transaction -> event -> indexed response fixture. + +### Non-functional + +- Database migration duration and lock profile. +- Query plans and API/search load. +- Worker queue throughput, retry amplification, and provider budgets. +- SSRF, URI sanitization, response-size, secret/redaction, and malformed-bytes tests. +- Kill-switch and rollback drill. + +## Backfill plan + +1. Count candidate historical `int:` raw atoms and all context events from activation block. +2. Run a read-only classification report: valid, invalid, unregistered, polymorphic, already projected. +3. Backfill in deterministic chunks with checkpoint, version, attempt, and error ledger. +4. Use the same domain functions and projections as live processing. +5. Rate-limit external resolution separately from local recognition/projection. +6. Reconcile counts among raw events, typed events, KG identities/context, artifacts, and resolved views. +7. Re-run a sample and then the full backfill to prove idempotency. + +## Go/no-go dashboard + +Minimum signals: + +- chain ingestion and projection lag; +- context event stored/projected/orphan/error counts; +- IID parse valid/invalid by scheme/profile/version; +- classification unknown/polymorphic rates; +- resolution queue depth, age, success, retry, terminal failure by provider/scheme; +- API error/latency and search fallback rate; +- seed builder rejection, simulation, transaction, confirmation, and indexing outcomes; +- contract/package/fixture version deployed. + +## Canary sequence + +1. Readers and migrations deployed, writer disabled. +2. Historical backfill and reconciliation within budget. +3. Internal actor, one P0 scheme, very small volume. +4. Inspect each atom end to end and compare manifest. +5. Expand volume for same scheme. +6. Add schemes one at a time; typed/polymorphic profiles last. +7. Enable approved external producers. +8. Change default only after observation window and consumer sign-off. + +## Stop and rollback criteria + +Immediately disable IID/URI writers when any of these occur: + +- atom ID differs from the preflight manifest; +- ABI/event decode mismatch or lost context; +- persistent reader/API failure for newly created atoms; +- duplicate creation caused by ledger/reconciliation behavior; +- material provider abuse, SSRF, credential exposure, or uncontrolled retry amplification; +- database or projection lag exceeds agreed error budget. + +Rollback means writer flags off and producer jobs paused. Keep additive schema, parsing, context ingestion, and reader support deployed unless they themselves cause the incident. Reconcile submitted transactions from the ledger before resuming. + +## Acceptance + +- All master checklist gates have evidence and named sign-off. +- Package versions have passed Core's release-age and integrity policies. +- Backfill and double-replay reconcile exactly. +- Canary metrics remain within the agreed observation budget. +- Rollback drill completes without data deletion or schema reversal. +- Operations runbooks identify on-call owner, dashboards, common failure actions, and escalation path. diff --git a/apps/explorer/src/components/term-chip.tsx b/apps/explorer/src/components/term-chip.tsx index e4e94bb..8a0773f 100644 --- a/apps/explorer/src/components/term-chip.tsx +++ b/apps/explorer/src/components/term-chip.tsx @@ -1,8 +1,9 @@ import { Link } from '@tanstack/react-router'; import type { TermSummary } from '@/lib/api'; +import { termDisplayLabel } from '@/lib/atom-presentation'; import { classificationClasses } from '@/lib/classification'; import { cn } from '@/lib/cn'; -import { formatId, previewData } from '@/lib/format'; +import { formatId } from '@/lib/format'; /** * One term of a triple as a linked chip: `[classification] data-preview`. @@ -17,7 +18,7 @@ export function TermChip({ term?: TermSummary; highlight?: boolean; }) { - const label = term?.data ? previewData(term.data, 42) : formatId(termId); + const label = term ? termDisplayLabel(term, 42) || formatId(termId) : formatId(termId); return ( ; + +export const atomResolutionSchema = z.looseObject({ + status: z.string(), + updatedAt: z.string().nullable().optional(), +}); + +export const atomDisplaySchema = z.looseObject({ + name: z.string().optional(), + description: z.string().optional(), + image: z.string().optional(), +}); + export const atomListItemSchema = z.looseObject({ id: z.string(), createdAt: z.string(), isOnchain: z.boolean(), rawType: z.string(), data: z.string().nullable(), + iid: z.string().nullable().optional(), dataResolved: anyJson, classificationType: z.string(), parseStatus: z.string(), classificationStatus: z.string(), enrichmentStatus: z.string(), + raw: atomRawSchema.optional(), + identity: atomIdentitySchema.optional(), + classification: atomClassificationViewSchema.optional(), + context: z.array(atomContextItemSchema).optional(), + resolution: atomResolutionSchema.optional(), + display: atomDisplaySchema.optional(), }); export type AtomListItem = z.infer; @@ -53,8 +115,12 @@ export const termSummarySchema = z .looseObject({ id: z.string(), data: z.string().nullable(), + dataResolved: anyJson.optional(), classificationType: z.string(), rawType: z.string(), + raw: atomRawSchema.optional(), + classification: atomClassificationViewSchema.optional(), + display: atomDisplaySchema.optional(), }) .nullable(); export type TermSummary = z.infer; @@ -193,6 +259,10 @@ const one = (item: T) => z.object({ data: item }); export type Page = { limit?: number; offset?: number }; +export function iidAtomsPath(iid: string): string { + return `/api/iids/${encodeURIComponent(iid)}/atoms`; +} + export const api = { stats: () => request(one(statsSchema), '/api/stats'), @@ -203,6 +273,9 @@ export const api = { atom: (id: string) => request(one(atomDetailSchema), `/api/atoms/${id}`), + iidAtoms: (iid: string, params: Page = {}) => + request(listOf(atomListItemSchema), iidAtomsPath(iid), { params }), + atomTriples: (id: string, params: Page = {}) => request(listOf(tripleSchema), `/api/atoms/${id}/triples`, { params: { ...params, expand: 'terms' }, diff --git a/apps/explorer/src/lib/atom-presentation.test.ts b/apps/explorer/src/lib/atom-presentation.test.ts new file mode 100644 index 0000000..c0b5235 --- /dev/null +++ b/apps/explorer/src/lib/atom-presentation.test.ts @@ -0,0 +1,164 @@ +import { describe, expect, test } from 'bun:test'; +import { atomContextItemSchema, atomIdentitySchema, iidAtomsPath } from './api'; +import { + atomDisplayImage, + atomDisplayLabel, + atomSecondaryIdentity, + contextDisplayValue, + parseSemanticReadsFlag, + safeContextHref, +} from './atom-presentation'; + +type GoldenFixture = { + schemaVersion: string; + canonicalizationPolicy: string; + identityCases: Array<{ id: string; input: string }>; + contextCases: Array<{ + id: string; + entries: Array<{ + ordinal: number; + uri: string | null; + raw?: string; + source: string; + expectedLinkable: boolean; + expectedDisplay?: string; + }>; + }>; + resolutionCases: Array<{ + id: string; + source: { + id: string; + data: string; + dataResolved: unknown; + }; + expected: { + displayName: string | null; + displayImage?: string; + }; + }>; +}; + +const golden = (await Bun.file( + new URL('../../../../tests/fixtures/atom-semantic-read-model.v1.json', import.meta.url) +).json()) as GoldenFixture; + +function byId(cases: T[], id: string): T { + const found = cases.find((entry) => entry.id === id); + if (!found) { + throw new Error(`missing golden fixture case: ${id}`); + } + return found; +} + +const unresolved = byId(golden.resolutionCases, 'unresolved-isrc'); +const resolved = byId(golden.resolutionCases, 'resolved-isrc'); +const baseAtom = unresolved.source; + +describe('atom presentation', () => { + test('prefers the resolved display name and image over a raw IID', () => { + const atom = { + ...resolved.source, + display: { + name: resolved.expected.displayName ?? undefined, + image: resolved.expected.displayImage, + }, + }; + + expect(atomDisplayLabel(atom, 96, true)).toBe( + resolved.expected.displayName ?? resolved.source.data + ); + expect(atomDisplayImage(atom, true)).toBe(resolved.expected.displayImage ?? null); + }); + + test('falls back through resolved payload, raw data, and atom id', () => { + expect( + atomDisplayLabel({ ...baseAtom, dataResolved: { name: 'Resolved name' } }, 96, true) + ).toBe('Resolved name'); + expect(atomDisplayLabel(baseAtom, 96, true)).toBe('int:isrc:USQX91300108'); + expect(atomDisplayLabel({ ...baseAtom, data: null }, 96, true)).toBe(baseAtom.id); + }); + + test('retains legacy presentation while the Explorer exposure gate is off', () => { + const atom = { + ...baseAtom, + dataResolved: { name: 'Resolved name', image: 'https://images.example/legacy.jpg' }, + display: { name: 'Semantic name', image: 'https://images.example/semantic.jpg' }, + }; + + expect(atomDisplayLabel(atom, 96, false)).toBe('int:isrc:USQX91300108'); + expect(atomDisplayImage(atom, false)).toBe('https://images.example/legacy.jpg'); + expect( + atomSecondaryIdentity( + { ...atom, identity: { raw: baseAtom.data, canonical: baseAtom.data } }, + false + ) + ).toBeNull(); + }); + + test('loads a versioned, package-neutral identity corpus', () => { + expect(golden.schemaVersion).toBe('1.0.0'); + expect(golden.canonicalizationPolicy).toBe('deferred-to-public-iid-package'); + expect(golden.identityCases.map((entry) => entry.id)).toEqual([ + 'p0-isrc-recording', + 'typed-mbid-recording', + 'polymorphic-wikidata', + 'invalid-isrc-value', + 'iid-lookalike', + ]); + }); + + test('accepts the complete Core producer identity envelope', () => { + const identity = (unresolved.source as { parseResult?: { identity?: unknown } }).parseResult + ?.identity; + + const parsed = atomIdentitySchema.parse(identity); + expect(parsed).toEqual(identity as typeof parsed); + }); + + test('parses only explicit exposure values', () => { + expect(parseSemanticReadsFlag('true')).toBe(true); + expect(parseSemanticReadsFlag('1')).toBe(true); + expect(parseSemanticReadsFlag('false')).toBe(false); + expect(parseSemanticReadsFlag('yes')).toBe(false); + expect(parseSemanticReadsFlag(undefined)).toBe(false); + }); + + test('builds an exact, encoded same-IID cluster path', () => { + expect(iidAtomsPath('int:mbid:recording:a/b')).toBe( + '/api/iids/int%3Ambid%3Arecording%3Aa%2Fb/atoms' + ); + }); +}); + +describe('safeContextHref', () => { + test('accepts persisted ordering provenance while keeping opaque bytes non-clickable', () => { + const context = atomContextItemSchema.parse({ + eventSequence: '42', + ordinal: 3, + uri: null, + raw: '0xff00', + source: 'onchain', + registrant: '0xregistrant', + transactionHash: '0xtx', + logIndex: 7, + }); + + expect(contextDisplayValue(context)).toBe('0xff00'); + expect(safeContextHref(context)).toBeNull(); + }); + + test('applies link safety without reordering or deduplicating context evidence', () => { + const fixture = byId(golden.contextCases, 'ordered-duplicate-and-unsafe-context'); + + expect(fixture.entries.map((entry) => entry.ordinal)).toEqual([0, 1, 2, 3, 4, 5, 6]); + expect(fixture.entries[0]?.uri).toBe(fixture.entries[1]?.uri); + for (const entry of fixture.entries) { + expect(safeContextHref(entry) !== null, `context ordinal ${entry.ordinal}`).toBe( + entry.expectedLinkable + ); + if (entry.expectedDisplay) { + expect(contextDisplayValue(entry)).toBe(entry.expectedDisplay); + } + } + }); +}); diff --git a/apps/explorer/src/lib/atom-presentation.ts b/apps/explorer/src/lib/atom-presentation.ts new file mode 100644 index 0000000..b79c01e --- /dev/null +++ b/apps/explorer/src/lib/atom-presentation.ts @@ -0,0 +1,104 @@ +import type { AtomContextItem, AtomListItem, TermSummary } from './api'; +import { previewData } from './format'; +import { extractImageFromRecord } from './images'; + +type PresentableAtom = Pick; + +export function parseSemanticReadsFlag(value: string | boolean | undefined): boolean { + return ( + value === true || (typeof value === 'string' && ['true', '1'].includes(value.toLowerCase())) + ); +} + +/** Independent UI gate: an enabled API cannot silently change Explorer labels. */ +export const ATOM_SEMANTIC_READS_ENABLED = parseSemanticReadsFlag( + import.meta.env.VITE_ATOM_SEMANTIC_READS_ENABLED +); + +/** Human-facing label priority: resolved display, resolved payload, raw bytes, term id. */ +export function atomDisplayLabel( + atom: PresentableAtom, + maxLength = 96, + semanticReadsEnabled = ATOM_SEMANTIC_READS_ENABLED +): string { + const resolvedName = semanticReadsEnabled + ? readString(toRecord(atom.dataResolved)?.name) + : undefined; + const label = + (semanticReadsEnabled ? (atom.display?.name ?? resolvedName) : undefined) ?? + atom.data ?? + atom.id; + return previewData(label, maxLength) || atom.id; +} + +/** A canonical IID remains useful as a secondary identity, never as the preferred label. */ +export function atomSecondaryIdentity( + atom: PresentableAtom, + semanticReadsEnabled = ATOM_SEMANTIC_READS_ENABLED +): string | null { + return semanticReadsEnabled ? (atom.identity?.canonical ?? atom.identity?.raw ?? null) : null; +} + +export function atomDisplayImage( + atom: Pick, + semanticReadsEnabled = ATOM_SEMANTIC_READS_ENABLED +): string | null { + return ( + (semanticReadsEnabled ? atom.display?.image : undefined) ?? + extractImageFromRecord(atom.dataResolved) + ); +} + +export function termDisplayLabel( + term: NonNullable, + maxLength = 42, + semanticReadsEnabled = ATOM_SEMANTIC_READS_ENABLED +): string { + const resolvedName = semanticReadsEnabled + ? readString(toRecord(term.dataResolved)?.name) + : undefined; + return previewData( + (semanticReadsEnabled ? (term.display?.name ?? resolvedName) : undefined) ?? term.data, + maxLength + ); +} + +/** + * Context entries are untrusted on-chain evidence. Only web URLs without + * embedded credentials become clickable; all other values remain visible as + * text for inspection and copying. + */ +export function safeContextHref(context: Pick): string | null { + if (!context.uri) { + return null; + } + try { + const url = new URL(context.uri); + if ((url.protocol !== 'https:' && url.protocol !== 'http:') || url.username || url.password) { + return null; + } + return url.href; + } catch { + return null; + } +} + +export function contextDisplayValue(context: AtomContextItem): string { + return context.uri ?? context.raw ?? '(unreadable context bytes)'; +} + +function toRecord(value: unknown): Record | undefined { + return typeof value === 'object' && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +function readString(value: unknown): string | undefined { + if (typeof value === 'string' && value.trim()) { + return value.trim(); + } + if (Array.isArray(value)) { + return value.map(readString).find((entry): entry is string => entry !== undefined); + } + return undefined; +} diff --git a/apps/explorer/src/routes/atoms.$atomId.tsx b/apps/explorer/src/routes/atoms.$atomId.tsx index a5fe0c4..c311c0a 100644 --- a/apps/explorer/src/routes/atoms.$atomId.tsx +++ b/apps/explorer/src/routes/atoms.$atomId.tsx @@ -17,7 +17,14 @@ import { Skeleton, } from '@/components/ui/primitives'; import { api } from '@/lib/api'; -import { formatId, formatRelativeTime, previewData } from '@/lib/format'; +import { + ATOM_SEMANTIC_READS_ENABLED, + atomDisplayImage, + atomDisplayLabel, + contextDisplayValue, + safeContextHref, +} from '@/lib/atom-presentation'; +import { formatId, formatRelativeTime } from '@/lib/format'; import { extractImageFromRecord } from '@/lib/images'; export const Route = createFileRoute('/atoms/$atomId')({ @@ -28,6 +35,12 @@ function AtomDetailPage() { const { atomId } = Route.useParams(); const atom = useQuery({ queryKey: ['atom', atomId], queryFn: () => api.atom(atomId) }); + const canonicalIid = atom.data?.data.identity?.canonical ?? null; + const iidCluster = useQuery({ + queryKey: ['iid-atoms', canonicalIid], + queryFn: () => api.iidAtoms(canonicalIid as string, { limit: 25 }), + enabled: ATOM_SEMANTIC_READS_ENABLED && Boolean(canonicalIid), + }); const artifacts = useQuery({ queryKey: ['atom-artifacts', atomId], queryFn: () => api.atomArtifacts(atomId, { limit: 50 }), @@ -75,16 +88,14 @@ function AtomDetailPage() { a.extracted).find(extractImageFromRecord) ) } size={36} /> - - {previewData(data.data, 80) || formatId(data.id)} - + {atomDisplayLabel(data, 80)} @@ -104,6 +115,99 @@ function AtomDetailPage() { + {ATOM_SEMANTIC_READS_ENABLED && data?.identity ? ( + + +
+ +
+ + ) : null} + + {ATOM_SEMANTIC_READS_ENABLED && canonicalIid ? ( + + + {iidCluster.isLoading ? ( +
+ +
+ ) : iidCluster.error ? ( +
+ +
+ ) : iidCluster.data && iidCluster.data.data.length > 0 ? ( +
    + {iidCluster.data.data.map((member) => ( +
  • + + + + {atomDisplayLabel(member, 64)} + + {member.id === atomId ? ( + current + ) : null} + +
    • + ))} +
+ ) : ( + No public atoms currently share this exact IID. + )} + + ) : null} + + {ATOM_SEMANTIC_READS_ENABLED && data?.context ? ( + + + {data.context.length > 0 ? ( +
    + {data.context.map((entry) => { + const label = contextDisplayValue(entry); + const href = safeContextHref(entry); + return ( +
  1. + + {entry.eventSequence ? `${entry.eventSequence}:` : ''} + {entry.ordinal} + + {href ? ( + + {label} + + ) : ( + + {label} + + )} +
    2. + ); + })} +
+ ) : ( + No context URIs were registered for this atom. + )} + + ) : null} + {data?.dataResolved != null && Object.keys(data.dataResolved as object).length > 0 ? ( diff --git a/apps/explorer/src/routes/atoms.index.tsx b/apps/explorer/src/routes/atoms.index.tsx index 45503e3..6fcc16e 100644 --- a/apps/explorer/src/routes/atoms.index.tsx +++ b/apps/explorer/src/routes/atoms.index.tsx @@ -9,8 +9,8 @@ import { PageHeader } from '@/components/layout/app-shell'; import { DataTable, Pager } from '@/components/ui/data-table'; import { Card, ErrorNote, IdChip, Input } from '@/components/ui/primitives'; import { type AtomListItem, api } from '@/lib/api'; -import { formatId, formatRelativeTime, previewData } from '@/lib/format'; -import { extractImageFromRecord } from '@/lib/images'; +import { atomDisplayImage, atomDisplayLabel } from '@/lib/atom-presentation'; +import { formatId, formatRelativeTime } from '@/lib/format'; const PAGE_SIZE = 25; @@ -29,19 +29,14 @@ const columns: ColumnDef[] = [ { id: 'thumb', header: '', - cell: ({ row }) => ( - - ), + cell: ({ row }) => , }, { id: 'data', - header: 'Data', + header: 'Name', cell: ({ row }) => ( - {previewData(row.original.data, 80) || (empty)} + {atomDisplayLabel(row.original, 80) || (empty)} ), }, diff --git a/apps/explorer/src/routes/index.tsx b/apps/explorer/src/routes/index.tsx index 78a675e..f0069da 100644 --- a/apps/explorer/src/routes/index.tsx +++ b/apps/explorer/src/routes/index.tsx @@ -10,8 +10,8 @@ import { PipelineBars } from '@/components/pipeline-bars'; import { StatCard } from '@/components/stat-card'; import { Card, CardHeader, EmptyState, SkeletonRows } from '@/components/ui/primitives'; import { api } from '@/lib/api'; -import { formatRelativeTime, previewData } from '@/lib/format'; -import { extractImageFromRecord } from '@/lib/images'; +import { atomDisplayImage, atomDisplayLabel } from '@/lib/atom-presentation'; +import { formatRelativeTime } from '@/lib/format'; export const Route = createFileRoute('/')({ component: DashboardPage, @@ -105,9 +105,9 @@ function DashboardPage() { params={{ atomId: atom.id }} to="/atoms/$atomId" > - + - {previewData(atom.data, 64) || atom.id} + {atomDisplayLabel(atom, 64)} , diff --git a/crates/projections/src/coordinator.rs b/crates/projections/src/coordinator.rs index d9003b2..0792d08 100644 --- a/crates/projections/src/coordinator.rs +++ b/crates/projections/src/coordinator.rs @@ -97,8 +97,8 @@ fn create_pg_projection( total_shards, ), )), - // Dual projectors manage their own kg_pool internally via with_kg_pool(). - // The PgWorker passes the legacy pool; kg writes happen inside process_parsed_batch. + // Dual projectors own their KG pool internally. The PgWorker passes the + // legacy pool; KG writes happen inside process_parsed_batch. "vault_state:dual" => { let mut proj = projection::dual::vault_state::VaultStateDualProjection::new( shard_id.unwrap_or(0), @@ -117,6 +117,10 @@ fn create_pg_projection( } Some(Box::new(proj)) } + "atom_context:dual" => kg_pool.map(|kp| { + Box::new(projection::dual::atom_context::AtomContextDualProjection::new(kp.clone())) + as Box + }), _ => None, } } @@ -777,6 +781,17 @@ mod tests { assert_eq!(proj.unwrap().name(), "vault_holders_index:dual"); } + #[tokio::test] + async fn pg_factory_requires_kg_pool_for_atom_context_dual() { + assert!(create_pg_projection("atom_context:dual", None, 1, None).is_none()); + + let kg_pool = sqlx::PgPool::connect_lazy("postgres://localhost/kg") + .expect("lazy connect must not fail"); + let proj = create_pg_projection("atom_context:dual", None, 1, Some(&kg_pool)); + assert!(proj.is_some()); + assert_eq!(proj.unwrap().name(), "atom_context:dual"); + } + #[test] fn batch_factory_returns_leaderboard_refresh() { let proj = create_batch_projection("leaderboard_refresh"); diff --git a/crates/projections/src/event/typed_reader.rs b/crates/projections/src/event/typed_reader.rs index bf692fa..bb94bcd 100644 --- a/crates/projections/src/event/typed_reader.rs +++ b/crates/projections/src/event/typed_reader.rs @@ -55,6 +55,22 @@ fn sql_fragment(event_type: &str) -> Option<&'static str> { FROM atom_created_events WHERE sequence_number > $1"#, ), + "AtomContextRegistered" => Some( + r#"SELECT sequence_number, block_number, block_timestamp, block_hash, + transaction_hash, log_index, + 'AtomContextRegistered'::TEXT AS event_type, + jsonb_build_object( + 'registrant', registrant, + 'term_id', term_id_hex, + 'uris', uris + ) AS event_data, + term_id_hex AS term_id, + NULL::TEXT AS entity_id, + true AS is_canonical, + block_timestamp AS ingested_at + FROM atom_context_registered_events + WHERE sequence_number > $1"#, + ), "TripleCreated" => Some( r#"SELECT sequence_number, block_number, block_timestamp, block_hash, transaction_hash, log_index, @@ -236,17 +252,18 @@ mod tests { } #[test] - fn all_six_event_types() { + fn all_seven_event_types() { let q = build_union_query(&[ "AtomCreated", + "AtomContextRegistered", "TripleCreated", "Deposited", "Redeemed", "SharePriceChanged", "ProtocolFeeAccrued", ]); - // 5 UNION ALL connectors for 6 fragments - assert_eq!(q.matches("UNION ALL").count(), 5); + // 6 UNION ALL connectors for 7 fragments + assert_eq!(q.matches("UNION ALL").count(), 6); } #[test] @@ -288,4 +305,17 @@ mod tests { assert!(!q.contains("predicate_id::TEXT")); assert!(!q.contains("object_id::TEXT")); } + + #[test] + fn atom_context_registered_reconstructs_ordered_opaque_uri_array() { + let q = build_union_query(&["AtomContextRegistered"]); + + assert!(q.contains("FROM atom_context_registered_events")); + assert!(q.contains("'term_id', term_id_hex")); + assert!(q.contains("'uris', uris")); + assert!(!q.contains("uris->")); + assert!(!q.contains("jsonb_array_elements")); + assert!(!q.contains("convert_from")); + assert!(!q.contains("decode(")); + } } diff --git a/crates/projections/src/main.rs b/crates/projections/src/main.rs index 1dac4e2..33e4fa1 100644 --- a/crates/projections/src/main.rs +++ b/crates/projections/src/main.rs @@ -77,6 +77,7 @@ fn build_pool_partitioner(config: &ProjectionsConfig) -> Arc { ("vault_holders_index", ConnectionTier::Standard), ("vault_state:dual", ConnectionTier::Critical), ("vault_holders_index:dual", ConnectionTier::Standard), + ("atom_context:dual", ConnectionTier::Standard), ("signals_analytics", ConnectionTier::Standard), ("term_aggregates", ConnectionTier::Standard), ("protocol_stats", ConnectionTier::Standard), @@ -291,8 +292,10 @@ fn build_pg_projections( } } - // Dual projectors — vault_state:dual and vault_holders_index:dual. - // Each dual projector manages its own kg_pool internally via with_kg_pool(). + // Dual projectors — vault_state:dual, vault_holders_index:dual, and + // atom_context:dual. + // Each dual projector owns its KG pool internally (the vault projectors + // attach it via with_kg_pool; atom_context requires it in its constructor). // The PgWorker passes the legacy pool; kg writes happen inside process_parsed_batch. // Sharding for vault_state:dual mirrors vault_state (same shard count, same hash key). if config.is_projection_enabled("vault_state:dual") { @@ -336,6 +339,19 @@ fn build_pg_projections( pg_projections.push(Box::new(proj)); } + if config.is_projection_enabled("atom_context:dual") { + if let Some(kp) = kg_pool { + pg_projections.push(Box::new( + projection::dual::atom_context::AtomContextDualProjection::new(kp.clone()), + )); + } else { + // Unlike the vault dual projectors, atom_context has no legacy + // write side. Never spawn a no-op worker that would advance its + // checkpoint while discarding context events. + info!("atom_context:dual enabled but DATABASE_KG_URL not set — projection not spawned"); + } + } + pg_projections } @@ -476,7 +492,7 @@ fn spawn_shutdown_handler(token: CancellationToken) { /// is logged and propagated — misconfigured URLs should fail fast at startup. async fn connect_kg_if_configured(config: &ProjectionsConfig) -> anyhow::Result> { let Some(kg_url) = config.database_kg_url.as_deref() else { - info!("DATABASE_KG_URL not set — kg.nodes writes are disabled"); + info!("DATABASE_KG_URL not set — KG node and atom-context writes are disabled"); return Ok(None); }; @@ -486,7 +502,7 @@ async fn connect_kg_if_configured(config: &ProjectionsConfig) -> anyhow::Result< .await .map_err(|e| anyhow::anyhow!("Failed to connect to DATABASE_KG_URL: {e}"))?; - info!("Connected to KG database (kg.nodes writes enabled)"); + info!("Connected to KG database (node and atom-context writes enabled)"); Ok(Some(pool)) } diff --git a/crates/projections/src/projection/dual/atom_context.rs b/crates/projections/src/projection/dual/atom_context.rs new file mode 100644 index 0000000..ee40f98 --- /dev/null +++ b/crates/projections/src/projection/dual/atom_context.rs @@ -0,0 +1,306 @@ +//! Projection of `AtomContextRegistered` into the KG database. +//! +//! `atom_context:dual` deliberately owns an independent event-stream +//! checkpoint. It joins context events to `kg.nodes` by the exact on-chain +//! `term_id`, writes one immutable row per URI, and appends one `kg.events` +//! evidence row per registration event. It never mutates the atom payload or +//! attempts to resolve, normalize, fetch, or trust a URI. + +use async_trait::async_trait; +use serde_json::json; +use shared::models::{AtomContextRegisteredRecord, StoredEvent}; +use shared::parsed_event::{EventMetadata, ParsedEvent}; +use shared::types::EventType; +use sqlx::{PgPool, Postgres, Transaction}; +use tracing::warn; + +use crate::error::ProjectionError; +use crate::projection::pg::PgProjection; + +const PROJECTION_NAME: &str = "atom_context:dual"; +const NODE_LOOKUP_SQL: &str = "SELECT classification_type FROM kg.nodes WHERE id = $1"; +const CONTEXT_INSERT_SQL: &str = "INSERT INTO kg.node_contexts + (node_id, event_sequence, block_number, block_timestamp, + block_hash, transaction_hash, log_index, ordinal, + registrant, uri_hex, uri_text) + VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) + ON CONFLICT (node_id, transaction_hash, log_index, ordinal) DO NOTHING"; +const EVIDENCE_INSERT_SQL: &str = "INSERT INTO kg.events + (event_time, id, actor_id, entity_kind, entity_id, event_type, + classification_type, is_onchain, block_number, tx_hash, + payload, created_at) + VALUES ($1, $2, $3, 'node', $4, 'atom_context_registered', + $5, true, $6, $7, $8, now()) + ON CONFLICT (event_time, id) DO NOTHING"; + +/// A byte-preserving URI representation ready for storage. +#[derive(Debug, Clone, PartialEq, Eq)] +struct ProjectedUri { + ordinal: i32, + uri_hex: String, + uri_text: Option, +} + +/// KG-only context projector. The legacy pool supplied by [`PgProjection`] +/// is intentionally ignored; it remains the event/checkpoint database only. +pub struct AtomContextDualProjection { + kg_pool: PgPool, +} + +impl AtomContextDualProjection { + pub fn new(kg_pool: PgPool) -> Self { + Self { kg_pool } + } +} + +#[async_trait] +impl PgProjection for AtomContextDualProjection { + fn name(&self) -> &str { + PROJECTION_NAME + } + + fn event_types(&self) -> &'static [EventType] { + &[EventType::AtomContextRegistered] + } + + fn uses_typed_events(&self) -> bool { + true + } + + async fn process_parsed_batch( + &self, + _event_store_pool: &PgPool, + events: &[ParsedEvent], + ) -> Result<(), ProjectionError> { + let mut tx = self.kg_pool.begin().await?; + + for event in events { + match event { + ParsedEvent::AtomContextRegistered { metadata, data } => { + project_context_event(&mut tx, metadata, data).await?; + } + ParsedEvent::Unknown(raw) + if raw.event_type == EventType::AtomContextRegistered.as_str() => + { + // A malformed context payload must pin this projection's + // checkpoint. Advancing would permanently lose URI bytes. + return Err(ProjectionError::InvalidEventData(format!( + "sequence {} could not be parsed as AtomContextRegistered", + raw.sequence_number + ))); + } + _ => { + // The worker filters by event_types(), so unrelated typed + // variants are only possible in direct unit/test calls. + } + } + } + + tx.commit().await?; + Ok(()) + } + + async fn process_batch( + &self, + pool: &PgPool, + events: &[StoredEvent], + ) -> Result<(), ProjectionError> { + let parsed: Vec<_> = events + .iter() + .map(|event| ParsedEvent::parse_or_unknown(event.clone()).0) + .collect(); + self.process_parsed_batch(pool, &parsed).await + } +} + +async fn project_context_event( + tx: &mut Transaction<'_, Postgres>, + metadata: &EventMetadata, + data: &AtomContextRegisteredRecord, +) -> Result<(), ProjectionError> { + // This exact equality is the dependency barrier between atom creation and + // context registration. RowNotFound is transient to the worker, so the + // transaction rolls back and the dedicated checkpoint remains pinned. + let classification_type = sqlx::query_scalar::<_, String>(NODE_LOOKUP_SQL) + .bind(&data.term_id) + .fetch_optional(&mut **tx) + .await?; + let Some(classification_type) = classification_type else { + warn!( + projection = PROJECTION_NAME, + sequence = metadata.sequence_number, + term_id = %data.term_id, + "Context dependency not ready: exact kg.nodes row is missing" + ); + return Err(sqlx::Error::RowNotFound.into()); + }; + + let projected_uris = project_uris(&data.uris)?; + + for uri in &projected_uris { + sqlx::query(CONTEXT_INSERT_SQL) + .bind(&data.term_id) + .bind(metadata.sequence_number) + .bind(metadata.block_number) + .bind(metadata.block_timestamp) + .bind(&metadata.block_hash) + .bind(&metadata.transaction_hash) + .bind(metadata.log_index) + .bind(uri.ordinal) + .bind(&data.registrant) + .bind(&uri.uri_hex) + .bind(&uri.uri_text) + .execute(&mut **tx) + .await?; + } + + let event_id = context_event_id(&metadata.transaction_hash, metadata.log_index); + let uri_hex: Vec<&str> = projected_uris + .iter() + .map(|uri| uri.uri_hex.as_str()) + .collect(); + let payload = json!({ + "event_sequence": metadata.sequence_number, + "block_hash": metadata.block_hash, + "log_index": metadata.log_index, + "is_canonical": metadata.is_canonical, + "uris": uri_hex, + }); + + sqlx::query(EVIDENCE_INSERT_SQL) + .bind(metadata.block_timestamp) + .bind(event_id) + .bind(&data.registrant) + .bind(&data.term_id) + .bind(classification_type) + .bind(metadata.block_number) + .bind(&metadata.transaction_hash) + .bind(payload) + .execute(&mut **tx) + .await?; + + Ok(()) +} + +fn project_uris(uris: &[String]) -> Result, ProjectionError> { + uris.iter() + .enumerate() + .map(|(ordinal, raw)| { + let ordinal = i32::try_from(ordinal).map_err(|_| { + ProjectionError::InvalidEventData( + "AtomContextRegistered URI ordinal exceeds int32".to_owned(), + ) + })?; + let encoded = raw.strip_prefix("0x").ok_or_else(|| { + ProjectionError::InvalidEventData(format!( + "context URI at ordinal {ordinal} is not 0x-prefixed hex" + )) + })?; + let bytes = hex::decode(encoded).map_err(|_| { + ProjectionError::InvalidEventData(format!( + "context URI at ordinal {ordinal} is not valid even-length hex" + )) + })?; + + // Re-encoding makes the stored representation canonical without + // changing a byte of the underlying opaque value. + let uri_hex = format!("0x{}", hex::encode(&bytes)); + let uri_text = String::from_utf8(bytes) + .ok() + .filter(|value| !value.contains('\0')); + + Ok(ProjectedUri { + ordinal, + uri_hex, + uri_text, + }) + }) + .collect() +} + +fn context_event_id(transaction_hash: &str, log_index: i32) -> String { + format!("atom_context:{transaction_hash}:{log_index}") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn projection_preserves_order_and_duplicates() { + let input = vec![ + "0x69703a666f6f".to_owned(), + "0x69703a626172".to_owned(), + "0x69703a666f6f".to_owned(), + ]; + + let rows = project_uris(&input).expect("valid opaque bytes"); + + assert_eq!( + rows.iter().map(|row| row.ordinal).collect::>(), + [0, 1, 2] + ); + assert_eq!(rows[0].uri_hex, rows[2].uri_hex); + assert_eq!(rows[0].uri_text.as_deref(), Some("ip:foo")); + assert_eq!(rows[1].uri_text.as_deref(), Some("ip:bar")); + } + + #[test] + fn unsafe_bytes_remain_in_hex_without_lossy_text() { + let rows = project_uris(&[ + "0xfffe".to_owned(), + "0x610062".to_owned(), + "0x68747470733a2f2f6578616d706c652e636f6d".to_owned(), + ]) + .expect("all values are valid bytes"); + + assert_eq!(rows[0].uri_hex, "0xfffe"); + assert_eq!(rows[0].uri_text, None); + assert_eq!(rows[1].uri_hex, "0x610062"); + assert_eq!(rows[1].uri_text, None); + assert_eq!(rows[2].uri_text.as_deref(), Some("https://example.com")); + } + + #[test] + fn canonicalizes_hex_representation_without_normalizing_uri_bytes() { + let rows = project_uris(&["0x49503A414243".to_owned()]).expect("valid bytes"); + + assert_eq!(rows[0].uri_hex, "0x49503a414243"); + assert_eq!(rows[0].uri_text.as_deref(), Some("IP:ABC")); + } + + #[test] + fn malformed_hex_is_rejected_so_checkpoint_cannot_advance() { + assert!(matches!( + project_uris(&["ip:abc".to_owned()]), + Err(ProjectionError::InvalidEventData(_)) + )); + assert!(matches!( + project_uris(&["0xabc".to_owned()]), + Err(ProjectionError::InvalidEventData(_)) + )); + } + + #[test] + fn replay_keys_are_deterministic() { + let first = project_uris(&["0x61".to_owned(), "0x61".to_owned()]).unwrap(); + let replay = project_uris(&["0x61".to_owned(), "0x61".to_owned()]).unwrap(); + + assert_eq!(first, replay); + assert_eq!( + context_event_id("0xdeadbeef", 7), + context_event_id("0xdeadbeef", 7) + ); + assert_ne!(first[0].ordinal, first[1].ordinal); + assert!(CONTEXT_INSERT_SQL.contains("ON CONFLICT")); + assert!(CONTEXT_INSERT_SQL.contains("DO NOTHING")); + assert!(EVIDENCE_INSERT_SQL.contains("ON CONFLICT")); + assert!(EVIDENCE_INSERT_SQL.contains("DO NOTHING")); + } + + #[test] + fn missing_node_is_a_transient_database_dependency() { + let error = ProjectionError::from(sqlx::Error::RowNotFound); + assert_eq!(error.classify(), crate::error::ErrorClass::Transient); + } +} diff --git a/crates/projections/src/projection/dual/mod.rs b/crates/projections/src/projection/dual/mod.rs index 3fcc8ce..f113696 100644 --- a/crates/projections/src/projection/dual/mod.rs +++ b/crates/projections/src/projection/dual/mod.rs @@ -1,3 +1,4 @@ +pub mod atom_context; pub mod core_entities; pub mod vault_holders_index; pub mod vault_state; diff --git a/crates/projections/src/projection/dual/vault_holders_index.rs b/crates/projections/src/projection/dual/vault_holders_index.rs index 7f83fa5..8b7a003 100644 --- a/crates/projections/src/projection/dual/vault_holders_index.rs +++ b/crates/projections/src/projection/dual/vault_holders_index.rs @@ -144,6 +144,7 @@ impl PgProjection for VaultHoldersIndexDualProjection { process_redeem_typed(&mut legacy_tx, kg_tx_opt.as_mut(), metadata, data).await } ParsedEvent::AtomCreated { .. } + | ParsedEvent::AtomContextRegistered { .. } | ParsedEvent::TripleCreated { .. } | ParsedEvent::SharePriceChanged { .. } | ParsedEvent::ProtocolFeeAccrued { .. } => { diff --git a/crates/projections/src/projection/dual/vault_state.rs b/crates/projections/src/projection/dual/vault_state.rs index ea2fed2..669f77a 100644 --- a/crates/projections/src/projection/dual/vault_state.rs +++ b/crates/projections/src/projection/dual/vault_state.rs @@ -230,6 +230,7 @@ impl PgProjection for VaultStateDualProjection { // Exhaustive match — adding a new ParsedEvent variant must // force a compile-time decision here, not silently drop. ParsedEvent::AtomCreated { .. } + | ParsedEvent::AtomContextRegistered { .. } | ParsedEvent::TripleCreated { .. } | ParsedEvent::ProtocolFeeAccrued { .. } => { continue; diff --git a/crates/projections/src/projection/timescaledb/account_registry.rs b/crates/projections/src/projection/timescaledb/account_registry.rs index 1ac9ce3..3781f76 100644 --- a/crates/projections/src/projection/timescaledb/account_registry.rs +++ b/crates/projections/src/projection/timescaledb/account_registry.rs @@ -7,6 +7,7 @@ //! //! Addresses are extracted from: //! - `AtomCreated` — `creator` +//! - `AtomContextRegistered` — `registrant` //! - `TripleCreated` — `creator` //! - `Deposited` — `sender`, `receiver` //! - `Redeemed` — `sender`, `receiver` @@ -46,6 +47,9 @@ fn extract_addresses_typed(event: &ParsedEvent) -> Vec<(String, DateTime)> ParsedEvent::AtomCreated { metadata, data } => { vec![(data.creator.clone(), metadata.block_timestamp)] } + ParsedEvent::AtomContextRegistered { metadata, data } => { + vec![(data.registrant.clone(), metadata.block_timestamp)] + } ParsedEvent::TripleCreated { metadata, data } => { vec![(data.creator.clone(), metadata.block_timestamp)] } @@ -91,6 +95,7 @@ impl PgProjection for AccountRegistryProjection { // SharePriceChanged is intentionally absent — it carries no addresses. &[ EventType::AtomCreated, + EventType::AtomContextRegistered, EventType::TripleCreated, EventType::Deposited, EventType::Redeemed, @@ -218,6 +223,17 @@ mod tests { vec![] } }, + "AtomContextRegistered" => match get_str(data, "registrant") { + Ok(addr) => vec![(addr, ts)], + Err(_) => { + tracing::warn!( + sequence_number = seq, + event_type = %event.event_type, + "Missing registrant field; skipping address extraction" + ); + vec![] + } + }, "Deposited" | "Redeemed" => { let sender = get_str(data, "sender"); let receiver = get_str(data, "receiver"); @@ -279,9 +295,10 @@ mod tests { #[test] fn event_types_excludes_share_price_changed() { let types = AccountRegistryProjection.event_types(); - assert_eq!(types.len(), 5); + assert_eq!(types.len(), 6); assert!(!types.contains(&EventType::SharePriceChanged)); assert!(types.contains(&EventType::AtomCreated)); + assert!(types.contains(&EventType::AtomContextRegistered)); assert!(types.contains(&EventType::TripleCreated)); assert!(types.contains(&EventType::Deposited)); assert!(types.contains(&EventType::Redeemed)); @@ -296,6 +313,17 @@ mod tests { assert_eq!(addrs[0].0, "0xCreator"); } + #[test] + fn extract_addresses_atom_context_registered() { + let event = make_event( + "AtomContextRegistered", + json!({ "registrant": "0xRegistrant" }), + ); + let addrs = extract_addresses(&event); + assert_eq!(addrs.len(), 1); + assert_eq!(addrs[0].0, "0xRegistrant"); + } + #[test] fn extract_addresses_triple_created() { let event = make_event("TripleCreated", json!({ "creator": "0xTripleCreator" })); @@ -420,6 +448,22 @@ mod tests { assert_eq!(addrs[0].0, "0xCreator"); } + #[test] + fn typed_atom_context_registered_returns_registrant_and_timestamp() { + let event = make_parsed( + "AtomContextRegistered", + serde_json::json!({ + "registrant": "0xRegistrant", + "term_id": "0x07", + "uris": ["0xff00", "0xff00"] + }), + ); + let expected_timestamp = event.metadata().block_timestamp(); + let addrs = extract_addresses_typed(&event); + + assert_eq!(addrs, vec![("0xRegistrant".to_owned(), expected_timestamp)]); + } + #[test] fn typed_deposited_returns_sender_and_receiver() { let event = make_parsed( diff --git a/crates/projections/src/projection/timescaledb/activity_marker.rs b/crates/projections/src/projection/timescaledb/activity_marker.rs index bb889c9..585f809 100644 --- a/crates/projections/src/projection/timescaledb/activity_marker.rs +++ b/crates/projections/src/projection/timescaledb/activity_marker.rs @@ -209,7 +209,9 @@ impl PgProjection for ActivityMarkerProjection { ParsedEvent::Redeemed { metadata, data } => { handle_redeemed_typed(&mut tx, metadata, data).await } - ParsedEvent::SharePriceChanged { .. } | ParsedEvent::ProtocolFeeAccrued { .. } => { + ParsedEvent::AtomContextRegistered { .. } + | ParsedEvent::SharePriceChanged { .. } + | ParsedEvent::ProtocolFeeAccrued { .. } => { // Filtered by event_types(). continue; } diff --git a/crates/projections/src/projection/timescaledb/event_log.rs b/crates/projections/src/projection/timescaledb/event_log.rs index 6d112c3..25f8570 100644 --- a/crates/projections/src/projection/timescaledb/event_log.rs +++ b/crates/projections/src/projection/timescaledb/event_log.rs @@ -48,6 +48,7 @@ impl PgProjection for EventLogProjection { fn event_types(&self) -> &'static [EventType] { &[ EventType::AtomCreated, + EventType::AtomContextRegistered, EventType::TripleCreated, EventType::Deposited, EventType::Redeemed, @@ -63,7 +64,7 @@ impl PgProjection for EventLogProjection { /// Process a batch of pre-parsed typed events, writing to `event` and the /// three financial fact tables. /// - /// Handles all six event types: `Deposited` and `Redeemed` write fact rows; + /// Handles all seven event types: `Deposited` and `Redeemed` write fact rows; /// `ProtocolFeeAccrued` writes a fee row; `AtomCreated`, `TripleCreated`, /// and `SharePriceChanged` write only the canonical event row. `Unknown` /// events are warned and skipped. @@ -122,8 +123,9 @@ impl PgProjection for EventLogProjection { ParsedEvent::ProtocolFeeAccrued { metadata, data } => { insert_fee_transfer_fact_typed(&mut tx, metadata, data, &event_id).await } - // AtomCreated, TripleCreated, SharePriceChanged: canonical row only. + // Creation/context/price events only contribute the canonical row. ParsedEvent::AtomCreated { .. } + | ParsedEvent::AtomContextRegistered { .. } | ParsedEvent::TripleCreated { .. } | ParsedEvent::SharePriceChanged { .. } => { continue; @@ -332,10 +334,11 @@ mod tests { } #[test] - fn event_types_contains_all_six() { + fn event_types_contains_all_seven() { let types = EventLogProjection.event_types(); - assert_eq!(types.len(), 6); + assert_eq!(types.len(), 7); assert!(types.contains(&EventType::AtomCreated)); + assert!(types.contains(&EventType::AtomContextRegistered)); assert!(types.contains(&EventType::TripleCreated)); assert!(types.contains(&EventType::Deposited)); assert!(types.contains(&EventType::Redeemed)); diff --git a/crates/projections/src/projection/timescaledb/leaderboard_marker.rs b/crates/projections/src/projection/timescaledb/leaderboard_marker.rs index fa044fc..d0c3de9 100644 --- a/crates/projections/src/projection/timescaledb/leaderboard_marker.rs +++ b/crates/projections/src/projection/timescaledb/leaderboard_marker.rs @@ -303,6 +303,7 @@ impl PgProjection for LeaderboardMarkerProjection { handle_share_price_changed_typed(&mut tx, metadata, data).await } ParsedEvent::AtomCreated { .. } + | ParsedEvent::AtomContextRegistered { .. } | ParsedEvent::TripleCreated { .. } | ParsedEvent::ProtocolFeeAccrued { .. } => { // Filtered by event_types(). diff --git a/crates/projections/src/projection/timescaledb/position_tracking.rs b/crates/projections/src/projection/timescaledb/position_tracking.rs index 4a8ae7a..7ff59a6 100644 --- a/crates/projections/src/projection/timescaledb/position_tracking.rs +++ b/crates/projections/src/projection/timescaledb/position_tracking.rs @@ -125,6 +125,7 @@ impl PgProjection for PositionTrackingProjection { process_redeemed_typed(&mut tx, metadata, data).await } ParsedEvent::AtomCreated { .. } + | ParsedEvent::AtomContextRegistered { .. } | ParsedEvent::TripleCreated { .. } | ParsedEvent::SharePriceChanged { .. } | ParsedEvent::ProtocolFeeAccrued { .. } => { diff --git a/crates/projections/src/projection/timescaledb/protocol_stats.rs b/crates/projections/src/projection/timescaledb/protocol_stats.rs index 5d0ca2b..c8e5fed 100644 --- a/crates/projections/src/projection/timescaledb/protocol_stats.rs +++ b/crates/projections/src/projection/timescaledb/protocol_stats.rs @@ -220,8 +220,8 @@ fn accumulate_typed(d: &mut Deltas, event: &ParsedEvent) { // `amount` is already `BigDecimal` — no parse needed. d.fees += &data.amount; } - ParsedEvent::SharePriceChanged { .. } => { - // SharePriceChanged has no stats impact. + ParsedEvent::AtomContextRegistered { .. } | ParsedEvent::SharePriceChanged { .. } => { + // Context registration and share-price updates have no stats impact. } ParsedEvent::Unknown(raw) => { warn!( diff --git a/crates/projections/src/projection/timescaledb/signals_analytics.rs b/crates/projections/src/projection/timescaledb/signals_analytics.rs index e06b97e..3c49748 100644 --- a/crates/projections/src/projection/timescaledb/signals_analytics.rs +++ b/crates/projections/src/projection/timescaledb/signals_analytics.rs @@ -73,6 +73,7 @@ impl PgProjection for SignalsAnalyticsProjection { insert_redemption_signal_typed(&mut tx, metadata, data).await } ParsedEvent::AtomCreated { .. } + | ParsedEvent::AtomContextRegistered { .. } | ParsedEvent::TripleCreated { .. } | ParsedEvent::SharePriceChanged { .. } | ParsedEvent::ProtocolFeeAccrued { .. } => { diff --git a/crates/projections/src/projection/timescaledb/term_aggregates.rs b/crates/projections/src/projection/timescaledb/term_aggregates.rs index a9305b6..fdb9829 100644 --- a/crates/projections/src/projection/timescaledb/term_aggregates.rs +++ b/crates/projections/src/projection/timescaledb/term_aggregates.rs @@ -71,6 +71,7 @@ impl PgProjection for TermAggregatesProjection { process_share_price_changed_typed(metadata, data, &mut tx).await } ParsedEvent::AtomCreated { .. } + | ParsedEvent::AtomContextRegistered { .. } | ParsedEvent::Deposited { .. } | ParsedEvent::Redeemed { .. } | ParsedEvent::ProtocolFeeAccrued { .. } => { diff --git a/crates/projections/src/projection/timescaledb/vault_holders_index.rs b/crates/projections/src/projection/timescaledb/vault_holders_index.rs index e43a1ba..4a8507a 100644 --- a/crates/projections/src/projection/timescaledb/vault_holders_index.rs +++ b/crates/projections/src/projection/timescaledb/vault_holders_index.rs @@ -80,6 +80,7 @@ impl PgProjection for VaultHoldersIndexProjection { process_redeem_typed(&mut tx, metadata, data).await } ParsedEvent::AtomCreated { .. } + | ParsedEvent::AtomContextRegistered { .. } | ParsedEvent::TripleCreated { .. } | ParsedEvent::SharePriceChanged { .. } | ParsedEvent::ProtocolFeeAccrued { .. } => { diff --git a/crates/rindexer-ingestion/README.md b/crates/rindexer-ingestion/README.md index 08dbca7..575fbce 100644 --- a/crates/rindexer-ingestion/README.md +++ b/crates/rindexer-ingestion/README.md @@ -6,5 +6,11 @@ This crate decodes MultiVault events through rindexer-generated code and writes the append-only event store consumed by projections. It is distributed as a container image target, not as a public library crate. +`event_store.event_type` is intentionally forward-compatible rather than a +database whitelist: generated event decoding and the per-event typed tables are +the canonical contract. `AtomContextRegistered` URI values are stored in +contract order as opaque `0x`-prefixed byte strings; ingestion does not decode, +normalize, fetch, or validate URI schemes. + Operators should configure event/block scope through the rindexer manifest and environment variables documented in `../../docs/indexing-scope.md`. diff --git a/crates/rindexer-ingestion/abi/MultiVault.json b/crates/rindexer-ingestion/abi/MultiVault.json index 501829c..8b9ccf5 100644 --- a/crates/rindexer-ingestion/abi/MultiVault.json +++ b/crates/rindexer-ingestion/abi/MultiVault.json @@ -69,6 +69,19 @@ ], "stateMutability": "view" }, + { + "type": "function", + "name": "PAUSER_ROLE", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "TRIPLE_SALT", @@ -136,7 +149,7 @@ } ], "outputs": [], - "stateMutability": "nonpayable" + "stateMutability": "payable" }, { "type": "function", @@ -175,6 +188,44 @@ ], "stateMutability": "view" }, + { + "type": "function", + "name": "atomCreatedAt", + "inputs": [ + { + "name": "atomId", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "createdAt", + "type": "uint48", + "internalType": "uint48" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "atomCreators", + "inputs": [ + { + "name": "atomId", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "atomDepositFractionAmount", @@ -403,6 +454,69 @@ ], "stateMutability": "payable" }, + { + "type": "function", + "name": "createAtomsFor", + "inputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + }, + { + "name": "data", + "type": "bytes[]", + "internalType": "bytes[]" + }, + { + "name": "assets", + "type": "uint256[]", + "internalType": "uint256[]" + } + ], + "outputs": [ + { + "name": "", + "type": "bytes32[]", + "internalType": "bytes32[]" + } + ], + "stateMutability": "payable" + }, + { + "type": "function", + "name": "createAtomsWithUris", + "inputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + }, + { + "name": "data", + "type": "bytes[]", + "internalType": "bytes[]" + }, + { + "name": "assets", + "type": "uint256[]", + "internalType": "uint256[]" + }, + { + "name": "uris", + "type": "bytes[][]", + "internalType": "bytes[][]" + } + ], + "outputs": [ + { + "name": "", + "type": "bytes32[]", + "internalType": "bytes32[]" + } + ], + "stateMutability": "payable" + }, { "type": "function", "name": "createTriples", @@ -437,6 +551,45 @@ ], "stateMutability": "payable" }, + { + "type": "function", + "name": "createTriplesFor", + "inputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + }, + { + "name": "subjectIds", + "type": "bytes32[]", + "internalType": "bytes32[]" + }, + { + "name": "predicateIds", + "type": "bytes32[]", + "internalType": "bytes32[]" + }, + { + "name": "objectIds", + "type": "bytes32[]", + "internalType": "bytes32[]" + }, + { + "name": "assets", + "type": "uint256[]", + "internalType": "uint256[]" + } + ], + "outputs": [ + { + "name": "", + "type": "bytes32[]", + "internalType": "bytes32[]" + } + ], + "stateMutability": "payable" + }, { "type": "function", "name": "currentEpoch", @@ -540,7 +693,7 @@ ], "outputs": [ { - "name": "shares", + "name": "", "type": "uint256[]", "internalType": "uint256[]" } @@ -690,6 +843,62 @@ ], "stateMutability": "view" }, + { + "type": "function", + "name": "getAtomCreatedAt", + "inputs": [ + { + "name": "termId", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "", + "type": "uint48", + "internalType": "uint48" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "getAtomCreator", + "inputs": [ + { + "name": "termId", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "", + "type": "address", + "internalType": "address" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "getAtomUriConfig", + "inputs": [], + "outputs": [ + { + "name": "maxUriCount", + "type": "uint32", + "internalType": "uint32" + }, + { + "name": "maxUriLength", + "type": "uint32", + "internalType": "uint32" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "getAtomWarden", @@ -1370,6 +1579,78 @@ "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "isApprovedToCreate", + "inputs": [ + { + "name": "sender", + "type": "address", + "internalType": "address" + }, + { + "name": "creator", + "type": "address", + "internalType": "address" + } + ], + "outputs": [ + { + "name": "approved", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "isApprovedToDeposit", + "inputs": [ + { + "name": "sender", + "type": "address", + "internalType": "address" + }, + { + "name": "receiver", + "type": "address", + "internalType": "address" + } + ], + "outputs": [ + { + "name": "approved", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "isApprovedToRedeem", + "inputs": [ + { + "name": "sender", + "type": "address", + "internalType": "address" + }, + { + "name": "receiver", + "type": "address", + "internalType": "address" + } + ], + "outputs": [ + { + "name": "approved", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "isAtom", @@ -1446,6 +1727,19 @@ ], "stateMutability": "view" }, + { + "type": "function", + "name": "lastSystemUtilizationEpoch", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "uint256", + "internalType": "uint256" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "maxRedeem", @@ -1475,6 +1769,30 @@ ], "stateMutability": "view" }, + { + "type": "function", + "name": "multicall", + "inputs": [ + { + "name": "data", + "type": "bytes[]", + "internalType": "bytes[]" + }, + { + "name": "values", + "type": "uint256[]", + "internalType": "uint256[]" + } + ], + "outputs": [ + { + "name": "results", + "type": "bytes[]", + "internalType": "bytes[]" + } + ], + "stateMutability": "payable" + }, { "type": "function", "name": "pause", @@ -1711,7 +2029,7 @@ "internalType": "uint256" } ], - "stateMutability": "nonpayable" + "stateMutability": "payable" }, { "type": "function", @@ -1745,11 +2063,24 @@ ], "outputs": [ { - "name": "received", + "name": "", "type": "uint256[]", "internalType": "uint256[]" } ], + "stateMutability": "payable" + }, + { + "type": "function", + "name": "reinitialize", + "inputs": [ + { + "name": "_timelock", + "type": "address", + "internalType": "address" + } + ], + "outputs": [], "stateMutability": "nonpayable" }, { @@ -1813,6 +2144,24 @@ "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "setAtomUriConfig", + "inputs": [ + { + "name": "maxUriCount", + "type": "uint32", + "internalType": "uint32" + }, + { + "name": "maxUriLength", + "type": "uint32", + "internalType": "uint32" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, { "type": "function", "name": "setBondingCurveConfig", @@ -1893,6 +2242,19 @@ "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "setTimelock", + "inputs": [ + { + "name": "_timelock", + "type": "address", + "internalType": "address" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, { "type": "function", "name": "setTripleConfig", @@ -2015,6 +2377,19 @@ "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "timelock", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "address", + "internalType": "address" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "totalTermsCreated", @@ -2220,6 +2595,31 @@ ], "anonymous": false }, + { + "type": "event", + "name": "AtomContextRegistered", + "inputs": [ + { + "name": "termId", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "registrant", + "type": "address", + "indexed": true, + "internalType": "address" + }, + { + "name": "uris", + "type": "bytes[]", + "indexed": false, + "internalType": "bytes[]" + } + ], + "anonymous": false + }, { "type": "event", "name": "AtomCreated", @@ -2251,6 +2651,25 @@ ], "anonymous": false }, + { + "type": "event", + "name": "AtomUriConfigUpdated", + "inputs": [ + { + "name": "maxUriCount", + "type": "uint32", + "indexed": false, + "internalType": "uint32" + }, + { + "name": "maxUriLength", + "type": "uint32", + "indexed": false, + "internalType": "uint32" + } + ], + "anonymous": false + }, { "type": "event", "name": "AtomWalletDepositFeeCollected", @@ -2753,6 +3172,19 @@ ], "anonymous": false }, + { + "type": "event", + "name": "TimelockSet", + "inputs": [ + { + "name": "timelock", + "type": "address", + "indexed": true, + "internalType": "address" + } + ], + "anonymous": false + }, { "type": "event", "name": "TotalUtilizationAdded", @@ -3065,6 +3497,16 @@ } ] }, + { + "type": "error", + "name": "MultiVault_AtomUriCountExceeded", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_AtomUriLengthExceeded", + "inputs": [] + }, { "type": "error", "name": "MultiVault_BurnFromZeroAddress", @@ -3085,6 +3527,11 @@ "name": "MultiVault_CannotDirectlyInitializeCounterTriple", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_CreatorNotApproved", + "inputs": [] + }, { "type": "error", "name": "MultiVault_DefaultCurveMustBeInitializedViaCreatePaths", @@ -3146,11 +3593,26 @@ "name": "MultiVault_InvalidArrayLength", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_InvalidAtomUriConfig", + "inputs": [] + }, { "type": "error", "name": "MultiVault_InvalidEpoch", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_MulticallValueMismatch", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_NestedMulticall", + "inputs": [] + }, { "type": "error", "name": "MultiVault_NoAtomDataProvided", @@ -3161,6 +3623,16 @@ "name": "MultiVault_OnlyAssociatedAtomWallet", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_OnlyTimelock", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_RedeemYieldsNoAssets", + "inputs": [] + }, { "type": "error", "name": "MultiVault_RedeemerNotApproved", @@ -3218,6 +3690,16 @@ } ] }, + { + "type": "error", + "name": "MultiVault_UnexpectedValue", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_ZeroAddress", + "inputs": [] + }, { "type": "error", "name": "NotInitializing", diff --git a/crates/rindexer-ingestion/rindexer.yaml b/crates/rindexer-ingestion/rindexer.yaml index 22efb4a..8ea9339 100644 --- a/crates/rindexer-ingestion/rindexer.yaml +++ b/crates/rindexer-ingestion/rindexer.yaml @@ -52,6 +52,7 @@ ${MULTIVAULT_END_BLOCK_LINE} abi: ./abi/MultiVault.json include_events: - name: AtomCreated + - name: AtomContextRegistered - name: TripleCreated - name: Deposited - name: Redeemed diff --git a/crates/rindexer-ingestion/src/handlers.rs b/crates/rindexer-ingestion/src/handlers.rs index be5874e..6c54023 100644 --- a/crates/rindexer-ingestion/src/handlers.rs +++ b/crates/rindexer-ingestion/src/handlers.rs @@ -17,14 +17,14 @@ use tracing::{error, info}; use crate::metrics; use crate::rindexer_lib::typings::be_v_3_indexer::events::multi_vault::{ - AtomCreatedEvent, AtomCreatedResult, DepositedEvent, DepositedResult, EventContext, - MultiVaultEventType, ProtocolFeeAccruedEvent, ProtocolFeeAccruedResult, RedeemedEvent, - RedeemedResult, SharePriceChangedEvent, SharePriceChangedResult, TripleCreatedEvent, - TripleCreatedResult, + AtomContextRegisteredEvent, AtomContextRegisteredResult, AtomCreatedEvent, AtomCreatedResult, + DepositedEvent, DepositedResult, EventContext, MultiVaultEventType, ProtocolFeeAccruedEvent, + ProtocolFeeAccruedResult, RedeemedEvent, RedeemedResult, SharePriceChangedEvent, + SharePriceChangedResult, TripleCreatedEvent, TripleCreatedResult, }; use crate::storage::{ - AtomCreatedTyped, DepositedTyped, EventRecord, EventStoreStorage, ProtocolFeeAccruedTyped, - RedeemedTyped, SharePriceChangedTyped, TripleCreatedTyped, + AtomContextRegisteredTyped, AtomCreatedTyped, DepositedTyped, EventRecord, EventStoreStorage, + ProtocolFeeAccruedTyped, RedeemedTyped, SharePriceChangedTyped, TripleCreatedTyped, }; /// Start the rindexer with custom event handlers @@ -56,6 +56,7 @@ pub async fn start_indexer( // Register all event handlers register_atom_created_handler(&manifest_path, &mut registry, storage.clone()).await; + register_atom_context_registered_handler(&manifest_path, &mut registry, storage.clone()).await; register_triple_created_handler(&manifest_path, &mut registry, storage.clone()).await; register_deposited_handler(&manifest_path, &mut registry, storage.clone()).await; register_redeemed_handler(&manifest_path, &mut registry, storage.clone()).await; @@ -137,6 +138,22 @@ fn b256_to_bd(b: &alloy::primitives::FixedBytes<32>) -> BigDecimal { BigDecimal::from_str(&num.to_string()).expect("U256 always produces valid decimal") } +/// Encode contract bytes without interpreting, normalizing, or validating them. +fn opaque_bytes_to_hex(bytes: &[u8]) -> String { + format!("0x{}", hex::encode(bytes)) +} + +#[cfg(test)] +mod tests { + use super::opaque_bytes_to_hex; + + #[test] + fn opaque_uri_bytes_are_preserved_as_lowercase_hex() { + assert_eq!(opaque_bytes_to_hex(&[0xff, 0x00, b':', 0x80]), "0xff003a80"); + assert_eq!(opaque_bytes_to_hex(&[]), "0x"); + } +} + /// Maximum results to convert + insert per sub-batch inside a handler callback. /// Keeps peak memory bounded when rindexer delivers 400k+ events in one call. /// Override with HANDLER_CHUNK_SIZE env var (default: 50000). @@ -250,6 +267,111 @@ async fn register_atom_created_handler( .await; } +/// Register AtomContextRegistered event handler. +async fn register_atom_context_registered_handler( + manifest_path: &PathBuf, + registry: &mut EventCallbackRegistry, + storage: Arc, +) { + let handler = AtomContextRegisteredEvent::handler( + |results: Vec, + context: Arc>>| async move { + if results.is_empty() { + return Ok(()); + } + + let total = results.len(); + let storage = &context.extensions; + + for chunk in results.chunks(handler_chunk_size()) { + let mut events: Vec = Vec::with_capacity(chunk.len()); + let mut typed: Vec = Vec::with_capacity(chunk.len()); + + for result in chunk { + let tx = &result.tx_information; + let event = &result.event_data; + + let registrant = format!("{:?}", event.registrant); + let term_id_bd = b256_to_bd(&event.termId); + let term_id_hex = format!("{:?}", event.termId); + // Preserve contract order and exact bytes. These values are + // deliberately not decoded as UTF-8 or treated as URLs. + let uris: Vec = event + .uris + .iter() + .map(|uri| opaque_bytes_to_hex(uri.as_ref())) + .collect(); + let block_ts = timestamp_to_datetime(tx); + let block_hash = format!("{:?}", tx.block_hash); + let transaction_hash = format!("{:?}", tx.transaction_hash); + let block_number = tx.block_number as i64; + let log_index = tx.log_index.to::(); + + let event_data = json!({ + "term_id": &term_id_hex, + "registrant": ®istrant, + "uris": &uris, + }); + + events.push(EventRecord { + block_number, + block_timestamp: block_ts, + block_hash: block_hash.clone(), + transaction_hash: transaction_hash.clone(), + log_index, + event_type: "AtomContextRegistered".to_string(), + event_data, + }); + + typed.push(AtomContextRegisteredTyped { + block_number, + block_timestamp: block_ts, + block_hash, + transaction_hash, + log_index, + registrant, + term_id: term_id_bd, + term_id_hex, + uris: json!(uris), + }); + } + + if let Err(e) = storage + .insert_atom_context_registered_events(events, typed) + .await + { + error!("Failed to insert AtomContextRegistered events: {}", e); + return Err(e.to_string()); + } + } + + let max_block = results + .iter() + .map(|r| r.tx_information.block_number) + .max() + .unwrap_or(0); + metrics::record_events_with_block( + "AtomContextRegistered", + total as u64, + max_block, + None, + ); + + info!( + "AtomContextRegistered - INDEXED {} events (block {})", + total, max_block + ); + Ok(()) + }, + storage.clone(), + ) + .await; + + MultiVaultEventType::AtomContextRegistered(handler) + .register(manifest_path, registry) + .await; +} + /// Register TripleCreated event handler async fn register_triple_created_handler( manifest_path: &PathBuf, diff --git a/crates/rindexer-ingestion/src/main.rs b/crates/rindexer-ingestion/src/main.rs index 83958a1..7ef0bae 100644 --- a/crates/rindexer-ingestion/src/main.rs +++ b/crates/rindexer-ingestion/src/main.rs @@ -79,10 +79,12 @@ async fn main() -> Result<(), Box> { // Set the latest block for all event types for event_type in &[ "AtomCreated", + "AtomContextRegistered", "TripleCreated", "Deposited", "Redeemed", "SharePriceChanged", + "ProtocolFeeAccrued", ] { metrics::set_latest_block(event_type, latest_block); } diff --git a/crates/rindexer-ingestion/src/metrics.rs b/crates/rindexer-ingestion/src/metrics.rs index 8adf444..ae95b26 100644 --- a/crates/rindexer-ingestion/src/metrics.rs +++ b/crates/rindexer-ingestion/src/metrics.rs @@ -267,6 +267,7 @@ pub async fn start_metrics_server( // Initialize metrics with default values for all event types let event_types = [ "AtomCreated", + "AtomContextRegistered", "TripleCreated", "Deposited", "Redeemed", diff --git a/crates/rindexer-ingestion/src/rindexer_lib/indexers/be_v_3_indexer/multi_vault.rs b/crates/rindexer-ingestion/src/rindexer_lib/indexers/be_v_3_indexer/multi_vault.rs index 133a81c..b2f84a6 100644 --- a/crates/rindexer-ingestion/src/rindexer_lib/indexers/be_v_3_indexer/multi_vault.rs +++ b/crates/rindexer-ingestion/src/rindexer_lib/indexers/be_v_3_indexer/multi_vault.rs @@ -1,16 +1,43 @@ #![allow(non_snake_case)] use super::super::super::typings::be_v_3_indexer::events::multi_vault::{ - no_extensions, AtomCreatedEvent, DepositedEvent, MultiVaultEventType, RedeemedEvent, - SharePriceChangedEvent, TripleCreatedEvent, + no_extensions, AtomContextRegisteredEvent, AtomCreatedEvent, DepositedEvent, + MultiVaultEventType, ProtocolFeeAccruedEvent, RedeemedEvent, SharePriceChangedEvent, + TripleCreatedEvent, }; use alloy::primitives::{I256, U256, U64}; use rindexer::{ event::callback_registry::EventCallbackRegistry, rindexer_error, rindexer_info, - EthereumSqlTypeWrapper, PgType, RindexerColorize, + EthereumSqlTypeWrapper, PgType, }; use std::path::PathBuf; use std::sync::Arc; +async fn atom_context_registered_handler( + manifest_path: &PathBuf, + registry: &mut EventCallbackRegistry, +) { + let handler = AtomContextRegisteredEvent::handler( + |results, context| async move { + if results.is_empty() { + return Ok(()); + } + + rindexer_info!( + "MultiVault::AtomContextRegistered - INDEXED - {} events", + results.len(), + ); + + Ok(()) + }, + no_extensions(), + ) + .await; + + MultiVaultEventType::AtomContextRegistered(handler) + .register(manifest_path, registry) + .await; +} + async fn atom_created_handler(manifest_path: &PathBuf, registry: &mut EventCallbackRegistry) { let handler = AtomCreatedEvent::handler( |results, context| async move { @@ -19,8 +46,7 @@ async fn atom_created_handler(manifest_path: &PathBuf, registry: &mut EventCallb } rindexer_info!( - "MultiVault::AtomCreated - {} - {} events", - "INDEXED".green(), + "MultiVault::AtomCreated - INDEXED - {} events", results.len(), ); @@ -37,14 +63,36 @@ async fn atom_created_handler(manifest_path: &PathBuf, registry: &mut EventCallb async fn deposited_handler(manifest_path: &PathBuf, registry: &mut EventCallbackRegistry) { let handler = DepositedEvent::handler( + |results, context| async move { + if results.is_empty() { + return Ok(()); + } + + rindexer_info!("MultiVault::Deposited - INDEXED - {} events", results.len(),); + + Ok(()) + }, + no_extensions(), + ) + .await; + + MultiVaultEventType::Deposited(handler) + .register(manifest_path, registry) + .await; +} + +async fn protocol_fee_accrued_handler( + manifest_path: &PathBuf, + registry: &mut EventCallbackRegistry, +) { + let handler = ProtocolFeeAccruedEvent::handler( |results, context| async move { if results.is_empty() { return Ok(()); } rindexer_info!( - "MultiVault::Deposited - {} - {} events", - "INDEXED".green(), + "MultiVault::ProtocolFeeAccrued - INDEXED - {} events", results.len(), ); @@ -54,7 +102,7 @@ async fn deposited_handler(manifest_path: &PathBuf, registry: &mut EventCallback ) .await; - MultiVaultEventType::Deposited(handler) + MultiVaultEventType::ProtocolFeeAccrued(handler) .register(manifest_path, registry) .await; } @@ -66,11 +114,7 @@ async fn redeemed_handler(manifest_path: &PathBuf, registry: &mut EventCallbackR return Ok(()); } - rindexer_info!( - "MultiVault::Redeemed - {} - {} events", - "INDEXED".green(), - results.len(), - ); + rindexer_info!("MultiVault::Redeemed - INDEXED - {} events", results.len(),); Ok(()) }, @@ -94,8 +138,7 @@ async fn share_price_changed_handler( } rindexer_info!( - "MultiVault::SharePriceChanged - {} - {} events", - "INDEXED".green(), + "MultiVault::SharePriceChanged - INDEXED - {} events", results.len(), ); @@ -118,8 +161,7 @@ async fn triple_created_handler(manifest_path: &PathBuf, registry: &mut EventCal } rindexer_info!( - "MultiVault::TripleCreated - {} - {} events", - "INDEXED".green(), + "MultiVault::TripleCreated - INDEXED - {} events", results.len(), ); @@ -134,10 +176,14 @@ async fn triple_created_handler(manifest_path: &PathBuf, registry: &mut EventCal .await; } pub async fn multi_vault_handlers(manifest_path: &PathBuf, registry: &mut EventCallbackRegistry) { + atom_context_registered_handler(manifest_path, registry).await; + atom_created_handler(manifest_path, registry).await; deposited_handler(manifest_path, registry).await; + protocol_fee_accrued_handler(manifest_path, registry).await; + redeemed_handler(manifest_path, registry).await; share_price_changed_handler(manifest_path, registry).await; diff --git a/crates/rindexer-ingestion/src/rindexer_lib/typings/be_v_3_indexer/events/multi_vault.rs b/crates/rindexer-ingestion/src/rindexer_lib/typings/be_v_3_indexer/events/multi_vault.rs index f7cabe9..15e7c29 100644 --- a/crates/rindexer-ingestion/src/rindexer_lib/typings/be_v_3_indexer/events/multi_vault.rs +++ b/crates/rindexer-ingestion/src/rindexer_lib/typings/be_v_3_indexer/events/multi_vault.rs @@ -70,6 +70,20 @@ impl HasTxInformation for AtomConfigUpdatedResult { } } +pub type AtomContextRegisteredData = RindexerMultiVaultGen::AtomContextRegistered; + +#[derive(Debug, Clone)] +pub struct AtomContextRegisteredResult { + pub event_data: AtomContextRegisteredData, + pub tx_information: TxInformation, +} + +impl HasTxInformation for AtomContextRegisteredResult { + fn tx_information(&self) -> &TxInformation { + &self.tx_information + } +} + pub type AtomCreatedData = RindexerMultiVaultGen::AtomCreated; #[derive(Debug, Clone)] @@ -84,6 +98,20 @@ impl HasTxInformation for AtomCreatedResult { } } +pub type AtomUriConfigUpdatedData = RindexerMultiVaultGen::AtomUriConfigUpdated; + +#[derive(Debug, Clone)] +pub struct AtomUriConfigUpdatedResult { + pub event_data: AtomUriConfigUpdatedData, + pub tx_information: TxInformation, +} + +impl HasTxInformation for AtomUriConfigUpdatedResult { + fn tx_information(&self) -> &TxInformation { + &self.tx_information + } +} + pub type AtomWalletDepositFeeCollectedData = RindexerMultiVaultGen::AtomWalletDepositFeeCollected; #[derive(Debug, Clone)] @@ -308,6 +336,20 @@ impl HasTxInformation for SharePriceChangedResult { } } +pub type TimelockSetData = RindexerMultiVaultGen::TimelockSet; + +#[derive(Debug, Clone)] +pub struct TimelockSetResult { + pub event_data: TimelockSetData, + pub tx_information: TxInformation, +} + +impl HasTxInformation for TimelockSetResult { + fn tx_information(&self) -> &TxInformation { + &self.tx_information + } +} + pub type TotalUtilizationAddedData = RindexerMultiVaultGen::TotalUtilizationAdded; #[derive(Debug, Clone)] @@ -427,6 +469,101 @@ pub fn no_extensions() -> NoExtensions { NoExtensions {} } +pub fn atomcontextregistered_handler( + custom_logic: F, +) -> AtomContextRegisteredEventCallbackType +where + AtomContextRegisteredResult: Clone + 'static, + F: for<'a> Fn(Vec, Arc>) -> Fut + + Send + + Sync + + 'static + + Clone, + Fut: Future> + Send + 'static, + TExtensions: Send + Sync + 'static, +{ + Arc::new(move |results, context| { + let custom_logic = custom_logic.clone(); + let results = results.clone(); + let context = Arc::clone(&context); + async move { (custom_logic)(results, context).await }.boxed() + }) +} + +type AtomContextRegisteredEventCallbackType = Arc< + dyn for<'a> Fn( + &'a Vec, + Arc>, + ) -> BoxFuture<'a, EventCallbackResult<()>> + + Send + + Sync, +>; + +pub struct AtomContextRegisteredEvent +where + TExtensions: Send + Sync + 'static, +{ + callback: AtomContextRegisteredEventCallbackType, + context: Arc>, +} + +impl AtomContextRegisteredEvent +where + TExtensions: Send + Sync + 'static, +{ + pub async fn handler(closure: F, extensions: TExtensions) -> Self + where + AtomContextRegisteredResult: Clone + 'static, + F: for<'a> Fn(Vec, Arc>) -> Fut + + Send + + Sync + + 'static + + Clone, + Fut: Future> + Send + 'static, + { + Self { + callback: atomcontextregistered_handler(closure), + context: Arc::new(EventContext { + extensions: Arc::new(extensions), + }), + } + } +} + +#[async_trait] +impl EventCallback for AtomContextRegisteredEvent +where + TExtensions: Send + Sync, +{ + async fn call(&self, events: Vec) -> EventCallbackResult<()> { + let events_len = events.len(); + + // note some can not downcast because it cant decode + // this happens on events which failed decoding due to + // not having the right abi for example + // transfer events with 2 indexed topics cant decode + // transfer events with 3 indexed topics + let result: Vec = events + .into_iter() + .filter_map(|item| { + item.decoded_data + .downcast::() + .ok() + .map(|arc| AtomContextRegisteredResult { + event_data: (*arc).clone(), + tx_information: item.tx_information, + }) + }) + .collect(); + + if result.len() == events_len { + (self.callback)(&result, Arc::clone(&self.context)).await + } else { + panic!("AtomContextRegisteredEvent: Unexpected data type - expected: AtomContextRegisteredData") + } + } +} + pub fn atomcreated_handler( custom_logic: F, ) -> AtomCreatedEventCallbackType @@ -1003,6 +1140,7 @@ pub enum MultiVaultEventType where TExtensions: 'static + Send + Sync, { + AtomContextRegistered(AtomContextRegisteredEvent), AtomCreated(AtomCreatedEvent), Deposited(DepositedEvent), ProtocolFeeAccrued(ProtocolFeeAccruedEvent), @@ -1014,7 +1152,7 @@ where pub async fn multi_vault_contract( network: &str, ) -> RindexerMultiVaultGenInstance, AnyNetwork> { - let address: Address = "0xebc49d356b7f64d888130d85cc6d17114a6843ec" + let address: Address = "0x0000000000000000000000000000000000000001" .parse() .expect("Invalid address"); RindexerMultiVaultGen::new( @@ -1047,6 +1185,9 @@ where { pub fn topic_id(&self) -> &'static str { match self { + MultiVaultEventType::AtomContextRegistered(_) => { + "0x006dfca493b1686f1cc639fa9675dbd6f4a694a9d3230c346f3879d925382097" + } MultiVaultEventType::AtomCreated(_) => { "0xfd579ad7468b1720e08f84efe16900074f3ffdaaeaa82e675e5aec69bf393189" } @@ -1070,6 +1211,7 @@ where pub fn event_name(&self) -> &'static str { match self { + MultiVaultEventType::AtomContextRegistered(_) => "AtomContextRegistered", MultiVaultEventType::AtomCreated(_) => "AtomCreated", MultiVaultEventType::Deposited(_) => "Deposited", MultiVaultEventType::ProtocolFeeAccrued(_) => "ProtocolFeeAccrued", @@ -1094,6 +1236,18 @@ where let decoder_contract = decoder_contract(network); match self { + MultiVaultEventType::AtomContextRegistered(_) => { + Arc::new(move |topics: Vec, data: Bytes| { + match AtomContextRegisteredData::decode_raw_log(topics, &data[0..]) { + Ok(event) => { + let result: AtomContextRegisteredData = event; + Arc::new(result) as Arc + } + Err(error) => Arc::new(error) as Arc, + } + }) + } + MultiVaultEventType::AtomCreated(_) => Arc::new( move |topics: Vec, data: Bytes| match AtomCreatedData::decode_raw_log( topics, @@ -1191,7 +1345,7 @@ where let index_event_in_order = contract_details .index_event_in_order .as_ref() - .is_some_and(|vec| vec.contains(&event_name.to_string())); + .map_or(false, |vec| vec.contains(&event_name.to_string())); // Expect providers to have been initialized, but it's an async init so this should // be fast but for correctness we must await each future. @@ -1231,8 +1385,7 @@ where .networks .iter() .find(|n| n.name == c.network) - .map(|n| n.disable_logs_bloom_checks.unwrap_or_default()) - .unwrap_or(false), + .map_or(false, |n| n.disable_logs_bloom_checks.unwrap_or_default()), } }) .collect(), @@ -1243,6 +1396,14 @@ where let callback: Arc< dyn Fn(Vec) -> BoxFuture<'static, EventCallbackResult<()>> + Send + Sync, > = match self { + MultiVaultEventType::AtomContextRegistered(event) => { + let event = Arc::new(event); + Arc::new(move |result| { + let event = Arc::clone(&event); + async move { event.call(result).await }.boxed() + }) + } + MultiVaultEventType::AtomCreated(event) => { let event = Arc::new(event); Arc::new(move |result| { diff --git a/crates/rindexer-ingestion/src/rindexer_lib/typings/be_v_3_indexer/events/multi_vault_abi_gen.rs b/crates/rindexer-ingestion/src/rindexer_lib/typings/be_v_3_indexer/events/multi_vault_abi_gen.rs index 0151757..dc84721 100644 --- a/crates/rindexer-ingestion/src/rindexer_lib/typings/be_v_3_indexer/events/multi_vault_abi_gen.rs +++ b/crates/rindexer-ingestion/src/rindexer_lib/typings/be_v_3_indexer/events/multi_vault_abi_gen.rs @@ -1,5 +1,3 @@ -#![allow(clippy::too_many_arguments)] - use alloy::sol; sol!( @@ -7,8 +5,9 @@ sol!( RindexerMultiVaultGen, r#"[ { - "type": "receive", - "stateMutability": "payable" + "type": "constructor", + "inputs": [], + "stateMutability": "nonpayable" }, { "type": "function", @@ -77,7 +76,7 @@ sol!( }, { "type": "function", - "name": "MIGRATOR_ROLE", + "name": "PAUSER_ROLE", "inputs": [], "outputs": [ { @@ -155,7 +154,7 @@ sol!( } ], "outputs": [], - "stateMutability": "nonpayable" + "stateMutability": "payable" }, { "type": "function", @@ -196,128 +195,60 @@ sol!( }, { "type": "function", - "name": "atomDepositFractionAmount", + "name": "atomCreatedAt", "inputs": [ { - "name": "assets", - "type": "uint256", - "internalType": "uint256" + "name": "atomId", + "type": "bytes32", + "internalType": "bytes32" } ], "outputs": [ { - "name": "", - "type": "uint256", - "internalType": "uint256" + "name": "createdAt", + "type": "uint48", + "internalType": "uint48" } ], "stateMutability": "view" }, { "type": "function", - "name": "batchSetAtomData", + "name": "atomCreators", "inputs": [ { - "name": "creators", - "type": "address[]", - "internalType": "address[]" - }, - { - "name": "atomDataArray", - "type": "bytes[]", - "internalType": "bytes[]" + "name": "atomId", + "type": "bytes32", + "internalType": "bytes32" } ], - "outputs": [], - "stateMutability": "nonpayable" - }, - { - "type": "function", - "name": "batchSetTripleData", - "inputs": [ - { - "name": "creators", - "type": "address[]", - "internalType": "address[]" - }, + "outputs": [ { - "name": "tripleAtomIds", - "type": "bytes32[3][]", - "internalType": "bytes32[3][]" + "name": "creator", + "type": "address", + "internalType": "address" } ], - "outputs": [], - "stateMutability": "nonpayable" + "stateMutability": "view" }, { "type": "function", - "name": "batchSetUserBalances", + "name": "atomDepositFractionAmount", "inputs": [ { - "name": "params", - "type": "tuple", - "internalType": "struct MultiVaultMigrationMode.BatchSetUserBalancesParams", - "components": [ - { - "name": "termIds", - "type": "bytes32[][]", - "internalType": "bytes32[][]" - }, - { - "name": "bondingCurveId", - "type": "uint256", - "internalType": "uint256" - }, - { - "name": "users", - "type": "address[]", - "internalType": "address[]" - }, - { - "name": "userBalances", - "type": "uint256[][]", - "internalType": "uint256[][]" - } - ] + "name": "assets", + "type": "uint256", + "internalType": "uint256" } ], - "outputs": [], - "stateMutability": "nonpayable" - }, - { - "type": "function", - "name": "batchSetVaultTotals", - "inputs": [ - { - "name": "termIds", - "type": "bytes32[]", - "internalType": "bytes32[]" - }, + "outputs": [ { - "name": "bondingCurveId", + "name": "", "type": "uint256", "internalType": "uint256" - }, - { - "name": "vaultTotals", - "type": "tuple[]", - "internalType": "struct MultiVaultMigrationMode.VaultTotals[]", - "components": [ - { - "name": "totalAssets", - "type": "uint256", - "internalType": "uint256" - }, - { - "name": "totalShares", - "type": "uint256", - "internalType": "uint256" - } - ] } ], - "outputs": [], - "stateMutability": "nonpayable" + "stateMutability": "view" }, { "type": "function", @@ -528,6 +459,69 @@ sol!( ], "stateMutability": "payable" }, + { + "type": "function", + "name": "createAtomsFor", + "inputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + }, + { + "name": "data", + "type": "bytes[]", + "internalType": "bytes[]" + }, + { + "name": "assets", + "type": "uint256[]", + "internalType": "uint256[]" + } + ], + "outputs": [ + { + "name": "", + "type": "bytes32[]", + "internalType": "bytes32[]" + } + ], + "stateMutability": "payable" + }, + { + "type": "function", + "name": "createAtomsWithUris", + "inputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + }, + { + "name": "data", + "type": "bytes[]", + "internalType": "bytes[]" + }, + { + "name": "assets", + "type": "uint256[]", + "internalType": "uint256[]" + }, + { + "name": "uris", + "type": "bytes[][]", + "internalType": "bytes[][]" + } + ], + "outputs": [ + { + "name": "", + "type": "bytes32[]", + "internalType": "bytes32[]" + } + ], + "stateMutability": "payable" + }, { "type": "function", "name": "createTriples", @@ -562,6 +556,45 @@ sol!( ], "stateMutability": "payable" }, + { + "type": "function", + "name": "createTriplesFor", + "inputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + }, + { + "name": "subjectIds", + "type": "bytes32[]", + "internalType": "bytes32[]" + }, + { + "name": "predicateIds", + "type": "bytes32[]", + "internalType": "bytes32[]" + }, + { + "name": "objectIds", + "type": "bytes32[]", + "internalType": "bytes32[]" + }, + { + "name": "assets", + "type": "uint256[]", + "internalType": "uint256[]" + } + ], + "outputs": [ + { + "name": "", + "type": "bytes32[]", + "internalType": "bytes32[]" + } + ], + "stateMutability": "payable" + }, { "type": "function", "name": "currentEpoch", @@ -665,7 +698,7 @@ sol!( ], "outputs": [ { - "name": "shares", + "name": "", "type": "uint256[]", "internalType": "uint256[]" } @@ -815,6 +848,62 @@ sol!( ], "stateMutability": "view" }, + { + "type": "function", + "name": "getAtomCreatedAt", + "inputs": [ + { + "name": "termId", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "", + "type": "uint48", + "internalType": "uint48" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "getAtomCreator", + "inputs": [ + { + "name": "termId", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "", + "type": "address", + "internalType": "address" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "getAtomUriConfig", + "inputs": [], + "outputs": [ + { + "name": "maxUriCount", + "type": "uint32", + "internalType": "uint32" + }, + { + "name": "maxUriLength", + "type": "uint32", + "internalType": "uint32" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "getAtomWarden", @@ -1120,7 +1209,7 @@ sol!( }, { "type": "function", - "name": "getUserUtilization", + "name": "getUserUtilizationForEpoch", "inputs": [ { "name": "user", @@ -1144,7 +1233,7 @@ sol!( }, { "type": "function", - "name": "getUserUtilizationForEpoch", + "name": "getUserUtilizationInEpoch", "inputs": [ { "name": "user", @@ -1321,6 +1410,25 @@ sol!( ], "stateMutability": "view" }, + { + "type": "function", + "name": "hasRolledOverSystemUtilization", + "inputs": [ + { + "name": "epoch", + "type": "uint256", + "internalType": "uint256" + } + ], + "outputs": [ + { + "name": "hasRolledOver", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "initialize", @@ -1476,6 +1584,78 @@ sol!( "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "isApprovedToCreate", + "inputs": [ + { + "name": "sender", + "type": "address", + "internalType": "address" + }, + { + "name": "creator", + "type": "address", + "internalType": "address" + } + ], + "outputs": [ + { + "name": "approved", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "isApprovedToDeposit", + "inputs": [ + { + "name": "sender", + "type": "address", + "internalType": "address" + }, + { + "name": "receiver", + "type": "address", + "internalType": "address" + } + ], + "outputs": [ + { + "name": "approved", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "isApprovedToRedeem", + "inputs": [ + { + "name": "sender", + "type": "address", + "internalType": "address" + }, + { + "name": "receiver", + "type": "address", + "internalType": "address" + } + ], + "outputs": [ + { + "name": "approved", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "isAtom", @@ -1552,6 +1732,19 @@ sol!( ], "stateMutability": "view" }, + { + "type": "function", + "name": "lastSystemUtilizationEpoch", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "uint256", + "internalType": "uint256" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "maxRedeem", @@ -1581,6 +1774,30 @@ sol!( ], "stateMutability": "view" }, + { + "type": "function", + "name": "multicall", + "inputs": [ + { + "name": "data", + "type": "bytes[]", + "internalType": "bytes[]" + }, + { + "name": "values", + "type": "uint256[]", + "internalType": "uint256[]" + } + ], + "outputs": [ + { + "name": "results", + "type": "bytes[]", + "internalType": "bytes[]" + } + ], + "stateMutability": "payable" + }, { "type": "function", "name": "pause", @@ -1817,7 +2034,7 @@ sol!( "internalType": "uint256" } ], - "stateMutability": "nonpayable" + "stateMutability": "payable" }, { "type": "function", @@ -1851,11 +2068,24 @@ sol!( ], "outputs": [ { - "name": "received", + "name": "", "type": "uint256[]", "internalType": "uint256[]" } ], + "stateMutability": "payable" + }, + { + "type": "function", + "name": "reinitialize", + "inputs": [ + { + "name": "_timelock", + "type": "address", + "internalType": "address" + } + ], + "outputs": [], "stateMutability": "nonpayable" }, { @@ -1919,6 +2149,24 @@ sol!( "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "setAtomUriConfig", + "inputs": [ + { + "name": "maxUriCount", + "type": "uint32", + "internalType": "uint32" + }, + { + "name": "maxUriLength", + "type": "uint32", + "internalType": "uint32" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, { "type": "function", "name": "setBondingCurveConfig", @@ -2001,12 +2249,12 @@ sol!( }, { "type": "function", - "name": "setTermCount", + "name": "setTimelock", "inputs": [ { - "name": "_termCount", - "type": "uint256", - "internalType": "uint256" + "name": "_timelock", + "type": "address", + "internalType": "address" } ], "outputs": [], @@ -2134,6 +2382,19 @@ sol!( "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "timelock", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "address", + "internalType": "address" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "totalTermsCreated", @@ -2339,6 +2600,31 @@ sol!( ], "anonymous": false }, + { + "type": "event", + "name": "AtomContextRegistered", + "inputs": [ + { + "name": "termId", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "registrant", + "type": "address", + "indexed": true, + "internalType": "address" + }, + { + "name": "uris", + "type": "bytes[]", + "indexed": false, + "internalType": "bytes[]" + } + ], + "anonymous": false + }, { "type": "event", "name": "AtomCreated", @@ -2370,6 +2656,25 @@ sol!( ], "anonymous": false }, + { + "type": "event", + "name": "AtomUriConfigUpdated", + "inputs": [ + { + "name": "maxUriCount", + "type": "uint32", + "indexed": false, + "internalType": "uint32" + }, + { + "name": "maxUriLength", + "type": "uint32", + "indexed": false, + "internalType": "uint32" + } + ], + "anonymous": false + }, { "type": "event", "name": "AtomWalletDepositFeeCollected", @@ -2872,6 +3177,19 @@ sol!( ], "anonymous": false }, + { + "type": "event", + "name": "TimelockSet", + "inputs": [ + { + "name": "timelock", + "type": "address", + "indexed": true, + "internalType": "address" + } + ], + "anonymous": false + }, { "type": "event", "name": "TotalUtilizationAdded", @@ -3184,6 +3502,16 @@ sol!( } ] }, + { + "type": "error", + "name": "MultiVault_AtomUriCountExceeded", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_AtomUriLengthExceeded", + "inputs": [] + }, { "type": "error", "name": "MultiVault_BurnFromZeroAddress", @@ -3204,6 +3532,11 @@ sol!( "name": "MultiVault_CannotDirectlyInitializeCounterTriple", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_CreatorNotApproved", + "inputs": [] + }, { "type": "error", "name": "MultiVault_DefaultCurveMustBeInitializedViaCreatePaths", @@ -3267,7 +3600,7 @@ sol!( }, { "type": "error", - "name": "MultiVault_InvalidBondingCurveId", + "name": "MultiVault_InvalidAtomUriConfig", "inputs": [] }, { @@ -3275,6 +3608,16 @@ sol!( "name": "MultiVault_InvalidEpoch", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_MulticallValueMismatch", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_NestedMulticall", + "inputs": [] + }, { "type": "error", "name": "MultiVault_NoAtomDataProvided", @@ -3285,6 +3628,16 @@ sol!( "name": "MultiVault_OnlyAssociatedAtomWallet", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_OnlyTimelock", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_RedeemYieldsNoAssets", + "inputs": [] + }, { "type": "error", "name": "MultiVault_RedeemerNotApproved", @@ -3342,6 +3695,11 @@ sol!( } ] }, + { + "type": "error", + "name": "MultiVault_UnexpectedValue", + "inputs": [] + }, { "type": "error", "name": "MultiVault_ZeroAddress", diff --git a/crates/rindexer-ingestion/src/storage.rs b/crates/rindexer-ingestion/src/storage.rs index 91ef261..b839c3d 100644 --- a/crates/rindexer-ingestion/src/storage.rs +++ b/crates/rindexer-ingestion/src/storage.rs @@ -47,6 +47,20 @@ pub struct AtomCreatedTyped { pub atom_wallet: String, } +#[derive(Debug, Clone)] +pub struct AtomContextRegisteredTyped { + pub block_number: i64, + pub block_timestamp: DateTime, + pub block_hash: String, + pub transaction_hash: String, + pub log_index: i32, + pub registrant: String, + pub term_id: BigDecimal, + pub term_id_hex: String, + /// Ordered JSON array of opaque 0x-prefixed URI byte strings. + pub uris: serde_json::Value, +} + #[derive(Debug, Clone)] pub struct TripleCreatedTyped { pub block_number: i64, @@ -372,6 +386,80 @@ impl EventStoreStorage { Ok(()) } + /// Insert AtomContextRegistered events into event_store and the typed table. + pub async fn insert_atom_context_registered_events( + &self, + events: Vec, + typed: Vec, + ) -> Result<()> { + if events.is_empty() { + return Ok(()); + } + info!( + "Dual-write inserting {} AtomContextRegistered events", + events.len() + ); + + let mut tx = self.pool.begin().await?; + + Self::bulk_insert_event_store(&mut tx, &events).await?; + + let all_tx_hashes: Vec = typed.iter().map(|t| t.transaction_hash.clone()).collect(); + let all_log_indices: Vec = typed.iter().map(|t| t.log_index).collect(); + let all_block_ts: Vec> = typed.iter().map(|t| t.block_timestamp).collect(); + let seq_numbers = + Self::fetch_sequence_numbers(&mut tx, &all_tx_hashes, &all_log_indices, &all_block_ts) + .await?; + + for (chunk_idx, chunk) in typed.chunks(bulk_chunk_size()).enumerate() { + let offset = chunk_idx * bulk_chunk_size(); + let seq_chunk: Vec = seq_numbers[offset..offset + chunk.len()].to_vec(); + let block_numbers: Vec = chunk.iter().map(|t| t.block_number).collect(); + let block_timestamps: Vec> = + chunk.iter().map(|t| t.block_timestamp).collect(); + let block_hashes: Vec = chunk.iter().map(|t| t.block_hash.clone()).collect(); + let tx_hashes: Vec = chunk.iter().map(|t| t.transaction_hash.clone()).collect(); + let log_indices: Vec = chunk.iter().map(|t| t.log_index).collect(); + let registrants: Vec = chunk.iter().map(|t| t.registrant.clone()).collect(); + let term_ids: Vec = chunk.iter().map(|t| t.term_id.clone()).collect(); + let term_id_hexes: Vec = chunk.iter().map(|t| t.term_id_hex.clone()).collect(); + let uris: Vec = chunk.iter().map(|t| t.uris.clone()).collect(); + + sqlx::query( + r#" + INSERT INTO atom_context_registered_events ( + block_number, block_timestamp, block_hash, + transaction_hash, log_index, + registrant, term_id, term_id_hex, uris, + sequence_number + ) + SELECT * FROM UNNEST( + $1::BIGINT[], $2::TIMESTAMPTZ[], $3::TEXT[], + $4::TEXT[], $5::INT[], + $6::TEXT[], $7::NUMERIC[], $8::TEXT[], $9::JSONB[], + $10::BIGINT[] + ) + ON CONFLICT (transaction_hash, log_index) DO NOTHING + "#, + ) + .bind(&block_numbers) + .bind(&block_timestamps) + .bind(&block_hashes) + .bind(&tx_hashes) + .bind(&log_indices) + .bind(®istrants) + .bind(&term_ids) + .bind(&term_id_hexes) + .bind(&uris) + .bind(&seq_chunk) + .execute(&mut *tx) + .await?; + } + + tx.commit().await?; + Ok(()) + } + /// Insert TripleCreated events into both event_store and triple_created_events. pub async fn insert_triple_created_events( &self, diff --git a/crates/shared/src/models.rs b/crates/shared/src/models.rs index 6022cc1..410ab7a 100644 --- a/crates/shared/src/models.rs +++ b/crates/shared/src/models.rs @@ -144,6 +144,18 @@ pub struct AtomCreatedRecord { pub atom_wallet: String, } +/// Parsed payload from an `AtomContextRegistered` event's `event_data` column. +/// +/// `uris` preserves contract order, duplicates, and exact byte values as +/// `0x`-prefixed opaque strings. Readers must not decode or normalize them. +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AtomContextRegisteredRecord { + pub registrant: String, + /// Keccak256 hash of the atom term, stored as a `0x`-prefixed hex string. + pub term_id: String, + pub uris: Vec, +} + /// Parsed payload from a `TripleCreated` event's `event_data` column. /// /// All ID fields (`term_id`, `subject_id`, `predicate_id`, `object_id`) are diff --git a/crates/shared/src/parsed_event.rs b/crates/shared/src/parsed_event.rs index c260b21..dd20f71 100644 --- a/crates/shared/src/parsed_event.rs +++ b/crates/shared/src/parsed_event.rs @@ -30,8 +30,8 @@ use chrono::{DateTime, Utc}; use crate::models::{ - AtomCreatedRecord, DepositedRecord, ProtocolFeeAccruedRecord, RedeemedRecord, - SharePriceChangedRecord, StoredEvent, TripleCreatedRecord, + AtomContextRegisteredRecord, AtomCreatedRecord, DepositedRecord, ProtocolFeeAccruedRecord, + RedeemedRecord, SharePriceChangedRecord, StoredEvent, TripleCreatedRecord, }; use crate::types::{BlockNumber, LogIndex, SequenceNumber}; @@ -112,6 +112,11 @@ pub enum ParsedEvent { metadata: EventMetadata, data: AtomCreatedRecord, }, + /// `AtomContextRegistered` event with ordered opaque URI bytes. + AtomContextRegistered { + metadata: EventMetadata, + data: AtomContextRegisteredRecord, + }, /// `TripleCreated` event with validated fields. TripleCreated { metadata: EventMetadata, @@ -266,6 +271,12 @@ impl ParsedEvent { event.event_data ) } + EventType::AtomContextRegistered => parse_variant!( + EventType::AtomContextRegistered, + AtomContextRegistered, + metadata, + event.event_data + ), EventType::TripleCreated => parse_variant!( EventType::TripleCreated, TripleCreated, @@ -301,6 +312,7 @@ impl ParsedEvent { pub fn metadata(&self) -> EventMetadataRef<'_> { match self { Self::AtomCreated { metadata, .. } + | Self::AtomContextRegistered { metadata, .. } | Self::TripleCreated { metadata, .. } | Self::Deposited { metadata, .. } | Self::Redeemed { metadata, .. } @@ -315,6 +327,7 @@ impl ParsedEvent { pub fn event_type(&self) -> &str { match self { Self::AtomCreated { metadata, .. } + | Self::AtomContextRegistered { metadata, .. } | Self::TripleCreated { metadata, .. } | Self::Deposited { metadata, .. } | Self::Redeemed { metadata, .. } @@ -329,6 +342,7 @@ impl ParsedEvent { pub fn sequence_number(&self) -> SequenceNumber { match self { Self::AtomCreated { metadata, .. } + | Self::AtomContextRegistered { metadata, .. } | Self::TripleCreated { metadata, .. } | Self::Deposited { metadata, .. } | Self::Redeemed { metadata, .. } @@ -359,6 +373,7 @@ impl ParsedEvent { match self { Self::Unknown(e) => Ok(e.clone()), Self::AtomCreated { metadata, data } => reserialise!(metadata, data), + Self::AtomContextRegistered { metadata, data } => reserialise!(metadata, data), Self::TripleCreated { metadata, data } => reserialise!(metadata, data), Self::Deposited { metadata, data } => reserialise!(metadata, data), Self::Redeemed { metadata, data } => reserialise!(metadata, data), @@ -502,6 +517,7 @@ fn _event_type_exhaustive_check(et: crate::types::EventType) -> &'static str { use crate::types::EventType; match et { EventType::AtomCreated => "AtomCreated", + EventType::AtomContextRegistered => "AtomContextRegistered", EventType::TripleCreated => "TripleCreated", EventType::Deposited => "Deposited", EventType::Redeemed => "Redeemed", @@ -515,6 +531,7 @@ fn _parsed_event_exhaustive_check(e: &ParsedEvent) -> Option Some(EventType::AtomCreated), + ParsedEvent::AtomContextRegistered { .. } => Some(EventType::AtomContextRegistered), ParsedEvent::TripleCreated { .. } => Some(EventType::TripleCreated), ParsedEvent::Deposited { .. } => Some(EventType::Deposited), ParsedEvent::Redeemed { .. } => Some(EventType::Redeemed), @@ -601,6 +618,21 @@ mod tests { ) } + fn atom_context_registered_event() -> StoredEvent { + make_stored( + "AtomContextRegistered", + json!({ + "registrant": "0xRegistrant", + "term_id": HEX_7, + "uris": [ + "0x68747470733a2f2f6578616d706c652e636f6d", + "0xff00", + "0x68747470733a2f2f6578616d706c652e636f6d" + ] + }), + ) + } + fn triple_created_event() -> StoredEvent { make_stored( "TripleCreated", @@ -706,6 +738,42 @@ mod tests { assert_eq!(data.atom_wallet, "0xWallet"); } + #[test] + fn parse_atom_context_registered_preserves_ordered_opaque_uri_bytes() { + let parsed = ParsedEvent::parse(atom_context_registered_event()) + .expect("AtomContextRegistered parse should succeed"); + let ParsedEvent::AtomContextRegistered { metadata, data } = parsed else { + panic!("expected AtomContextRegistered variant"); + }; + + assert_eq!(metadata.sequence_number, 42); + assert_eq!(data.registrant, "0xRegistrant"); + assert_eq!(data.term_id, HEX_7); + assert_eq!( + data.uris, + vec![ + "0x68747470733a2f2f6578616d706c652e636f6d", + "0xff00", + "0x68747470733a2f2f6578616d706c652e636f6d", + ] + ); + } + + #[test] + fn parse_atom_context_registered_rejects_scalar_uri_payload() { + let event = make_stored( + "AtomContextRegistered", + json!({ + "registrant": "0xRegistrant", + "term_id": HEX_7, + "uris": "0xff00" + }), + ); + + let error = ParsedEvent::parse(event).expect_err("scalar uris must not be accepted"); + assert_eq!(error.event_type, "AtomContextRegistered"); + } + #[test] fn parse_triple_created_yields_typed_variant() { let parsed = ParsedEvent::parse(triple_created_event()).expect("parse should succeed"); @@ -839,6 +907,11 @@ mod tests { assert_round_trip(atom_created_event()); } + #[test] + fn round_trip_atom_context_registered() { + assert_round_trip(atom_context_registered_event()); + } + #[test] fn round_trip_triple_created() { assert_round_trip(triple_created_event()); diff --git a/crates/shared/src/types.rs b/crates/shared/src/types.rs index 77cf1da..f5df3f3 100644 --- a/crates/shared/src/types.rs +++ b/crates/shared/src/types.rs @@ -31,6 +31,7 @@ pub type EntityId = String; #[serde(rename_all = "PascalCase")] pub enum EventType { AtomCreated, + AtomContextRegistered, TripleCreated, Deposited, Redeemed, @@ -42,6 +43,7 @@ impl EventType { pub fn as_str(&self) -> &'static str { match self { EventType::AtomCreated => "AtomCreated", + EventType::AtomContextRegistered => "AtomContextRegistered", EventType::TripleCreated => "TripleCreated", EventType::Deposited => "Deposited", EventType::Redeemed => "Redeemed", @@ -82,6 +84,7 @@ impl std::str::FromStr for EventType { fn from_str(s: &str) -> Result { match s { "AtomCreated" => Ok(Self::AtomCreated), + "AtomContextRegistered" => Ok(Self::AtomContextRegistered), "TripleCreated" => Ok(Self::TripleCreated), "Deposited" => Ok(Self::Deposited), "Redeemed" => Ok(Self::Redeemed), @@ -103,6 +106,7 @@ mod event_type_tests { fn from_str_round_trips_every_variant() { let variants = [ EventType::AtomCreated, + EventType::AtomContextRegistered, EventType::TripleCreated, EventType::Deposited, EventType::Redeemed, @@ -130,6 +134,27 @@ mod event_type_tests { assert!(EventType::from_str("atomcreated").is_err()); assert!(EventType::from_str("ATOMCREATED").is_err()); } + + #[test] + fn atom_context_registered_conversions_are_exact() { + let event_type = EventType::AtomContextRegistered; + + assert_eq!(event_type.as_str(), "AtomContextRegistered"); + assert_eq!(event_type.to_string(), "AtomContextRegistered"); + assert_eq!( + EventType::from_str("AtomContextRegistered").unwrap(), + event_type + ); + assert_eq!( + serde_json::to_value(event_type).unwrap(), + serde_json::json!("AtomContextRegistered") + ); + assert_eq!( + serde_json::from_value::(serde_json::json!("AtomContextRegistered")) + .unwrap(), + event_type + ); + } } /// Vault type classification diff --git a/docker-compose.yml b/docker-compose.yml index 5207a42..82893e0 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -65,6 +65,9 @@ services: # Mint keys with: bun --filter @0xintuition/api run keys:create API_AUTH: ${API_AUTH:-public-read} API_RATE_LIMIT_RPM: ${API_RATE_LIMIT_RPM:-120} + # Additive identity/context/resolution/display response envelope. Keep + # dark until reader compatibility has been verified end to end. + API_ATOM_SEMANTIC_READS_ENABLED: ${API_ATOM_SEMANTIC_READS_ENABLED:-false} ports: - "${API_HOST_PORT:-3000}:3000" depends_on: @@ -112,6 +115,9 @@ services: # full | music | podcasts | music-and-podcasts. Scoped modes only gate # enrichment; parse and classification intentionally remain broad. WORKERS_PROCESSING_SCOPE: ${WORKERS_PROCESSING_SCOPE:-full} + # IID read and provider-resolution paths are independently dark-launched. + WORKERS_IID_READ_ENABLED: ${WORKERS_IID_READ_ENABLED:-false} + WORKERS_IID_RESOLUTION_ENABLED: ${WORKERS_IID_RESOLUTION_ENABLED:-false} SPOTIFY_CLIENT_ID: ${SPOTIFY_CLIENT_ID:-} SPOTIFY_CLIENT_SECRET: ${SPOTIFY_CLIENT_SECRET:-} SPOTIFY_MARKET: ${SPOTIFY_MARKET:-} @@ -173,9 +179,11 @@ services: # MULTIVAULT_START_BLOCK=0 # and run: docker compose --profile devnet --profile indexing up - # --disable-code-size-limit: the contracts ship as the production build - # (optimizer_runs=10000); MultiVault's runtime exceeds EIP-170's 24576 bytes, - # which the Intuition L3 permits by raising the cap — anvil must do the same. + # --disable-code-size-limit: the standard v1.1 local path deploys and links + # MultiVaultLib into the production MultiVault implementation, whose runtime + # exceeds EIP-170's 24576 bytes. The Intuition L3 raises that cap, so this + # Anvil path mirrors it. The separate EIP-170 acceptance path uses the linked + # MultiVaultSizeFit artifact. anvil: image: ghcr.io/foundry-rs/foundry:stable profiles: [devnet] @@ -191,9 +199,10 @@ services: retries: 20 # One-shot: deploy the full protocol from the pinned @0xintuition/contracts-v2 - # npm package (production bytecode, no toolchain) and verify AtomCreated - # fires. Idempotent — skips deploy when MultiVault already has code on the - # anvil chain (state persists on the anvil_data volume). + # npm package (production bytecode, no toolchain), link MultiVaultLib, and + # verify AtomCreated plus the URI config/AtomContextRegistered path. Idempotent + # — skips deploy when MultiVault already has code on the Anvil chain (state + # persists on the anvil_data volume). devnet-deploy: build: context: . @@ -255,6 +264,10 @@ services: # own greenfield environments run) — graph writes go to PostgreSQL. SURREAL_DB_URL: "" PROJECTIONS_METRICS_PORT: "9092" + # Core's canonical event source is the per-event typed tables. Migration + # 050 creates/backfills AtomContextRegistered there, and the context + # projector must consume the same byte-preserving representation. + USE_TYPED_READER: ${USE_TYPED_READER:-true} # Lean open-source default: skip product analytics and market-schema dual writes. DISABLED_PROJECTIONS: ${DISABLED_PROJECTIONS:-funnel_tracker,user_activity_batch,vault_state:dual,vault_holders_index:dual} ports: diff --git a/docker/Dockerfile.api b/docker/Dockerfile.api index de0626f..452a194 100644 --- a/docker/Dockerfile.api +++ b/docker/Dockerfile.api @@ -17,7 +17,17 @@ LABEL org.opencontainers.image.title="Intuition Core API" \ org.opencontainers.image.revision="${VCS_REF}" \ org.opencontainers.image.created="${CREATED}" -RUN apk add --no-cache curl +# The upstream Bun `-alpine` tag has historically changed its base OS across +# architectures/releases. Keep the image build deterministic across either +# supported package manager instead of assuming the tag's implementation. +RUN if command -v apk >/dev/null 2>&1; then \ + apk add --no-cache curl; \ + elif command -v apt-get >/dev/null 2>&1; then \ + apt-get update && apt-get install -y --no-install-recommends curl ca-certificates && \ + rm -rf /var/lib/apt/lists/*; \ + else \ + echo "No supported package manager found for curl installation" >&2; exit 1; \ + fi WORKDIR /repo COPY . . diff --git a/docker/Dockerfile.atom-services b/docker/Dockerfile.atom-services index fa16346..1a9bbe6 100644 --- a/docker/Dockerfile.atom-services +++ b/docker/Dockerfile.atom-services @@ -17,7 +17,14 @@ LABEL org.opencontainers.image.title="Intuition Core Atom Services" \ org.opencontainers.image.revision="${VCS_REF}" \ org.opencontainers.image.created="${CREATED}" -RUN apk add --no-cache curl +RUN if command -v apk >/dev/null 2>&1; then \ + apk add --no-cache curl; \ + elif command -v apt-get >/dev/null 2>&1; then \ + apt-get update && apt-get install -y --no-install-recommends curl ca-certificates && \ + rm -rf /var/lib/apt/lists/*; \ + else \ + echo "No supported package manager found for curl installation" >&2; exit 1; \ + fi WORKDIR /repo COPY . . diff --git a/docker/Dockerfile.ingestion b/docker/Dockerfile.ingestion index 3ee9c9a..daf2cac 100644 --- a/docker/Dockerfile.ingestion +++ b/docker/Dockerfile.ingestion @@ -24,6 +24,7 @@ WORKDIR /app # Manifests first for dependency-layer caching. All workspace members must be # present for cargo to load the workspace, even though we only build one. COPY Cargo.toml ./ +COPY Cargo.lock ./ COPY crates/shared/Cargo.toml ./crates/shared/ COPY crates/rindexer-ingestion/Cargo.toml ./crates/rindexer-ingestion/ COPY crates/projections/Cargo.toml ./crates/projections/ @@ -36,17 +37,17 @@ RUN mkdir -p crates/shared/src && echo "" > crates/shared/src/lib.rs \ && mkdir -p crates/curves/src && echo "" > crates/curves/src/lib.rs # Build dependencies only (cached layer) -RUN cargo build --release -p rindexer-ingestion || true +RUN cargo build --locked --release -p rindexer-ingestion # Real sources, then rebuild. COPY crates/shared ./crates/shared COPY crates/curves ./crates/curves COPY crates/rindexer-ingestion ./crates/rindexer-ingestion RUN touch crates/shared/src/lib.rs crates/curves/src/lib.rs crates/rindexer-ingestion/src/main.rs \ - && cargo build --release -p rindexer-ingestion + && cargo build --locked --release -p rindexer-ingestion # Runtime stage -FROM debian:trixie-slim +FROM debian:trixie-slim@sha256:3a39a0592364683e6bab97937b72cad5a8fa6dcbbee90edb3bb48c7f8e94f258 ARG VERSION=dev ARG VCS_REF=unknown diff --git a/docker/Dockerfile.projections b/docker/Dockerfile.projections index 3e6b589..7eae9a8 100644 --- a/docker/Dockerfile.projections +++ b/docker/Dockerfile.projections @@ -1,7 +1,10 @@ # projections — event store → typed read models + graph. # Build context: repo root. -FROM rust:1-slim AS builder +# Match the pinned toolchain used by the ingestion image and CI. A floating +# `rust:1-slim` tag makes two services in the same release build with +# potentially different compilers. +FROM rust:1.97.1-slim-bookworm@sha256:99e09cb2284e2ddbb73a995deee3e91783fd04d177602ccf6eab326d778ee777 AS builder RUN apt-get update && apt-get install -y \ pkg-config \ @@ -12,6 +15,7 @@ RUN apt-get update && apt-get install -y \ WORKDIR /app COPY Cargo.toml ./ +COPY Cargo.lock ./ COPY crates/shared/Cargo.toml ./crates/shared/ COPY crates/rindexer-ingestion/Cargo.toml ./crates/rindexer-ingestion/ COPY crates/projections/Cargo.toml ./crates/projections/ @@ -22,15 +26,15 @@ RUN mkdir -p crates/shared/src && echo "" > crates/shared/src/lib.rs \ && mkdir -p crates/projections/src && echo "fn main() {}" > crates/projections/src/main.rs \ && mkdir -p crates/curves/src && echo "" > crates/curves/src/lib.rs -RUN cargo build --release -p projections || true +RUN cargo build --locked --release -p projections COPY crates/shared ./crates/shared COPY crates/curves ./crates/curves COPY crates/projections ./crates/projections RUN touch crates/shared/src/lib.rs crates/curves/src/lib.rs crates/projections/src/main.rs \ - && cargo build --release -p projections + && cargo build --locked --release -p projections -FROM debian:trixie-slim +FROM debian:trixie-slim@sha256:3a39a0592364683e6bab97937b72cad5a8fa6dcbbee90edb3bb48c7f8e94f258 ARG VERSION=dev ARG VCS_REF=unknown diff --git a/docker/Dockerfile.workers b/docker/Dockerfile.workers index 6e78a1a..6ed6c4a 100644 --- a/docker/Dockerfile.workers +++ b/docker/Dockerfile.workers @@ -17,7 +17,14 @@ LABEL org.opencontainers.image.title="Intuition Core Workers" \ org.opencontainers.image.revision="${VCS_REF}" \ org.opencontainers.image.created="${CREATED}" -RUN apk add --no-cache curl +RUN if command -v apk >/dev/null 2>&1; then \ + apk add --no-cache curl; \ + elif command -v apt-get >/dev/null 2>&1; then \ + apt-get update && apt-get install -y --no-install-recommends curl ca-certificates && \ + rm -rf /var/lib/apt/lists/*; \ + else \ + echo "No supported package manager found for curl installation" >&2; exit 1; \ + fi WORKDIR /repo COPY . . diff --git a/docs/atom-context-projection.md b/docs/atom-context-projection.md new file mode 100644 index 0000000..c72e7b5 --- /dev/null +++ b/docs/atom-context-projection.md @@ -0,0 +1,109 @@ +# Atom Context Projection Runbook + +`atom_context:dual` projects the protocol's `AtomContextRegistered` events into +the KG database without changing an atom's identity or payload. It is an +append-only evidence projection: it stores every URI byte value in contract +order and appends an associated `kg.events` row. + +The projection does not fetch a URI, normalize its text, infer a scheme, or +assign trust. Consumers must treat `uri_hex` as canonical. `uri_text` is a +nullable convenience rendering populated only when the bytes are valid UTF-8 +and contain no NUL byte. + +## Data and checkpoint contract + +- `kg.node_contexts` has one row per URI. Its primary key is + `(node_id, transaction_hash, log_index, ordinal)`, so duplicates at different + ordinals and original order are preserved. +- `kg.events` receives one `atom_context_registered` row per contract event. +- `kg.nodes.data`, `kg.nodes.data_hex`, and the atom ID are never updated. +- Node lookup is exact: event `term_id = kg.nodes.id`. +- If the node does not exist yet, the KG transaction fails with a retryable + dependency error. The batch rolls back and its checkpoint does not advance. +- The generic PostgreSQL worker stores the independent cursor as + `projection_name = 'atom_context:dual'`, `sink_name = 'pg'`. An absent row + reads as sequence `0`, which makes first enablement a complete backfill. +- URI rows and evidence use `ON CONFLICT DO NOTHING`, making replay after a + checkpoint-write failure idempotent. + +## Deployment order + +1. Deploy KG migrations through `0004_add-node-contexts.sql`. Migration `0003` + must run first because it adds IID support to `kg.nodes`. +2. Deploy the Timescale/rindexer migration that creates and fills + `atom_context_registered_events`. +3. Deploy a projections image containing the typed + `AtomContextRegistered` reader and this projection. +4. Confirm `DATABASE_KG_URL` is set. Without it, the service intentionally does + not spawn `atom_context:dual`; it never runs a no-op that could discard data. +5. Add `atom_context:dual` to `ENABLED_PROJECTIONS` when an allowlist is in use. + Keep `core_entities` enabled until all referenced atom nodes are present. +6. Observe the backfill from sequence zero, then run the verification queries + below before promoting the image. + +Example: + +```dotenv +DATABASE_KG_URL=postgresql://... +USE_TYPED_READER=true +ENABLED_PROJECTIONS=core_entities,atom_context:dual +``` + +## Verification + +Run the checkpoint query against the event-store/Timescale database: + +```sql +SELECT projection_name, sink_name, last_sequence_number, last_block_number, + last_updated_at +FROM projection_checkpoints +WHERE projection_name = 'atom_context:dual' AND sink_name = 'pg'; +``` + +Run the remaining queries against the KG database: + +```sql +-- URI rows retain their per-event order. +SELECT node_id, transaction_hash, log_index, ordinal, uri_hex, uri_text +FROM kg.node_contexts +ORDER BY event_sequence, ordinal +LIMIT 100; + +-- Every stored context remains attached to an existing canonical node. +SELECT count(*) AS orphan_contexts +FROM kg.node_contexts c +LEFT JOIN kg.nodes n ON n.id = c.node_id +WHERE n.id IS NULL; + +-- Evidence exists without atom-payload mutation. +SELECT event_time, id, actor_id, entity_id, event_type, payload +FROM kg.events +WHERE event_type = 'atom_context_registered' +ORDER BY event_time DESC +LIMIT 100; + +-- Duplicated URI bytes at distinct ordinals are retained intentionally. +SELECT node_id, transaction_hash, log_index, uri_hex, count(*) +FROM kg.node_contexts +GROUP BY node_id, transaction_hash, log_index, uri_hex +HAVING count(*) > 1; +``` + +Expected results: `orphan_contexts = 0`; checkpoint sequence increases during +backfill; URI ordinals are zero-based and contiguous per event; evidence +`entity_id` exactly matches the associated `node_id`. + +## Failure and rollback + +- Repeated `RowNotFound` errors mean context registration has caught up to an + atom that `core_entities` has not written. Check the `core_entities` + checkpoint and KG-node row. Do not bypass the join or manually insert context. +- Malformed/non-hex URI input pins the checkpoint to avoid silent byte loss. + Repair the upstream typed event only from authoritative chain data. +- To halt context projection, remove `atom_context:dual` from the allowlist (or + add it to `DISABLED_PROJECTIONS`) and restart projections. Existing rows are + append-only and need no rollback. +- Re-enabling resumes from the independent checkpoint. A controlled replay from + zero is safe because both tables use deterministic keys and conflict-ignore + inserts; changing or deleting a production checkpoint still requires normal + database-change approval. diff --git a/docs/configuration.md b/docs/configuration.md index 8eb96e0..011c55f 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -8,7 +8,7 @@ automatically. | Variable | Used by | Default (local) | Notes | | --- | --- | --- | --- | -| `DATABASE_KG_URL` | api, workers, kg migrate, projections (optional) | `postgresql://intuition:intuition@localhost:5432/intuition_kg` | knowledge graph | +| `DATABASE_KG_URL` | api, workers, kg migrate, projections (optional) | `postgresql://intuition:intuition@localhost:5432/intuition_kg` | knowledge graph; projections use it for canonical nodes and atom-context evidence | | `DATABASE_TIMESCALE_URL` | database-timescale tests | `postgresql://…@localhost:5433/intuition_timescale` | TS-package tests skip cleanly when unset | | `DATABASE_URL` | indexer, projections, timescale migrate | container DSN → `timescale:5432` | the event store | | `REDIS_URL` | indexer | `redis://localhost:6379` | leader election | @@ -31,8 +31,9 @@ automatically. | Variable | Default | Notes | | --- | --- | --- | | `SURREAL_DB_URL` | *(empty)* | **keep empty** — selects the no-op graph sink; Core is Postgres-only | -| `DATABASE_KG_URL` | unset | when set, `core_entities` writes atoms/triples into the KG | -| `ENABLED_PROJECTIONS` / `DISABLED_PROJECTIONS` | — / `funnel_tracker,user_activity_batch,vault_state:dual,vault_holders_index:dual` | CSV allow/deny lists | +| `DATABASE_KG_URL` | unset | when set, `core_entities` writes atoms/triples and `atom_context:dual` writes immutable URI/evidence rows into the KG | +| `USE_TYPED_READER` | `true` in Core Compose | use the per-event typed tables, including the byte-preserving `AtomContextRegistered` table; the standalone binary retains its legacy default of `false` | +| `ENABLED_PROJECTIONS` / `DISABLED_PROJECTIONS` | — / `funnel_tracker,user_activity_batch,vault_state:dual,vault_holders_index:dual` | CSV allow/deny lists; `atom_context:dual` is intentionally enabled by default after migration because it records chain truth | | `PROJECTIONS_BATCH_SIZE` / `PROJECTIONS_POLL_INTERVAL_MS` | `500` / `1000` | throughput tuning | | `PROJECTIONS_METRICS_PORT` | `9092` | health: `/health/live` | @@ -43,6 +44,14 @@ automatically. | `API_PORT` | `3000` | | | `API_AUTH` | `public-read` | `open` \| `public-read` \| `gated` — see run-your-own-node.md | | `API_ALLOWED_ORIGINS` | *(empty = allow all)* | comma-separated CORS origins | +| `API_ATOM_SEMANTIC_READS_ENABLED` | `false` | expose additive `raw`, `identity`, `classification`, `context`, `resolution`, and `display` atom fields; existing fields remain present | + +## Explorer (`apps/explorer`) + +| Variable | Default | Notes | +| --- | --- | --- | +| `VITE_API_URL` | `http://localhost:3000` | query API base URL | +| `VITE_ATOM_SEMANTIC_READS_ENABLED` | `false` | independently consume semantic atom fields and show identity/context UI; this is a Vite build/start setting | ## Workers (`services/workers`) @@ -56,6 +65,8 @@ automatically. | `WORKERS_PARSE_ALLOW_HTTP` | `false` | plain-http fetches off by default | | `WORKERS_PARSE_IPFS_GATEWAY_BASE_URL` | unset | optional IPFS gateway | | `WORKERS_PROCESSING_SCOPE` | `full` | `full`, `music`, `podcasts`, or `music-and-podcasts`; scoped modes gate enrichment only | +| `WORKERS_IID_READ_ENABLED` | `false` | enable IID parse/classification reads after package and persistence compatibility is verified | +| `WORKERS_IID_RESOLUTION_ENABLED` | `false` | independently enable IID provider resolution; keep off until IID reads are enabled and stable | ## Atom services (`services/atom-services`) diff --git a/docs/contracts.md b/docs/contracts.md index d0ea6c6..cedf91e 100644 --- a/docs/contracts.md +++ b/docs/contracts.md @@ -55,6 +55,14 @@ The facade is intentionally small. It gives backend code a stable import path and records which MultiVault events the indexer expects. It should not fork, edit, or reinterpret upstream contract artifacts. +The indexer-critical event surface includes `AtomCreated`, +`AtomContextRegistered`, `TripleCreated`, `Deposited`, `Redeemed`, +`SharePriceChanged`, and `ProtocolFeeAccrued`. The URI-aware +`createAtomsWithUris` entrypoint emits `AtomContextRegistered`; its ordered URI +bytes add creation-time context without participating in the atom's +deterministic ID. Consumers can read the active URI limits with +`getAtomUriConfig`. + ## Updating contract artifacts 1. Check the latest published version: diff --git a/docs/local-devnet.md b/docs/local-devnet.md index cc7d596..d15ef5c 100644 --- a/docs/local-devnet.md +++ b/docs/local-devnet.md @@ -28,16 +28,11 @@ roles) straight from the npm package, and finishes by creating a test atom and verifying the `AtomCreated` event fires. Re-runs detect the existing deployment and just re-verify. -Because Anvil runs with its default mnemonic, the deployment is deterministic: - -| Contract | Address (chain 31337) | -| --- | --- | -| **MultiVault proxy** | `0xa85233C63b9Ee964Add6F2cffe00Fd84eb32338f` | -| WrappedTrust (WTRUST) | `0x5FbDB2315678afecb367f032d93F642f64180aa3` | - -The full address set (plus the actual deploy block) is written to -`devnet/deployments-devnet.json`. Chain state persists on the `anvil_data` -volume across restarts. +The deployment is deterministic for a fresh Anvil state, but the persisted +volume may already contain transactions and deployments. Always treat +`devnet/deployments-devnet.json` as authoritative for the MultiVault address +and start block. Chain state persists on the `anvil_data` volume across +restarts. ## 2. Point the indexer at your chain @@ -46,9 +41,11 @@ In `.env`: ```bash INTUITION_RPC_URL=http://anvil:8545 CHAIN_ID=31337 -MULTIVAULT_CONTRACT_ADDRESS=0xa85233C63b9Ee964Add6F2cffe00Fd84eb32338f -MULTIVAULT_START_BLOCK=0 +# Copy these two values from devnet/deployments-devnet.json. +MULTIVAULT_CONTRACT_ADDRESS=0x... +MULTIVAULT_START_BLOCK=... MULTIVAULT_END_BLOCK= +USE_TYPED_READER=true ``` ```bash @@ -58,14 +55,46 @@ docker compose --profile devnet --profile indexing up The indexer follows your local chain head; every atom you create onchain shows up in the graph within seconds. -## 3. Create atoms onchain +## 3. Create canonical IID atoms with URI context + +The canonical acceptance command creates one music recording and one book: + +```bash +bun run devnet:fixtures:iid +``` + +It creates `int:isrc:USUM71703861` with MusicBrainz context and +`int:isbn:9780684832722` with OpenLibrary context. The command: + +- refuses to run on any chain except Anvil chain `31337`; +- reads the active deployment and URI limits from chain state; +- derives the term ID from IID bytes only; +- simulates `createAtomsWithUris` before sending; +- joins `AtomCreated` and `AtomContextRegistered` by exact `termId`; and +- is idempotent when the atoms already exist. + +With semantic API reads enabled, verify the final read models using the term +IDs printed by the command: + +```bash +API_ATOM_SEMANTIC_READS_ENABLED=true docker compose up -d api +curl "localhost:3000/api/atoms/" +curl "localhost:3000/api/iids/int%3Aisbn%3A9780684832722/atoms" +``` + +The atom detail must show identity, classification, resolution, display, and +on-chain context as separate fields. The context transaction hash and log +index must match the `AtomContextRegistered` event—not the adjacent +`AtomCreated` event. + +## 4. Create arbitrary atoms onchain Anvil's account #0 is pre-funded (key `0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80` — the universal Foundry dev key, safe only for local chains): ```bash -MV=0xa85233C63b9Ee964Add6F2cffe00Fd84eb32338f +MV=$(jq -r .MultiVault devnet/deployments-devnet.json) COST=$(cast call $MV "getAtomCost()(uint256)" --rpc-url http://localhost:8545 | awk '{print $1}') cast send $MV "createAtoms(bytes[],uint256[])" \ @@ -94,6 +123,7 @@ With [Foundry](https://getfoundry.sh)'s `anvil` installed: ```bash anvil --disable-code-size-limit & # terminal 1 bun run devnet:deploy # terminal 2 — deploy + acceptance test +bun run devnet:fixtures:iid # canonical ISRC + ISBN fixtures ``` Or run the whole native dev stack (datastores, API, workers, chain) under @@ -117,7 +147,8 @@ Requirements: a key funded with tTRUST for gas (~40 transactions — faucet at https://testnet.hub.intuition.systems). The canonical wrapped-TRUST token is reused by default (it's shared infra, like WETH; set `TRUST_TOKEN=fresh` for a fully isolated one). State lands in `devnet/deployments-testnet.json`, the -createAtoms acceptance test runs, and the CLI prints the `.env` block that +`createAtomsWithUris` acceptance test runs (including URI config and context-event +validation), and the CLI prints the `.env` block that points the Core indexer at your new instance. ## Upgrading the contracts @@ -138,10 +169,11 @@ The protocol version is pinned in **one place**: ## Gotchas we already hit for you -- **Contract size:** the published bytecode is the production build; MultiVault's - runtime (27,666 bytes) exceeds EIP-170's 24,576-byte cap, which the Intuition - L3 raises. Anvil must therefore run with `--disable-code-size-limit` — the - compose file and Process Compose overlay already do. +- **Contract size and linking:** the default local deployment uses + MultiVaultMigrationMode and requires Anvil's `--disable-code-size-limit`. + MultiVault and MultiVaultMigrationMode also contain a `MultiVaultLib` link + placeholder; the deployer deploys the library first and links its address + into the selected implementation bytecode. - **Proxies are mandatory:** the implementations call `_disableInitializers()` in their constructors; everything is initialized through `TransparentUpgradeableProxy` init-data, exactly like production. @@ -150,13 +182,17 @@ The protocol version is pinned in **one place**: - The deployer account must be the admin (the deploy makes admin-only calls); the deployer defaults to Anvil account #0 for both. -## Upstream follow-ups (tracked for `contracts-v2@1.0.0-alpha.1`) +## Upstream artifact follow-ups The npm package doesn't yet export everything a from-scratch deployment needs; -`packages/contracts/vendored/` fills the gaps (see its README for provenance). -Once upstream ships these, the vendored artifacts get deleted: +`packages/contracts/vendored/` fills the remaining gaps (see its README for +provenance). Follow-up cleanup is: -- Export `AtomWarden` + `WrappedTrust` ABIs/bytecodes. +- Move the remaining `AtomWarden` + `WrappedTrust` consumers to the exports now + available in `@0xintuition/contracts-v2@1.1.0-alpha.0` and delete those + vendored copies. - Ship the OZ infra bytecodes (`TransparentUpgradeableProxy`, `TimelockController`, `UpgradeableBeacon`) or a first-party deploy module. +- Publish an EIP-170-compatible MultiVault artifact (or a reproducible build + profile) alongside its library link references. - Add a `deployments` export (canonical addresses per chain). diff --git a/example.env b/example.env index 7794505..319ee61 100644 --- a/example.env +++ b/example.env @@ -13,7 +13,8 @@ NODE_ENV=development # ── Datastores ─────────────────────────────────────────────────────────────── # These defaults match docker-compose.datastores.yml — they work out of the box. -# PostgreSQL knowledge graph (atoms, triples, accounts, predicates) +# PostgreSQL knowledge graph (atoms, triples, accounts, predicates, and +# immutable AtomContextRegistered URI evidence via atom_context:dual) DATABASE_KG_URL=postgresql://intuition:intuition@localhost:5432/intuition_kg # TimescaleDB time-series (event store, markets, positions, leaderboards) @@ -38,6 +39,12 @@ API_AUTH=public-read # Requests/minute per caller (API key, or IP for anonymous reads). 0 = unlimited. # Per-key overrides: keys:create -- --rate-limit API_RATE_LIMIT_RPM=120 +# Additive identity/context/resolution/display response envelope (default off). +API_ATOM_SEMANTIC_READS_ENABLED=false + +# Explorer independently opts into consuming the semantic response envelope. +# Enable only after the API flag above is deployed and verified. +VITE_ATOM_SEMANTIC_READS_ENABLED=false # ── Chain / indexer ────────────────────────────────────────────────────────── # Point the indexer at the Intuition chain (or any EVM MultiVault deployment). @@ -56,9 +63,11 @@ MULTIVAULT_END_BLOCK= # then use: # INTUITION_RPC_URL=http://anvil:8545 # CHAIN_ID=31337 -# MULTIVAULT_CONTRACT_ADDRESS=0xa85233C63b9Ee964Add6F2cffe00Fd84eb32338f -# MULTIVAULT_START_BLOCK=0 +# MULTIVAULT_CONTRACT_ADDRESS= +# MULTIVAULT_START_BLOCK= # MULTIVAULT_END_BLOCK= +# Typed event tables are the canonical Core source, including URI context. +USE_TYPED_READER=true # ───────────────────────────────────────────────────────────────────────────── # OPTIONAL — none of the below are required for the minimal tier. @@ -68,6 +77,11 @@ MULTIVAULT_END_BLOCK= # Scoped modes gate enrichment only; parse and classification remain broad. WORKERS_PROCESSING_SCOPE=full +# IID parsing/classification reads and provider resolution are separate, +# default-off rollout gates. Enable reads before resolution. +WORKERS_IID_READ_ENABLED=false +WORKERS_IID_RESOLUTION_ENABLED=false + # Embeddings / semantic search (+ Search tier). Off by default. OPENAI_API_KEY= ANTHROPIC_API_KEY= diff --git a/migrations/timescale/050_add_atom_context_registered.sql b/migrations/timescale/050_add_atom_context_registered.sql new file mode 100644 index 0000000..12defa1 --- /dev/null +++ b/migrations/timescale/050_add_atom_context_registered.sql @@ -0,0 +1,44 @@ +-- Migration: Ingest AtomContextRegistered as a canonical typed event +-- Description: Extends the append-only event-store contract and creates the +-- regular dual-write target for ordered opaque atom URI bytes. + +BEGIN; + +-- The original event_type CHECK is a closed-world whitelist. TimescaleDB does +-- not support ADD CONSTRAINT on hypertables while columnstore is enabled, and +-- disabling columnstore requires decompressing every historical compressed +-- chunk. DROP CONSTRAINT is supported on compressed hypertables. Remove the +-- brittle whitelist once; generated event decoding plus the typed dual-write +-- tables remain the canonical event contract for this and future additions. +ALTER TABLE event_store + DROP CONSTRAINT IF EXISTS event_store_event_type_check; + +-- Regular table: context registration is creation-adjacent, low-volume data. +-- uris is an ordered JSON array of 0x-prefixed byte strings exactly as emitted +-- by the contract; ingestion performs no decoding or normalization. +CREATE TABLE IF NOT EXISTS atom_context_registered_events ( + block_number BIGINT NOT NULL, + block_timestamp TIMESTAMPTZ NOT NULL, + block_hash TEXT NOT NULL, + transaction_hash TEXT NOT NULL, + log_index INTEGER NOT NULL, + + registrant TEXT NOT NULL, + term_id NUMERIC NOT NULL, + term_id_hex TEXT NOT NULL, + uris JSONB NOT NULL CHECK (jsonb_typeof(uris) = 'array'), + sequence_number BIGINT NOT NULL, + + PRIMARY KEY (transaction_hash, log_index) +); + +CREATE UNIQUE INDEX IF NOT EXISTS ux_atom_context_registered_seq + ON atom_context_registered_events (sequence_number); + +CREATE INDEX IF NOT EXISTS idx_atom_context_registered_term + ON atom_context_registered_events (term_id, sequence_number); + +CREATE INDEX IF NOT EXISTS idx_atom_context_registered_term_hex + ON atom_context_registered_events (term_id_hex, sequence_number); + +COMMIT; diff --git a/package.json b/package.json index acfe9bd..12ef740 100644 --- a/package.json +++ b/package.json @@ -35,6 +35,7 @@ "guard:supply-chain": "node scripts/guard-supply-chain-policy.mjs", "preinstall": "node scripts/enforce-bun-install.mjs", "devnet:deploy": "bun run --cwd packages/contracts devnet:deploy", + "devnet:fixtures:iid": "bun run --cwd packages/contracts scripts/devnet-create-iid-fixtures.ts", "abis:sync": "bun run scripts/sync-abis.ts", "abis:check": "bun run scripts/sync-abis.ts --check", "testnet:deploy": "bun run --cwd packages/contracts testnet:deploy" diff --git a/packages/atom-enrichment/__tests__/classification-registry.contract.test.ts b/packages/atom-enrichment/__tests__/classification-registry.contract.test.ts index abf090f..eb4248b 100644 --- a/packages/atom-enrichment/__tests__/classification-registry.contract.test.ts +++ b/packages/atom-enrichment/__tests__/classification-registry.contract.test.ts @@ -128,12 +128,13 @@ describe('classification registry contract', () => { expect(result.success).toBe(false); }); - it('ships all 36 built-in classifications', () => { - expect(builtinClassificationDefinitions).toHaveLength(36); + it('ships all 37 built-in classifications', () => { + expect(builtinClassificationDefinitions).toHaveLength(37); const defaultRegistry = createDefaultClassificationRegistry(); - expect(defaultRegistry.list()).toHaveLength(36); + expect(defaultRegistry.list()).toHaveLength(37); expect(defaultRegistry.has('opengraph')).toBe(true); + expect(defaultRegistry.has('openlibrary')).toBe(true); expect(defaultRegistry.has('ai-entities')).toBe(true); expect(defaultRegistry.has('color-palette')).toBe(true); }); diff --git a/packages/atom-enrichment/__tests__/presets.contract.test.ts b/packages/atom-enrichment/__tests__/presets.contract.test.ts index f9a7fdc..f408df1 100644 --- a/packages/atom-enrichment/__tests__/presets.contract.test.ts +++ b/packages/atom-enrichment/__tests__/presets.contract.test.ts @@ -1,6 +1,12 @@ import { describe, expect, it } from 'bun:test'; -import { academicPreset, companyPreset, cryptoPreset, musicPreset } from '../src/presets'; +import { + academicPreset, + companyPreset, + cryptoPreset, + musicPreset, + serverDefaultPreset, +} from '../src/presets'; function pluginIds(plugins: Array<{ id: string }>): string[] { return plugins.map((plugin) => plugin.id); @@ -27,6 +33,12 @@ describe('preset bundles', () => { expect(pluginIds(plugins)).toEqual(['crossref', 'wikipedia']); }); + it('serverDefaultPreset registers OpenLibrary for identifier-first ISBN and OLID plans', () => { + const ids = pluginIds(serverDefaultPreset()); + expect(ids).toContain('openlibrary'); + expect(ids.indexOf('openlibrary')).toBe(ids.indexOf('musicbrainz') + 1); + }); + it('propagates options to underlying plugin manifests', () => { const [opengraph, brand, favicon] = companyPreset({ opengraph: { priority: 91, TTL: 12_345 }, diff --git a/packages/atom-enrichment/__tests__/provider-plan.test.ts b/packages/atom-enrichment/__tests__/provider-plan.test.ts new file mode 100644 index 0000000..4df2c99 --- /dev/null +++ b/packages/atom-enrichment/__tests__/provider-plan.test.ts @@ -0,0 +1,125 @@ +import { describe, expect, it } from 'bun:test'; +import { + createAppleMusicPlugin, + createCoinGeckoPlugin, + createCrossrefPlugin, + createEtherscanPlugin, + createGitHubPlugin, + createMusicBrainzPlugin, + createNpmPlugin, + createOpenLibraryPlugin, + createPodcastIndexPlugin, + createSpotifyPlugin, + createTmdbPlugin, + createWikidataPlugin, + createXProfilePlugin, +} from '../src/plugins/providers'; +import { + createIdentifierProviderPlan, + IID_PROVIDER_CAPABILITIES, + IID_PROVIDER_SLUGS, +} from '../src/provider-plan'; + +const REGISTRY_PROVIDER_SLUGS = [ + 'apple-music', + 'coingecko', + 'crossref', + 'etherscan', + 'github', + 'musicbrainz', + 'npm', + 'openlibrary', + 'podcast-index', + 'spotify', + 'tmdb', + 'wikidata', + 'x-profile', +] as const; + +describe('IID provider capability contract', () => { + it('is total over the public iid-registry provider vocabulary', () => { + expect(IID_PROVIDER_SLUGS).toEqual(REGISTRY_PROVIDER_SLUGS); + expect(Object.keys(IID_PROVIDER_CAPABILITIES).sort()).toEqual( + [...REGISTRY_PROVIDER_SLUGS].sort() + ); + }); + + it('backs every supported capability with a Core plugin implementation', () => { + const implemented = new Set( + [ + createAppleMusicPlugin(), + createCoinGeckoPlugin(), + createCrossrefPlugin(), + createEtherscanPlugin(), + createGitHubPlugin(), + createMusicBrainzPlugin(), + createNpmPlugin(), + createOpenLibraryPlugin(), + createPodcastIndexPlugin(), + createSpotifyPlugin(), + createTmdbPlugin(), + createWikidataPlugin(), + createXProfilePlugin(), + ].map((plugin) => plugin.id) + ); + + for (const capability of Object.values(IID_PROVIDER_CAPABILITIES)) { + if (capability.status === 'supported') { + expect(implemented.has(capability.pluginId), capability.pluginId).toBe(true); + } + } + }); + + it('preserves the open-first ISRC plan and canonical hint order', () => { + const plan = createIdentifierProviderPlan({ + providers: ['musicbrainz', 'spotify', 'apple-music'], + identifiers: { isrc: 'USUM71703861', secondary: 'keep-after-isrc' }, + registeredPluginIds: ['apple-music', 'musicbrainz', 'spotify'], + }); + + expect(plan.complete).toBe(true); + expect(plan.plugins).toEqual(['musicbrainz', 'spotify', 'apple-music']); + expect(plan.entries.map((entry) => entry.providerSlug)).toEqual([ + 'musicbrainz', + 'spotify', + 'apple-music', + ]); + expect(Object.keys(plan.identifiers)).toEqual(['isrc', 'secondary']); + expect(plan.identifiers).toEqual({ isrc: 'USUM71703861', secondary: 'keep-after-isrc' }); + }); + + it('marks deployment gaps retryable and unknown or duplicate slugs terminal', () => { + const plan = createIdentifierProviderPlan({ + providers: ['openlibrary', 'future-books', 'openlibrary'], + identifiers: { isbn: '9780684832722' }, + registeredPluginIds: [], + }); + + expect(plan.complete).toBe(false); + expect(plan.plugins).toEqual([]); + expect(plan.entries).toEqual([ + { + providerSlug: 'openlibrary', + pluginId: 'openlibrary', + ordinal: 0, + status: 'unavailable', + disposition: 'retry', + reason: 'plugin_not_registered', + }, + { + providerSlug: 'future-books', + ordinal: 1, + status: 'unsupported', + disposition: 'terminal', + reason: 'unknown_provider_slug', + }, + { + providerSlug: 'openlibrary', + ordinal: 2, + status: 'unsupported', + disposition: 'terminal', + reason: 'duplicate_provider_slug', + }, + ]); + }); +}); diff --git a/packages/atom-enrichment/__tests__/providers/openlibrary.test.ts b/packages/atom-enrichment/__tests__/providers/openlibrary.test.ts new file mode 100644 index 0000000..c076040 --- /dev/null +++ b/packages/atom-enrichment/__tests__/providers/openlibrary.test.ts @@ -0,0 +1,174 @@ +import { describe, expect, it } from 'bun:test'; + +import { createEnrichmentEngine } from '../../src/engine'; +import type { FetchLike } from '../../src/plugins/providers'; +import { + createOpenLibraryPlugin, + resolveOpenLibraryTarget, +} from '../../src/plugins/providers/openlibrary'; +import { createMockAtomInput, createMockPluginContext, createMockRequest } from '../../src/testing'; + +function identifierRequest(identifiers: Record) { + return createMockRequest({ + input: createMockAtomInput({ + atomType: 'thing', + jsonLd: { '@context': 'https://schema.org', '@type': 'Thing' }, + hints: { identifiers }, + }), + plugins: ['openlibrary'], + }); +} + +function jsonFetch(body: unknown, status = 200, inspect?: (url: string) => void): FetchLike { + return async (url) => { + inspect?.(url); + return new Response(JSON.stringify(body), { + status, + headers: { 'content-type': 'application/json' }, + }); + }; +} + +describe('OpenLibrary identifier routing', () => { + it('prefers the more specific OLID hint when both OLID and ISBN are present', () => { + expect( + resolveOpenLibraryTarget(identifierRequest({ isbn: '9780684832722', olid: 'ol45804w' })) + ).toEqual({ kind: 'olid', identifier: 'OL45804W', entityType: 'work' }); + }); + + it('accepts canonical ISBN and typed OLID values, but rejects malformed hints', () => { + const plugin = createOpenLibraryPlugin(); + expect(plugin.supports(identifierRequest({ isbn: '9780684832722' }))).toBe(true); + expect(plugin.supports(identifierRequest({ olid: 'OL7353617M' }))).toBe(true); + expect(plugin.supports(identifierRequest({ olid: 'OL26320A' }))).toBe(true); + expect(plugin.supports(identifierRequest({ isbn: 'not-isbn', olid: 'OL-nope' }))).toBe(false); + }); +}); + +describe('OpenLibrary plugin', () => { + it('uses the canonical Books API ISBN path and maps edition metadata', async () => { + const plugin = createOpenLibraryPlugin({ + fetch: jsonFetch( + { + 'ISBN:9780684832722': { + key: '/books/OL7721520M', + title: 'The Sovereign Individual', + authors: [{ name: 'James Dale Davidson' }, { name: 'William Rees-Mogg' }], + publishers: [{ name: 'Scribner' }], + publish_date: '1996', + number_of_pages: 180, + cover: { large: 'https://covers.openlibrary.org/b/id/8432047-L.jpg' }, + subjects: [{ name: 'Fiction' }], + }, + }, + 200, + (url) => { + expect(url).toContain('/api/books?'); + expect(url).toContain('bibkeys=ISBN%3A9780684832722'); + expect(url).toContain('jscmd=data'); + } + ), + }); + + const artifacts = await plugin.enrich( + identifierRequest({ isbn: '9780684832722' }), + createMockPluginContext() + ); + + expect(artifacts).toHaveLength(1); + expect(artifacts[0]).toMatchObject({ + artifact_type: 'openlibrary', + data: { + identifier: '9780684832722', + identifierType: 'isbn', + entityType: 'edition', + isbn: '9780684832722', + olid: 'OL7721520M', + title: 'The Sovereign Individual', + authors: ['James Dale Davidson', 'William Rees-Mogg'], + }, + meta: { pluginId: 'openlibrary', provider: 'openlibrary' }, + }); + }); + + it('maps work and author OLIDs through their typed canonical paths', async () => { + const work = createOpenLibraryPlugin({ + fetch: jsonFetch( + { + key: '/works/OL45804W', + title: 'Fantastic Mr Fox', + description: { value: 'A fox outwits three farmers.' }, + authors: [{ author: { key: '/authors/OL34184A' } }], + covers: [8739161], + subjects: ['Foxes'], + }, + 200, + (url) => expect(url).toBe('https://openlibrary.org/works/OL45804W.json') + ), + }); + const workArtifacts = await work.enrich( + identifierRequest({ olid: 'OL45804W' }), + createMockPluginContext() + ); + expect(workArtifacts[0]?.data).toMatchObject({ + entityType: 'work', + olid: 'OL45804W', + authorOlids: ['OL34184A'], + description: 'A fox outwits three farmers.', + }); + + const author = createOpenLibraryPlugin({ + fetch: jsonFetch( + { key: '/authors/OL26320A', name: 'J. R. R. Tolkien', bio: 'English author.' }, + 200, + (url) => expect(url).toBe('https://openlibrary.org/authors/OL26320A.json') + ), + }); + const authorArtifacts = await author.enrich( + identifierRequest({ olid: 'OL26320A' }), + createMockPluginContext() + ); + expect(authorArtifacts[0]?.data).toMatchObject({ + entityType: 'author', + olid: 'OL26320A', + title: 'J. R. R. Tolkien', + description: 'English author.', + }); + }); + + it('treats 404/no record as a terminal no-match instead of a retry error', async () => { + const plugin = createOpenLibraryPlugin({ fetch: jsonFetch({}, 404) }); + const engine = createEnrichmentEngine({ plugins: [plugin] }); + const result = await engine.enrich(identifierRequest({ isbn: '9780684832722' })); + + expect(result.status).toBe('success'); + expect(result.artifacts).toEqual([]); + expect(result.errors).toEqual([]); + }); + + it('surfaces rate limits and upstream failures as retryable', async () => { + for (const [status, code] of [ + [429, 'rate_limited'], + [503, 'upstream_error'], + ] as const) { + const engine = createEnrichmentEngine({ + plugins: [createOpenLibraryPlugin({ fetch: jsonFetch({}, status) })], + }); + const result = await engine.enrich(identifierRequest({ isbn: '9780684832722' })); + + expect(result.status).toBe('failed'); + expect(result.errors).toMatchObject([{ pluginId: 'openlibrary', code, retriable: true }]); + } + }); + + it('makes malformed identifiers terminally not applicable', async () => { + const engine = createEnrichmentEngine({ plugins: [createOpenLibraryPlugin()] }); + const result = await engine.enrich(identifierRequest({ isbn: 'bad' })); + + expect(result.errors).toEqual([]); + expect(result.skipped).toContainEqual({ + pluginId: 'openlibrary', + reason: 'not_applicable', + }); + }); +}); diff --git a/packages/atom-enrichment/__tests__/providers/universal-providers.conformance.test.ts b/packages/atom-enrichment/__tests__/providers/universal-providers.conformance.test.ts index 9573961..9daab4d 100644 --- a/packages/atom-enrichment/__tests__/providers/universal-providers.conformance.test.ts +++ b/packages/atom-enrichment/__tests__/providers/universal-providers.conformance.test.ts @@ -26,6 +26,7 @@ import { } from '../../src/plugins/providers'; import { createMockAtomInput, + createMockPluginContext, createMockRequest, runPluginConformanceSuite, } from '../../src/testing'; @@ -966,6 +967,32 @@ describe('v1 provider plugins conformance', () => { }); describe('musicbrainz', () => { + it('resolves canonical ISRC hints before classification-name fallback', async () => { + const response = await readFixture( + '../../src/plugins/providers/musicbrainz/__fixtures__/recording.json' + ); + let requestedUrl = ''; + const plugin = createMusicBrainzPlugin({ + fetch: async (url) => { + requestedUrl = url; + return new Response(JSON.stringify({ recordings: [JSON.parse(response)] }), { + headers: { 'content-type': 'application/json' }, + }); + }, + }); + const request = createMockRequest({ + input: createMockAtomInput({ + atomType: 'thing', + jsonLd: { '@context': 'https://schema.org', '@type': 'Thing' }, + hints: { identifiers: { isrc: 'usacm0000001' } }, + }), + }); + + expect(plugin.supports(request)).toBe(true); + const artifacts = await plugin.enrich(request, createMockPluginContext()); + expect(requestedUrl).toContain('query=isrc%3AUSACM0000001'); + expect(artifacts[0]?.artifact_type).toBe('musicbrainz'); + }); it('normalizes musicbrainz recording fixture', async () => { const response = await readFixture( '../../src/plugins/providers/musicbrainz/__fixtures__/recording.json' @@ -1816,6 +1843,7 @@ function assertRegistryCoverage(registryToCheck: ClassificationRegistry): void { expect(registryToCheck.has('github-user')).toBe(true); expect(registryToCheck.has('npm-package')).toBe(true); expect(registryToCheck.has('musicbrainz')).toBe(true); + expect(registryToCheck.has('openlibrary')).toBe(true); expect(registryToCheck.has('spotify')).toBe(true); expect(registryToCheck.has('tmdb')).toBe(true); expect(registryToCheck.has('youtube')).toBe(true); diff --git a/packages/atom-enrichment/package.json b/packages/atom-enrichment/package.json index 9f8ed82..46a9cec 100644 --- a/packages/atom-enrichment/package.json +++ b/packages/atom-enrichment/package.json @@ -19,6 +19,7 @@ "./classifications/registry": "./src/classifications/registry.ts", "./classifications/schemas": "./src/classifications/schemas.ts", "./provider-external-data": "./src/provider-external-data.ts", + "./provider-plan": "./src/provider-plan.ts", "./slug-aliases": "./src/slug-aliases.ts", "./extraction": "./src/extraction/index.ts" }, diff --git a/packages/atom-enrichment/src/classifications/defaults.ts b/packages/atom-enrichment/src/classifications/defaults.ts index f743230..0d5d4da 100644 --- a/packages/atom-enrichment/src/classifications/defaults.ts +++ b/packages/atom-enrichment/src/classifications/defaults.ts @@ -28,6 +28,7 @@ import { npmPackageDataSchema, oembedDataSchema, opengraphDataSchema, + openLibraryDataSchema, placesDataSchema, productListingDataSchema, pubmedDataSchema, @@ -374,6 +375,15 @@ export const builtinClassificationDefinitions = [ schemaVersion: '1.0.0', runtime: 'server', }, + { + slug: 'openlibrary', + displayName: 'OpenLibrary Metadata', + category: 'knowledge', + dataSchema: openLibraryDataSchema, + description: 'Book, work, edition, or author metadata resolved from ISBN and OLID identities.', + schemaVersion: '1.0.0', + runtime: 'universal', + }, { slug: 'isbn', displayName: 'ISBN Book Lookup', diff --git a/packages/atom-enrichment/src/classifications/index.ts b/packages/atom-enrichment/src/classifications/index.ts index f0a46f5..755e744 100644 --- a/packages/atom-enrichment/src/classifications/index.ts +++ b/packages/atom-enrichment/src/classifications/index.ts @@ -35,6 +35,7 @@ export type { NpmPackageData, OEmbedData, OpenGraphData, + OpenLibraryData, PlacesData, ProductListingData, PubmedData, @@ -75,6 +76,7 @@ export { npmPackageDataSchema, oembedDataSchema, opengraphDataSchema, + openLibraryDataSchema, placesDataSchema, productListingDataSchema, pubmedDataSchema, diff --git a/packages/atom-enrichment/src/classifications/schemas.ts b/packages/atom-enrichment/src/classifications/schemas.ts index bad5c5b..5c4a434 100644 --- a/packages/atom-enrichment/src/classifications/schemas.ts +++ b/packages/atom-enrichment/src/classifications/schemas.ts @@ -97,6 +97,10 @@ export { type OpenGraphData, opengraphDataSchema, } from '../plugins/providers/opengraph/schema'; +export { + type OpenLibraryData, + openLibraryDataSchema, +} from '../plugins/providers/openlibrary/schema'; export { type PlacesData, placesDataSchema } from '../plugins/providers/places/schema'; export { type ProductListingData, diff --git a/packages/atom-enrichment/src/index.ts b/packages/atom-enrichment/src/index.ts index 3b5f72e..7265bb7 100644 --- a/packages/atom-enrichment/src/index.ts +++ b/packages/atom-enrichment/src/index.ts @@ -63,6 +63,7 @@ export { createNpmPlugin, createOEmbedPlugin, createOpenGraphPlugin, + createOpenLibraryPlugin, createProductListingPlugin, createSpotifyPlugin, createTmdbPlugin, @@ -120,6 +121,18 @@ export { xUserLookupResponseSchema, xUserLookupUserSchema, } from './provider-external-data'; +export type { + CreateIdentifierProviderPlanInput, + IdentifierProviderPlan, + IdentifierProviderPlanEntry, + IidProviderCapability, + IidProviderSlug, +} from './provider-plan'; +export { + createIdentifierProviderPlan, + IID_PROVIDER_CAPABILITIES, + IID_PROVIDER_SLUGS, +} from './provider-plan'; export { canonicalizeEnrichmentSlug, canonicalizeEnrichmentSlugs, diff --git a/packages/atom-enrichment/src/plugins/providers/index.ts b/packages/atom-enrichment/src/plugins/providers/index.ts index f053f3b..37764d1 100644 --- a/packages/atom-enrichment/src/plugins/providers/index.ts +++ b/packages/atom-enrichment/src/plugins/providers/index.ts @@ -12,6 +12,7 @@ export { createMusicBrainzPlugin } from './musicbrainz'; export { createNpmPlugin } from './npm'; export { createOEmbedPlugin } from './oembed'; export { createOpenGraphPlugin } from './opengraph'; +export { createOpenLibraryPlugin, resolveOpenLibraryTarget } from './openlibrary'; export { createPlacesPlugin, parseMapsUrl } from './places'; export { createPodcastIndexPlugin, diff --git a/packages/atom-enrichment/src/plugins/providers/musicbrainz/index.ts b/packages/atom-enrichment/src/plugins/providers/musicbrainz/index.ts index 9278c26..ddf99c5 100644 --- a/packages/atom-enrichment/src/plugins/providers/musicbrainz/index.ts +++ b/packages/atom-enrichment/src/plugins/providers/musicbrainz/index.ts @@ -105,6 +105,11 @@ function resolveMusicBrainzRequest( return { kind: 'mbid', mbid }; } + const isrc = getIdentifier(request, 'isrc'); + if (isrc && /^[A-Z]{2}[A-Z0-9]{3}\d{7}$/i.test(isrc)) { + return { kind: 'search', query: `isrc:${isrc.toUpperCase()}` }; + } + if (!isMusicRecordingRequest(request)) { return undefined; } diff --git a/packages/atom-enrichment/src/plugins/providers/openlibrary/external.ts b/packages/atom-enrichment/src/plugins/providers/openlibrary/external.ts new file mode 100644 index 0000000..c8d20d2 --- /dev/null +++ b/packages/atom-enrichment/src/plugins/providers/openlibrary/external.ts @@ -0,0 +1,61 @@ +import { z } from 'zod/v4'; + +const openLibraryNamedValueSchema = z.object({ name: z.string().optional() }).passthrough(); +const openLibraryAuthorLinkSchema = z.object({ key: z.string().optional() }).passthrough(); +const openLibraryWorkAuthorSchema = z + .object({ author: openLibraryAuthorLinkSchema.optional() }) + .passthrough(); +const openLibraryDescriptionSchema = z.union([ + z.string(), + z.object({ value: z.string().optional() }).passthrough(), +]); + +export const openLibraryBooksApiEntrySchema = z + .object({ + key: z.string().optional(), + title: z.string().optional(), + authors: z.array(openLibraryNamedValueSchema).optional(), + publishers: z.array(openLibraryNamedValueSchema).optional(), + publish_date: z.string().optional(), + number_of_pages: z.number().optional(), + cover: z + .object({ + small: z.string().optional(), + medium: z.string().optional(), + large: z.string().optional(), + }) + .passthrough() + .optional(), + subjects: z.array(openLibraryNamedValueSchema).optional(), + identifiers: z.record(z.string(), z.array(z.string())).optional(), + }) + .passthrough(); + +export const openLibraryBooksApiResponseSchema = z + .record(z.string(), openLibraryBooksApiEntrySchema) + .default({}); + +export const openLibraryEntityResponseSchema = z + .object({ + key: z.string().optional(), + title: z.string().optional(), + name: z.string().optional(), + personal_name: z.string().optional(), + description: openLibraryDescriptionSchema.optional(), + bio: openLibraryDescriptionSchema.optional(), + publish_date: z.string().optional(), + publishers: z.array(z.string()).optional(), + number_of_pages: z.number().optional(), + covers: z.array(z.number()).optional(), + photos: z.array(z.number()).optional(), + subjects: z.array(z.string()).optional(), + isbn_10: z.array(z.string()).optional(), + isbn_13: z.array(z.string()).optional(), + authors: z + .array(z.union([openLibraryAuthorLinkSchema, openLibraryWorkAuthorSchema])) + .optional(), + }) + .passthrough(); + +export type OpenLibraryBooksApiEntry = z.infer; +export type OpenLibraryEntityResponse = z.infer; diff --git a/packages/atom-enrichment/src/plugins/providers/openlibrary/index.ts b/packages/atom-enrichment/src/plugins/providers/openlibrary/index.ts new file mode 100644 index 0000000..e2391ac --- /dev/null +++ b/packages/atom-enrichment/src/plugins/providers/openlibrary/index.ts @@ -0,0 +1,233 @@ +import { defineEnrichmentPlugin, type EnrichmentPlugin } from '../../../plugins'; +import type { EnrichmentRequest } from '../../../types'; +import type { FetchLike } from '../__shared__/http'; +import { getIdentifier } from '../__shared__/request'; +import { + type OpenLibraryBooksApiEntry, + type OpenLibraryEntityResponse, + openLibraryBooksApiResponseSchema, + openLibraryEntityResponseSchema, +} from './external'; +import { type OpenLibraryData, openLibraryDataSchema } from './schema'; + +type CreateOpenLibraryPluginOptions = { + fetch?: FetchLike; + priority?: number; + TTL?: number; +}; + +type OpenLibraryTarget = + | { kind: 'isbn'; identifier: string } + | { kind: 'olid'; identifier: string; entityType: 'edition' } + | { kind: 'olid'; identifier: string; entityType: 'work' } + | { kind: 'olid'; identifier: string; entityType: 'author' }; + +const ISBN_PATTERN = /^(?:\d{9}[\dX]|\d{13})$/i; +const OLID_PATTERN = /^OL\d+([AMW])$/i; + +export function createOpenLibraryPlugin( + options: CreateOpenLibraryPluginOptions = {} +): EnrichmentPlugin { + const fetcher = options.fetch ?? (globalThis.fetch as FetchLike); + + return defineEnrichmentPlugin({ + id: 'openlibrary', + version: '1.0.0', + runtime: 'universal', + artifactTypes: ['openlibrary'], + priority: options.priority ?? 30, + TTL: options.TTL ?? 43_200, + + supports(request) { + return !!resolveOpenLibraryTarget(request); + }, + + async enrich(request, ctx) { + const target = resolveOpenLibraryTarget(request); + if (!target) { + return []; + } + + const data = await fetchOpenLibraryData(fetcher, target, ctx.signal); + if (!data) { + return []; + } + + return [ + { + artifact_type: 'openlibrary', + data: openLibraryDataSchema.parse(data), + meta: { + pluginId: 'openlibrary', + provider: 'openlibrary', + fetchedAt: ctx.now(), + sourceUrl: data.sourceUrl, + }, + }, + ]; + }, + }); +} + +export function resolveOpenLibraryTarget( + request: EnrichmentRequest +): OpenLibraryTarget | undefined { + // OLID is the more specific OpenLibrary-native identity. Its precedence is + // intentional and tested when a migration supplies both hints. + const olid = getIdentifier(request, 'olid'); + if (olid) { + const canonical = olid.toUpperCase(); + const match = canonical.match(OLID_PATTERN); + const suffix = match?.[1]?.toUpperCase(); + if (suffix === 'M' || suffix === 'W' || suffix === 'A') { + return { + kind: 'olid', + identifier: canonical, + entityType: suffix === 'M' ? 'edition' : suffix === 'W' ? 'work' : 'author', + }; + } + } + + const isbn = getIdentifier(request, 'isbn'); + if (isbn) { + const canonical = isbn.replace(/[\s-]/g, '').toUpperCase(); + if (ISBN_PATTERN.test(canonical)) { + return { kind: 'isbn', identifier: canonical }; + } + } + + return undefined; +} + +async function fetchOpenLibraryData( + fetcher: FetchLike, + target: OpenLibraryTarget, + signal: AbortSignal +): Promise { + if (target.kind === 'isbn' || target.entityType === 'edition') { + const bibkey = `${target.kind === 'isbn' ? 'ISBN' : 'OLID'}:${target.identifier}`; + const url = `https://openlibrary.org/api/books?bibkeys=${encodeURIComponent(bibkey)}&jscmd=data&format=json`; + const response = await fetcher(url, { signal }); + if (response.status === 404) { + return undefined; + } + assertRetryableResponse(response, url); + const payload = openLibraryBooksApiResponseSchema.parse(await response.json()); + const entry = payload[bibkey]; + return entry ? normalizeBooksApiEntry(target, entry) : undefined; + } + + const segment = target.entityType === 'author' ? 'authors' : 'works'; + const url = `https://openlibrary.org/${segment}/${encodeURIComponent(target.identifier)}.json`; + const response = await fetcher(url, { signal }); + if (response.status === 404) { + return undefined; + } + assertRetryableResponse(response, url); + const payload = openLibraryEntityResponseSchema.parse(await response.json()); + return normalizeEntityResponse(target, payload); +} + +function normalizeBooksApiEntry( + target: + | Extract + | Extract, + entry: OpenLibraryBooksApiEntry +): OpenLibraryData | undefined { + if (!entry.title) { + return undefined; + } + + const olid = extractOlid(entry.key) ?? entry.identifiers?.openlibrary?.[0]; + const isbn = target.kind === 'isbn' ? target.identifier : firstIsbn(entry.identifiers); + const identifierType = target.kind; + const identifier = target.identifier; + const sourceUrl = olid + ? `https://openlibrary.org/books/${encodeURIComponent(olid)}` + : `https://openlibrary.org/isbn/${encodeURIComponent(identifier)}`; + + return openLibraryDataSchema.parse({ + identifier, + identifierType, + entityType: 'edition', + isbn, + olid, + title: entry.title, + authors: names(entry.authors), + publisher: names(entry.publishers)?.[0], + publishedDate: entry.publish_date, + pageCount: entry.number_of_pages, + coverUrl: entry.cover?.large ?? entry.cover?.medium ?? entry.cover?.small, + subjects: names(entry.subjects), + sourceUrl, + }); +} + +function normalizeEntityResponse( + target: Extract, + entity: OpenLibraryEntityResponse +): OpenLibraryData | undefined { + const title = entity.title ?? entity.name ?? entity.personal_name; + if (!title) { + return undefined; + } + + const imageId = entity.covers?.[0] ?? entity.photos?.[0]; + const sourceUrl = `https://openlibrary.org/${target.entityType === 'author' ? 'authors' : 'works'}/${encodeURIComponent(target.identifier)}`; + + return openLibraryDataSchema.parse({ + identifier: target.identifier, + identifierType: 'olid', + entityType: target.entityType, + isbn: entity.isbn_13?.[0] ?? entity.isbn_10?.[0], + olid: target.identifier, + title, + authorOlids: extractAuthorOlids(entity.authors), + publisher: entity.publishers?.[0], + publishedDate: entity.publish_date, + pageCount: entity.number_of_pages, + coverUrl: imageId + ? `https://covers.openlibrary.org/${target.entityType === 'author' ? 'a' : 'b'}/id/${imageId}-L.jpg` + : undefined, + description: descriptionValue(entity.description ?? entity.bio), + subjects: entity.subjects, + sourceUrl, + }); +} + +function assertRetryableResponse(response: Response, url: string): void { + if (response.ok) { + return; + } + if (response.status === 429) { + throw new Error(`Rate limited by OpenLibrary (HTTP 429 from ${url})`); + } + throw new Error(`Upstream HTTP ${response.status} from ${url}`); +} + +function names(values: Array<{ name?: string }> | undefined): string[] | undefined { + const result = values?.flatMap((value) => (value.name ? [value.name] : [])); + return result && result.length > 0 ? result : undefined; +} + +function extractOlid(key: string | undefined): string | undefined { + return key?.match(/\/(OL\d+[AMW])$/i)?.[1]?.toUpperCase(); +} + +function firstIsbn(identifiers: Record | undefined): string | undefined { + return identifiers?.isbn_13?.[0] ?? identifiers?.isbn_10?.[0]; +} + +function extractAuthorOlids(authors: OpenLibraryEntityResponse['authors']): string[] | undefined { + const result = authors?.flatMap((author) => { + const candidate = author as { key?: string; author?: { key?: string } }; + const key = candidate.author?.key ?? candidate.key; + const olid = extractOlid(key); + return olid ? [olid] : []; + }); + return result && result.length > 0 ? result : undefined; +} + +function descriptionValue(value: string | { value?: string } | undefined): string | undefined { + return typeof value === 'string' ? value : value?.value; +} diff --git a/packages/atom-enrichment/src/plugins/providers/openlibrary/schema.ts b/packages/atom-enrichment/src/plugins/providers/openlibrary/schema.ts new file mode 100644 index 0000000..c7ede3f --- /dev/null +++ b/packages/atom-enrichment/src/plugins/providers/openlibrary/schema.ts @@ -0,0 +1,23 @@ +import { z } from 'zod/v4'; + +export const openLibraryDataSchema = z + .object({ + identifier: z.string().min(1), + identifierType: z.enum(['isbn', 'olid']), + entityType: z.enum(['book', 'edition', 'work', 'author']), + isbn: z.string().optional(), + olid: z.string().optional(), + title: z.string().min(1), + authors: z.array(z.string()).optional(), + authorOlids: z.array(z.string()).optional(), + publisher: z.string().optional(), + publishedDate: z.string().optional(), + pageCount: z.number().int().nonnegative().optional(), + coverUrl: z.string().url().optional(), + description: z.string().optional(), + subjects: z.array(z.string()).optional(), + sourceUrl: z.string().url(), + }) + .strict(); + +export type OpenLibraryData = z.infer; diff --git a/packages/atom-enrichment/src/presets/bundles.ts b/packages/atom-enrichment/src/presets/bundles.ts index 82003a5..6e217fd 100644 --- a/packages/atom-enrichment/src/presets/bundles.ts +++ b/packages/atom-enrichment/src/presets/bundles.ts @@ -12,6 +12,7 @@ import { createNpmPlugin, createOEmbedPlugin, createOpenGraphPlugin, + createOpenLibraryPlugin, createPlacesPlugin, createPodcastIndexPlugin, createProductListingPlugin, @@ -35,6 +36,7 @@ type GitHubPluginOptions = NonNullable[0]> type MicrodataPluginOptions = NonNullable[0]>; type MusicBrainzPluginOptions = NonNullable[0]>; type OEmbedPluginOptions = NonNullable[0]>; +type OpenLibraryPluginOptions = NonNullable[0]>; type ProductListingPluginOptions = NonNullable[0]>; type SpotifyPluginOptions = NonNullable[0]>; type TmdbPluginOptions = NonNullable[0]>; @@ -80,6 +82,7 @@ export type ServerDefaultPresetOptions = { microdata?: MicrodataPluginOptions; musicbrainz?: MusicBrainzPluginOptions; oembed?: OEmbedPluginOptions; + openlibrary?: OpenLibraryPluginOptions; opengraph?: OpenGraphPluginOptions; places?: PlacesPluginOptions; podcastIndex?: PodcastIndexPluginOptions; @@ -116,6 +119,7 @@ export function createServerDefaultPresetOptions( microdata: {}, musicbrainz: {}, oembed: {}, + openlibrary: {}, opengraph: {}, places: { apiKey: env.GOOGLE_PLACES_API_KEY, @@ -185,6 +189,7 @@ export function serverDefaultPreset(options: ServerDefaultPresetOptions = {}): E createSpotifyPlugin(options.spotify), createAppleMusicPlugin(options.appleMusic), createMusicBrainzPlugin(options.musicbrainz), + createOpenLibraryPlugin(options.openlibrary), createTmdbPlugin(options.tmdb), createPlacesPlugin(options.places), createPodcastIndexPlugin(options.podcastIndex), diff --git a/packages/atom-enrichment/src/provider-data.ts b/packages/atom-enrichment/src/provider-data.ts index c377904..d37d5b0 100644 --- a/packages/atom-enrichment/src/provider-data.ts +++ b/packages/atom-enrichment/src/provider-data.ts @@ -59,6 +59,10 @@ export { type OpenGraphData, opengraphDataSchema, } from './plugins/providers/opengraph/schema'; +export { + type OpenLibraryData, + openLibraryDataSchema, +} from './plugins/providers/openlibrary/schema'; export { type PlacesData, placesDataSchema } from './plugins/providers/places/schema'; export { type ProductListingData, diff --git a/packages/atom-enrichment/src/provider-plan.ts b/packages/atom-enrichment/src/provider-plan.ts new file mode 100644 index 0000000..780a034 --- /dev/null +++ b/packages/atom-enrichment/src/provider-plan.ts @@ -0,0 +1,189 @@ +/** + * Provider-plan boundary for `@0xintuition/iid-registry` output. + * + * This package intentionally does not depend on the registry. The IID layer + * owns parsing/canonicalization and supplies its ordered provider slugs plus + * canonical identifier hints. This adapter intersects that desired plan with + * the plugins registered in the current enrichment deployment without ever + * silently dropping a provider. + */ + +/** Mirrored capability vocabulary from iid-registry `PROVIDER_SLUGS`. */ +export const IID_PROVIDER_SLUGS = [ + 'apple-music', + 'coingecko', + 'crossref', + 'etherscan', + 'github', + 'musicbrainz', + 'npm', + 'openlibrary', + 'podcast-index', + 'spotify', + 'tmdb', + 'wikidata', + 'x-profile', +] as const; + +export type IidProviderSlug = (typeof IID_PROVIDER_SLUGS)[number]; + +export type IidProviderCapability = + | { + pluginId: string; + status: 'supported'; + } + | { + status: 'unsupported'; + reason: string; + }; + +/** + * Total compile-time contract: adding a mirrored registry slug requires an + * explicit Core capability decision before TypeScript will compile. + */ +export const IID_PROVIDER_CAPABILITIES: Readonly> = { + 'apple-music': { pluginId: 'apple-music', status: 'supported' }, + coingecko: { pluginId: 'coingecko', status: 'supported' }, + crossref: { pluginId: 'crossref', status: 'supported' }, + etherscan: { pluginId: 'etherscan', status: 'supported' }, + github: { pluginId: 'github', status: 'supported' }, + musicbrainz: { pluginId: 'musicbrainz', status: 'supported' }, + npm: { pluginId: 'npm', status: 'supported' }, + openlibrary: { pluginId: 'openlibrary', status: 'supported' }, + 'podcast-index': { pluginId: 'podcast-index', status: 'supported' }, + spotify: { pluginId: 'spotify', status: 'supported' }, + tmdb: { pluginId: 'tmdb', status: 'supported' }, + wikidata: { pluginId: 'wikidata', status: 'supported' }, + 'x-profile': { pluginId: 'x-profile', status: 'supported' }, +}; + +export type ScheduledProviderPlanEntry = { + providerSlug: IidProviderSlug; + pluginId: string; + ordinal: number; + status: 'scheduled'; + disposition: 'execute'; +}; + +export type UnavailableProviderPlanEntry = { + providerSlug: IidProviderSlug; + pluginId: string; + ordinal: number; + status: 'unavailable'; + disposition: 'retry'; + reason: 'plugin_not_registered'; +}; + +export type UnsupportedProviderPlanEntry = { + providerSlug: string; + ordinal: number; + status: 'unsupported'; + disposition: 'terminal'; + reason: 'unknown_provider_slug' | 'duplicate_provider_slug' | 'provider_not_implemented'; + detail?: string; +}; + +export type IdentifierProviderPlanEntry = + | ScheduledProviderPlanEntry + | UnavailableProviderPlanEntry + | UnsupportedProviderPlanEntry; + +export type IdentifierProviderPlan = { + /** Canonical hints, copied in caller-provided insertion order. */ + identifiers: Record; + /** One entry for every requested provider, in registry order. */ + entries: IdentifierProviderPlanEntry[]; + /** Ordered allowlist suitable for `EnrichmentRequest.plugins`. */ + plugins: string[]; + complete: boolean; +}; + +export type CreateIdentifierProviderPlanInput = { + providers: readonly string[]; + identifiers: Readonly>; + registeredPluginIds: Iterable; +}; + +export function createIdentifierProviderPlan( + input: CreateIdentifierProviderPlanInput +): IdentifierProviderPlan { + const registered = new Set(input.registeredPluginIds); + const seen = new Set(); + const entries: IdentifierProviderPlanEntry[] = []; + const plugins: string[] = []; + + for (const [ordinal, providerSlug] of input.providers.entries()) { + if (seen.has(providerSlug)) { + entries.push({ + providerSlug, + ordinal, + status: 'unsupported', + disposition: 'terminal', + reason: 'duplicate_provider_slug', + }); + continue; + } + seen.add(providerSlug); + + if (!isIidProviderSlug(providerSlug)) { + entries.push({ + providerSlug, + ordinal, + status: 'unsupported', + disposition: 'terminal', + reason: 'unknown_provider_slug', + }); + continue; + } + + const capability: IidProviderCapability = IID_PROVIDER_CAPABILITIES[providerSlug]; + if (capability.status === 'unsupported') { + entries.push({ + providerSlug, + ordinal, + status: 'unsupported', + disposition: 'terminal', + reason: 'provider_not_implemented', + detail: capability.reason, + }); + continue; + } + + const pluginId = capability.pluginId; + if (!registered.has(pluginId)) { + entries.push({ + providerSlug, + pluginId, + ordinal, + status: 'unavailable', + disposition: 'retry', + reason: 'plugin_not_registered', + }); + continue; + } + + entries.push({ + providerSlug, + pluginId, + ordinal, + status: 'scheduled', + disposition: 'execute', + }); + plugins.push(pluginId); + } + + return { + identifiers: copyCanonicalHints(input.identifiers), + entries, + plugins, + complete: entries.every((entry) => entry.status === 'scheduled'), + }; +} + +function isIidProviderSlug(value: string): value is IidProviderSlug { + return (IID_PROVIDER_SLUGS as readonly string[]).includes(value); +} + +function copyCanonicalHints(hints: Readonly>): Record { + return Object.fromEntries(Object.entries(hints)); +} diff --git a/packages/atom-parser/__tests__/fixtures/atom-parser-contract-fixtures.json b/packages/atom-parser/__tests__/fixtures/atom-parser-contract-fixtures.json index bf3a9ea..bb88502 100644 --- a/packages/atom-parser/__tests__/fixtures/atom-parser-contract-fixtures.json +++ b/packages/atom-parser/__tests__/fixtures/atom-parser-contract-fixtures.json @@ -125,6 +125,40 @@ "remote": null, "warnings": [] } + }, + { + "name": "custom_scheme_colon_bearing_identifier", + "input": "did:pkh:eip155:1:0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045", + "options": { + "remoteFetch": true + }, + "expected": { + "kind": "plain_string", + "normalizedInput": "did:pkh:eip155:1:0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045", + "data": { + "original": "did:pkh:eip155:1:0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045", + "trimmed": "did:pkh:eip155:1:0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045" + }, + "remote": null, + "warnings": [] + } + }, + { + "name": "unsupported_ftp_scheme", + "input": "ftp://example.com/archive.bin", + "options": { + "remoteFetch": true + }, + "expected": { + "kind": "plain_string", + "normalizedInput": "ftp://example.com/archive.bin", + "data": { + "original": "ftp://example.com/archive.bin", + "trimmed": "ftp://example.com/archive.bin" + }, + "remote": null, + "warnings": [] + } } ], "remoteCases": [ diff --git a/packages/atom-parser/__tests__/local-detection.test.ts b/packages/atom-parser/__tests__/local-detection.test.ts index f568323..1118995 100644 --- a/packages/atom-parser/__tests__/local-detection.test.ts +++ b/packages/atom-parser/__tests__/local-detection.test.ts @@ -196,6 +196,24 @@ describe('local detection: URL', () => { const result = await parseAtom('https://example.com/9780306406157', localOnly); expect(result.kind).toBe('url'); }); + + it('does not classify custom URI schemes as remotely inspectable URLs', async () => { + for (const input of [ + 'did:pkh:eip155:1:0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045', + 'urn:isbn:9780306406157', + 'ftp://example.com/archive.bin', + ]) { + const result = await parseAtom(input, localOnly); + expect(result.kind).toBe('plain_string'); + } + }); + + it('keeps IID-shaped input out of the generic URL lane', async () => { + for (const input of ['int:isrc:USQX91300108', 'int:isrc:US:QX9:1300108']) { + const result = await parseAtom(input, localOnly); + expect(result.kind).not.toBe('url'); + } + }); }); describe('local detection: ISBN', () => { diff --git a/packages/atom-parser/__tests__/remote-inspection.test.ts b/packages/atom-parser/__tests__/remote-inspection.test.ts index 0c0805c..46b23d2 100644 --- a/packages/atom-parser/__tests__/remote-inspection.test.ts +++ b/packages/atom-parser/__tests__/remote-inspection.test.ts @@ -216,11 +216,12 @@ describe('remote inspection: redirects', () => { }); describe('remote inspection: safety controls', () => { - it('denies unsupported scheme (ftp)', async () => { + it('does not send unsupported schemes to remote inspection', async () => { const result = await parseAtom('ftp://example.com/archive.bin', { remoteFetch: true, }); - expect(result.remote?.outcome).toBe('denied'); + expect(result.kind).toBe('plain_string'); + expect('remote' in result).toBe(false); }); it('denies private network targets by default', async () => { @@ -303,6 +304,18 @@ describe('remote inspection: IPFS gateway', () => { }); describe('remote inspection: non-remote kinds', () => { + it('does not attempt remote inspection for IID-shaped or colon-bearing identifiers', async () => { + for (const input of [ + 'int:isrc:USQX91300108', + 'int:isrc:US:QX9:1300108', + 'did:pkh:eip155:1:0xd8dA6BF26964aF9D7eEd9e03E53415D37aA96045', + ]) { + const result = await parseAtom(input, { remoteFetch: true }); + expect(result.kind).not.toBe('url'); + expect('remote' in result).toBe(false); + } + }); + it('does not attempt remote for plain strings', async () => { const result = await parseAtom('hello world', { remoteFetch: true }); expect(result.kind).toBe('plain_string'); diff --git a/packages/atom-parser/src/detect.ts b/packages/atom-parser/src/detect.ts index 43dcd3b..ce12244 100644 --- a/packages/atom-parser/src/detect.ts +++ b/packages/atom-parser/src/detect.ts @@ -202,6 +202,15 @@ function tryUrl(normalizedInput: string): UrlData | undefined { return undefined; } + // `URL` accepts any syntactically valid scheme (for example `did:`, + // `urn:`, and `int:`). In this parser, however, the `url` kind is the + // remotely inspectable HTTP(S) lane. Treating arbitrary identifiers as + // URLs would send them through URL classification and remote-fetch policy + // before their owning parser gets a chance to recognize them. + if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') { + return undefined; + } + return { canonicalUrl: parsed.href, scheme: parsed.protocol.replace(/:$/, ''), diff --git a/packages/contracts/__tests__/multivault.test.ts b/packages/contracts/__tests__/multivault.test.ts index 20f5e77..dab7ce5 100644 --- a/packages/contracts/__tests__/multivault.test.ts +++ b/packages/contracts/__tests__/multivault.test.ts @@ -24,8 +24,11 @@ describe('MultiVault contract artifacts', () => { }, {}); expect(namesByType.function?.has('createAtoms')).toBe(true); + expect(namesByType.function?.has('createAtomsWithUris')).toBe(true); + expect(namesByType.function?.has('getAtomUriConfig')).toBe(true); expect(namesByType.function?.has('createTriples')).toBe(true); expect(namesByType.event?.has('AtomCreated')).toBe(true); + expect(namesByType.event?.has('AtomContextRegistered')).toBe(true); expect(namesByType.event?.has('TripleCreated')).toBe(true); for (const eventName of MULTIVAULT_RINDEXER_EVENTS) { diff --git a/packages/contracts/package.json b/packages/contracts/package.json index e495c4a..4bb1671 100644 --- a/packages/contracts/package.json +++ b/packages/contracts/package.json @@ -18,7 +18,7 @@ "typecheck": "tsc --noEmit --emitDeclarationOnly false -p tsconfig.typecheck.json", "test": "bun test src __tests__", "devnet:deploy": "DEPLOY_TARGET=anvil bun run src/deploy/cli.ts", - "vendored:regen": "scripts/regen-vendored.sh", + "vendored:regen": "bash scripts/regen-vendored.sh", "ci": "biome check", "lint": "biome lint", "format": "biome format", @@ -32,7 +32,7 @@ "typescript": "catalog:" }, "dependencies": { - "@0xintuition/contracts-v2": "1.0.0-alpha.0", + "@0xintuition/contracts-v2": "1.1.0-alpha.0", "viem": "^2.23.2" } } diff --git a/packages/contracts/scripts/devnet-create-iid-fixtures.ts b/packages/contracts/scripts/devnet-create-iid-fixtures.ts new file mode 100644 index 0000000..37c5539 --- /dev/null +++ b/packages/contracts/scripts/devnet-create-iid-fixtures.ts @@ -0,0 +1,144 @@ +import { + type Address, + createPublicClient, + createWalletClient, + defineChain, + http, + parseEventLogs, + toHex, +} from 'viem'; +import { privateKeyToAccount } from 'viem/accounts'; +import { MultiVaultAbi } from '../src/multivault'; + +const ANVIL_CHAIN_ID = 31_337; +const ANVIL_PRIVATE_KEY = '0xac0974bec39a17e36ba4a6b4d238ff944bacb478cbed5efcae784d7bf4f2ff80'; +const rpcUrl = process.env.RPC_URL?.trim() || 'http://127.0.0.1:8545'; +const stateFile = process.env.STATE_FILE?.trim() || '../../devnet/deployments-devnet.json'; + +const fixtures = [ + { + iid: 'int:isrc:USUM71703861', + uris: ['https://musicbrainz.org/search?query=isrc%3AUSUM71703861&type=recording'], + }, + { + iid: 'int:isbn:9780684832722', + uris: ['https://openlibrary.org/isbn/9780684832722'], + }, +] as const; + +type DeploymentState = { + chainId: number; + MultiVault: Address; +}; + +const state = (await Bun.file(stateFile).json()) as DeploymentState; +if (state.chainId !== ANVIL_CHAIN_ID) { + throw new Error( + `Refusing to create deterministic development fixtures on chain ${state.chainId}; expected Anvil ${ANVIL_CHAIN_ID}.` + ); +} + +const chain = defineChain({ + id: ANVIL_CHAIN_ID, + name: 'Anvil', + nativeCurrency: { name: 'Ether', symbol: 'ETH', decimals: 18 }, + rpcUrls: { default: { http: [rpcUrl] } }, +}); +const account = privateKeyToAccount(ANVIL_PRIVATE_KEY); +const publicClient = createPublicClient({ chain, transport: http(rpcUrl) }); +const walletClient = createWalletClient({ account, chain, transport: http(rpcUrl) }); +const atomCost = await publicClient.readContract({ + address: state.MultiVault, + abi: MultiVaultAbi, + functionName: 'getAtomCost', +}); +const [maxUriCount, maxUriLength] = await publicClient.readContract({ + address: state.MultiVault, + abi: MultiVaultAbi, + functionName: 'getAtomUriConfig', +}); + +const results: Array> = []; +for (const fixture of fixtures) { + const atomData = toHex(fixture.iid); + const uris = fixture.uris.map((uri) => toHex(uri)); + if (uris.length > maxUriCount || uris.some((uri) => (uri.length - 2) / 2 > maxUriLength)) { + throw new Error(`Fixture URI context exceeds the on-chain configuration for ${fixture.iid}.`); + } + + const termId = await publicClient.readContract({ + address: state.MultiVault, + abi: MultiVaultAbi, + functionName: 'calculateAtomId', + args: [atomData], + }); + const exists = await publicClient.readContract({ + address: state.MultiVault, + abi: MultiVaultAbi, + functionName: 'isTermCreated', + args: [termId], + }); + if (exists) { + results.push({ iid: fixture.iid, termId, status: 'already-created' }); + continue; + } + + const request = await publicClient.simulateContract({ + account, + address: state.MultiVault, + abi: MultiVaultAbi, + functionName: 'createAtomsWithUris', + args: [account.address, [atomData], [atomCost], [uris]], + value: atomCost, + }); + const transactionHash = await walletClient.writeContract(request.request); + const receipt = await publicClient.waitForTransactionReceipt({ hash: transactionHash }); + if (receipt.status !== 'success') { + throw new Error(`Fixture creation reverted for ${fixture.iid}: ${transactionHash}`); + } + + const atomEvent = parseEventLogs({ + abi: MultiVaultAbi, + logs: receipt.logs, + eventName: 'AtomCreated', + }).find((event) => event.args.termId === termId); + const contextEvent = parseEventLogs({ + abi: MultiVaultAbi, + logs: receipt.logs, + eventName: 'AtomContextRegistered', + }).find((event) => event.args.termId === termId); + if (!atomEvent || !contextEvent) { + throw new Error( + `Creation receipt did not contain joined atom/context events for ${fixture.iid}.` + ); + } + if ( + contextEvent.args.uris.length !== uris.length || + contextEvent.args.uris.some((uri, index) => uri !== uris[index]) + ) { + throw new Error(`Creation receipt did not preserve URI bytes for ${fixture.iid}.`); + } + + results.push({ + iid: fixture.iid, + termId, + status: 'created', + transactionHash, + blockNumber: receipt.blockNumber.toString(), + uris: fixture.uris, + }); +} + +console.log( + JSON.stringify( + { + chainId: state.chainId, + multiVault: state.MultiVault, + atomCost: atomCost.toString(), + uriConfig: { maxUriCount, maxUriLength }, + fixtures: results, + }, + null, + 2 + ) +); diff --git a/packages/contracts/scripts/regen-vendored.sh b/packages/contracts/scripts/regen-vendored.sh index 1f727b9..9f69e36 100644 --- a/packages/contracts/scripts/regen-vendored.sh +++ b/packages/contracts/scripts/regen-vendored.sh @@ -105,14 +105,13 @@ for (const [name, source] of Object.entries(contracts)) { JS # ── Stage 2: size-fit MultiVault (optimizer_runs=200) ─────────────────────── -# EIP-170 chains (Intuition Sepolia enforces the 24,576-byte runtime cap; the -# canonical impl there is 23,926 B) cannot take the package's production -# optimizer_runs=10000 bytecode. Recompile plain MultiVault at 200 runs — the -# proxy's steady-state implementation; MultiVaultMigrationMode does not fit -# even at 200 runs. +# EIP-170 chains (including Intuition Sepolia) use a separately verified +# optimizer_runs=200 build of plain MultiVault. MultiVaultMigrationMode does +# not fit under the cap. Both package and size-fit MultiVault bytecodes contain +# a MultiVaultLib link placeholder which the deployer resolves at runtime. SIZEFIT_DIR="$PKG_DIR/.vendor-build-sizefit" rm -rf "$SIZEFIT_DIR" -mkdir -p "$SIZEFIT_DIR/src/interfaces" "$SIZEFIT_DIR/src/protocol" +mkdir -p "$SIZEFIT_DIR/src/interfaces" "$SIZEFIT_DIR/src/libraries" "$SIZEFIT_DIR/src/protocol" cd "$SIZEFIT_DIR" echo '{ "name": "sizefit-build", "private": true }' > package.json @@ -132,14 +131,14 @@ cat > foundry.toml < { - test('MultiVault ABI carries the six indexer-critical events', () => { + test('MultiVault ABI carries the seven indexer-critical events', () => { const events = new Set( MultiVaultAbi.filter((item) => item.type === 'event').map((item) => item.name) ); for (const required of [ 'AtomCreated', + 'AtomContextRegistered', 'TripleCreated', 'Deposited', 'Redeemed', @@ -29,8 +37,22 @@ describe('abis', () => { } }); + test('MultiVault ABI carries URI-aware atom APIs', () => { + const functions = new Set( + MultiVaultAbi.filter((item) => item.type === 'function').map((item) => item.name) + ); + + expect(functions).toContain('createAtomsWithUris'); + expect(functions).toContain('getAtomUriConfig'); + }); + test('vendored ABIs are surfaced', () => { - expect(AtomWardenAbi.some((i) => i.type === 'function' && i.name === 'initialize')).toBe(true); + const atomWardenInitialize = AtomWardenAbi.find( + (i) => i.type === 'function' && i.name === 'initialize' + ); + expect( + atomWardenInitialize && 'inputs' in atomWardenInitialize ? atomWardenInitialize.inputs : [] + ).toHaveLength(9); expect(WrappedTrustAbi.some((i) => i.type === 'function' && i.name === 'deposit')).toBe(true); }); }); @@ -42,7 +64,6 @@ describe('vendored artifacts', () => { UpgradeableBeaconArtifact, AtomWardenArtifact, WrappedTrustArtifact, - MultiVaultSizeFitArtifact, ]; test('every artifact has creation bytecode and a non-empty ABI', () => { @@ -54,25 +75,52 @@ describe('vendored artifacts', () => { }); test('size-fit MultiVault creation code fits under the EIP-170 ballpark', () => { - // Creation ≈ runtime + dispatcher; the real runtime (24,033 B) is - // asserted at regen time and verified live on Intuition Sepolia. - expect((MultiVaultSizeFitArtifact.bytecode.length - 2) / 2).toBeLessThan(24_576); + const linked = linkMultiVaultLibraryBytecode( + MultiVaultSizeFitArtifact.bytecode, + MultiVaultLinkReferences, + zeroAddress + ); + expect(isHex(linked)).toBe(true); + // Creation is slightly larger than runtime; the exact 20,379 B runtime is + // asserted by the regeneration script before it writes the artifact. + expect((linked.length - 2) / 2).toBeLessThan(24_576); + }); + + test('deployer links every MultiVault implementation path', () => { + const cases = [ + [MultiVaultBytecode, MultiVaultLinkReferences], + [MultiVaultMigrationModeBytecode, MultiVaultMigrationModeLinkReferences], + [MultiVaultSizeFitArtifact.bytecode, MultiVaultLinkReferences], + ] as const; + + for (const [bytecode, references] of cases) { + const linked = linkMultiVaultLibraryBytecode(bytecode, references, zeroAddress); + expect(isHex(linked)).toBe(true); + expect(linked).not.toContain('__$'); + } }); test('bytecode is stable (guards accidental vendored edits)', () => { // Regenerate deliberately with scripts/regen-vendored.sh, then update - // these pins (compiled from @0xintuition/contracts-v2@1.0.0-alpha.0 + OZ 5.4.0). + // these pins (compiled from @0xintuition/contracts-v2@1.1.0-alpha.0 + OZ 5.4.0). const hashes = Object.fromEntries( artifacts.map((artifact) => [artifact.contractName, keccak256(artifact.bytecode)]) ); + hashes.MultiVault = keccak256( + linkMultiVaultLibraryBytecode( + MultiVaultSizeFitArtifact.bytecode, + MultiVaultLinkReferences, + zeroAddress + ) + ); expect(hashes).toEqual({ TransparentUpgradeableProxy: '0x24a1dc4b18e0740872e78682f81be920c06d8625cd543aefc3d65150ae7c9203', TimelockController: '0x295f5901c1ae5f2745efea24250154c118928b67537e11c33629729160fa6a7d', UpgradeableBeacon: '0xd294f3707414f1a846cac6ca25db062b4dd427de5f5eabe04398182bd8f587bd', - AtomWarden: '0x44dd4b03f2d46d96e59f3e697239dd02182210d85495ddff6c81674514ee70a1', + AtomWarden: '0x2bf1f6ee2afce3b8593a26ad2b732b1e46a7a2a978d37e864aaf31a94cc34c73', WrappedTrust: '0x8941ce5213265a502cd85b8b6819ae4982c640d837e097add799a0c011b972c5', - MultiVault: '0x8863d73da7057f9d2cdb7654c669f30acd547726b115c35e68da27301f955b04', + MultiVault: '0x88a9e4380c4238d54b04f01c20e4085efa7318dcac025c49257c7e62004e49c4', }); }); }); diff --git a/packages/contracts/src/abis.ts b/packages/contracts/src/abis.ts index 8a51d32..bae5641 100644 --- a/packages/contracts/src/abis.ts +++ b/packages/contracts/src/abis.ts @@ -2,9 +2,9 @@ * Single import point for protocol ABIs across the monorepo. * * Everything re-exported here comes from the pinned `@0xintuition/contracts-v2` - * package (typed `as const`, viem-ready). Contracts the package does not yet - * export (AtomWarden, WrappedTrust) are surfaced from the vendored artifacts — - * see `../vendored/README.md`. + * package (typed `as const`, viem-ready). AtomWarden and WrappedTrust remain + * surfaced from the regenerated vendored pins until their consumers migrate + * to the equivalent 1.1 package exports; see `../vendored/README.md`. */ export { AtomWalletAbi, diff --git a/packages/contracts/src/addresses.ts b/packages/contracts/src/addresses.ts index 17ab965..ad5dc1c 100644 --- a/packages/contracts/src/addresses.ts +++ b/packages/contracts/src/addresses.ts @@ -33,6 +33,8 @@ export type DeploymentAddresses = { SatelliteEmissionsController?: Address; TrustBonding?: Address; BaseEmissionsController?: Address; + /** Library address linked into the 1.1+ MultiVault implementation. */ + MultiVaultLib?: Address; MultiVaultImplementation?: Address; deployer?: Address; }; diff --git a/packages/contracts/src/deploy/acceptance.ts b/packages/contracts/src/deploy/acceptance.ts index fb302c6..43a57f5 100644 --- a/packages/contracts/src/deploy/acceptance.ts +++ b/packages/contracts/src/deploy/acceptance.ts @@ -1,6 +1,7 @@ /** - * Deployment acceptance test: creating an atom on the deployed MultiVault - * must emit `AtomCreated`. Port of the check at the end of the legacy + * Deployment acceptance test: creating an atom with URI context on the + * deployed MultiVault must emit `AtomCreated` and `AtomContextRegistered`. + * Port of the check at the end of the legacy * `devnet/devnet-deploy.sh`, generalized to any deploy target. */ @@ -23,9 +24,12 @@ export type AcceptanceResult = { termId: `0x${string}`; creator: Address; atomCost: bigint; + uris: readonly `0x${string}`[]; + maxUriCount: number; + maxUriLength: number; }; -/** Create a unique throwaway atom and assert the `AtomCreated` event fires. */ +/** Create a unique URI-backed atom and assert both creation events fire. */ export async function runCreateAtomAcceptance(options: { rpcUrl: string; account: PrivateKeyAccount; @@ -44,22 +48,44 @@ export async function runCreateAtomAcceptance(options: { functionName: 'getAtomCost', }); log(` getAtomCost() = ${atomCost} wei`); + const [maxUriCount, maxUriLength] = await publicClient.readContract({ + address: options.multiVault, + abi: MultiVaultAbi, + functionName: 'getAtomUriConfig', + }); + if (maxUriCount < 1 || maxUriLength < 1) { + throw new Error( + `acceptance: invalid atom URI config (maxUriCount=${maxUriCount}, maxUriLength=${maxUriLength})` + ); + } + log(` getAtomUriConfig() = ${maxUriCount} URIs × ${maxUriLength} bytes`); // Unique per run so re-runs against persistent chain state never collide // with an already-created atom. - const atomData = toHex(`devnet-atom-${Date.now()}-${Math.floor(Math.random() * 1e9)}`); + const nonce = `${Date.now()}-${Math.floor(Math.random() * 1e9)}`; + // This probe validates the protocol's URI-aware creation path, not IID + // semantics. Keep its throwaway payload visibly outside the `int:` namespace + // so test data cannot be mistaken for a registered Intuition Identifier. + const atomData = toHex(`devnet-uri-atom-${nonce}`); + const uris = [toHex(`https://example.test/atoms/${nonce}`)] as const; + const expectedTermId = await publicClient.readContract({ + address: options.multiVault, + abi: MultiVaultAbi, + functionName: 'calculateAtomId', + args: [atomData], + }); const hash = await walletClient.writeContract({ address: options.multiVault, abi: MultiVaultAbi, - functionName: 'createAtoms', - args: [[atomData], [atomCost]], + functionName: 'createAtomsWithUris', + args: [options.account.address, [atomData], [atomCost], [uris]], value: atomCost, account: options.account, chain, }); const receipt = await publicClient.waitForTransactionReceipt({ hash }); if (receipt.status !== 'success') { - throw new Error(`acceptance: createAtoms reverted (tx ${hash})`); + throw new Error(`acceptance: createAtomsWithUris reverted (tx ${hash})`); } const events = parseEventLogs({ @@ -71,8 +97,35 @@ export async function runCreateAtomAcceptance(options: { if (!event) { throw new Error(`acceptance: AtomCreated event NOT found in receipt logs (tx ${hash})`); } + const contextEvents = parseEventLogs({ + abi: MultiVaultAbi, + logs: receipt.logs, + eventName: 'AtomContextRegistered', + }); + const contextEvent = contextEvents.find( + (e) => + e.address.toLowerCase() === options.multiVault.toLowerCase() && + e.args.termId === event.args.termId + ); + if (!contextEvent) { + throw new Error( + `acceptance: AtomContextRegistered event NOT found in receipt logs (tx ${hash})` + ); + } + if (event.args.termId !== expectedTermId) { + throw new Error( + `acceptance: AtomCreated term ID does not match calculateAtomId(atomData) (tx ${hash})` + ); + } + if (contextEvent.args.uris.length !== 1 || contextEvent.args.uris[0] !== uris[0]) { + throw new Error( + `acceptance: AtomContextRegistered contained unexpected URI context (tx ${hash})` + ); + } - log(` ACCEPTANCE PASSED: AtomCreated emitted in tx ${hash} (block ${receipt.blockNumber})`); + log( + ` ACCEPTANCE PASSED: AtomCreated + AtomContextRegistered emitted in tx ${hash} (block ${receipt.blockNumber})` + ); log(` creator: ${event.args.creator}`); log(` termId: ${event.args.termId}`); return { @@ -81,5 +134,8 @@ export async function runCreateAtomAcceptance(options: { termId: event.args.termId, creator: event.args.creator, atomCost, + uris, + maxUriCount, + maxUriLength, }; } diff --git a/packages/contracts/src/deploy/cli.ts b/packages/contracts/src/deploy/cli.ts index 2a62db5..36927d0 100644 --- a/packages/contracts/src/deploy/cli.ts +++ b/packages/contracts/src/deploy/cli.ts @@ -23,14 +23,17 @@ import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs'; import { dirname, resolve } from 'node:path'; import { fileURLToPath } from 'node:url'; -import { MultiVaultMigrationModeBytecode } from '@0xintuition/contracts-v2/bytecodes'; -import { type Address, createPublicClient, formatEther, http } from 'viem'; +import { + MultiVaultMigrationModeBytecode, + MultiVaultMigrationModeLinkReferences, +} from '@0xintuition/contracts-v2/bytecodes'; +import { type Address, createPublicClient, formatEther, http, zeroAddress } from 'viem'; import { privateKeyToAccount } from 'viem/accounts'; import { parseDeploymentState } from '../addresses'; import { runCreateAtomAcceptance } from './acceptance'; import { DEPLOY_TARGETS, type DeployTarget } from './config'; -import { deployIntuitionSystem, targetChain } from './system'; +import { deployIntuitionSystem, linkMultiVaultLibraryBytecode, targetChain } from './system'; // Anvil dev account #0 — the universal Foundry dev key, safe only for local chains. // gitleaks:allow (publicly documented, pre-funded only on local anvil) @@ -93,7 +96,14 @@ if (isLocal) { try { await publicClient.estimateGas({ account: account.address, - data: MultiVaultMigrationModeBytecode, + // The published 1.1 bytecode contains a MultiVaultLib placeholder. + // Linking to any address is sufficient for this code-size-only probe; + // the deployment path links to the actual library address. + data: linkMultiVaultLibraryBytecode( + MultiVaultMigrationModeBytecode, + MultiVaultMigrationModeLinkReferences, + zeroAddress + ), }); } catch (error) { const message = error instanceof Error ? error.message : String(error); @@ -153,7 +163,7 @@ if (state) { } console.log(`==> MultiVault proxy: ${state.MultiVault}`); -console.log(`==> Acceptance: createAtoms on ${state.MultiVault}`); +console.log(`==> Acceptance: createAtomsWithUris on ${state.MultiVault}`); await runCreateAtomAcceptance({ rpcUrl, account, multiVault: state.MultiVault, target }); console.log('==> DONE'); diff --git a/packages/contracts/src/deploy/config.ts b/packages/contracts/src/deploy/config.ts index a1c8713..9de7f19 100644 --- a/packages/contracts/src/deploy/config.ts +++ b/packages/contracts/src/deploy/config.ts @@ -38,6 +38,13 @@ export type DeployConfig = { entryFee: bigint; exitFee: bigint; protocolFee: bigint; + atomWardenClaimWindow: bigint; + atomWardenMinFeeThreshold: bigint; + atomWardenSignatureThreshold: bigint; + atomWardenMaxValidAfter: bigint; + atomWardenMaxValidUntil: bigint; + atomWardenMaxClaimsPerWindow: bigint; + atomWardenClaimCapWindow: bigint; bondingEpochLength: bigint; bondingSystemUtilizationLowerBound: bigint; bondingPersonalUtilizationLowerBound: bigint; @@ -66,6 +73,14 @@ const NETWORK_AGNOSTIC = { entryFee: 50n, // 0.5% exitFee: 75n, // 0.75% protocolFee: 125n, // 1.25% + // IntuitionDeployAndSetup.s.sol defaults for a fresh AtomWarden. + atomWardenClaimWindow: ONE_DAY * 365n, + atomWardenMinFeeThreshold: 0n, + atomWardenSignatureThreshold: 1n, + atomWardenMaxValidAfter: 3600n, + atomWardenMaxValidUntil: ONE_DAY, + atomWardenMaxClaimsPerWindow: 0n, // cap disabled + atomWardenClaimCapWindow: ONE_DAY, bondingSystemUtilizationLowerBound: 5000n, // 50% bondingPersonalUtilizationLowerBound: 2500n, // 25% bondingStartOffsetSeconds: 100n, // block.timestamp + 100 for fresh instances @@ -114,9 +129,8 @@ export type DeployTarget = { */ canonicalWrappedTrust?: Address; /** - * Chain enforces the EIP-170 24,576-byte runtime cap → the MultiVault - * implementation must be the size-fit build (the production - * optimizer_runs=10000 bytecode only deploys on raised-cap chains). + * Chain enforces the EIP-170 24,576-byte runtime cap → use the verified + * size-fit plain-MultiVault build instead of MultiVaultMigrationMode. */ eip170: boolean; /** State file name under devnet/. */ diff --git a/packages/contracts/src/deploy/system.ts b/packages/contracts/src/deploy/system.ts index 4021130..32a04c9 100644 --- a/packages/contracts/src/deploy/system.ts +++ b/packages/contracts/src/deploy/system.ts @@ -9,9 +9,9 @@ * profiles in `config.ts` cover the local anvil devnet (31337) and fresh * self-owned instances on Intuition Sepolia (13579). * - * The published bytecode is the production build (optimizer_runs=10000); - * MultiVault's runtime exceeds EIP-170, which the Intuition chains permit by - * raising the code-size cap — a local anvil must therefore run with + * The published bytecode is the production build (optimizer_runs=10000) and + * must be linked to a separately deployed MultiVaultLib. The default local + * deployment uses MultiVaultMigrationMode, so Anvil must run with * `--disable-code-size-limit`. */ @@ -20,6 +20,7 @@ import { BaseEmissionsControllerAbi, BondingCurveRegistryAbi, LinearCurveAbi, + MultiVaultLibAbi, MultiVaultMigrationModeAbi, SatelliteEmissionsControllerAbi, TrustBondingAbi, @@ -31,7 +32,10 @@ import { BondingCurveRegistryBytecode, LinearCurveBytecode, MultiVaultBytecode, + MultiVaultLibBytecode, + MultiVaultLinkReferences, MultiVaultMigrationModeBytecode, + MultiVaultMigrationModeLinkReferences, SatelliteEmissionsControllerBytecode, TrustBondingBytecode, } from '@0xintuition/contracts-v2/bytecodes'; @@ -45,6 +49,7 @@ import { getAddress, type Hex, http, + isHex, keccak256, type PublicClient, stringToBytes, @@ -65,6 +70,28 @@ import { import type { DeployTarget } from './config'; const MIGRATOR_ROLE = keccak256(stringToBytes('MIGRATOR_ROLE')); +const MULTIVAULT_LIB_FQN = 'src/libraries/MultiVaultLib.sol:MultiVaultLib'; + +/** Link a package-generated MultiVault bytecode artifact to its deployed library. */ +export function linkMultiVaultLibraryBytecode( + bytecode: string, + linkReferences: Readonly>, + libraryAddress: Address +): Hex { + const placeholder = linkReferences[MULTIVAULT_LIB_FQN]; + if (!placeholder) { + throw new Error(`missing link reference for ${MULTIVAULT_LIB_FQN}`); + } + if (!bytecode.includes(placeholder)) { + throw new Error(`MultiVault bytecode does not contain link placeholder ${placeholder}`); + } + + const linked = bytecode.replaceAll(placeholder, libraryAddress.slice(2).toLowerCase()); + if (!isHex(linked) || linked.includes('__$')) { + throw new Error('MultiVault bytecode remains invalid after library linking'); + } + return linked; +} /** Minimal viem chain for any deploy target. */ export function targetChain(target: DeployTarget, rpcUrl: string): Chain { @@ -89,7 +116,7 @@ export type DeploySystemOptions = { /** * Use the plain MultiVault bytecode as the proxy implementation instead of * the production-faithful MultiVaultMigrationMode (escape hatch; both pass - * the createAtoms acceptance test). + * the URI-backed atom acceptance test). */ plainMultiVaultImplementation?: boolean; log?: (message: string) => void; @@ -267,25 +294,37 @@ export async function deployIntuitionSystem( }) ); - log('==> [3/4] MultiVault proxy implementation'); - // EIP-170 chains cannot take the production optimizer_runs=10000 bytecode - // (MultiVault runtime 27,666 B; MigrationMode 30,926 B) — deploy the - // size-fit plain-MultiVault build instead (runtime 24,033 B; MigrationMode - // does not fit even at 200 runs, and plain MultiVault is a strict subset - // that the proxy would be upgraded to anyway). + log('==> [3/4] MultiVault library + proxy implementation'); + const multiVaultLib = await deployContract( + clients, + 'MultiVaultLib', + MultiVaultLibAbi, + MultiVaultLibBytecode + ); + // EIP-170 targets use the verified size-fit plain-MultiVault build. + // MultiVaultMigrationMode does not fit under the cap, and plain MultiVault + // is the steady-state implementation the migration proxy upgrades to. const useMigrationMode = !(options.target.eip170 || options.plainMultiVaultImplementation); const multiVaultImplementation = options.target.eip170 ? await deployContract( clients, 'MultiVault implementation (size-fit, optimizer_runs=200)', MultiVaultSizeFitArtifact.abi, - MultiVaultSizeFitArtifact.bytecode + linkMultiVaultLibraryBytecode( + MultiVaultSizeFitArtifact.bytecode, + MultiVaultLinkReferences, + multiVaultLib + ) ) : await deployContract( clients, useMigrationMode ? 'MultiVaultMigrationMode implementation' : 'MultiVault implementation', MultiVaultMigrationModeAbi, - useMigrationMode ? MultiVaultMigrationModeBytecode : MultiVaultBytecode + linkMultiVaultLibraryBytecode( + useMigrationMode ? MultiVaultMigrationModeBytecode : MultiVaultBytecode, + useMigrationMode ? MultiVaultMigrationModeLinkReferences : MultiVaultLinkReferences, + multiVaultLib + ) ); log('==> [4/4] IntuitionDeployAndSetup (full system)'); @@ -509,11 +548,21 @@ export async function deployIntuitionSystem( ); await write( clients, - 'AtomWarden.initialize(admin, MultiVault)', + 'AtomWarden.initialize(admin, MultiVault, claim policy)', atomWarden, AtomWardenArtifact.abi, 'initialize', - [admin, multiVault] + [ + admin, + multiVault, + cfg.atomWardenClaimWindow, + cfg.atomWardenMinFeeThreshold, + cfg.atomWardenSignatureThreshold, + cfg.atomWardenMaxValidAfter, + cfg.atomWardenMaxValidUntil, + cfg.atomWardenMaxClaimsPerWindow, + cfg.atomWardenClaimCapWindow, + ] ); await write( clients, @@ -539,6 +588,7 @@ export async function deployIntuitionSystem( chainId: options.target.chainId, WrappedTrust: wrappedTrust, BaseEmissionsController: baseEmissionsController, + MultiVaultLib: multiVaultLib, MultiVaultImplementation: multiVaultImplementation, MultiVault: multiVault, AtomWalletFactory: atomWalletFactory, diff --git a/packages/contracts/src/multivault.ts b/packages/contracts/src/multivault.ts index 7a4edf6..241a445 100644 --- a/packages/contracts/src/multivault.ts +++ b/packages/contracts/src/multivault.ts @@ -3,6 +3,7 @@ import { MultiVaultAbi, MultiVaultBytecode } from '@0xintuition/contracts-v2'; export const MULTIVAULT_CONTRACT_NAME = 'MultiVault' as const; export const MULTIVAULT_RINDEXER_EVENTS = [ 'AtomCreated', + 'AtomContextRegistered', 'TripleCreated', 'Deposited', 'Redeemed', diff --git a/packages/contracts/src/vendored.ts b/packages/contracts/src/vendored.ts index 4d89fc2..19d6b70 100644 --- a/packages/contracts/src/vendored.ts +++ b/packages/contracts/src/vendored.ts @@ -18,6 +18,13 @@ export type VendoredArtifact = { bytecode: Hex; }; +export type LinkableVendoredArtifact = { + contractName: string; + abi: Abi; + /** Solidity library placeholders must be linked before deployment. */ + bytecode: string; +}; + function artifact(raw: { contractName: string; abi: unknown; bytecode: string }): VendoredArtifact { return { contractName: raw.contractName, @@ -32,7 +39,11 @@ export const UpgradeableBeaconArtifact = artifact(upgradeableBeacon); export const AtomWardenArtifact = artifact(atomWarden); export const WrappedTrustArtifact = artifact(wrappedTrust); /** optimizer_runs=200 build whose runtime fits EIP-170 chains (see the JSON's note). */ -export const MultiVaultSizeFitArtifact = artifact(multiVaultSizeFit); +export const MultiVaultSizeFitArtifact: LinkableVendoredArtifact = { + contractName: multiVaultSizeFit.contractName, + abi: multiVaultSizeFit.abi as Abi, + bytecode: multiVaultSizeFit.bytecode, +}; export const AtomWardenAbi = AtomWardenArtifact.abi; export const WrappedTrustAbi = WrappedTrustArtifact.abi; diff --git a/packages/contracts/vendored/AtomWarden.json b/packages/contracts/vendored/AtomWarden.json index ffa2f0e..ef637b6 100644 --- a/packages/contracts/vendored/AtomWarden.json +++ b/packages/contracts/vendored/AtomWarden.json @@ -1,6 +1,6 @@ { "contractName": "AtomWarden", - "source": "@0xintuition/contracts-v2@1.0.0-alpha.0 src/protocol/wallet/AtomWarden.sol", + "source": "@0xintuition/contracts-v2@1.1.0-alpha.0 src/protocol/wallet/AtomWarden.sol", "compiler": { "solc": "0.8.29", "optimizerRuns": 10000, @@ -15,28 +15,131 @@ }, { "type": "function", - "name": "acceptOwnership", + "name": "CLAIM_AUTHORIZATION_TYPEHASH", "inputs": [], + "outputs": [ + { + "name": "", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "DEFAULT_ADMIN_ROLE", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "MAX_BATCH_SIZE", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "uint256", + "internalType": "uint256" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "OPERATOR_ROLE", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "SIGNER_ROLE", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "batchGrantAtomWalletOwnership", + "inputs": [ + { + "name": "atomIds", + "type": "bytes32[]", + "internalType": "bytes32[]" + }, + { + "name": "newOwners", + "type": "address[]", + "internalType": "address[]" + } + ], "outputs": [], "stateMutability": "nonpayable" }, { "type": "function", - "name": "claimOwnership", + "name": "claimAsCreatorAfterExpiry", "inputs": [ { "name": "atomId", "type": "bytes32", "internalType": "bytes32" - }, + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "claimCapWindow", + "inputs": [], + "outputs": [ { - "name": "newOwner", + "name": "", + "type": "uint256", + "internalType": "uint256" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "claimNonces", + "inputs": [ + { + "name": "claimant", "type": "address", "internalType": "address" } ], - "outputs": [], - "stateMutability": "nonpayable" + "outputs": [ + { + "name": "nonce", + "type": "uint256", + "internalType": "uint256" + } + ], + "stateMutability": "view" }, { "type": "function", @@ -53,17 +156,62 @@ }, { "type": "function", - "name": "initialize", + "name": "claimWindow", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "uint256", + "internalType": "uint256" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "claimWithAuthorization", "inputs": [ { - "name": "admin", - "type": "address", - "internalType": "address" + "name": "authorization", + "type": "tuple", + "internalType": "struct IAtomWarden.ClaimAuthorization", + "components": [ + { + "name": "claimant", + "type": "address", + "internalType": "address" + }, + { + "name": "atomId", + "type": "bytes32", + "internalType": "bytes32" + }, + { + "name": "claimType", + "type": "uint8", + "internalType": "uint8" + }, + { + "name": "nonce", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "validAfter", + "type": "uint48", + "internalType": "uint48" + }, + { + "name": "validUntil", + "type": "uint48", + "internalType": "uint48" + } + ] }, { - "name": "_multiVault", - "type": "address", - "internalType": "address" + "name": "signature", + "type": "bytes", + "internalType": "bytes" } ], "outputs": [], @@ -71,56 +219,103 @@ }, { "type": "function", - "name": "multiVault", + "name": "claimsInWindow", "inputs": [], "outputs": [ { "name": "", - "type": "address", - "internalType": "address" + "type": "uint256", + "internalType": "uint256" } ], "stateMutability": "view" }, { "type": "function", - "name": "owner", + "name": "currentClaimWindowId", "inputs": [], "outputs": [ { "name": "", - "type": "address", - "internalType": "address" + "type": "uint256", + "internalType": "uint256" } ], "stateMutability": "view" }, { "type": "function", - "name": "pendingOwner", + "name": "eip712Domain", "inputs": [], "outputs": [ { - "name": "", + "name": "fields", + "type": "bytes1", + "internalType": "bytes1" + }, + { + "name": "name", + "type": "string", + "internalType": "string" + }, + { + "name": "version", + "type": "string", + "internalType": "string" + }, + { + "name": "chainId", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "verifyingContract", "type": "address", "internalType": "address" + }, + { + "name": "salt", + "type": "bytes32", + "internalType": "bytes32" + }, + { + "name": "extensions", + "type": "uint256[]", + "internalType": "uint256[]" } ], "stateMutability": "view" }, { "type": "function", - "name": "renounceOwnership", - "inputs": [], - "outputs": [], - "stateMutability": "nonpayable" + "name": "getRoleAdmin", + "inputs": [ + { + "name": "role", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "stateMutability": "view" }, { "type": "function", - "name": "setMultiVault", + "name": "grantAtomWalletOwnership", "inputs": [ { - "name": "_multiVault", + "name": "atomId", + "type": "bytes32", + "internalType": "bytes32" + }, + { + "name": "newOwner", "type": "address", "internalType": "address" } @@ -130,10 +325,15 @@ }, { "type": "function", - "name": "transferOwnership", + "name": "grantRole", "inputs": [ { - "name": "newOwner", + "name": "role", + "type": "bytes32", + "internalType": "bytes32" + }, + { + "name": "account", "type": "address", "internalType": "address" } @@ -142,108 +342,892 @@ "stateMutability": "nonpayable" }, { - "type": "event", - "name": "AtomWalletOwnershipClaimed", + "type": "function", + "name": "hasRole", "inputs": [ { - "name": "atomId", + "name": "role", "type": "bytes32", - "indexed": false, "internalType": "bytes32" }, { - "name": "pendingOwner", + "name": "account", "type": "address", - "indexed": false, "internalType": "address" } ], - "anonymous": false - }, - { - "type": "event", - "name": "Initialized", - "inputs": [ + "outputs": [ { - "name": "version", - "type": "uint64", - "indexed": false, - "internalType": "uint64" + "name": "", + "type": "bool", + "internalType": "bool" } ], - "anonymous": false + "stateMutability": "view" }, { - "type": "event", - "name": "MultiVaultSet", + "type": "function", + "name": "incrementNonce", "inputs": [ { - "name": "multiVault", + "name": "claimant", "type": "address", - "indexed": false, "internalType": "address" } ], - "anonymous": false + "outputs": [], + "stateMutability": "nonpayable" }, { - "type": "event", - "name": "OwnershipTransferStarted", + "type": "function", + "name": "initialize", "inputs": [ { - "name": "previousOwner", + "name": "admin", "type": "address", - "indexed": true, "internalType": "address" }, { - "name": "newOwner", + "name": "_multiVault", "type": "address", - "indexed": true, "internalType": "address" + }, + { + "name": "_claimWindow", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "_minFeeThreshold", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "_signatureThreshold", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "_maxValidAfter", + "type": "uint48", + "internalType": "uint48" + }, + { + "name": "_maxValidUntil", + "type": "uint48", + "internalType": "uint48" + }, + { + "name": "_maxClaimsPerWindow", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "_claimCapWindow", + "type": "uint256", + "internalType": "uint256" } ], - "anonymous": false + "outputs": [], + "stateMutability": "nonpayable" }, { - "type": "event", - "name": "OwnershipTransferred", - "inputs": [ - { - "name": "previousOwner", - "type": "address", - "indexed": true, - "internalType": "address" - }, + "type": "function", + "name": "maxClaimsPerWindow", + "inputs": [], + "outputs": [ { - "name": "newOwner", - "type": "address", - "indexed": true, - "internalType": "address" + "name": "", + "type": "uint256", + "internalType": "uint256" } ], - "anonymous": false + "stateMutability": "view" }, { - "type": "error", - "name": "AtomWarden_AtomIdDoesNotExist", - "inputs": [] + "type": "function", + "name": "maxValidAfter", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "uint48", + "internalType": "uint48" + } + ], + "stateMutability": "view" }, { - "type": "error", - "name": "AtomWarden_AtomWalletNotDeployed", - "inputs": [] + "type": "function", + "name": "maxValidUntil", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "uint48", + "internalType": "uint48" + } + ], + "stateMutability": "view" }, { - "type": "error", - "name": "AtomWarden_ClaimOwnershipFailed", - "inputs": [] - }, + "type": "function", + "name": "minFeeThreshold", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "uint256", + "internalType": "uint256" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "multiVault", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "address", + "internalType": "address" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "pause", + "inputs": [], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "paused", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "reinitialize", + "inputs": [ + { + "name": "_claimWindow", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "_minFeeThreshold", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "_signatureThreshold", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "_maxValidAfter", + "type": "uint48", + "internalType": "uint48" + }, + { + "name": "_maxValidUntil", + "type": "uint48", + "internalType": "uint48" + }, + { + "name": "_maxClaimsPerWindow", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "_claimCapWindow", + "type": "uint256", + "internalType": "uint256" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "renounceRole", + "inputs": [ + { + "name": "role", + "type": "bytes32", + "internalType": "bytes32" + }, + { + "name": "callerConfirmation", + "type": "address", + "internalType": "address" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "revokeRole", + "inputs": [ + { + "name": "role", + "type": "bytes32", + "internalType": "bytes32" + }, + { + "name": "account", + "type": "address", + "internalType": "address" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "setClaimCapWindow", + "inputs": [ + { + "name": "newValue", + "type": "uint256", + "internalType": "uint256" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "setClaimWindow", + "inputs": [ + { + "name": "newClaimWindow", + "type": "uint256", + "internalType": "uint256" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "setMaxClaimsPerWindow", + "inputs": [ + { + "name": "newValue", + "type": "uint256", + "internalType": "uint256" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "setMaxValidAfter", + "inputs": [ + { + "name": "newValue", + "type": "uint48", + "internalType": "uint48" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "setMaxValidUntil", + "inputs": [ + { + "name": "newValue", + "type": "uint48", + "internalType": "uint48" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "setMinFeeThreshold", + "inputs": [ + { + "name": "newThreshold", + "type": "uint256", + "internalType": "uint256" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "setMultiVault", + "inputs": [ + { + "name": "_multiVault", + "type": "address", + "internalType": "address" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "setSignatureThreshold", + "inputs": [ + { + "name": "newThreshold", + "type": "uint256", + "internalType": "uint256" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "function", + "name": "signatureThreshold", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "uint256", + "internalType": "uint256" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "signerCount", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "uint256", + "internalType": "uint256" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "supportsInterface", + "inputs": [ + { + "name": "interfaceId", + "type": "bytes4", + "internalType": "bytes4" + } + ], + "outputs": [ + { + "name": "", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "unpause", + "inputs": [], + "outputs": [], + "stateMutability": "nonpayable" + }, + { + "type": "event", + "name": "AtomWalletOwnershipClaimed", + "inputs": [ + { + "name": "atomId", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "claimant", + "type": "address", + "indexed": true, + "internalType": "address" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "AtomWalletOwnershipClaimedByAuthorization", + "inputs": [ + { + "name": "atomId", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "claimant", + "type": "address", + "indexed": true, + "internalType": "address" + }, + { + "name": "firstSigner", + "type": "address", + "indexed": true, + "internalType": "address" + }, + { + "name": "claimType", + "type": "uint8", + "indexed": false, + "internalType": "uint8" + }, + { + "name": "signers", + "type": "uint16", + "indexed": false, + "internalType": "uint16" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "AtomWalletOwnershipClaimedByCreator", + "inputs": [ + { + "name": "atomId", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "creator", + "type": "address", + "indexed": true, + "internalType": "address" + }, + { + "name": "accumulatedFees", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "AtomWalletOwnershipGranted", + "inputs": [ + { + "name": "atomId", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "newOwner", + "type": "address", + "indexed": true, + "internalType": "address" + }, + { + "name": "operator", + "type": "address", + "indexed": true, + "internalType": "address" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "ClaimCapWindowSet", + "inputs": [ + { + "name": "oldValue", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + }, + { + "name": "newValue", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "ClaimNonceIncremented", + "inputs": [ + { + "name": "claimant", + "type": "address", + "indexed": true, + "internalType": "address" + }, + { + "name": "newNonce", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "ClaimWindowSet", + "inputs": [ + { + "name": "oldValue", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + }, + { + "name": "newValue", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "EIP712DomainChanged", + "inputs": [], + "anonymous": false + }, + { + "type": "event", + "name": "Initialized", + "inputs": [ + { + "name": "version", + "type": "uint64", + "indexed": false, + "internalType": "uint64" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "MaxClaimsPerWindowSet", + "inputs": [ + { + "name": "oldValue", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + }, + { + "name": "newValue", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "MaxValidAfterSet", + "inputs": [ + { + "name": "oldValue", + "type": "uint48", + "indexed": false, + "internalType": "uint48" + }, + { + "name": "newValue", + "type": "uint48", + "indexed": false, + "internalType": "uint48" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "MaxValidUntilSet", + "inputs": [ + { + "name": "oldValue", + "type": "uint48", + "indexed": false, + "internalType": "uint48" + }, + { + "name": "newValue", + "type": "uint48", + "indexed": false, + "internalType": "uint48" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "MinFeeThresholdSet", + "inputs": [ + { + "name": "oldValue", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + }, + { + "name": "newValue", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "MultiVaultSet", + "inputs": [ + { + "name": "multiVault", + "type": "address", + "indexed": false, + "internalType": "address" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "Paused", + "inputs": [ + { + "name": "account", + "type": "address", + "indexed": false, + "internalType": "address" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "RoleAdminChanged", + "inputs": [ + { + "name": "role", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "previousAdminRole", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "newAdminRole", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "RoleGranted", + "inputs": [ + { + "name": "role", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "account", + "type": "address", + "indexed": true, + "internalType": "address" + }, + { + "name": "sender", + "type": "address", + "indexed": true, + "internalType": "address" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "RoleRevoked", + "inputs": [ + { + "name": "role", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "account", + "type": "address", + "indexed": true, + "internalType": "address" + }, + { + "name": "sender", + "type": "address", + "indexed": true, + "internalType": "address" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "SignatureThresholdSet", + "inputs": [ + { + "name": "oldValue", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + }, + { + "name": "newValue", + "type": "uint256", + "indexed": false, + "internalType": "uint256" + } + ], + "anonymous": false + }, + { + "type": "event", + "name": "Unpaused", + "inputs": [ + { + "name": "account", + "type": "address", + "indexed": false, + "internalType": "address" + } + ], + "anonymous": false + }, + { + "type": "error", + "name": "AccessControlBadConfirmation", + "inputs": [] + }, + { + "type": "error", + "name": "AccessControlUnauthorizedAccount", + "inputs": [ + { + "name": "account", + "type": "address", + "internalType": "address" + }, + { + "name": "neededRole", + "type": "bytes32", + "internalType": "bytes32" + } + ] + }, + { + "type": "error", + "name": "AtomWarden_AlreadyClaimed", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_ArrayLengthMismatch", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_AtomIdDoesNotExist", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_AtomWalletNotDeployed", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_BatchTooLarge", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_ClaimCapExceeded", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_ClaimOwnershipFailed", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_ClaimWindowNotElapsed", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_CreatorClaimDisabled", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_CreatorUnknown", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_InsufficientSigners", + "inputs": [] + }, { "type": "error", "name": "AtomWarden_InvalidAddress", "inputs": [] }, + { + "type": "error", + "name": "AtomWarden_InvalidClaimCapWindow", + "inputs": [] + }, { "type": "error", "name": "AtomWarden_InvalidNewOwnerAddress", @@ -251,36 +1235,95 @@ }, { "type": "error", - "name": "InvalidInitialization", + "name": "AtomWarden_InvalidNonce", "inputs": [] }, { "type": "error", - "name": "NotInitializing", + "name": "AtomWarden_InvalidSignature", "inputs": [] }, { "type": "error", - "name": "OwnableInvalidOwner", - "inputs": [ - { - "name": "owner", - "type": "address", - "internalType": "address" - } - ] + "name": "AtomWarden_InvalidThreshold", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_InvalidTimeWindow", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_MinFeeThresholdNotMet", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_NonCanonicalSignerOrder", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_NotAtomCreator", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_SignatureLengthInvalid", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_UnauthorizedClaimant", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_UnauthorizedReinitializer", + "inputs": [] + }, + { + "type": "error", + "name": "AtomWarden_ValidityWindowTooLong", + "inputs": [] + }, + { + "type": "error", + "name": "EnforcedPause", + "inputs": [] + }, + { + "type": "error", + "name": "ExpectedPause", + "inputs": [] + }, + { + "type": "error", + "name": "InvalidInitialization", + "inputs": [] + }, + { + "type": "error", + "name": "NotInitializing", + "inputs": [] }, { "type": "error", - "name": "OwnableUnauthorizedAccount", + "name": "StringsInsufficientHexLength", "inputs": [ { - "name": "account", - "type": "address", - "internalType": "address" + "name": "value", + "type": "uint256", + "internalType": "uint256" + }, + { + "name": "length", + "type": "uint256", + "internalType": "uint256" } ] } ], - "bytecode": "0x6080604052348015600e575f5ffd5b5060156019565b60c9565b7ff0c57e16840df040f15088dc2f81fe391c3923bec73e23a9662efc9c229c6a00805468010000000000000000900460ff161560685760405163f92ee8a960e01b815260040160405180910390fd5b80546001600160401b039081161460c65780546001600160401b0319166001600160401b0390811782556040519081527fc7f505b2f371ae2175ee4913f4499e1f2633a7b5936321eed1cdaeb6115181d29060200160405180910390a15b50565b611363806100d65f395ff3fe608060405234801561000f575f5ffd5b50600436106100b9575f3560e01c80638da5cb5b11610072578063ed50d91c11610058578063ed50d91c1461014d578063f2fde38b14610160578063fe74e1ca14610173575f5ffd5b80638da5cb5b1461013d578063e30c397814610145575f5ffd5b8063715018a6116100a2578063715018a6146100e557806379ba5097146100ed5780638c3ecc45146100f5575f5ffd5b8063485cc955146100bd5780634d554c22146100d2575b5f5ffd5b6100d06100cb3660046110d2565b610186565b005b6100d06100e0366004611109565b6102f9565b6100d06105c7565b6100d06105da565b5f546101149073ffffffffffffffffffffffffffffffffffffffff1681565b60405173ffffffffffffffffffffffffffffffffffffffff909116815260200160405180910390f35b61011461065a565b61011461069b565b6100d061015b36600461112c565b6106c3565b6100d061016e36600461112c565b6106d4565b6100d061018136600461114e565b61078b565b5f61018f610b12565b805490915060ff68010000000000000000820416159067ffffffffffffffff165f811580156101bb5750825b90505f8267ffffffffffffffff1660011480156101d75750303b155b9050811580156101e5575080155b1561021c576040517ff92ee8a900000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b84547fffffffffffffffffffffffffffffffffffffffffffffffff0000000000000000166001178555831561027d5784547fffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffff16680100000000000000001785555b61028687610b3c565b61028f86610b4d565b83156102f05784547fffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffff168555604051600181527fc7f505b2f371ae2175ee4913f4499e1f2633a7b5936321eed1cdaeb6115181d29060200160405180910390a15b50505050505050565b610301610c12565b73ffffffffffffffffffffffffffffffffffffffff811661034e576040517f1cc0e0ea00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f546040517ffc4a75f80000000000000000000000000000000000000000000000000000000081526004810184905273ffffffffffffffffffffffffffffffffffffffff9091169063fc4a75f890602401602060405180830381865afa1580156103ba573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906103de9190611165565b610414576040517f792f079000000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f80546040517f218e250d0000000000000000000000000000000000000000000000000000000081526004810185905273ffffffffffffffffffffffffffffffffffffffff9091169063218e250d90602401602060405180830381865afa158015610481573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906104a59190611184565b90508073ffffffffffffffffffffffffffffffffffffffff163b5f036104f7576040517f7b7aa15100000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b6040517ff2fde38b00000000000000000000000000000000000000000000000000000000815273ffffffffffffffffffffffffffffffffffffffff838116600483015282169063f2fde38b906024015f604051808303815f87803b15801561055d575f5ffd5b505af115801561056f573d5f5f3e3d5ffd5b50506040805186815273ffffffffffffffffffffffffffffffffffffffff861660208201527f370ab9259d16fd6cc8f5a9f36513a6e5160e35e804cf7434a717ae9ecf2ae815935001905060405180910390a1505050565b6105cf610c12565b6105d85f610c6a565b565b33806105e461069b565b73ffffffffffffffffffffffffffffffffffffffff161461064e576040517f118cdaa700000000000000000000000000000000000000000000000000000000815273ffffffffffffffffffffffffffffffffffffffff821660048201526024015b60405180910390fd5b61065781610c6a565b50565b5f807f9016d09d72d40fdae2fd8ceac6b6234c7706214fd39c1cd1e609a0528c1993005b5473ffffffffffffffffffffffffffffffffffffffff1692915050565b5f807f237e158222e3e6968b72b9db0d8043aacf074ad9f650f0d1606b4d82ee432c0061067e565b6106cb610c12565b61065781610b4d565b6106dc610c12565b7f237e158222e3e6968b72b9db0d8043aacf074ad9f650f0d1606b4d82ee432c0080547fffffffffffffffffffffffff00000000000000000000000000000000000000001673ffffffffffffffffffffffffffffffffffffffff8316908117825561074561065a565b73ffffffffffffffffffffffffffffffffffffffff167f38d16b8cac22d99fc7c124b9cd0de2d3fa1faef420bfe791d8c362d765e2270060405160405180910390a35050565b5f546040517ffc4a75f80000000000000000000000000000000000000000000000000000000081526004810183905273ffffffffffffffffffffffffffffffffffffffff9091169063fc4a75f890602401602060405180830381865afa1580156107f7573d5f5f3e3d5ffd5b505050506040513d601f19601f8201168201806040525081019061081b9190611165565b610851576040517f792f079000000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f80546040517f6961cd1c0000000000000000000000000000000000000000000000000000000081526004810184905273ffffffffffffffffffffffffffffffffffffffff90911690636961cd1c906024015f60405180830381865afa1580156108bd573d5f5f3e3d5ffd5b505050506040513d5f823e601f3d9081017fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe016820160405261090291908101906111cc565b90505f61090e33610cbe565b60405160200161091e91906112bc565b60405160208183030381529060405290508080519060200120828051906020012014610976576040517f559f821700000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f80546040517f218e250d0000000000000000000000000000000000000000000000000000000081526004810186905273ffffffffffffffffffffffffffffffffffffffff9091169063218e250d90602401602060405180830381865afa1580156109e3573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190610a079190611184565b90508073ffffffffffffffffffffffffffffffffffffffff163b5f03610a59576040517f7b7aa15100000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b6040517ff2fde38b00000000000000000000000000000000000000000000000000000000815233600482015273ffffffffffffffffffffffffffffffffffffffff82169063f2fde38b906024015f604051808303815f87803b158015610abd575f5ffd5b505af1158015610acf573d5f5f3e3d5ffd5b5050604080518781523360208201527f370ab9259d16fd6cc8f5a9f36513a6e5160e35e804cf7434a717ae9ecf2ae815935001905060405180910390a150505050565b5f807ff0c57e16840df040f15088dc2f81fe391c3923bec73e23a9662efc9c229c6a005b92915050565b610b44610f69565b61065781610fa7565b73ffffffffffffffffffffffffffffffffffffffff8116610b9a576040517f3aa0b87700000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f80547fffffffffffffffffffffffff00000000000000000000000000000000000000001673ffffffffffffffffffffffffffffffffffffffff83169081179091556040519081527f52d161c0b054ff0f476f54f34dd34c480edb4a541cb443cf5a0f73d96acf23509060200160405180910390a150565b33610c1b61065a565b73ffffffffffffffffffffffffffffffffffffffff16146105d8576040517f118cdaa7000000000000000000000000000000000000000000000000000000008152336004820152602401610645565b7f237e158222e3e6968b72b9db0d8043aacf074ad9f650f0d1606b4d82ee432c0080547fffffffffffffffffffffffff0000000000000000000000000000000000000000168155610cba82610ffe565b5050565b604080518082018252601081527f303132333435363738396162636465660000000000000000000000000000000060208201528151602a808252606082810190945284841b915f916020820181803683370190505090507f3000000000000000000000000000000000000000000000000000000000000000815f81518110610d4857610d486112d2565b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff191690815f1a9053507f780000000000000000000000000000000000000000000000000000000000000081600181518110610daa57610daa6112d2565b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff191690815f1a9053505f5b6014811015610f6057836004848360148110610df857610df86112d2565b1a60f81b7effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff1916901c60f81c60ff1681518110610e3657610e366112d2565b01602001517fff000000000000000000000000000000000000000000000000000000000000001682610e6983600261132c565b610e74906002611343565b81518110610e8457610e846112d2565b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff191690815f1a90535083838260148110610ec557610ec56112d2565b825191901a600f16908110610edc57610edc6112d2565b01602001517fff000000000000000000000000000000000000000000000000000000000000001682610f0f83600261132c565b610f1a906003611343565b81518110610f2a57610f2a6112d2565b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff191690815f1a905350600101610dda565b50949350505050565b610f71611093565b6105d8576040517fd7e6bcf800000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b610faf610f69565b73ffffffffffffffffffffffffffffffffffffffff811661064e576040517f1e4fbdf70000000000000000000000000000000000000000000000000000000081525f6004820152602401610645565b7f9016d09d72d40fdae2fd8ceac6b6234c7706214fd39c1cd1e609a0528c19930080547fffffffffffffffffffffffff0000000000000000000000000000000000000000811673ffffffffffffffffffffffffffffffffffffffff848116918217845560405192169182907f8be0079c531659141344cd1fd0a4f28419497f9722a3daafe3b4186f6b6457e0905f90a3505050565b5f61109c610b12565b5468010000000000000000900460ff16919050565b73ffffffffffffffffffffffffffffffffffffffff81168114610657575f5ffd5b5f5f604083850312156110e3575f5ffd5b82356110ee816110b1565b915060208301356110fe816110b1565b809150509250929050565b5f5f6040838503121561111a575f5ffd5b8235915060208301356110fe816110b1565b5f6020828403121561113c575f5ffd5b8135611147816110b1565b9392505050565b5f6020828403121561115e575f5ffd5b5035919050565b5f60208284031215611175575f5ffd5b81518015158114611147575f5ffd5b5f60208284031215611194575f5ffd5b8151611147816110b1565b7f4e487b71000000000000000000000000000000000000000000000000000000005f52604160045260245ffd5b5f602082840312156111dc575f5ffd5b815167ffffffffffffffff8111156111f2575f5ffd5b8201601f81018413611202575f5ffd5b805167ffffffffffffffff81111561121c5761121c61119f565b6040517fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0603f7fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0601f8501160116810181811067ffffffffffffffff821117156112885761128861119f565b60405281815282820160200186101561129f575f5ffd5b8160208401602083015e5f91810160200191909152949350505050565b5f82518060208501845e5f920191825250919050565b7f4e487b71000000000000000000000000000000000000000000000000000000005f52603260045260245ffd5b7f4e487b71000000000000000000000000000000000000000000000000000000005f52601160045260245ffd5b8082028115828204841417610b3657610b366112ff565b80820180821115610b3657610b366112ff56fea164736f6c634300081d000a" + "bytecode": "0x6080604052348015600e575f5ffd5b5060156019565b60c9565b7ff0c57e16840df040f15088dc2f81fe391c3923bec73e23a9662efc9c229c6a00805468010000000000000000900460ff161560685760405163f92ee8a960e01b815260040160405180910390fd5b80546001600160401b039081161460c65780546001600160401b0319166001600160401b0390811782556040519081527fc7f505b2f371ae2175ee4913f4499e1f2633a7b5936321eed1cdaeb6115181d29060200160405180910390a15b50565b613bb3806100d65f395ff3fe608060405234801561000f575f5ffd5b50600436106102d8575f3560e01c80638c3ecc4511610187578063d547741f116100dd578063eeb58fa211610093578063f5743c4c1161006e578063f5743c4c14610677578063f5b541a61461068a578063fe74e1ca146106b1575f5ffd5b8063eeb58fa21461063e578063f0a128a314610651578063f4be0a9f14610664575f5ffd5b8063db6c7d3e116100c3578063db6c7d3e146105f9578063e88725f414610618578063ed50d91c1461062b575f5ffd5b8063d547741f146105d3578063d8cbc8ac146105e6575f5ffd5b8063b47b05fc1161013d578063c5d37ae111610118578063c5d37ae1146105af578063c80b62fd146105b8578063cfdbf254146105cb575f5ffd5b8063b47b05fc14610581578063bbde537414610593578063bccc672a146105a6575f5ffd5b8063a1ebf35d1161016d578063a1ebf35d1461054a578063a217fddf14610571578063a82f2e2614610578575f5ffd5b80638c3ecc45146104c957806391d14854146104f3575f5ffd5b80633f4ba83a1161023c5780635c975abb116101f25780637ca548c6116101cd5780637ca548c61461049d5780638456cb59146104a657806384b0196e146104ae575f5ffd5b80635c975abb1461044d57806367a0d84714610477578063779f528b1461048a575f5ffd5b806344be73801161022257806344be7380146104285780635706a93f146104315780635a26eb7f1461043a575f5ffd5b80633f4ba83a1461040d578063433970d614610415575f5ffd5b80631c0d9a61116102915780632c35650a116102775780632c35650a146103de5780632f2ff15d146103e757806336568abe146103fa575f5ffd5b80631c0d9a6114610394578063248a9ca31461039d575f5ffd5b806316606a31116102c157806316606a311461031957806317b2d4f51461034c5780631802be7f1461035f575f5ffd5b806301ffc9a7146102dc5780630bcd44a414610304575b5f5ffd5b6102ef6102ea366004613357565b6106c4565b60405190151581526020015b60405180910390f35b6103176103123660046133c4565b61075c565b005b600654610335906601000000000000900465ffffffffffff1681565b60405165ffffffffffff90911681526020016102fb565b61031761035a36600461344e565b610a07565b6103867f4921076d5d938fe26df63349746edbe28c18a74bcf69f980b8858c1e568dc9c481565b6040519081526020016102fb565b61038660075481565b6103866103ab3660046134b1565b5f9081527f02dd7bc7dec4dceedda775e58dd541e08a116c6c53815c0bd028192f7b626800602052604090206001015490565b610386600a5481565b6103176103f53660046134c8565b610d27565b6103176104083660046134c8565b610d70565b610317610dc1565b6103176104233660046134b1565b610dd6565b61038660035481565b61038660085481565b61031761044836600461353e565b610ded565b7fcd5ed15c6e187e77e9aee88184c21f4f2182ab5827cb3b7e07fbedcd63f033005460ff166102ef565b6103176104853660046134b1565b610eef565b6103176104983660046135aa565b610f02565b61038660055481565b610317611166565b6104b6611178565b6040516102fb979695949392919061367b565b5f546104db906001600160a01b031681565b6040516001600160a01b0390911681526020016102fb565b6102ef6105013660046134c8565b5f9182527f02dd7bc7dec4dceedda775e58dd541e08a116c6c53815c0bd028192f7b626800602090815260408084206001600160a01b0393909316845291905290205460ff1690565b6103867fe2f4eaae4a9751e85a3e4a7b9587827a877f29914755229b07a7b2da98285f7081565b6103865f81565b61038660045481565b6006546103359065ffffffffffff1681565b6103176105a13660046134b1565b611277565b61038660095481565b61038660025481565b6103176105c636600461372d565b61130c565b610386609681565b6103176105e13660046134c8565b61131f565b6103176105f43660046134b1565b611362565b610386610607366004613748565b60016020525f908152604090205481565b6103176106263660046134c8565b611375565b610317610639366004613748565b6113a9565b61031761064c3660046134b1565b6113bc565b61031761065f36600461372d565b6113cf565b6103176106723660046134b1565b6113e2565b610317610685366004613748565b61178e565b6103867f97667070c54ef182b0f5858b034beac1b6f3089aa2d3188bb1e8929f4fa9b92981565b6103176106bf3660046134b1565b611850565b5f7fffffffff0000000000000000000000000000000000000000000000000000000082167f7965db0b00000000000000000000000000000000000000000000000000000000148061075657507f01ffc9a7000000000000000000000000000000000000000000000000000000007fffffffff000000000000000000000000000000000000000000000000000000008316145b92915050565b5f610765611aea565b805490915060ff68010000000000000000820416159067ffffffffffffffff165f811580156107915750825b90505f8267ffffffffffffffff1660011480156107ad5750303b155b9050811580156107bb575080155b156107f2576040517ff92ee8a900000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b84547fffffffffffffffffffffffffffffffffffffffffffffffff000000000000000016600117855583156108535784547fffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffff16680100000000000000001785555b895f0361088c576040517fb1d1f9ae00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b610894611b12565b6109086040518060400160405280600a81526020017f41746f6d57617264656e000000000000000000000000000000000000000000008152506040518060400160405280600181526020017f3200000000000000000000000000000000000000000000000000000000000000815250611b1c565b610910611b12565b6109198e611b2e565b6109228d611ba3565b61092b8c611c4f565b6109348b611c95565b61093d89611cd3565b61094688611d44565b61094f86611dc3565b61095887611e47565b60048a9055604080515f8152602081018c90527f86f674391b876df89514893e9ffbba76e944695b1513b5607e057254499d7ab7910160405180910390a183156109f75784547fffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffff168555604051600181527fc7f505b2f371ae2175ee4913f4499e1f2633a7b5936321eed1cdaeb6115181d29060200160405180910390a15b5050505050505050505050505050565b60025f610a12611aea565b805490915068010000000000000000900460ff1680610a3f5750805467ffffffffffffffff808416911610155b15610a76576040517ff92ee8a900000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b80547fffffffffffffffffffffffffffffffffffffffffffffff0000000000000000001667ffffffffffffffff831617680100000000000000001781555f8054604080517ffa32161100000000000000000000000000000000000000000000000000000000815290516001600160a01b039092169163fa32161191600480820192610100929091908290030181865afa158015610b15573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190610b39919061379b565b519050336001600160a01b03821614610b7e576040517f3374a91900000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b875f03610bb7576040517fb1d1f9ae00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b610bbf611b12565b610c336040518060400160405280600a81526020017f41746f6d57617264656e000000000000000000000000000000000000000000008152506040518060400160405280600181526020017f3200000000000000000000000000000000000000000000000000000000000000815250611b1c565b610c3b611b12565b610c4481611b2e565b610c4d8a611c4f565b610c5689611c95565b610c5f87611cd3565b610c6886611d44565b610c7184611dc3565b610c7a85611e47565b6004889055604080515f8152602081018a90527f86f674391b876df89514893e9ffbba76e944695b1513b5607e057254499d7ab7910160405180910390a15080547fffffffffffffffffffffffffffffffffffffffffffffff00ffffffffffffffff16815560405167ffffffffffffffff831681527fc7f505b2f371ae2175ee4913f4499e1f2633a7b5936321eed1cdaeb6115181d29060200160405180910390a1505050505050505050565b5f8281527f02dd7bc7dec4dceedda775e58dd541e08a116c6c53815c0bd028192f7b6268006020526040902060010154610d6081611ecb565b610d6a8383611ed5565b50505050565b6001600160a01b0381163314610db2576040517f6697b23200000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b610dbc8282611f22565b505050565b5f610dcb81611ecb565b610dd3611f7c565b50565b5f610de081611ecb565b610de982611c95565b5050565b7f97667070c54ef182b0f5858b034beac1b6f3089aa2d3188bb1e8929f4fa9b929610e1781611ecb565b83828114610e51576040517f637d545700000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b6096811115610e8c576040517f8de7b57100000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f5b81811015610ee657610ede878783818110610eab57610eab61383d565b90506020020135868684818110610ec457610ec461383d565b9050602002016020810190610ed99190613748565b612006565b600101610e8e565b50505050505050565b5f610ef981611ecb565b610de982611e47565b610f0a61210f565b610f176020840184613748565b6001600160a01b0316336001600160a01b031614610f61576040517f76f1dd6a00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b610f6e836020013561216b565b5f610f7c8460200135612224565b9050610f87816122f2565b60015f610f976020870187613748565b6001600160a01b03166001600160a01b031681526020019081526020015f2054846060013514610ff3576040517f994c33b100000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b61101b61100660a086016080870161372d565b61101660c0870160a0880161372d565b612389565b5f5f6110288686866124ad565b915091506110346126b2565b60015f6110446020890189613748565b6001600160a01b03908116825260208083019390935260409091015f208054600101905584169063481379cc9061107d90890189613748565b6040517fffffffff0000000000000000000000000000000000000000000000000000000060e084901b1681526001600160a01b0390911660048201526024015f604051808303815f87803b1580156110d3575f5ffd5b505af11580156110e5573d5f5f3e3d5ffd5b5050506001600160a01b03831690506111016020880188613748565b6001600160a01b031660208801357fbf95bdd8e776795f5ee66a948ca1f64c00ffe9d063780c563ad095c0958702a861114060608b0160408c0161386a565b6040805160ff909216825261ffff871660208301520160405180910390a4505050505050565b5f61117081611ecb565b610dd361272c565b5f60608082808083817fa16a46d94261c7517cc8ff89f61c0ce93598e3c849801011dee649a6a557d10080549091501580156111b657506001810154155b611221576040517f08c379a000000000000000000000000000000000000000000000000000000000815260206004820152601560248201527f4549503731323a20556e696e697469616c697a6564000000000000000000000060448201526064015b60405180910390fd5b6112296127a5565b611231612878565b604080515f808252602082019092527f0f000000000000000000000000000000000000000000000000000000000000009c939b5091995046985030975095509350915050565b5f61128181611ecb565b81158061128f575060055482115b156112c6576040517fb1d1f9ae00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b600480549083905560408051828152602081018590527f86f674391b876df89514893e9ffbba76e944695b1513b5607e057254499d7ab7910160405180910390a1505050565b5f61131681611ecb565b610de982611d44565b5f8281527f02dd7bc7dec4dceedda775e58dd541e08a116c6c53815c0bd028192f7b626800602052604090206001015461135881611ecb565b610d6a8383611f22565b5f61136c81611ecb565b610de982611dc3565b7f97667070c54ef182b0f5858b034beac1b6f3089aa2d3188bb1e8929f4fa9b92961139f81611ecb565b610dbc8383612006565b5f6113b381611ecb565b610de982611ba3565b5f6113c681611ecb565b610de982611c4f565b5f6113d981611ecb565b610de982611cd3565b6113ea61210f565b6002545f03611425576040517f39c1afbe00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b61142e8161216b565b5f80546040517f0f924db7000000000000000000000000000000000000000000000000000000008152600481018490526001600160a01b0390911690630f924db790602401602060405180830381865afa15801561148e573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906114b2919061388a565b90506001600160a01b0381166114f4576040517fe057efb000000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b6001600160a01b0381163314611536576040517fbe31360400000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f61154083612224565b905061154b816122f2565b5f80546040517f5ecb42450000000000000000000000000000000000000000000000000000000081526001600160a01b03848116600483015290911690635ecb424590602401602060405180830381865afa1580156115ac573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906115d091906138a5565b905060035481101561160e576040517f85cff25d00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f80546040517f970a5fc5000000000000000000000000000000000000000000000000000000008152600481018790526001600160a01b039091169063970a5fc590602401602060405180830381865afa15801561166e573d5f5f3e3d5ffd5b505050506040513d601f19601f8201168201806040525081019061169291906138bc565b90506002548165ffffffffffff166116aa9190613904565b4210156116e3576040517fd71001ab00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b6040517f481379cc0000000000000000000000000000000000000000000000000000000081523360048201526001600160a01b0384169063481379cc906024015f604051808303815f87803b15801561173a575f5ffd5b505af115801561174c573d5f5f3e3d5ffd5b50506040518481523392508791507fcb0f018854207708af6a96d4a78924c15e6a23e3876fd07e8a0284e0431101669060200160405180910390a35050505050565b7f97667070c54ef182b0f5858b034beac1b6f3089aa2d3188bb1e8929f4fa9b9296117b881611ecb565b6001600160a01b0382166117f8576040517f3aa0b87700000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b6001600160a01b0382165f8181526001602081815260409283902080549092019182905591519081527f993a190cf6f1cb07dd137577c5476514c04303c193c5f54b61e85131f8764e82910160405180910390a25050565b61185861210f565b5f611862336128c9565b6040516020016118729190613917565b60405160208183030381529060405290505f61188d336128df565b60405160200161189d9190613917565b604080517fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0818403018152908290525f80547f6828caa6000000000000000000000000000000000000000000000000000000008452919350916001600160a01b0390911690636828caa69061191690869060040161392d565b602060405180830381865afa158015611931573d5f5f3e3d5ffd5b505050506040513d601f19601f8201168201806040525081019061195591906138a5565b5f80546040517f6828caa600000000000000000000000000000000000000000000000000000000815292935090916001600160a01b0390911690636828caa6906119a390869060040161392d565b602060405180830381865afa1580156119be573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906119e291906138a5565b90508185141580156119f45750808514155b15611a2b576040517f559f821700000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b611a348561216b565b5f611a3e86612224565b9050611a49816122f2565b6040517f481379cc0000000000000000000000000000000000000000000000000000000081523360048201526001600160a01b0382169063481379cc906024015f604051808303815f87803b158015611aa0575f5ffd5b505af1158015611ab2573d5f5f3e3d5ffd5b50506040513392508891507f370ab9259d16fd6cc8f5a9f36513a6e5160e35e804cf7434a717ae9ecf2ae815905f90a3505050505050565b5f807ff0c57e16840df040f15088dc2f81fe391c3923bec73e23a9662efc9c229c6a00610756565b611b1a6129a5565b565b611b246129a5565b610de982826129e3565b6001600160a01b038116611b6e576040517f3aa0b87700000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b611b785f82611ed5565b50610de97f97667070c54ef182b0f5858b034beac1b6f3089aa2d3188bb1e8929f4fa9b92982611ed5565b6001600160a01b038116611be3576040517f3aa0b87700000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f80547fffffffffffffffffffffffff0000000000000000000000000000000000000000166001600160a01b0383169081179091556040519081527f52d161c0b054ff0f476f54f34dd34c480edb4a541cb443cf5a0f73d96acf2350906020015b60405180910390a150565b600280549082905560408051828152602081018490527f6f88ee581d7bf536cf22db87b1ba056c0cccc44c5a37341bdae366f5339ecad891015b60405180910390a15050565b600380549082905560408051828152602081018490527f45f6a7099afef2c6ad29593291427f5017eab5a69ba80d5a4bb0af867b83688d9101611c89565b6006805465ffffffffffff8381167fffffffffffffffffffffffffffffffffffffffffffffffffffff00000000000083168117909355604080519190921680825260208201939093527fdb39f112360f067da46193a919af96901465f8daab44d4e5836e73f064daf47e9101611c89565b6006805465ffffffffffff83811666010000000000008181027fffffffffffffffffffffffffffffffffffffffff000000000000ffffffffffff85161790945560408051949093049091168084526020840191909152917f3f871a1f772c61791592de2868f905a1576c85c4ab01cb28608c31d61b3dc4379101611c89565b805f03611dfc576040517fdab99eaa00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b6008805490829055611e0e824261396c565b60095560408051828152602081018490527f03937ef8050dfab2453c44b9ed5d6e208c397eabbb9894c77096a479278a76899101611c89565b8015801590611e565750600854155b15611e8d576040517fdab99eaa00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b600780549082905560408051828152602081018490527f761fb31df50b45330bcc1030f1eaf9b4f02625d670a9cb50e9d1c8e0204359c29101611c89565b610dd38133612a55565b5f611ee08383612ae1565b9050808015611f0e57507fe2f4eaae4a9751e85a3e4a7b9587827a877f29914755229b07a7b2da98285f7083145b156107565760058054600101905592915050565b5f611f2d8383612bcb565b9050808015611f5b57507fe2f4eaae4a9751e85a3e4a7b9587827a877f29914755229b07a7b2da98285f7083145b8015611f6857505f600554115b1561075657600580545f1901905592915050565b611f84612c8d565b7fcd5ed15c6e187e77e9aee88184c21f4f2182ab5827cb3b7e07fbedcd63f0330080547fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff001681557f5db9ee0a495bf2e6ff9c91a7834c1ba4fdd244a5e8aa4e537bd38aeae4b073aa335b6040516001600160a01b039091168152602001611c44565b6001600160a01b038116612046576040517f1cc0e0ea00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b61204f8261216b565b5f61205983612224565b9050612064816122f2565b6040517f481379cc0000000000000000000000000000000000000000000000000000000081526001600160a01b03838116600483015282169063481379cc906024015f604051808303815f87803b1580156120bd575f5ffd5b505af11580156120cf573d5f5f3e3d5ffd5b50506040513392506001600160a01b038516915085907f27dc5e2041543e9fa3d5cf76a96a47134117b453b8791dd5c5d23cacd1cbc501905f90a4505050565b7fcd5ed15c6e187e77e9aee88184c21f4f2182ab5827cb3b7e07fbedcd63f033005460ff1615611b1a576040517fd93c066500000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f546040517ffc4a75f8000000000000000000000000000000000000000000000000000000008152600481018390526001600160a01b039091169063fc4a75f890602401602060405180830381865afa1580156121ca573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906121ee919061397f565b610dd3576040517f792f079000000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f80546040517f218e250d000000000000000000000000000000000000000000000000000000008152600481018490526001600160a01b039091169063218e250d90602401602060405180830381865afa158015612284573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906122a8919061388a565b9050806001600160a01b03163b5f036122ed576040517f7b7aa15100000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b919050565b806001600160a01b03166357c9ca146040518163ffffffff1660e01b8152600401602060405180830381865afa15801561232e573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190612352919061397f565b15610dd3576040517fe80396ae00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b60065461239e9065ffffffffffff1642613904565b8265ffffffffffff1611156123df576040517faae24fc500000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b6006546123fe906601000000000000900465ffffffffffff1642613904565b8165ffffffffffff16111561243f576040517faae24fc500000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b8165ffffffffffff168165ffffffffffff16108061246457508165ffffffffffff1642105b8061247657508065ffffffffffff1642115b15610de9576040517f32a8f11f00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f80806124b984612ce8565b600454909150808210156124f9576040517fd3cdb9cf00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f61250b61250689612d7e565b612e52565b90505f805b848110156126a2575f61252482604161399e565b90505f80612580868d858e61253a826041613904565b92612547939291906139b5565b8080601f0160208091040260200160405190810160405280939291908181526020018383808284375f92019190915250612e9992505050565b5090925090505f816003811115612599576125996139dc565b146125d0576040517f16d3c96600000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b846001600160a01b0316826001600160a01b03161161261b576040517f3e0dd18900000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b6001600160a01b0382165f9081527fed82e8858f919528fd86c81da277f0812ef4876fae8bc5251645af9640d3f49f602052604090205460ff1661268b576040517f16d3c96600000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b835f03612696578199505b50925050600101612510565b5083945050505050935093915050565b6007545f8190036126c05750565b5f600854426126cf919061396c565b905060095481146126e45760098190555f600a555b81600a541061271f576040517f1b650d9000000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5050600a80546001019055565b61273461210f565b7fcd5ed15c6e187e77e9aee88184c21f4f2182ab5827cb3b7e07fbedcd63f0330080547fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff001660011781557f62e78cea01bee320cd4e420270b5ea74000d11b0c9f74754ebdbfc544b05a25833611fee565b7fa16a46d94261c7517cc8ff89f61c0ce93598e3c849801011dee649a6a557d10280546060917fa16a46d94261c7517cc8ff89f61c0ce93598e3c849801011dee649a6a557d100916127f690613a09565b80601f016020809104026020016040519081016040528092919081815260200182805461282290613a09565b801561286d5780601f106128445761010080835404028352916020019161286d565b820191905f5260205f20905b81548152906001019060200180831161285057829003601f168201915b505050505091505090565b7fa16a46d94261c7517cc8ff89f61c0ce93598e3c849801011dee649a6a557d10380546060917fa16a46d94261c7517cc8ff89f61c0ce93598e3c849801011dee649a6a557d100916127f690613a09565b60606107566001600160a01b0383166014612ee2565b60605f6128eb836128c9565b6028602282012090915060601c60295b600181111561299c57600782600f16118015612930575060608382815181106129265761292661383d565b016020015160f81c115b1561298557602060f81b83828151811061294c5761294c61383d565b0160200180517fff00000000000000000000000000000000000000000000000000000000000000908116909218909116905f82901a9053505b60049190911c9061299581613a5a565b90506128fb565b50909392505050565b6129ad613102565b611b1a576040517fd7e6bcf800000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b6129eb6129a5565b7fa16a46d94261c7517cc8ff89f61c0ce93598e3c849801011dee649a6a557d1007fa16a46d94261c7517cc8ff89f61c0ce93598e3c849801011dee649a6a557d102612a378482613aba565b5060038101612a468382613aba565b505f8082556001909101555050565b5f8281527f02dd7bc7dec4dceedda775e58dd541e08a116c6c53815c0bd028192f7b626800602090815260408083206001600160a01b038516845290915290205460ff16610de9576040517fe2517d3f0000000000000000000000000000000000000000000000000000000081526001600160a01b038216600482015260248101839052604401611218565b5f8281527f02dd7bc7dec4dceedda775e58dd541e08a116c6c53815c0bd028192f7b626800602081815260408084206001600160a01b038616855290915282205460ff16612bc2575f848152602082815260408083206001600160a01b0387168452909152902080547fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00166001179055612b783390565b6001600160a01b0316836001600160a01b0316857f2f8788117e7eff1d82e926ec794901d17c78024a50270940304540a733656f0d60405160405180910390a46001915050610756565b5f915050610756565b5f8281527f02dd7bc7dec4dceedda775e58dd541e08a116c6c53815c0bd028192f7b626800602081815260408084206001600160a01b038616855290915282205460ff1615612bc2575f848152602082815260408083206001600160a01b038716808552925280832080547fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0016905551339287917ff6391f5c32d9c69d2a47ea670b442974b53935d1edc7fd64eb21e047a839171b9190a46001915050610756565b7fcd5ed15c6e187e77e9aee88184c21f4f2182ab5827cb3b7e07fbedcd63f033005460ff16611b1a576040517f8dfc202b00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f811580612cff5750612cfc604183613b93565b15155b15612d36576040517f8ae4cbcd00000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b612d4160418361396c565b905060968111156122ed576040517f8de7b57100000000000000000000000000000000000000000000000000000000815260040160405180910390fd5b5f7f4921076d5d938fe26df63349746edbe28c18a74bcf69f980b8858c1e568dc9c4612dad6020840184613748565b6020840135612dc2606086016040870161386a565b6060860135612dd760a088016080890161372d565b612de760c0890160a08a0161372d565b6040805160208101989098526001600160a01b0390961695870195909552606086019390935260ff909116608085015260a084015265ffffffffffff90811660c08401521660e082015261010001604051602081830303815290604052805190602001209050919050565b5f610756612e5e613120565b836040517f19010000000000000000000000000000000000000000000000000000000000008152600281019290925260228201526042902090565b5f5f5f8351604103612ed0576020840151604085015160608601515f1a612ec28882858561312e565b955095509550505050612edb565b505081515f91506002905b9250925092565b6060825f612ef184600261399e565b612efc906002613904565b67ffffffffffffffff811115612f1457612f14613763565b6040519080825280601f01601f191660200182016040528015612f3e576020820181803683370190505b5090507f3000000000000000000000000000000000000000000000000000000000000000815f81518110612f7457612f7461383d565b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff191690815f1a9053507f780000000000000000000000000000000000000000000000000000000000000081600181518110612fd657612fd661383d565b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff191690815f1a9053505f61301085600261399e565b61301b906001613904565b90505b60018111156130b7577f303132333435363738396162636465660000000000000000000000000000000083600f166010811061305c5761305c61383d565b1a60f81b8282815181106130725761307261383d565b60200101907effffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff191690815f1a90535060049290921c916130b081613a5a565b905061301e565b5081156130fa576040517fe22e27eb0000000000000000000000000000000000000000000000000000000081526004810186905260248101859052604401611218565b949350505050565b5f61310b611aea565b5468010000000000000000900460ff16919050565b5f613129613214565b905090565b5f80807f7fffffffffffffffffffffffffffffff5d576e7357a4501ddfe92f46681b20a084111561316757505f9150600390508261320a565b604080515f808252602082018084528a905260ff891692820192909252606081018790526080810186905260019060a0016020604051602081039080840390855afa1580156131b8573d5f5f3e3d5ffd5b50506040517fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe001519150506001600160a01b03811661320157505f92506001915082905061320a565b92505f91508190505b9450945094915050565b5f7f8b73c3c69bb8fe3d512ecc4cf759cc79239f7b179b0ffacaa9a75d522b39400f61323e613287565b613246613302565b60408051602081019490945283019190915260608201524660808201523060a082015260c00160405160208183030381529060405280519060200120905090565b5f7fa16a46d94261c7517cc8ff89f61c0ce93598e3c849801011dee649a6a557d100816132b26127a5565b8051909150156132ca57805160209091012092915050565b815480156132d9579392505050565b7fc5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470935050505090565b5f7fa16a46d94261c7517cc8ff89f61c0ce93598e3c849801011dee649a6a557d1008161332d612878565b80519091501561334557805160209091012092915050565b600182015480156132d9579392505050565b5f60208284031215613367575f5ffd5b81357fffffffff0000000000000000000000000000000000000000000000000000000081168114613396575f5ffd5b9392505050565b6001600160a01b0381168114610dd3575f5ffd5b65ffffffffffff81168114610dd3575f5ffd5b5f5f5f5f5f5f5f5f5f6101208a8c0312156133dd575f5ffd5b89356133e88161339d565b985060208a01356133f88161339d565b975060408a0135965060608a0135955060808a0135945060a08a013561341d816133b1565b935060c08a013561342d816133b1565b989b979a50959894979396929550929360e081013593506101000135919050565b5f5f5f5f5f5f5f60e0888a031215613464575f5ffd5b8735965060208801359550604088013594506060880135613484816133b1565b93506080880135613494816133b1565b9699959850939692959460a0840135945060c09093013592915050565b5f602082840312156134c1575f5ffd5b5035919050565b5f5f604083850312156134d9575f5ffd5b8235915060208301356134eb8161339d565b809150509250929050565b5f5f83601f840112613506575f5ffd5b50813567ffffffffffffffff81111561351d575f5ffd5b6020830191508360208260051b8501011115613537575f5ffd5b9250929050565b5f5f5f5f60408587031215613551575f5ffd5b843567ffffffffffffffff811115613567575f5ffd5b613573878288016134f6565b909550935050602085013567ffffffffffffffff811115613592575f5ffd5b61359e878288016134f6565b95989497509550505050565b5f5f5f83850360e08112156135bd575f5ffd5b60c08112156135ca575f5ffd5b5083925060c084013567ffffffffffffffff8111156135e7575f5ffd5b8401601f810186136135f7575f5ffd5b803567ffffffffffffffff81111561360d575f5ffd5b86602082840101111561361e575f5ffd5b939660209190910195509293505050565b5f81518084528060208401602086015e5f6020828601015260207fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0601f83011685010191505092915050565b7fff000000000000000000000000000000000000000000000000000000000000008816815260e060208201525f6136b560e083018961362f565b82810360408401526136c7818961362f565b606084018890526001600160a01b038716608085015260a0840186905283810360c0850152845180825260208087019350909101905f5b8181101561371c5783518352602093840193909201916001016136fe565b50909b9a5050505050505050505050565b5f6020828403121561373d575f5ffd5b8135613396816133b1565b5f60208284031215613758575f5ffd5b81356133968161339d565b7f4e487b71000000000000000000000000000000000000000000000000000000005f52604160045260245ffd5b80516122ed8161339d565b5f6101008284031280156137ad575f5ffd5b50604051610100810167ffffffffffffffff811182821017156137d2576137d2613763565b6040526137de83613790565b81526137ec60208401613790565b60208201526040838101519082015261380760608401613790565b60608201526080838101519082015260a0808401519082015260c0808401519082015260e0928301519281019290925250919050565b7f4e487b71000000000000000000000000000000000000000000000000000000005f52603260045260245ffd5b5f6020828403121561387a575f5ffd5b813560ff81168114613396575f5ffd5b5f6020828403121561389a575f5ffd5b81516133968161339d565b5f602082840312156138b5575f5ffd5b5051919050565b5f602082840312156138cc575f5ffd5b8151613396816133b1565b7f4e487b71000000000000000000000000000000000000000000000000000000005f52601160045260245ffd5b80820180821115610756576107566138d7565b5f82518060208501845e5f920191825250919050565b602081525f613396602083018461362f565b7f4e487b71000000000000000000000000000000000000000000000000000000005f52601260045260245ffd5b5f8261397a5761397a61393f565b500490565b5f6020828403121561398f575f5ffd5b81518015158114613396575f5ffd5b8082028115828204841417610756576107566138d7565b5f5f858511156139c3575f5ffd5b838611156139cf575f5ffd5b5050820193919092039150565b7f4e487b71000000000000000000000000000000000000000000000000000000005f52602160045260245ffd5b600181811c90821680613a1d57607f821691505b602082108103613a54577f4e487b71000000000000000000000000000000000000000000000000000000005f52602260045260245ffd5b50919050565b5f81613a6857613a686138d7565b505f190190565b601f821115610dbc57805f5260205f20601f840160051c81016020851015613a945750805b601f840160051c820191505b81811015613ab3575f8155600101613aa0565b5050505050565b815167ffffffffffffffff811115613ad457613ad4613763565b613ae881613ae28454613a09565b84613a6f565b6020601f821160018114613b1a575f8315613b035750848201515b5f19600385901b1c1916600184901b178455613ab3565b5f848152602081207fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe08516915b82811015613b675787850151825560209485019460019092019101613b47565b5084821015613b8457868401515f19600387901b60f8161c191681555b50505050600190811b01905550565b5f82613ba157613ba161393f565b50069056fea164736f6c634300081d000a" } diff --git a/packages/contracts/vendored/MultiVaultSizeFit.json b/packages/contracts/vendored/MultiVaultSizeFit.json index 80de33d..c65aeee 100644 --- a/packages/contracts/vendored/MultiVaultSizeFit.json +++ b/packages/contracts/vendored/MultiVaultSizeFit.json @@ -1,13 +1,13 @@ { "contractName": "MultiVault", - "source": "@0xintuition/contracts-v2@1.0.0-alpha.0 src/protocol/MultiVault.sol (size-fit build)", + "source": "@0xintuition/contracts-v2@1.1.0-alpha.0 src/protocol/MultiVault.sol (size-fit build)", "compiler": { "solc": "0.8.29", "optimizerRuns": 200, "evmVersion": "cancun", "bytecodeHash": "none" }, - "note": "optimizer_runs=200 so the runtime (24,033 B) fits EIP-170 chains like Intuition Sepolia; the package-published production build (optimizer_runs=10000, 27,666 B runtime) only deploys on chains with a raised code-size cap.", + "note": "optimizer_runs=200 so the runtime (20379 B) fits EIP-170 chains like Intuition Sepolia; deploy only after linking the separately deployed MultiVaultLib address.", "abi": [ { "type": "constructor", @@ -79,6 +79,19 @@ ], "stateMutability": "view" }, + { + "type": "function", + "name": "PAUSER_ROLE", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "TRIPLE_SALT", @@ -146,7 +159,7 @@ } ], "outputs": [], - "stateMutability": "nonpayable" + "stateMutability": "payable" }, { "type": "function", @@ -185,6 +198,44 @@ ], "stateMutability": "view" }, + { + "type": "function", + "name": "atomCreatedAt", + "inputs": [ + { + "name": "atomId", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "createdAt", + "type": "uint48", + "internalType": "uint48" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "atomCreators", + "inputs": [ + { + "name": "atomId", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "atomDepositFractionAmount", @@ -413,6 +464,69 @@ ], "stateMutability": "payable" }, + { + "type": "function", + "name": "createAtomsFor", + "inputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + }, + { + "name": "data", + "type": "bytes[]", + "internalType": "bytes[]" + }, + { + "name": "assets", + "type": "uint256[]", + "internalType": "uint256[]" + } + ], + "outputs": [ + { + "name": "", + "type": "bytes32[]", + "internalType": "bytes32[]" + } + ], + "stateMutability": "payable" + }, + { + "type": "function", + "name": "createAtomsWithUris", + "inputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + }, + { + "name": "data", + "type": "bytes[]", + "internalType": "bytes[]" + }, + { + "name": "assets", + "type": "uint256[]", + "internalType": "uint256[]" + }, + { + "name": "uris", + "type": "bytes[][]", + "internalType": "bytes[][]" + } + ], + "outputs": [ + { + "name": "", + "type": "bytes32[]", + "internalType": "bytes32[]" + } + ], + "stateMutability": "payable" + }, { "type": "function", "name": "createTriples", @@ -447,6 +561,45 @@ ], "stateMutability": "payable" }, + { + "type": "function", + "name": "createTriplesFor", + "inputs": [ + { + "name": "creator", + "type": "address", + "internalType": "address" + }, + { + "name": "subjectIds", + "type": "bytes32[]", + "internalType": "bytes32[]" + }, + { + "name": "predicateIds", + "type": "bytes32[]", + "internalType": "bytes32[]" + }, + { + "name": "objectIds", + "type": "bytes32[]", + "internalType": "bytes32[]" + }, + { + "name": "assets", + "type": "uint256[]", + "internalType": "uint256[]" + } + ], + "outputs": [ + { + "name": "", + "type": "bytes32[]", + "internalType": "bytes32[]" + } + ], + "stateMutability": "payable" + }, { "type": "function", "name": "currentEpoch", @@ -550,7 +703,7 @@ ], "outputs": [ { - "name": "shares", + "name": "", "type": "uint256[]", "internalType": "uint256[]" } @@ -700,6 +853,62 @@ ], "stateMutability": "view" }, + { + "type": "function", + "name": "getAtomCreatedAt", + "inputs": [ + { + "name": "termId", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "", + "type": "uint48", + "internalType": "uint48" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "getAtomCreator", + "inputs": [ + { + "name": "termId", + "type": "bytes32", + "internalType": "bytes32" + } + ], + "outputs": [ + { + "name": "", + "type": "address", + "internalType": "address" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "getAtomUriConfig", + "inputs": [], + "outputs": [ + { + "name": "maxUriCount", + "type": "uint32", + "internalType": "uint32" + }, + { + "name": "maxUriLength", + "type": "uint32", + "internalType": "uint32" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "getAtomWarden", @@ -1380,6 +1589,78 @@ "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "isApprovedToCreate", + "inputs": [ + { + "name": "sender", + "type": "address", + "internalType": "address" + }, + { + "name": "creator", + "type": "address", + "internalType": "address" + } + ], + "outputs": [ + { + "name": "approved", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "isApprovedToDeposit", + "inputs": [ + { + "name": "sender", + "type": "address", + "internalType": "address" + }, + { + "name": "receiver", + "type": "address", + "internalType": "address" + } + ], + "outputs": [ + { + "name": "approved", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, + { + "type": "function", + "name": "isApprovedToRedeem", + "inputs": [ + { + "name": "sender", + "type": "address", + "internalType": "address" + }, + { + "name": "receiver", + "type": "address", + "internalType": "address" + } + ], + "outputs": [ + { + "name": "approved", + "type": "bool", + "internalType": "bool" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "isAtom", @@ -1456,6 +1737,19 @@ ], "stateMutability": "view" }, + { + "type": "function", + "name": "lastSystemUtilizationEpoch", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "uint256", + "internalType": "uint256" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "maxRedeem", @@ -1485,6 +1779,30 @@ ], "stateMutability": "view" }, + { + "type": "function", + "name": "multicall", + "inputs": [ + { + "name": "data", + "type": "bytes[]", + "internalType": "bytes[]" + }, + { + "name": "values", + "type": "uint256[]", + "internalType": "uint256[]" + } + ], + "outputs": [ + { + "name": "results", + "type": "bytes[]", + "internalType": "bytes[]" + } + ], + "stateMutability": "payable" + }, { "type": "function", "name": "pause", @@ -1721,7 +2039,7 @@ "internalType": "uint256" } ], - "stateMutability": "nonpayable" + "stateMutability": "payable" }, { "type": "function", @@ -1755,11 +2073,24 @@ ], "outputs": [ { - "name": "received", + "name": "", "type": "uint256[]", "internalType": "uint256[]" } ], + "stateMutability": "payable" + }, + { + "type": "function", + "name": "reinitialize", + "inputs": [ + { + "name": "_timelock", + "type": "address", + "internalType": "address" + } + ], + "outputs": [], "stateMutability": "nonpayable" }, { @@ -1823,6 +2154,24 @@ "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "setAtomUriConfig", + "inputs": [ + { + "name": "maxUriCount", + "type": "uint32", + "internalType": "uint32" + }, + { + "name": "maxUriLength", + "type": "uint32", + "internalType": "uint32" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, { "type": "function", "name": "setBondingCurveConfig", @@ -1903,6 +2252,19 @@ "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "setTimelock", + "inputs": [ + { + "name": "_timelock", + "type": "address", + "internalType": "address" + } + ], + "outputs": [], + "stateMutability": "nonpayable" + }, { "type": "function", "name": "setTripleConfig", @@ -2025,6 +2387,19 @@ "outputs": [], "stateMutability": "nonpayable" }, + { + "type": "function", + "name": "timelock", + "inputs": [], + "outputs": [ + { + "name": "", + "type": "address", + "internalType": "address" + } + ], + "stateMutability": "view" + }, { "type": "function", "name": "totalTermsCreated", @@ -2230,6 +2605,31 @@ ], "anonymous": false }, + { + "type": "event", + "name": "AtomContextRegistered", + "inputs": [ + { + "name": "termId", + "type": "bytes32", + "indexed": true, + "internalType": "bytes32" + }, + { + "name": "registrant", + "type": "address", + "indexed": true, + "internalType": "address" + }, + { + "name": "uris", + "type": "bytes[]", + "indexed": false, + "internalType": "bytes[]" + } + ], + "anonymous": false + }, { "type": "event", "name": "AtomCreated", @@ -2261,6 +2661,25 @@ ], "anonymous": false }, + { + "type": "event", + "name": "AtomUriConfigUpdated", + "inputs": [ + { + "name": "maxUriCount", + "type": "uint32", + "indexed": false, + "internalType": "uint32" + }, + { + "name": "maxUriLength", + "type": "uint32", + "indexed": false, + "internalType": "uint32" + } + ], + "anonymous": false + }, { "type": "event", "name": "AtomWalletDepositFeeCollected", @@ -2763,6 +3182,19 @@ ], "anonymous": false }, + { + "type": "event", + "name": "TimelockSet", + "inputs": [ + { + "name": "timelock", + "type": "address", + "indexed": true, + "internalType": "address" + } + ], + "anonymous": false + }, { "type": "event", "name": "TotalUtilizationAdded", @@ -3075,6 +3507,16 @@ } ] }, + { + "type": "error", + "name": "MultiVault_AtomUriCountExceeded", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_AtomUriLengthExceeded", + "inputs": [] + }, { "type": "error", "name": "MultiVault_BurnFromZeroAddress", @@ -3095,6 +3537,11 @@ "name": "MultiVault_CannotDirectlyInitializeCounterTriple", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_CreatorNotApproved", + "inputs": [] + }, { "type": "error", "name": "MultiVault_DefaultCurveMustBeInitializedViaCreatePaths", @@ -3156,11 +3603,26 @@ "name": "MultiVault_InvalidArrayLength", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_InvalidAtomUriConfig", + "inputs": [] + }, { "type": "error", "name": "MultiVault_InvalidEpoch", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_MulticallValueMismatch", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_NestedMulticall", + "inputs": [] + }, { "type": "error", "name": "MultiVault_NoAtomDataProvided", @@ -3171,6 +3633,16 @@ "name": "MultiVault_OnlyAssociatedAtomWallet", "inputs": [] }, + { + "type": "error", + "name": "MultiVault_OnlyTimelock", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_RedeemYieldsNoAssets", + "inputs": [] + }, { "type": "error", "name": "MultiVault_RedeemerNotApproved", @@ -3228,6 +3700,16 @@ } ] }, + { + "type": "error", + "name": "MultiVault_UnexpectedValue", + "inputs": [] + }, + { + "type": "error", + "name": "MultiVault_ZeroAddress", + "inputs": [] + }, { "type": "error", "name": "NotInitializing", @@ -3239,5 +3721,5 @@ "inputs": [] } ], - "bytecode": "0x6080604052348015600e575f5ffd5b5060156019565b60c9565b7ff0c57e16840df040f15088dc2f81fe391c3923bec73e23a9662efc9c229c6a00805468010000000000000000900460ff161560685760405163f92ee8a960e01b815260040160405180910390fd5b80546001600160401b039081161460c65780546001600160401b0319166001600160401b0390811782556040519081527fc7f505b2f371ae2175ee4913f4499e1f2633a7b5936321eed1cdaeb6115181d29060200160405180910390a15b50565b615de1806100d65f395ff3fe6080604052600436106104bb575f3560e01c80637d4eaa911161026d578063c5a909451161014a578063ee3abe38116100be578063f679bf0911610083578063f679bf0914611181578063f87d29ac146111a0578063f9d320da146111bf578063fa321611146111de578063fc4a75f81461125e578063fccc28131461127d575f5ffd5b8063ee3abe38146110a7578063f22df312146110c6578063f5719008146110e5578063f5da42f314611104578063f5e6bfb914611156575f5ffd5b8063d34ddc051161010f578063d34ddc0514610f8b578063d547741f14610ff0578063d91c360b1461100f578063dea9423a14611042578063e9576f2314611055578063ecedd54114611074575f5ffd5b8063c5a9094514610efb578063c69f03bc14610f1a578063c8b1792814610f39578063c9cedcd014610f58578063cfdbf25414610f77575f5ffd5b8063983dda1e116101e1578063a3a9f885116101a6578063a3a9f88514610e1e578063a3bb4f6e14610e3d578063a3d81fdc14610e7f578063a814c1fe14610e9e578063bb0b5ebb14610ebd578063c12f794714610edc575f5ffd5b8063983dda1e14610cfa5780639c93ef9614610d195780639e4cb31114610dcd578063a18dc31914610dec578063a217fddf14610e0b575f5ffd5b806384e100db1161023257806384e100db14610bbd57806385a095c414610bda5780638938639f14610c055780638bb444bc14610c2457806391d1485414610c50578063927a97a114610c6f575f5ffd5b80637d4eaa9114610b1e5780637e6c54db14610b3d5780637f833f0414610b5c578063844cc15714610b765780638456cb5914610ba9575f5ffd5b80633f4ba83a1161039b578063614033091161030f5780636961cd1c116102d45780636961cd1c14610a575780636ced5d3f14610a8357806372188e3f14610aa25780637667180814610ac0578063768bc3e214610ad45780637762b6a614610af3575f5ffd5b806361403309146109c457806362a80f3d146109d757806362b3bd4a146109f6578063678eae2314610a0a5780636828caa614610a38575f5ffd5b806358717e0d1161036057806358717e0d146108e65780635b6dd6bc146109055780635b7946f1146109385780635c975abb146109575780635ecb42451461097a5780636001ab68146109a5575f5ffd5b80633f4ba83a146108265780634342e9661461083a57806344fcc1f1146108595780634523e2bc1461088f57806345ab4716146108ae575f5ffd5b806324e73572116104325780632f2ff15d116103f75780632f2ff15d146107605780632fb1d2701461077f57806333332d391461079257806336568abe146107a65780633c6bbf45146107c55780633e27173c146107e5575f5ffd5b806324e73572146106885780632747a57e146106a75780632db27075146106e15780632e1aa0f2146107005780632eddd6771461071f575f5ffd5b80631a2385de116104835780631a2385de146105a05780631e19e2c8146105bf5780631f5575fb146105d95780631fdc812e14610604578063218e250d14610632578063248a9ca314610669575f5ffd5b806301a21760146104bf57806301ffc9a7146104f85780630d65c91c14610527578063139d4fa51461054957806313ee9df41461056a575b5f5ffd5b3480156104ca575f5ffd5b506104de6104d9366004615128565b611292565b604080519283526020830191909152015b60405180910390f35b348015610503575f5ffd5b50610517610512366004615148565b6112ba565b60405190151581526020016104ef565b348015610532575f5ffd5b5061053b6112f0565b6040519081526020016104ef565b348015610554575f5ffd5b50610568610563366004615256565b6112fe565b005b348015610575575f5ffd5b5061057e611368565b60408051825181526020808401519082015291810151908201526060016104ef565b3480156105ab575f5ffd5b5061053b6105ba366004615128565b6113ae565b3480156105ca575f5ffd5b50600b54600c546104de919082565b3480156105e4575f5ffd5b5061053b6105f3366004615270565b601c6020525f908152604090205481565b34801561060f575f5ffd5b5061051761061e366004615270565b5f9081526018602052604090205460ff1690565b34801561063d575f5ffd5b5061065161064c366004615270565b611452565b6040516001600160a01b0390911681526020016104ef565b348015610674575f5ffd5b5061053b610683366004615270565b61145c565b348015610693575f5ffd5b506105686106a23660046152b6565b61147c565b3480156106b2575f5ffd5b506106c66106c1366004615128565b6114d3565b604080519384526020840192909252908201526060016104ef565b3480156106ec575f5ffd5b506104de6106fb3660046152d0565b6114ed565b34801561070b575f5ffd5b5061053b61071a3660046152d0565b611534565b34801561072a575f5ffd5b506040805180820182525f80825260209182015281518083019092526009548252600a54908201525b6040516104ef91906152f9565b34801561076b575f5ffd5b5061056861077a366004615310565b61154a565b61053b61078d36600461533e565b61156c565b34801561079d575f5ffd5b5061053b6115d4565b3480156107b1575f5ffd5b506105686107c0366004615310565b6115dd565b6107d86107d33660046153b6565b611615565b6040516104ef9190615480565b3480156107f0575f5ffd5b5061053b6107ff3660046154c2565b6001600160a01b03919091165f908152601f60209081526040808320938352929052205490565b348015610831575f5ffd5b5061056861166a565b348015610845575f5ffd5b506105686108543660046154ec565b611687565b348015610864575f5ffd5b5061053b6108733660046154c2565b601f60209081525f928352604080842090915290825290205481565b34801561089a575f5ffd5b5061053b6108a93660046152d0565b611793565b3480156108b9575f5ffd5b506040805180820182525f8082526020918201528151808301909252600b548252600c5490820152610753565b3480156108f1575f5ffd5b5061053b6109003660046152d0565b6117c8565b348015610910575f5ffd5b5061053b61091f36600461551b565b6001600160a01b03165f90815260208052604090205490565b348015610943575f5ffd5b5061056861095236600461558c565b6117fd565b348015610962575f5ffd5b505f516020615d955f395f51905f525460ff16610517565b348015610985575f5ffd5b5061053b61099436600461551b565b601d6020525f908152604090205481565b3480156109b0575f5ffd5b5061053b6109bf366004615270565b61185d565b6107d86109d23660046155a6565b61186d565b3480156109e2575f5ffd5b5061053b6109f1366004615270565b6118b2565b348015610a01575f5ffd5b5061053b5f5481565b348015610a15575f5ffd5b50610517610a24366004615270565b60216020525f908152604090205460ff1681565b348015610a43575f5ffd5b5061053b610a52366004615610565b6118bc565b348015610a62575f5ffd5b50610a76610a71366004615270565b6118c6565b6040516104ef91906156a2565b348015610a8e575f5ffd5b5061053b610a9d3660046156d7565b611965565b348015610aad575f5ffd5b506011546012546013546106c692919083565b348015610acb575f5ffd5b5061053b611971565b348015610adf575f5ffd5b5061053b610aee366004615270565b61197a565b348015610afe575f5ffd5b5061053b610b0d366004615270565b601e6020525f908152604090205481565b348015610b29575f5ffd5b506104de610b383660046152d0565b611989565b348015610b48575f5ffd5b50610568610b57366004615813565b6119db565b348015610b67575f5ffd5b50600954600a546104de919082565b348015610b81575f5ffd5b5061053b7f23ad11f0a1505378b82984192ad0461e6a012820fc5bf2e4ba16513f8e43055281565b348015610bb4575f5ffd5b50610568611b02565b348015610bc8575f5ffd5b50600e546001600160a01b0316610651565b348015610be5575f5ffd5b5061053b610bf4366004615270565b5f9081526019602052604090205490565b348015610c10575f5ffd5b5061053b610c1f3660046152d0565b611b1c565b348015610c2f575f5ffd5b50610c43610c3e366004615270565b611b3d565b6040516104ef91906158b7565b348015610c5b575f5ffd5b50610517610c6a366004615310565b611b47565b348015610c7a575f5ffd5b50600154600254600354600454600554600654600754600854610caf976001600160a01b039081169781169695169392919088565b604080516001600160a01b03998a1681529789166020890152870195909552959092166060850152608084015260a083015260c082019290925260e0810191909152610100016104ef565b348015610d05575f5ffd5b50610568610d143660046158c5565b611b7d565b348015610d24575f5ffd5b50610d8a604080516080810182525f8082526020820181905291810182905260608101919091525060408051608081018252600d546001600160a01b039081168252600e5481166020830152600f54811692820192909252601054909116606082015290565b6040516104ef919081516001600160a01b039081168252602080840151821690830152604080840151821690830152606092830151169181019190915260800190565b348015610dd8575f5ffd5b50610a76610de7366004615270565b611c1b565b348015610df7575f5ffd5b506106c6610e06366004615270565b611c26565b348015610e16575f5ffd5b5061053b5f81565b348015610e29575f5ffd5b50610568610e38366004615270565b611c9a565b348015610e48575f5ffd5b50601454601554610e60916001600160a01b03169082565b604080516001600160a01b0390931683526020830191909152016104ef565b348015610e8a575f5ffd5b50610568610e99366004615270565b611dcb565b348015610ea9575f5ffd5b5061053b610eb83660046158df565b611dd4565b348015610ec8575f5ffd5b5061053b610ed73660046154c2565b611e43565b348015610ee7575f5ffd5b506106c6610ef6366004615270565b611e64565b348015610f06575f5ffd5b506106c6610f15366004615128565b611f02565b348015610f25575f5ffd5b5061053b610f34366004615270565b611f0f565b348015610f44575f5ffd5b50610568610f5336600461591f565b611f1f565b348015610f63575f5ffd5b5061053b610f72366004615270565b611fc4565b348015610f82575f5ffd5b5061053b609681565b348015610f96575f5ffd5b50600d54600e54600f54601054610fbd936001600160a01b03908116938116928116911684565b604080516001600160a01b03958616815293851660208501529184169183019190915290911660608201526080016104ef565b348015610ffb575f5ffd5b5061056861100a366004615310565b611fce565b34801561101a575f5ffd5b5061053b7fc50959b2b0264fed58f3489f13cdf8345df0911245cc2b741070787ee7aceaa281565b6107d861105036600461593a565b611fea565b348015611060575f5ffd5b5061051761106f366004615270565b6121a5565b34801561107f575f5ffd5b5061053b7fe7cbc1eb0e9b3f8688b0bc91a8278f7d2867f14f2a10dc3f0d9fcfdc32dada1281565b3480156110b2575f5ffd5b5061053b6110c13660046156d7565b6121ba565b3480156110d1575f5ffd5b506105686110e03660046152b6565b6121ee565b3480156110f0575f5ffd5b506105176110ff366004615270565b61223d565b34801561110f575f5ffd5b506040805180820182525f808252602091820152815180830183526014546001600160a01b03168082526015549183019182528351908152905191810191909152016104ef565b348015611161575f5ffd5b5061053b611170366004615270565b5f908152601e602052604090205490565b34801561118c575f5ffd5b506107d861119b36600461593a565b612247565b3480156111ab575f5ffd5b5061053b6111ba3660046154c2565b6123fd565b3480156111ca575f5ffd5b5061053b6111d9366004615270565b61252f565b3480156111e9575f5ffd5b506111f261253f565b6040516104ef919081516001600160a01b03908116825260208084015182169083015260408084015190830152606080840151909116908201526080808301519082015260a0828101519082015260c0808301519082015260e091820151918101919091526101000190565b348015611269575f5ffd5b50610517611278366004615270565b6125f8565b348015611288575f5ffd5b5061065161dead81565b5f828152601b60209081526040808320848452909152902080546001909101545b9250929050565b5f6001600160e01b03198216637965db0b60e01b14806112ea57506301ffc9a760e01b6001600160e01b03198316145b92915050565b5f6112f9612602565b905090565b5f6113088161261f565b8151601480546001600160a01b0319166001600160a01b03909216918217905560208084015160158190556040519081527f8e32e306972875584ae78a6586b19f2b97a9dbc1a78a73ea8ff3b207c7da23cf910160405180910390a25050565b61138960405180606001604052805f81526020015f81526020015f81525090565b5060408051606081018252601154815260125460208201526013549181019190915290565b5f828152601b60209081526040808320848452909152808220601454600182015482549351631f04758960e21b8152600481018790526024810191909152604481019390935290916001600160a01b0390911690637c11d62490606401602060405180830381865afa158015611426573d5f5f3e3d5ffd5b505050506040513d601f19601f8201168201806040525081019061144a9190615a18565b949350505050565b5f6112ea82612629565b5f9081525f516020615d755f395f51905f52602052604090206001015490565b5f6114868161261f565b81516009819055602080840151600a81905560408051938452918301527f6c59cf3d8d700a9538f44d5c8acf1889183727b7cf4669f0141f7ab689bdc81091015b60405180910390a15050565b5f5f5f6114e08585612694565b9250925092509250925092565b5f5f6114f885612744565b61151d57604051634762af7d60e01b8152600481018690526024015b60405180910390fd5b61152885858561277d565b91509150935093915050565b5f6115408484846127eb565b90505b9392505050565b6115538261145c565b61155c8161261f565b611566838361284d565b50505050565b5f6115756128f5565b61157d612927565b611587338661295e565b6115a457604051631aededff60e31b815260040160405180910390fd5b6115ae85346129af565b6115bc338686863487612af2565b905061144a60015f516020615db55f395f51905f5255565b5f6112f9612d62565b6001600160a01b03811633146116065760405163334bd91960e11b815260040160405180910390fd5b6116108282612d73565b505050565b606061161f6128f5565b611627612927565b5f6116328484612dec565b90506116458a8a8a8a8a8a8a8a89612e72565b91505061165e60015f516020615db55f395f51905f5255565b98975050505050505050565b5f6116748161261f565b61167c61301c565b61168461304b565b50565b336001600160a01b03831681036116b157604051638163594d60e01b815260040160405180910390fd5b5f8260038111156116c4576116c461588f565b036116fa576001600160a01b038082165f908152601a60209081526040808320938716835292905220805460ff19169055611743565b81600381111561170c5761170c61588f565b6001600160a01b038281165f908152601a60209081526040808320938816835292905220805460ff191660ff929092169190911790555b806001600160a01b0316836001600160a01b03167f82a44452b8f9b854115b84acf31076a4deb9edd2530d246cf0d96c97a6ae619b846040516117869190615a2f565b60405180910390a3505050565b5f61179d84612744565b6117bd57604051634762af7d60e01b815260048101859052602401611514565b6115408484846130aa565b5f6117d284612744565b6117f257604051634762af7d60e01b815260048101859052602401611514565b61154084848461314f565b5f6118078161261f565b81516011819055602080840151601281905560408086015160138190558151948552928401919091528201527f1456f0760ace81355304bceb3062ae05afa5fbb02ec5460188d98fed953e6d41906060016114c7565b5f6112ea826011600101546131b8565b60606118776128f5565b61187f612927565b5f61188a8484612dec565b905061189986868686856131ca565b91505061144a60015f516020615db55f395f51905f5255565b5f6112ea826132fa565b5f6112ea8261332d565b5f8181526016602052604090208054606091906118e290615a5d565b80601f016020809104026020016040519081016040528092919081815260200182805461190e90615a5d565b80156119595780601f1061193057610100808354040283529160200191611959565b820191905f5260205f20905b81548152906001019060200180831161193c57829003601f168201915b50505050509050919050565b5f6115408484846121ba565b5f6112f961338e565b5f6112ea8260115f01546131b8565b5f5f61199485612744565b6119b457604051634762af7d60e01b815260048101869052602401611514565b5f6119be866133fa565b90506119cc8686868461341b565b91989197509095505050505050565b5f6119e461344f565b805490915060ff600160401b82041615906001600160401b03165f81158015611a0a5750825b90505f826001600160401b03166001148015611a255750303b155b905081158015611a33575080155b15611a515760405163f92ee8a960e01b815260040160405180910390fd5b845467ffffffffffffffff191660011785558315611a7b57845460ff60401b1916600160401b1785555b611a83613477565b611a8b61347f565b611a93613477565b611aa18b8b8b8b8b8b61348f565b8a51611aae905f9061284d565b508315611af557845460ff60401b19168555604051600181527fc7f505b2f371ae2175ee4913f4499e1f2633a7b5936321eed1cdaeb6115181d29060200160405180910390a15b5050505050505050505050565b5f611b0c8161261f565b611b146128f5565b611684613545565b5f5f611b298585856127eb565b9050611b348161358d565b95945050505050565b5f6112ea826135c7565b5f9182525f516020615d755f395f51905f52602090815260408084206001600160a01b0393909316845291905290205460ff1690565b5f611b878161261f565b8151600d80546001600160a01b039283166001600160a01b03199182168117909255602080860151600e805491861691841682179055604080880151600f80549188169186168217905560608901516010805491909816951685179096555192835292917fa56701aea90c1cdd1c40fe625d4bc2f0d52b88214278fea3ae2db7b703f3681691015b60405180910390a45050565b60606112ea82613658565b5f81815260176020526040808220815160608101928390528392839283929160039082845b815481526020019060010190808311611c4b5750505050509050805f60038110611c7757611c77615a49565b602002015181600160200201518260026020020151935093509350509193909250565b611ca2612927565b5f611cac82612629565b9050336001600160a01b03821614611cd757604051630bfa39ff60e21b815260040160405180910390fd5b6001600160a01b0381165f908152601d60205260409020548015611db3576001600160a01b0382165f818152601d602090815260408083208390558051638da5cb5b60e01b81529051929392638da5cb5b926004808401939192918290030181865afa158015611d49573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190611d6d9190615a95565b9050611d798183613717565b81816001600160a01b0316857f93a8f3b2bae86deadc28b666f9f65297764642ef13b22d1de09b2125e163bd8460405160405180910390a4505b505061168460015f516020615db55f395f51905f5255565b611684816137a3565b5f611ddd6128f5565b611de5612927565b611def338761382a565b611e0c576040516312eab73160e11b815260040160405180910390fd5b5f5f611e1c33898989898961384d565b91509150611e2a888361393c565b915050611b3460015f516020615db55f395f51905f5255565b60208052815f5260405f208160038110611e5b575f80fd5b01549150829050565b5f81815260176020526040808220815160608101928390528392839283929160039082845b815481526020019060010190808311611e8957505050505090505f5f1b815f60038110611eb857611eb8615a49565b6020020151148015611ecc57506020810151155b8015611eda57506040810151155b15611efb576040516308848f3b60e01b815260048101869052602401611514565b805f611c77565b5f5f5f6114e08585613a75565b5f6112ea82600b600101546131b8565b5f611f298161261f565b611f3282613b09565b81606001516001600160a01b031682602001516001600160a01b0316835f01516001600160a01b03167faf8d85bc3313be057acad92fcdd8829b909273359b6a033f2f4348787b8df3f3856040015186608001518760a001518860c001518960e00151604051611c0f959493929190948552602085019390935260408401919091526060830152608082015260a00190565b5f6112ea8261358d565b611fd78261145c565b611fe08161261f565b6115668383612d73565b6060611ff46128f5565b611ffc612927565b5f6120078686612dec565b9050888015806120175750609681115b15612035576040516392cbb3c960e01b815260040160405180910390fd5b806001600160401b0381111561204d5761204d61516f565b604051908082528060200260200182016040528015612076578160200160208202803683370190505b50925080881415806120885750808614155b806120935750808414155b156120b15760405163479ca36960e01b815260040160405180910390fd5b6120bb338d61295e565b6120d857604051631aededff60e31b815260040160405180910390fd5b5f5b8181101561217557612150338e8e8e858181106120f9576120f9615a49565b905060200201358d8d8681811061211257612112615a49565b905060200201358c8c8781811061212b5761212b615a49565b905060200201358b8b8881811061214457612144615a49565b90506020020135612af2565b84828151811061216257612162615a49565b60209081029190910101526001016120da565b506121808c836129af565b505061219860015f516020615db55f395f51905f5255565b9998505050505050505050565b5f8181526019602052604081205415156112ea565b5f828152601b6020908152604080832084845282528083206001600160a01b03871684526002019091529020549392505050565b5f6121f88161261f565b8151600b819055602080840151600c81905560408051938452918301527f4a883a35415345b4e144c4eb6e7a784a553b46997702b6889f682e4dea64c35f91016114c7565b5f6112ea82612744565b60606122516128f5565b612259612927565b8715806122665750609688115b15612284576040516392cbb3c960e01b815260040160405180910390fd5b876001600160401b0381111561229c5761229c61516f565b6040519080825280602002602001820160405280156122c5578160200160208202803683370190505b50905087861415806122d75750878414155b806122e25750878214155b156123005760405163479ca36960e01b815260040160405180910390fd5b61230a338b61382a565b61232757604051631aededff60e31b815260040160405180910390fd5b5f805b898110156123db575f5f6123a2338f8f8f8781811061234b5761234b615a49565b905060200201358e8e8881811061236457612364615a49565b905060200201358d8d8981811061237d5761237d615a49565b905060200201358c8c8a81811061239657612396615a49565b9050602002013561384d565b90925090506123b18285615ac4565b9350808584815181106123c6576123c6615a49565b6020908102919091010152505060010161232a565b506123e68b8261393c565b5061219860015f516020615db55f395f51905f5255565b5f5f61240761338e565b90508083111561242a5760405163176ddfd960e11b815260040160405180910390fd5b6001600160a01b0384165f908152602080526040808220815160608101928390529160039082845b815481526020019060010190808311612452575050505050905083815f6003811061247f5761247f615a49565b6020020151116124c0576001600160a01b0385165f908152601f602052604081209082815b602002015181526020019081526020015f2054925050506112ea565b602081015184106124eb576001600160a01b0385165f908152601f60205260408120908260016124a4565b60408101518410612516576001600160a01b0385165f908152601f60205260408120908260026124a4565b604051630de728b960e41b815260040160405180910390fd5b5f6112ea826011600201546131b8565b61259a6040518061010001604052805f6001600160a01b031681526020015f6001600160a01b031681526020015f81526020015f6001600160a01b031681526020015f81526020015f81526020015f81526020015f81525090565b5060408051610100810182526001546001600160a01b03908116825260025481166020830152600354928201929092526004549091166060820152600554608082015260065460a082015260075460c082015260085460e082015290565b5f6112ea826133fa565b6006545f90612612906002615ad7565b600b546112f99190615ac4565b6116848133613ba2565b60105460405163218e250d60e01b8152600481018390525f916001600160a01b03169063218e250d90602401602060405180830381865afa158015612670573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906112ea9190615a95565b6015545f9081908190816126a6612602565b9050808610156126c9576040516301f4ca1f60e61b815260040160405180910390fd5b5f6126d48288615aee565b90505f6126e6826011600201546131b8565b90505f6126f28a613bdf565b6126fc575f61270b565b61270b83600b600101546131b8565b90505f816127198486615aee565b6127239190615aee565b90505f6127318c88846130aa565b9950939750955050505050509250925092565b5f818152601660205260408120805482919061275f90615a5d565b905011806112ea5750505f9081526018602052604090205460ff1690565b5f5f5f61278b86868661314f565b90505f61279d826011600201546131b8565b90505f6127ab888888613c5f565b6127b5575f6127c4565b6127c4836011600101546131b8565b90505f816127d28486615aee565b6127dc9190615aee565b99969850959650505050505050565b604080517f23ad11f0a1505378b82984192ad0461e6a012820fc5bf2e4ba16513f8e430552602082015290810184905260608101839052608081018290525f9060a0016040516020818303038152906040528051906020012090509392505050565b5f5f516020615d755f395f51905f526128668484611b47565b6128e5575f848152602082815260408083206001600160a01b03871684529091529020805460ff1916600117905561289b3390565b6001600160a01b0316836001600160a01b0316857f2f8788117e7eff1d82e926ec794901d17c78024a50270940304540a733656f0d60405160405180910390a460019150506112ea565b5f9150506112ea565b5092915050565b5f516020615d955f395f51905f525460ff16156129255760405163d93c066560e01b815260040160405180910390fd5b565b5f516020615db55f395f51905f5280546001190161295857604051633ee5aeb560e01b815260040160405180910390fd5b60029055565b5f816001600160a01b0316836001600160a01b03161480611543575060015b6001600160a01b038084165f908152601a60209081526040808320938816835292905220541660ff1615159392505050565b6129b882613cbf565b5f6129c161338e565b6001600160a01b0384165f908152602080526040902080549192509082146129fe578054156129fa576001810180546002830155815490555b8181555b5f828152601e602052604081208054859290612a1b908490615b01565b90915550505f828152601e60205260408082205490519091859185917f75ba148093b67b99a3a73383328e124a6ff020639d126ec6a0adb55766d3221791a46001600160a01b0384165f908152601f6020908152604080832085845290915281208054859290612a8c908490615b01565b90915550506001600160a01b0384165f818152601f602090815260408083208684528252918290205491519182528592859290917ffbc12b3b70d38b479dc83423a10420167e484f3fe4f16391b77b925194168a3391015b60405180910390a450505050565b5f612afc83613dfb565b5f612b06866135c7565b5f878152601b602090815260408083208984529091528120600101546015549293501591871490836002811115612b3f57612b3f61588f565b14612ba457612b4f88888b613e1e565b15612b6d5760405163332c26cd60e01b815260040160405180910390fd5b818015612b8657505f8881526019602052604090205415155b15612ba457604051634994110960e11b815260040160405180910390fd5b818015612bae5750805b15612bcc5760405163dc2025b560e01b815260040160405180910390fd5b5f8080612bee8b8b8b848a6002811115612be857612be861588f565b1461341b565b925092509250612c038b8b8b8686868e613e8f565b612c0c8261409c565b612c158b614112565b15612c3257612c328b612c2c8460115f01546131b8565b88614151565b5f866002811115612c4557612c4561588f565b03612c5a57612c548b83614193565b50612c80565b612c638b613bdf565b15612c8057612c808b612c7b84600b600101546131b8565b614220565b5f858015612c8c575084155b15612cd657612c9f8d8d8d85888c614278565b90505f876002811115612cb457612cb461588f565b14612cd1575f612cc38d61358d565b9050612ccf818d614308565b505b612ce7565b612ce48d8d8d85888c614363565b90505b8b8d6001600160a01b03168f6001600160a01b03167ff717428d3271f4ca44ee92a96ecda708463de1c200f30db4e295def5e8db6b4a8e8e878a888f604051612d3596959493929190615b28565b60405180910390a450919c9b505050505050505050505050565b60015f516020615db55f395f51905f5255565b6006546009545f916112f991615ac4565b5f5f516020615d755f395f51905f52612d8c8484611b47565b156128e5575f848152602082815260408083206001600160a01b0387168085529252808320805460ff1916905551339287917ff6391f5c32d9c69d2a47ea670b442974b53935d1edc7fd64eb21e047a839171b9190a460019150506112ea565b5f81801580612dfb5750609681115b15612e19576040516392cbb3c960e01b815260040160405180910390fd5b5f5b81811015612e5157848482818110612e3557612e35615a49565b9050602002013583612e479190615ac4565b9250600101612e1b565b508134146128ee57604051637b0a37cf60e01b815260040160405180910390fd5b6060885f84612e7f612602565b612e899190615ad7565b9050815f03612eab576040516392cbb3c960e01b815260040160405180910390fd5b8882141580612eba5750868214155b80612ec55750848214155b15612ee35760405163479ca36960e01b815260040160405180910390fd5b80841015612f0457604051637b0a37cf60e01b815260040160405180910390fd5b5f826001600160401b03811115612f1d57612f1d61516f565b604051908082528060200260200182016040528015612f46578160200160208202803683370190505b5090505f5b83811015612fe557612fc0338f8f84818110612f6957612f69615a49565b905060200201358e8e85818110612f8257612f82615a49565b905060200201358d8d86818110612f9b57612f9b615a49565b905060200201358c8c87818110612fb457612fb4615a49565b905060200201356143d3565b828281518110612fd257612fd2615a49565b6020908102919091010152600101612f4b565b50600b545f90612ff6908590615ad7565b905061300181614560565b61300b33876129af565b509c9b505050505050505050505050565b5f516020615d955f395f51905f525460ff1661292557604051638dfc202b60e01b815260040160405180910390fd5b61305361301c565b5f516020615d955f395f51905f52805460ff191681557f5db9ee0a495bf2e6ff9c91a7834c1ba4fdd244a5e8aa4e537bd38aeae4b073aa335b6040516001600160a01b03909116815260200160405180910390a150565b6014545f848152601b6020908152604080832086845290915280822080546001909101549151636199b7ff60e01b815260048101869052602481019190915260448101919091526064810185905290916001600160a01b0316908190636199b7ff906084015b602060405180830381865afa15801561312b573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190611b349190615a18565b6014545f848152601b60209081526040808320868452909152808220600181015490549151633e3cfc1760e11b815260048101869052602481019190915260448101919091526064810185905290916001600160a01b0316908190637c79f82e90608401613110565b6003545f9061154390849084906145c4565b6060845f8190036131ee57604051637a291d9160e01b815260040160405180910390fd5b80841461320e5760405163479ca36960e01b815260040160405180910390fd5b5f816001600160401b038111156132275761322761516f565b604051908082528060200260200182016040528015613250578160200160208202803683370190505b5090505f5b828110156132c8576132a3338a8a8481811061327357613273615a49565b90506020028101906132859190615b57565b8a8a8681811061329757613297615a49565b905060200201356145f1565b8282815181106132b5576132b5615a49565b6020908102919091010152600101613255565b506009545f906132d9908490615ad7565b90506132e481614560565b6132ee33866129af565b50979650505050505050565b5f818152601960205260408120541561331f57505f9081526019602052604090205490565b6112ea8261358d565b919050565b5f7fc50959b2b0264fed58f3489f13cdf8345df0911245cc2b741070787ee7aceaa28280519060200120604051602001613371929190918252602082015260400190565b604051602081830303815290604052805190602001209050919050565b6004805460408051630ecce30160e31b815290515f936001600160a01b03909316926376671808928082019260209290918290030181865afa1580156133d6573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906112f99190615a18565b5f818152601660205260408120805461341290615a5d565b15159392505050565b5f5f5f831561343a5761342f8787876147c6565b925092509250613445565b61342f87878761497a565b9450945094915050565b5f807ff0c57e16840df040f15088dc2f81fe391c3923bec73e23a9662efc9c229c6a006112ea565b612925614a8f565b613487614a8f565b612925614ab4565b613497614a8f565b6134a086613b09565b8451600955602094850151600a558351600b5592840151600c558151600d80546001600160a01b03199081166001600160a01b039384161790915583860151600e80548316918416919091179055604080850151600f8054841691851691909117905560609094015160108054831691841691909117905582516011558286015160125591909201516013558251601480549092169216919091179055015160155550565b61354d6128f5565b5f516020615d955f395f51905f52805460ff191660011781557f62e78cea01bee320cd4e420270b5ea74000d11b0c9f74754ebdbfc544b05a2583361308c565b604080517fe7cbc1eb0e9b3f8688b0bc91a8278f7d2867f14f2a10dc3f0d9fcfdc32dada1260208201529081018290525f90606001613371565b5f5f6135d2836133fa565b5f8481526018602090815260408083205460199092529091205491925060ff1690151582158015613601575081155b801561360b575080155b1561362c5760405163bdd4a69960e01b815260048101869052602401611514565b821561363c57505f949350505050565b801561364d57506002949350505050565b506001949350505050565b5f8181526016602052604081208054606092919061367590615a5d565b80601f01602080910402602001604051908101604052809291908181526020018280546136a190615a5d565b80156136ec5780601f106136c3576101008083540402835291602001916136ec565b820191905f5260205f20905b8154815290600101906020018083116136cf57829003601f168201915b5050505050905080515f036112ea5760405163b615632f60e01b815260048101849052602401611514565b804710156137415760405163cf47918160e01b815247600482015260248101829052604401611514565b5f5f836001600160a01b0316836040515f6040518083038185875af1925050503d805f811461378b576040519150601f19603f3d011682016040523d82523d5f602084013e613790565b606091505b5091509150816115665761156681614abc565b5f818152601c6020526040812054908190036137bd575050565b5f828152601c60205260408120556002546137e1906001600160a01b031682613717565b6002546040518281526001600160a01b039091169083907f0e19f21371647f79bb7c3f1e363266315b211fcca3836e7a71425cc0c4ab6a8e906020015b60405180910390a35050565b5f816001600160a01b0316836001600160a01b031614806115435750600261297d565b5f5f5f613859876135c7565b905061386887878a8888614ae4565b5f61387488888861314f565b90505f61388289898961277d565b50905061388e8261409c565b613899898989613c5f565b156138b7576138b7896138b1846011600101546131b8565b85614151565b5f6138c68b8b8b868c89614bb8565b90506138d28b83613717565b896001600160a01b03808d16908e167f5ab85658e5658520d47644e1417584159fc8edb9712149792a9337ff5e1048a48c8c8688613910818c615aee565b8c60405161392396959493929190615b28565b60405180910390a450909a909950975050505050505050565b61394582613cbf565b5f61394e61338e565b6001600160a01b0384165f9081526020805260409020805491925090821461398b57805415613987576001810180546002830155815490555b8181555b5f828152601e6020526040812080548592906139a8908490615b99565b90915550505f828152601e60205260408082205490519091859185917f15347e028ad43baa634bf63b5062f97a54a8b497ec496dd31c9b2da27fa3640491a46001600160a01b0384165f908152601f6020908152604080832085845290915281208054859290613a19908490615b99565b90915550506001600160a01b0384165f818152601f602090815260408083208684528252918290205491519182528592859290917f62c25a075382083421af11238f1e987f397e822e4d32d143ce60ba2ff98910f69101612ae4565b6015545f908190819081613a87612d62565b905080861015613aaa576040516301f4ca1f60e61b815260040160405180910390fd5b613ab48187615aee565b93505f613ac6856011600201546131b8565b90505f613ad8866009600101546131b8565b905080613ae58388615aee565b613aef9190615aee565b9450613afc8985876130aa565b9650505050509250925092565b80516001600160a01b0316613b315760405163c9f9ba1560e01b815260040160405180910390fd5b8051600180546001600160a01b039283166001600160a01b0319918216179091556020830151600280549184169183169190911790556040830151600355606083015160048054919093169116179055608081015160055560a081015160065560c081015160075560e00151600855565b613bac8282611b47565b613bdb5760405163e2517d3f60e01b81526001600160a01b038216600482015260248101839052604401611514565b5050565b5f8181526017602052604080822081516060810192839052839290919060039082845b815481526020019060010190808311613c025750505050509050613c3b815f60038110613c3157613c31615a49565b6020020151614112565b8015613c4d5750613c4d816001613c31565b80156115435750611543816002613c31565b6015545f848152601b6020908152604080832084845290915281206001015490919082828603613c9a57613c938583615aee565b9050613c9d565b50805b600854811015613cb2575f9350505050611543565b5060019695505050505050565b5f613cc861338e565b6001600160a01b0383165f9081526020805260409020549091508115801590613cff57505f8281526021602052604090205460ff16155b15613d6d575f828152602160205260408120805460ff19166001908117909155613d299084615aee565b5f818152601e60205260409020549091508015801590613d5457505f848152601e6020526040902054155b15613d6a575f848152601e602052604090208190555b50505b818103613d7957505050565b6001600160a01b0383165f908152601f602090815260408083208484529091529020548015801590613dcb57506001600160a01b0384165f908152601f60209081526040808320868452909152902054155b15611566576001600160a01b0384165f908152601f60209081526040808320868452909152902081905550505050565b6005548110156116845760405163e219a6cd60e01b815260040160405180910390fd5b5f8381526018602052604081205460ff16613e4c57604051635f916b8760e01b815260040160405180910390fd5b5f613e56856132fa565b5f908152601b6020908152604080832087845282528083206001600160a01b038716845260020190915290205415159150509392505050565b6014546001600160a01b03165f859003613ebc57604051630f1babf160e31b815260040160405180910390fd5b5f888152601b602090815260408083208a84529091528120600101541590613ee48689615aee565b5f8b8152601b602090815260408083208d8452909152812054919250908290613f0e908890615ac4565b613f189190615ac4565b60405163a0760cad60e01b8152600481018c90529091506001600160a01b0385169063a0760cad90602401602060405180830381865afa158015613f5e573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190613f829190615a18565b811115613fa257604051635ebfe68f60e11b815260040160405180910390fd5b5f83613fae575f613fb2565b6006545b5f8d8152601b602090815260408083208f8452909152902060010154613fd9908b90615ac4565b613fe39190615ac4565b6040516323693bcd60e11b8152600481018d90529091506001600160a01b038616906346d2779a90602401602060405180830381865afa158015614029573d5f5f3e3d5ffd5b505050506040513d601f19601f8201168201806040525081019061404d9190615a18565b81111561406d57604051634a308f9960e01b815260040160405180910390fd5b8589101561408e57604051632050746960e11b815260040160405180910390fd5b505050505050505050505050565b5f6140ac826011600201546131b8565b90505f6140b761338e565b905081601c5f8381526020019081526020015f205f8282546140d99190615ac4565b9091555050604051828152339082907fc7a5742f069dec143091f9dc7c9ca18f1212ac3f76cd16f5f8aa6f5cab9b2afc90602001611786565b6015545f828152601b6020908152604080832084845290915281206001015460085491929181101561414757505f9392505050565b5060019392505050565b6015545f848152601b602090815260408083208484529091529020805461418c9086908490614181908890615ac4565b846001015487614c07565b5050505050565b5f5f61419e84612629565b90505f6141b0846009600101546131b8565b6001600160a01b0383165f908152601d60205260408120805492935083929091906141dc908490615ac4565b9091555050604051818152339086907f2ef7be1e4972d779c5501d1e4658106fc78e92a99800d2f69140c78b84068e199060200160405180910390a3509392505050565b5f5f5f61422c85611e64565b919450925090505f61423f600386615bb8565b9050614254848261424f876135c7565b614151565b614262838261424f866135c7565b614270828261424f856135c7565b505050505050565b6006545f868152601b6020908152604080832088845290915281209091906142dd88886142a58186614e0d565b84546142b2908b90615ac4565b6142bc9190615ac4565b858986600101546142cd9190615ac4565b6142d79190615ac4565b88614c07565b5f6142ea8a8a8a89614e8d565b90506142fa61dead8a8a86614e8d565b509998505050505050505050565b5f828152601b60209081526040808320848452909152902060065461435584846143328185614e0d565b855461433e9190615ac4565b84866001015461434e9190615ac4565b6002614c07565b61418c61dead858584614e8d565b5f858152601b602090815260408083208784529091528120546143bc908790879061438f908890615ac4565b5f8a8152601b602090815260408083208c84529091529020600101546143b6908890615ac4565b86614c07565b6143c887878786614e8d565b979650505050505050565b5f6143df8585856127eb565b90506143ed81868686614f06565b6143f685614f4a565b6143ff84614f4a565b61440883614f4a565b60408051606081018252868152602081018690529081018490525f61442c8361358d565b9050614439838284614f73565b6015545f80806144498789612694565b9250925092506144588261409c565b5f6144688d898785886001614278565b905061447388613bdf565b1561448b5761448b88612c7b85600b600101546131b8565b6144958686614308565b604080518d8152602081018d90529081018b905288906001600160a01b038f16907ff13505b910c49b286cf7fbaf1a78620cf7d8bda0e8e39c1baed620c08af686e79060600160405180910390a3878d6001600160a01b03168e6001600160a01b03167ff717428d3271f4ca44ee92a96ecda708463de1c200f30db4e295def5e8db6b4a888d878a88600160405161453296959493929190615b28565b60405180910390a460025f5f82825461454b9190615ac4565b90915550505050505050505095945050505050565b5f61456961338e565b905081601c5f8381526020019081526020015f205f82825461458b9190615ac4565b9091555050604051828152339082907fc7a5742f069dec143091f9dc7c9ca18f1212ac3f76cd16f5f8aa6f5cab9b2afc9060200161381e565b828202831584820484141782026145e25763ad251c275f526004601cfd5b81810615159190040192915050565b5f8280820361461357604051637a291d9160e01b815260040160405180910390fd5b6007548111156146365760405163977c5b1160e01b815260040160405180910390fd5b61467485858080601f0160208091040260200160405190810160405280939291908181526020018383808284375f9201919091525061332d92505050565b5f81815260166020526040902080549193509061469090615a5d565b1590506146b4578484604051632d215baf60e21b8152600401611514929190615bff565b5f8281526016602052604090206146cc858783615c56565b506015545f80806146dd8688613a75565b9250925092506146ec8261409c565b5f6146f78784614193565b90505f6147088c898886895f614278565b9050878c6001600160a01b03167ffd579ad7468b1720e08f84efe16900074f3ffdaaeaa82e675e5aec69bf3931898d8d8660405161474893929190615d0f565b60405180910390a3878c6001600160a01b03168d6001600160a01b03167ff717428d3271f4ca44ee92a96ecda708463de1c200f30db4e295def5e8db6b4a898d888b885f60405161479e96959493929190615b28565b60405180910390a45f5f81546147b390615d3a565b9091555050505050505050949350505050565b5f808080846147f088885f918252601b602090815260408084209284529190529020600101541590565b15614830575f6148005f89614fe1565b905080871161482257604051638f7bebd560e01b815260040160405180910390fd5b61482c8183615aee565b9150505b5f614840826011600201546131b8565b90505f61484c8a614112565b614856575f614864565b6148648360115f01546131b8565b90505f614876846009600101546131b8565b905080826148848587615aee565b61488e9190615aee565b6148989190615aee565b5f8c8152601b602090815260408083208e845290915281206001015491965090156148cd576148c88c8c886130aa565b614967565b6014546006546001600160a01b0390911690636199b7ff9088906148f2908f90614e0d565b6006546040516001600160e01b031960e086901b168152600481019390935260248301919091526044820152606481018e9052608401602060405180830381865afa158015614943573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906149679190615a18565b9c949b5094995092975050505050505050565b5f808080846149a488885f918252601b602090815260408084209284529190529020600101541590565b80156149bc57505f8881526019602052604090205415155b156149da57604051634994110960e11b815260040160405180910390fd5b5f888152601b602090815260408083208a8452909152902060010154614a36575f614a06600189614fe1565b9050808711614a2857604051638f7bebd560e01b815260040160405180910390fd5b614a328183615aee565b9150505b5f614a46826011600201546131b8565b90505f614a528a614112565b614a5c575f614a6a565b614a6a8360115f01546131b8565b90505f614a768b613bdf565b614a80575f614876565b61487684600b600101546131b8565b614a9761501c565b61292557604051631afcd79f60e31b815260040160405180910390fd5b612d4f614a8f565b805115614acb57805160208201fd5b60405163d6bda27560e01b815260040160405180910390fd5b815f03614b0457604051630f1babf160e31b815260040160405180910390fd5b81614b108487876121ba565b1015614b2f5760405163e5e7340760e01b815260040160405180910390fd5b5f858152601b60209081526040808320878452909152812060010154614b56908490615aee565b600654909150811015614b7f5760405163f89b912f60e01b815260048101829052602401611514565b5f614b8b87878661277d565b50905082811015614baf57604051632050746960e11b815260040160405180910390fd5b50505050505050565b5f858152601b6020908152604080832087845290915281208054614bfb9088908890614be5908990615aee565b878560010154614bf59190615aee565b87614c07565b61165e88888887615035565b60145460405163a0760cad60e01b8152600481018690526001600160a01b03909116905f90829063a0760cad90602401602060405180830381865afa158015614c52573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190614c769190615a18565b6040516323693bcd60e11b8152600481018890529091505f906001600160a01b038416906346d2779a90602401602060405180830381865afa158015614cbe573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190614ce29190615a18565b905081861115614d0557604051635ebfe68f60e11b815260040160405180910390fd5b80851115614d2657604051634a308f9960e01b815260040160405180910390fd5b5f888152601b602090815260408083208a8452909152808220888155600181018890559051631f04758960e21b8152600481018a905260248101889052604481018990529091906001600160a01b03861690637c11d62490606401602060405180830381865afa158015614d9c573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190614dc09190615a18565b9050888a7fcc9bc63a4b047be4850fcbada8d7ed7aa097ed4f6c3b55bebc7e108ea1d2f824838b8b8b604051614df99493929190615d52565b60405180910390a350505050505050505050565b60145460405163cdd9e57360e01b8152600481018390525f602482018190526044820181905260648201859052916001600160a01b03169063cdd9e57390608401602060405180830381865afa158015614e69573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906115439190615a18565b5f838152601b6020908152604080832085845282528083206001600160a01b0388168452600201909152812080548391908390614ecb908490615ac4565b9091555050505f838152601b6020908152604080832085845282528083206001600160a01b0388168452600201909152902054949350505050565b5f848152601760205260409020541561156657604051632231995960e01b815260048101859052602481018490526044810183905260648101829052608401611514565b614f5381612744565b61168457604051634762af7d60e01b815260048101829052602401611514565b5f838152601760205260409020614f8c908260036150d6565b505f8381526018602090815260408083208054600160ff1991821681179092558685528285208054909116909117905560179091529020614fcf908260036150d6565b50505f90815260196020526040902055565b5f5f614ff283600160050154614e0d565b90505f8460028111156150075761500761588f565b1461154357615017816002615ad7565b61144a565b5f61502561344f565b54600160401b900460ff16919050565b5f6001600160a01b03851661505d57604051639d3907e360e01b815260040160405180910390fd5b5f848152601b6020908152604080832086845282528083206001600160a01b03891684526002019182905290912054838110156150ad57604051634e9cc70f60e11b815260040160405180910390fd5b6001600160a01b03969096165f9081526020919091526040902091909403908190559392505050565b8260038101928215615104579160200282015b828111156151045782518255916020019190600101906150e9565b50615110929150615114565b5090565b5b80821115615110575f8155600101615115565b5f5f60408385031215615139575f5ffd5b50508035926020909101359150565b5f60208284031215615158575f5ffd5b81356001600160e01b031981168114611543575f5ffd5b634e487b7160e01b5f52604160045260245ffd5b604080519081016001600160401b03811182821017156151a5576151a561516f565b60405290565b60405161010081016001600160401b03811182821017156151a5576151a561516f565b604051601f8201601f191681016001600160401b03811182821017156151f6576151f661516f565b604052919050565b6001600160a01b0381168114611684575f5ffd5b8035613328816151fe565b5f6040828403121561522d575f5ffd5b615235615183565b90508135615242816151fe565b815260209182013591810191909152919050565b5f60408284031215615266575f5ffd5b611543838361521d565b5f60208284031215615280575f5ffd5b5035919050565b5f60408284031215615297575f5ffd5b61529f615183565b823581526020928301359281019290925250919050565b5f604082840312156152c6575f5ffd5b6115438383615287565b5f5f5f606084860312156152e2575f5ffd5b505081359360208301359350604090920135919050565b8151815260208083015190820152604081016112ea565b5f5f60408385031215615321575f5ffd5b823591506020830135615333816151fe565b809150509250929050565b5f5f5f5f60808587031215615351575f5ffd5b843561535c816151fe565b966020860135965060408601359560600135945092505050565b5f5f83601f840112615386575f5ffd5b5081356001600160401b0381111561539c575f5ffd5b6020830191508360208260051b85010111156112b3575f5ffd5b5f5f5f5f5f5f5f5f6080898b0312156153cd575f5ffd5b88356001600160401b038111156153e2575f5ffd5b6153ee8b828c01615376565b90995097505060208901356001600160401b0381111561540c575f5ffd5b6154188b828c01615376565b90975095505060408901356001600160401b03811115615436575f5ffd5b6154428b828c01615376565b90955093505060608901356001600160401b03811115615460575f5ffd5b61546c8b828c01615376565b999c989b5096995094979396929594505050565b602080825282518282018190525f918401906040840190835b818110156154b7578351835260209384019390920191600101615499565b509095945050505050565b5f5f604083850312156154d3575f5ffd5b82356154de816151fe565b946020939093013593505050565b5f5f604083850312156154fd575f5ffd5b8235615508816151fe565b9150602083013560048110615333575f5ffd5b5f6020828403121561552b575f5ffd5b8135611543816151fe565b5f60608284031215615546575f5ffd5b604051606081016001600160401b03811182821017156155685761556861516f565b60409081528335825260208085013590830152928301359281019290925250919050565b5f6060828403121561559c575f5ffd5b6115438383615536565b5f5f5f5f604085870312156155b9575f5ffd5b84356001600160401b038111156155ce575f5ffd5b6155da87828801615376565b90955093505060208501356001600160401b038111156155f8575f5ffd5b61560487828801615376565b95989497509550505050565b5f60208284031215615620575f5ffd5b81356001600160401b03811115615635575f5ffd5b8201601f81018413615645575f5ffd5b80356001600160401b0381111561565e5761565e61516f565b615671601f8201601f19166020016151ce565b818152856020838501011115615685575f5ffd5b816020840160208301375f91810160200191909152949350505050565b602081525f82518060208401528060208501604085015e5f604082850101526040601f19601f83011684010191505092915050565b5f5f5f606084860312156156e9575f5ffd5b83356156f4816151fe565b95602085013595506040909401359392505050565b5f610100828403121561571a575f5ffd5b6157226151ab565b9050813561572f816151fe565b815261573d60208301615212565b60208201526040828101359082015261575860608301615212565b60608201526080828101359082015260a0808301359082015260c0808301359082015260e09182013591810191909152919050565b5f6080828403121561579d575f5ffd5b604051608081016001600160401b03811182821017156157bf576157bf61516f565b60405290508082356157d0816151fe565b815260208301356157e0816151fe565b602082015260408301356157f3816151fe565b60408201526060830135615806816151fe565b6060919091015292915050565b5f5f5f5f5f5f6102a08789031215615829575f5ffd5b6158338888615709565b9550615843886101008901615287565b9450615853886101408901615287565b935061586388610180890161578d565b9250615873886102008901615536565b915061588388610260890161521d565b90509295509295509295565b634e487b7160e01b5f52602160045260245ffd5b600381106158b3576158b361588f565b9052565b602081016112ea82846158a3565b5f608082840312156158d5575f5ffd5b611543838361578d565b5f5f5f5f5f60a086880312156158f3575f5ffd5b85356158fe816151fe565b97602087013597506040870135966060810135965060800135945092505050565b5f6101008284031215615930575f5ffd5b6115438383615709565b5f5f5f5f5f5f5f5f5f60a08a8c031215615952575f5ffd5b893561595d816151fe565b985060208a01356001600160401b03811115615977575f5ffd5b6159838c828d01615376565b90995097505060408a01356001600160401b038111156159a1575f5ffd5b6159ad8c828d01615376565b90975095505060608a01356001600160401b038111156159cb575f5ffd5b6159d78c828d01615376565b90955093505060808a01356001600160401b038111156159f5575f5ffd5b615a018c828d01615376565b915080935050809150509295985092959850929598565b5f60208284031215615a28575f5ffd5b5051919050565b6020810160048310615a4357615a4361588f565b91905290565b634e487b7160e01b5f52603260045260245ffd5b600181811c90821680615a7157607f821691505b602082108103615a8f57634e487b7160e01b5f52602260045260245ffd5b50919050565b5f60208284031215615aa5575f5ffd5b8151611543816151fe565b634e487b7160e01b5f52601160045260245ffd5b808201808211156112ea576112ea615ab0565b80820281158282048414176112ea576112ea615ab0565b818103818111156112ea576112ea615ab0565b8082018281125f831280158216821582161715615b2057615b20615ab0565b505092915050565b5f60c0820190508782528660208301528560408301528460608301528360808301526143c860a08301846158a3565b5f5f8335601e19843603018112615b6c575f5ffd5b8301803591506001600160401b03821115615b85575f5ffd5b6020019150368190038213156112b3575f5ffd5b8181035f8312801583831316838312821617156128ee576128ee615ab0565b5f82615bd257634e487b7160e01b5f52601260045260245ffd5b500490565b81835281816020850137505f828201602090810191909152601f909101601f19169091010190565b602081525f611540602083018486615bd7565b601f82111561161057805f5260205f20601f840160051c81016020851015615c375750805b601f840160051c820191505b8181101561418c575f8155600101615c43565b6001600160401b03831115615c6d57615c6d61516f565b615c8183615c7b8354615a5d565b83615c12565b5f601f841160018114615cb2575f8515615c9b5750838201355b5f19600387901b1c1916600186901b17835561418c565b5f83815260208120601f198716915b82811015615ce15786850135825560209485019460019092019101615cc1565b5086821015615cfd575f1960f88860031b161c19848701351681555b505060018560011b0183555050505050565b604081525f615d22604083018587615bd7565b905060018060a01b0383166020830152949350505050565b5f60018201615d4b57615d4b615ab0565b5060010190565b848152602081018490526040810183905260808101611b3460608301846158a356fe02dd7bc7dec4dceedda775e58dd541e08a116c6c53815c0bd028192f7b626800cd5ed15c6e187e77e9aee88184c21f4f2182ab5827cb3b7e07fbedcd63f033009b779b17422d0df92223018b32b4d1fa46e071723d6817e2486d003becc55f00a164736f6c634300081d000a" + "bytecode": "0x6080604052348015600e575f5ffd5b5060156019565b60c9565b7ff0c57e16840df040f15088dc2f81fe391c3923bec73e23a9662efc9c229c6a00805468010000000000000000900460ff161560685760405163f92ee8a960e01b815260040160405180910390fd5b80546001600160401b039081161460c65780546001600160401b0319166001600160401b0390811782556040519081527fc7f505b2f371ae2175ee4913f4499e1f2633a7b5936321eed1cdaeb6115181d29060200160405180910390a15b50565b614f9b806100d65f395ff3fe608060405260043610610617575f3560e01c8063844cc1571161031b578063c69f03bc116101ae578063e9576f23116100fd578063f679bf091161009d578063f9d320da11610078578063f9d320da146115b6578063fa321611146115d5578063fc4a75f814611655578063fccc281314611674575f5ffd5b8063f679bf0914611565578063f7e7d1fd14611578578063f87d29ac14611597575f5ffd5b8063f22df312116100d8578063f22df312146114aa578063f5719008146114c9578063f5da42f3146114e8578063f5e6bfb91461153a575f5ffd5b8063e9576f231461140a578063ecedd54114611429578063ee3abe381461145c575f5ffd5b8063d38119ef11610168578063dea9423a11610143578063dea9423a1461137d578063df62fa8814611390578063e63ab1e9146113a3578063e731a481146113d6575f5ffd5b8063d38119ef146112fa578063d547741f1461132b578063d91c360b1461134a575f5ffd5b8063c69f03bc14611205578063c8b1792814611224578063c9cedcd014611243578063cfdbf25414611262578063d33219b414611276578063d34ddc0514611295575f5ffd5b8063a18dc3191161026a578063ab19bcd411610224578063bb0b5ebb116101ff578063bb0b5ebb14611189578063bdacb303146111a8578063c12f7947146111c7578063c5a90945146111e6575f5ffd5b8063ab19bcd414611142578063b15d757814611161578063b7a9f5b214611174575f5ffd5b8063a18dc3191461107d578063a217fddf1461109c578063a3a9f885146110af578063a3bb4f6e146110ce578063a3d81fdc14611110578063a814c1fe1461112f575f5ffd5b806391d14854116102d5578063983dda1e116102b0578063983dda1e14610f6c5780639c93ef9614610f8b5780639e4cb3111461103f578063a0ae55c71461105e575f5ffd5b806391d1485414610e8f578063927a97a114610eae578063970a5fc514610f39575f5ffd5b8063844cc15714610db55780638456cb5914610de857806384e100db14610dfc57806385a095c414610e195780638938639f14610e445780638bb444bc14610e63575f5ffd5b806344fcc1f1116104ad57806362b3bd4a116103fc578063768bc3e21161039c5780637d4eaa91116103775780637d4eaa9114610d3e5780637e66abfd14610d5d5780637e6c54db14610d7c5780637f833f0414610d9b575f5ffd5b8063768bc3e214610ce15780637762b6a614610d005780637a9d980f14610d2b575f5ffd5b80636961cd1c116103d75780636961cd1c14610c645780636ced5d3f14610c9057806372188e3f14610caf5780637667180814610ccd575f5ffd5b806362b3bd4a14610c03578063678eae2314610c175780636828caa614610c45575f5ffd5b80635b6dd6bc116104675780635ecb4245116104425780635ecb424514610b875780636001ab6814610bb25780636140330914610bd157806362a80f3d14610be4575f5ffd5b80635b6dd6bc14610b125780635b7946f114610b455780635c975abb14610b64575f5ffd5b806344fcc1f1146109fd5780634523e2bc14610a3357806345ab471614610a525780634656c5f114610a8a57806347bf425214610aa957806358717e0d14610af3575f5ffd5b80632747a57e1161056957806333332d39116105235780633c6bbf45116104fe5780633c6bbf45146109755780633e27173c146109955780633f4ba83a146109d65780634342e966146109ea575f5ffd5b806333332d391461092257806336568abe146109365780633696ac7214610955575f5ffd5b80632747a57e146108375780632db27075146108715780632e1aa0f2146108905780632eddd677146108af5780632f2ff15d146108f05780632fb1d2701461090f575f5ffd5b80631a2385de116105d45780631fdc812e116105af5780631fdc812e146107ac578063218e250d146107da578063248a9ca3146107f957806324e7357214610818575f5ffd5b80631a2385de146107485780631e19e2c8146107675780631f5575fb14610781575f5ffd5b806301a217601461061b57806301ffc9a7146106545780630d65c91c146106835780630f924db7146106a5578063139d4fa5146106f157806313ee9df414610712575b5f5ffd5b348015610626575f5ffd5b5061063a610635366004613e47565b611689565b604080519283526020830191909152015b60405180910390f35b34801561065f575f5ffd5b5061067361066e366004613e67565b6116b1565b604051901515815260200161064b565b34801561068e575f5ffd5b506106976116e7565b60405190815260200161064b565b3480156106b0575f5ffd5b506106d96106bf366004613e8e565b5f908152602360205260409020546001600160a01b031690565b6040516001600160a01b03909116815260200161064b565b3480156106fc575f5ffd5b5061071061070b366004613f8c565b6116f5565b005b34801561071d575f5ffd5b5061072661175c565b604080518251815260208084015190820152918101519082015260600161064b565b348015610753575f5ffd5b50610697610762366004613e47565b6117a2565b348015610772575f5ffd5b50600b54600c5461063a919082565b34801561078c575f5ffd5b5061069761079b366004613e8e565b601c6020525f908152604090205481565b3480156107b7575f5ffd5b506106736107c6366004613e8e565b5f9081526018602052604090205460ff1690565b3480156107e5575f5ffd5b506106d96107f4366004613e8e565b611825565b348015610804575f5ffd5b50610697610813366004613e8e565b61182f565b348015610823575f5ffd5b50610710610832366004613fd5565b61184f565b348015610842575f5ffd5b50610856610851366004613e47565b6118a3565b6040805193845260208401929092529082015260600161064b565b34801561087c575f5ffd5b5061063a61088b366004613fef565b61192f565b34801561089b575f5ffd5b506106976108aa366004613fef565b611a55565b3480156108ba575f5ffd5b506040805180820182525f80825260209182015281518083019092526009548252600a54908201525b60405161064b9190614018565b3480156108fb575f5ffd5b5061071061090a36600461402f565b611a69565b61069761091d36600461405d565b611a8b565b34801561092d575f5ffd5b50610697611b5a565b348015610941575f5ffd5b5061071061095036600461402f565b611b63565b6109686109633660046140d5565b611b9b565b60405161064b919061416d565b6109886109833660046141d0565b611d87565b60405161064b919061429a565b3480156109a0575f5ffd5b506106976109af3660046142dc565b6001600160a01b03919091165f908152601f60209081526040808320938352929052205490565b3480156109e1575f5ffd5b50610710611e4c565b6107106109f8366004614306565b611e69565b348015610a08575f5ffd5b50610697610a173660046142dc565b601f60209081525f928352604080842090915290825290205481565b348015610a3e575f5ffd5b50610697610a4d366004613fef565b611f9b565b348015610a5d575f5ffd5b506040805180820182525f8082526020918201528151808301909252600b548252600c54908201526108e3565b348015610a95575f5ffd5b50610673610aa4366004614335565b6120b0565b348015610ab4575f5ffd5b50610adc610ac3366004613e8e565b60246020525f908152604090205465ffffffffffff1681565b60405165ffffffffffff909116815260200161064b565b348015610afe575f5ffd5b50610697610b0d366004613fef565b612135565b348015610b1d575f5ffd5b50610697610b2c366004614361565b6001600160a01b03165f90815260208052604090205490565b348015610b50575f5ffd5b50610710610b5f3660046143d2565b6121d4565b348015610b6f575f5ffd5b505f516020614f4f5f395f51905f525460ff16610673565b348015610b92575f5ffd5b50610697610ba1366004614361565b601d6020525f908152604090205481565b348015610bbd575f5ffd5b50610697610bcc366004613e8e565b612232565b610988610bdf3660046140d5565b612242565b348015610bef575f5ffd5b50610697610bfe366004613e8e565b6122db565b348015610c0e575f5ffd5b506106975f5481565b348015610c22575f5ffd5b50610673610c31366004613e8e565b60216020525f908152604090205460ff1681565b348015610c50575f5ffd5b50610697610c5f3660046143ec565b6122e5565b348015610c6f575f5ffd5b50610c83610c7e366004613e8e565b6122ef565b60405161064b919061447e565b348015610c9b575f5ffd5b50610697610caa366004614490565b61238e565b348015610cba575f5ffd5b5060115460125460135461085692919083565b348015610cd8575f5ffd5b506106976123de565b348015610cec575f5ffd5b50610697610cfb366004613e8e565b6123e7565b348015610d0b575f5ffd5b50610697610d1a366004613e8e565b601e6020525f908152604090205481565b610988610d393660046144c2565b6123f6565b348015610d49575f5ffd5b5061063a610d58366004613fef565b6124be565b348015610d68575f5ffd5b50610710610d773660046145b3565b6125f8565b348015610d87575f5ffd5b50610710610d963660046146ee565b6126ab565b348015610da6575f5ffd5b50600954600a5461063a919082565b348015610dc0575f5ffd5b506106977f23ad11f0a1505378b82984192ad0461e6a012820fc5bf2e4ba16513f8e43055281565b348015610df3575f5ffd5b506107106127d2565b348015610e07575f5ffd5b50600e546001600160a01b03166106d9565b348015610e24575f5ffd5b50610697610e33366004613e8e565b5f9081526019602052604090205490565b348015610e4f575f5ffd5b50610697610e5e366004613fef565b61280c565b348015610e6e575f5ffd5b50610e82610e7d366004613e8e565b61282d565b60405161064b919061477e565b348015610e9a575f5ffd5b50610673610ea936600461402f565b612837565b348015610eb9575f5ffd5b50600154600254600354600454600554600654600754600854610eee976001600160a01b039081169781169695169392919088565b604080516001600160a01b03998a1681529789166020890152870195909552959092166060850152608084015260a083015260c082019290925260e08101919091526101000161064b565b348015610f44575f5ffd5b50610adc610f53366004613e8e565b5f9081526024602052604090205465ffffffffffff1690565b348015610f77575f5ffd5b50610710610f86366004614798565b61286d565b348015610f96575f5ffd5b50610ffc604080516080810182525f8082526020820181905291810182905260608101919091525060408051608081018252600d546001600160a01b039081168252600e5481166020830152600f54811692820192909252601054909116606082015290565b60405161064b919081516001600160a01b039081168252602080840151821690830152604080840151821690830152606092830151169181019190915260800190565b34801561104a575f5ffd5b50610c83611059366004613e8e565b612908565b348015611069575f5ffd5b50610673611078366004614335565b612913565b348015611088575f5ffd5b50610856611097366004613e8e565b61295d565b3480156110a7575f5ffd5b506106975f81565b3480156110ba575f5ffd5b506107106110c9366004613e8e565b6129d1565b3480156110d9575f5ffd5b506014546015546110f1916001600160a01b03169082565b604080516001600160a01b03909316835260208301919091520161064b565b34801561111b575f5ffd5b5061071061112a366004613e8e565b612b02565b61069761113d3660046147b2565b612ba6565b34801561114d575f5ffd5b5061067361115c366004614335565b612c6d565b61098861116f3660046147f2565b612cb7565b34801561117f575f5ffd5b5061069760255481565b348015611194575f5ffd5b506106976111a33660046142dc565b612d79565b3480156111b3575f5ffd5b506107106111c2366004614361565b612d9a565b3480156111d2575f5ffd5b506108566111e1366004613e8e565b612dab565b3480156111f1575f5ffd5b50610856611200366004613e47565b612e49565b348015611210575f5ffd5b5061069761121f366004613e8e565b612e8e565b34801561122f575f5ffd5b5061071061123e3660046148a0565b612e9e565b34801561124e575f5ffd5b5061069761125d366004613e8e565b612f41565b34801561126d575f5ffd5b50610697609681565b348015611281575f5ffd5b506022546106d9906001600160a01b031681565b3480156112a0575f5ffd5b50600d54600e54600f546010546112c7936001600160a01b03908116938116928116911684565b604080516001600160a01b039586168152938516602085015291841691830191909152909116606082015260800161064b565b348015611305575f5ffd5b5061130e612f4b565b6040805163ffffffff93841681529290911660208301520161064b565b348015611336575f5ffd5b5061071061134536600461402f565b612fad565b348015611355575f5ffd5b506106977fc50959b2b0264fed58f3489f13cdf8345df0911245cc2b741070787ee7aceaa281565b61098861138b3660046144c2565b612fc9565b61098861139e3660046148bb565b613006565b3480156113ae575f5ffd5b506106977f65d7a28e3265b37a6474929f336521b332c1681b933f6cb9f3376673440d862a81565b3480156113e1575f5ffd5b506106d96113f0366004613e8e565b60236020525f90815260409020546001600160a01b031681565b348015611415575f5ffd5b50610673611424366004613e8e565b6130a1565b348015611434575f5ffd5b506106977fe7cbc1eb0e9b3f8688b0bc91a8278f7d2867f14f2a10dc3f0d9fcfdc32dada1281565b348015611467575f5ffd5b50610697611476366004614490565b5f828152601b6020908152604080832084845282528083206001600160a01b03871684526002019091529020549392505050565b3480156114b5575f5ffd5b506107106114c4366004613fd5565b6130b6565b3480156114d4575f5ffd5b506106736114e3366004613e8e565b613103565b3480156114f3575f5ffd5b506040805180820182525f808252602091820152815180830183526014546001600160a01b031680825260155491830191825283519081529051918101919091520161064b565b348015611545575f5ffd5b50610697611554366004613e8e565b5f908152601e602052604090205490565b6109886115733660046144c2565b613177565b348015611583575f5ffd5b50610710611592366004614361565b6131d8565b3480156115a2575f5ffd5b506106976115b13660046142dc565b6132ec565b3480156115c1575f5ffd5b506106976115d0366004613e8e565b613335565b3480156115e0575f5ffd5b506115e9613345565b60405161064b919081516001600160a01b03908116825260208084015182169083015260408084015190830152606080840151909116908201526080808301519082015260a0828101519082015260c0808301519082015260e091820151918101919091526101000190565b348015611660575f5ffd5b5061067361166f366004613e8e565b6133fe565b34801561167f575f5ffd5b506106d961dead81565b5f828152601b60209081526040808320848452909152902080546001909101545b9250929050565b5f6001600160e01b03198216637965db0b60e01b14806116e157506301ffc9a760e01b6001600160e01b03198316145b92915050565b5f6116f0613408565b905090565b6116fd613425565b8051601480546001600160a01b0319166001600160a01b03909216918217905560208083015160158190556040519081527f8e32e306972875584ae78a6586b19f2b97a9dbc1a78a73ea8ff3b207c7da23cf910160405180910390a250565b61177d60405180606001604052805f81526020015f81526020015f81525090565b5060408051606081018252601154815260125460208201526013549181019190915290565b604051630d11c2ef60e11b815260048101839052602481018290525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__90631a2385de906044015b602060405180830381865af41580156117fa573d5f5f3e3d5ffd5b505050506040513d601f19601f8201168201806040525081019061181e919061493b565b9392505050565b5f6116e182613452565b5f9081525f516020614f2f5f395f51905f52602052604090206001015490565b611857613425565b80516009819055602080830151600a81905560408051938452918301527f6c59cf3d8d700a9538f44d5c8acf1889183727b7cf4669f0141f7ab689bdc81091015b60405180910390a150565b604051621e25f360e41b815260048101839052602481018290525f908190819073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__906301e25f30906044015b606060405180830381865af41580156118fe573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906119229190614952565b9250925092509250925092565b604051631eae320160e31b8152600481018490525f90819073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063f571900890602401602060405180830381865af4158015611981573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906119a5919061497d565b6119ca57604051634762af7d60e01b8152600481018690526024015b60405180910390fd5b60405163dcd2af4960e01b815260048101869052602481018590526044810184905273__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063dcd2af49906064016040805180830381865af4158015611a25573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190611a49919061499c565b91509150935093915050565b5f611a618484846134c6565b949350505050565b611a728261182f565b611a7b81613528565b611a858383613532565b50505050565b5f611a946135d3565b611a9c613603565b73__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__6358a19d5e86868686611ac261363a565b6040516001600160e01b031960e088901b1681526001600160a01b039095166004860152602485019390935260448401919091526064830152608482015260a401602060405180830381865af4158015611b1e573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190611b42919061493b565b9050611a6160015f516020614f6f5f395f51905f5255565b5f6116f0613663565b6001600160a01b0381163314611b8c5760405163334bd91960e11b815260040160405180910390fd5b611b968282613674565b505050565b606060ff5f5c1615611bc057604051632578e65d60e01b815260040160405180910390fd5b838214611be05760405163479ca36960e01b815260040160405180910390fd5b835f805b82811015611c1a57858582818110611bfe57611bfe6149be565b9050602002013582611c1091906149e6565b9150600101611be4565b50348114611c3b5760405163221c4cc160e21b815260040160405180910390fd5b60015f805c60ff19168217905d50816001600160401b03811115611c6157611c61613ea5565b604051908082528060200260200182016040528015611c9457816020015b6060815260200190600190039081611c7f5790505b5092505f5b82811015611d6c57858582818110611cb357611cb36149be565b90506020020135600181905d505f80308a8a85818110611cd557611cd56149be565b9050602002810190611ce791906149f9565b604051611cf5929190614a3b565b5f60405180830381855af49150503d805f8114611d2d576040519150601f19603f3d011682016040523d82523d5f602084013e611d32565b606091505b509150915081611d4457805160208201fd5b80868481518110611d5757611d576149be565b60209081029190910101525050600101611c99565b505f8060015d505f60ff19815c16815d505050949350505050565b6060611d916135d3565b611d99613603565b73__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__634f9a2e148a8a8a8a8a8a8a8a611dc361363a565b6040518a63ffffffff1660e01b8152600401611de799989796959493929190614a7a565b5f60405180830381865af4158015611e01573d5f5f3e3d5ffd5b505050506040513d5f823e601f3d908101601f19168201604052611e289190810190614b03565b9050611e4060015f516020614f6f5f395f51905f5255565b98975050505050505050565b5f611e5681613528565b611e5e6136ed565b611e6661371c565b50565b611e71613775565b611e79613603565b336001600160a01b0383168103611ea357604051638163594d60e01b815260040160405180910390fd5b5f826007811115611eb657611eb661476a565b03611eec576001600160a01b038082165f908152601a60209081526040808320938716835292905220805460ff19169055611f35565b816007811115611efe57611efe61476a565b6001600160a01b038281165f908152601a60209081526040808320938816835292905220805460ff191660ff929092169190911790555b806001600160a01b0316836001600160a01b03167f82a44452b8f9b854115b84acf31076a4deb9edd2530d246cf0d96c97a6ae619b84604051611f789190614b98565b60405180910390a350611f9760015f516020614f6f5f395f51905f5255565b5050565b604051631eae320160e31b8152600481018490525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063f571900890602401602060405180830381865af4158015611feb573d5f5f3e3d5ffd5b505050506040513d601f19601f8201168201806040525081019061200f919061497d565b61202f57604051634762af7d60e01b8152600481018590526024016119c1565b604051631148f8af60e21b815260048101859052602481018490526044810183905273__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__90634523e2bc906064015b602060405180830381865af415801561208c573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190611a61919061493b565b604051634656c5f160e01b81526001600160a01b038084166004830152821660248201525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__90634656c5f1906044015b602060405180830381865af4158015612111573d5f5f3e3d5ffd5b505050506040513d601f19601f8201168201806040525081019061181e919061497d565b604051631eae320160e31b8152600481018490525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063f571900890602401602060405180830381865af4158015612185573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906121a9919061497d565b6121c957604051634762af7d60e01b8152600481018590526024016119c1565b611a6184848461379b565b6121dc613425565b80516011819055602080830151601281905560408085015160138190558151948552928401919091528201527f1456f0760ace81355304bceb3062ae05afa5fbb02ec5460188d98fed953e6d4190606001611898565b5f6116e1826011600101546137e3565b606061224c6135d3565b612254613603565b73__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__63d93e99148686868661227a61363a565b6040518663ffffffff1660e01b815260040161229a959493929190614c6d565b5f60405180830381865af41580156122b4573d5f5f3e3d5ffd5b505050506040513d5f823e601f3d908101601f19168201604052611b429190810190614b03565b5f6116e1826137f5565b5f6116e182613828565b5f81815260166020526040902080546060919061230b90614ca6565b80601f016020809104026020016040519081016040528092919081815260200182805461233790614ca6565b80156123825780601f1061235957610100808354040283529160200191612382565b820191905f5260205f20905b81548152906001019060200180831161236557829003601f168201915b50505050509050919050565b604051636ced5d3f60e01b81526001600160a01b038416600482015260248101839052604481018290525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__90636ced5d3f90606401612071565b5f6116f0613889565b5f6116e18260115f01546137e3565b60606124006135d3565b612408613603565b73__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__63864c5fbf8b8b8b8b8b8b8b8b8b61243361363a565b6040518b63ffffffff1660e01b81526004016124589a99989796959493929190614cde565b5f60405180830381865af4158015612472573d5f5f3e3d5ffd5b505050506040513d5f823e601f3d908101601f191682016040526124999190810190614b03565b90506124b160015f516020614f6f5f395f51905f5255565b9998505050505050505050565b604051631eae320160e31b8152600481018490525f90819073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063f571900890602401602060405180830381865af4158015612510573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190612534919061497d565b61255457604051634762af7d60e01b8152600481018690526024016119c1565b5f61255e866138f5565b60405163d9dff30f60e01b8152600481018890526024810187905260448101869052811515606482015290915073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063d9dff30f90608401606060405180830381865af41580156125c5573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906125e99190614952565b91989197509095505050505050565b612600613425565b63ffffffff82161580612617575063ffffffff8116155b15612635576040516328d78f9f60e11b815260040160405180910390fd5b60408051808201825263ffffffff84811680835290841660209283018190526026805467ffffffffffffffff1916831764010000000083021790558351918252918101919091527f2b5bf708791497e53be4b967ae0d20fd869e30f845d57915ead6d434636546c8910160405180910390a15050565b5f6126b4613916565b805490915060ff600160401b82041615906001600160401b03165f811580156126da5750825b90505f826001600160401b031660011480156126f55750303b155b905081158015612703575080155b156127215760405163f92ee8a960e01b815260040160405180910390fd5b845467ffffffffffffffff19166001178555831561274b57845460ff60401b1916600160401b1785555b61275361393e565b61275b613946565b61276361393e565b6127718b8b8b8b8b8b613956565b8a5161277e905f90613532565b5083156127c557845460ff60401b19168555604051600181527fc7f505b2f371ae2175ee4913f4499e1f2633a7b5936321eed1cdaeb6115181d29060200160405180910390a15b5050505050505050505050565b7f65d7a28e3265b37a6474929f336521b332c1681b933f6cb9f3376673440d862a6127fc81613528565b6128046135d3565b611e66613a0c565b5f5f6128198585856134c6565b905061282481613a54565b95945050505050565b5f6116e182613a8e565b5f9182525f516020614f2f5f395f51905f52602090815260408084206001600160a01b0393909316845291905290205460ff1690565b612875613425565b8051600d80546001600160a01b039283166001600160a01b03199182168117909255602080850151600e805491861691841682179055604080870151600f80549188169186168217905560608801516010805491909816951685179096555192835292917fa56701aea90c1cdd1c40fe625d4bc2f0d52b88214278fea3ae2db7b703f3681691015b60405180910390a450565b60606116e182613b1f565b60405163a0ae55c760e01b81526001600160a01b038084166004830152821660248201525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063a0ae55c7906044016120f6565b5f81815260176020526040808220815160608101928390528392839283929160039082845b8154815260200190600101908083116129825750505050509050805f600381106129ae576129ae6149be565b602002015181600160200201518260026020020151935093509350509193909250565b6129d9613603565b5f6129e382613452565b9050336001600160a01b03821614612a0e57604051630bfa39ff60e21b815260040160405180910390fd5b6001600160a01b0381165f908152601d60205260409020548015612aea576001600160a01b0382165f818152601d602090815260408083208390558051638da5cb5b60e01b81529051929392638da5cb5b926004808401939192918290030181865afa158015612a80573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190612aa49190614d57565b9050612ab08183613bde565b81816001600160a01b0316857f93a8f3b2bae86deadc28b666f9f65297764642ef13b22d1de09b2125e163bd8460405160405180910390a4505b5050611e6660015f516020614f6f5f395f51905f5255565b612b0a613603565b5f818152601c602052604081205490819003612b265750612b90565b5f828152601c6020526040812055600254612b4a906001600160a01b031682613bde565b6002546040518281526001600160a01b039091169083907f0e19f21371647f79bb7c3f1e363266315b211fcca3836e7a71425cc0c4ab6a8e9060200160405180910390a3505b611e6660015f516020614f6f5f395f51905f5255565b5f612baf613775565b612bb76135d3565b612bbf613603565b60405163540a60ff60e11b81526001600160a01b03871660048201526024810186905260448101859052606481018490526084810183905273__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063a814c1fe9060a401602060405180830381865af4158015612c31573d5f5f3e3d5ffd5b505050506040513d601f19601f82011682018060405250810190612c55919061493b565b905061282460015f516020614f6f5f395f51905f5255565b604051632ac66f3560e21b81526001600160a01b038084166004830152821660248201525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063ab19bcd4906044016120f6565b6060612cc16135d3565b612cc9613603565b73__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__63a713c75e89898989898989612cf261363a565b6040518963ffffffff1660e01b8152600401612d15989796959493929190614d72565b5f60405180830381865af4158015612d2f573d5f5f3e3d5ffd5b505050506040513d5f823e601f3d908101601f19168201604052612d569190810190614b03565b9050612d6e60015f516020614f6f5f395f51905f5255565b979650505050505050565b60208052815f5260405f208160038110612d91575f80fd5b01549150829050565b612da2613425565b611e6681613c6a565b5f81815260176020526040808220815160608101928390528392839283929160039082845b815481526020019060010190808311612dd057505050505090505f5f1b815f60038110612dff57612dff6149be565b6020020151148015612e1357506020810151155b8015612e2157506040810151155b15612e42576040516308848f3b60e01b8152600481018690526024016119c1565b805f6129ae565b6040516317ad4ef560e21b815260048101839052602481018290525f908190819073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__90635eb53bd4906044016118e3565b5f6116e182600b600101546137e3565b612ea6613425565b612eaf81613cda565b80606001516001600160a01b031681602001516001600160a01b0316825f01516001600160a01b03167faf8d85bc3313be057acad92fcdd8829b909273359b6a033f2f4348787b8df3f3846040015185608001518660a001518760c001518860e001516040516128fd959493929190948552602085019390935260408401919091526060830152608082015260a00190565b5f6116e182613a54565b6040805180820190915260265463ffffffff8082168084526401000000009092041660208301525f91829115612f82578051612f85565b60055b9250806020015163ffffffff165f14612fa2578060200151612fa6565b6102bc5b9150509091565b612fb68261182f565b612fbf81613528565b611a858383613674565b6060612fd36135d3565b612fdb613603565b73__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__63a20efbea8b8b8b8b8b8b8b8b8b61243361363a565b60606130106135d3565b613018613603565b73__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__6323683bbd878787878761303f61363a565b6040518763ffffffff1660e01b815260040161306096959493929190614e5b565b5f60405180830381865af415801561307a573d5f5f3e3d5ffd5b505050506040513d5f823e601f3d908101601f19168201604052612c559190810190614b03565b5f8181526019602052604081205415156116e1565b6130be613425565b8051600b819055602080830151600c81905560408051938452918301527f4a883a35415345b4e144c4eb6e7a784a553b46997702b6889f682e4dea64c35f9101611898565b604051631eae320160e31b8152600481018290525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063f571900890602401602060405180830381865af4158015613153573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906116e1919061497d565b6060613181613775565b6131896135d3565b613191613603565b60405163f679bf0960e01b815273__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063f679bf0990612458908d908d908d908d908d908d908d908d908d90600401614ea6565b5f6131e281613528565b60025f6131ed613916565b8054909150600160401b900460ff1680613214575080546001600160401b03808416911610155b156132325760405163f92ee8a960e01b815260040160405180910390fd5b805468ffffffffffffffffff19166001600160401b03831617600160401b17815561325c84613c6a565b600154613293907f65d7a28e3265b37a6474929f336521b332c1681b933f6cb9f3376673440d862a906001600160a01b0316613532565b5061329c613889565b602555805460ff60401b191681556040516001600160401b03831681527fc7f505b2f371ae2175ee4913f4499e1f2633a7b5936321eed1cdaeb6115181d29060200160405180910390a150505050565b604051633e1f4a6b60e21b81526001600160a01b0383166004820152602481018290525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063f87d29ac906044016117df565b5f6116e1826011600201546137e3565b6133a06040518061010001604052805f6001600160a01b031681526020015f6001600160a01b031681526020015f81526020015f6001600160a01b031681526020015f81526020015f81526020015f81526020015f81525090565b5060408051610100810182526001546001600160a01b03908116825260025481166020830152600354928201929092526004549091166060820152600554608082015260065460a082015260075460c082015260085460e082015290565b5f6116e1826138f5565b6006545f90613418906002614f17565b600b546116f091906149e6565b6022546001600160a01b0316331461345057604051632b30e7b760e11b815260040160405180910390fd5b565b60405163218e250d60e01b8152600481018290525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__9063218e250d90602401602060405180830381865af41580156134a2573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906116e19190614d57565b604080517f23ad11f0a1505378b82984192ad0461e6a012820fc5bf2e4ba16513f8e430552602082015290810184905260608101839052608081018290525f9060a0016040516020818303038152906040528051906020012090509392505050565b611e668133613d73565b5f5f516020614f2f5f395f51905f5261354b8484612837565b6135ca575f848152602082815260408083206001600160a01b03871684529091529020805460ff191660011790556135803390565b6001600160a01b0316836001600160a01b0316857f2f8788117e7eff1d82e926ec794901d17c78024a50270940304540a733656f0d60405160405180910390a460019150506116e1565b5f9150506116e1565b5f516020614f4f5f395f51905f525460ff16156134505760405163d93c066560e01b815260040160405180910390fd5b5f516020614f6f5f395f51905f5280546001190161363457604051633ee5aeb560e01b815260040160405180910390fd5b60029055565b5f60ff815c1661364957503490565b5060015c90565b60015f516020614f6f5f395f51905f5255565b6006546009545f916116f0916149e6565b5f5f516020614f2f5f395f51905f5261368d8484612837565b156135ca575f848152602082815260408083206001600160a01b0387168085529252808320805460ff1916905551339287917ff6391f5c32d9c69d2a47ea670b442974b53935d1edc7fd64eb21e047a839171b9190a460019150506116e1565b5f516020614f4f5f395f51905f525460ff1661345057604051638dfc202b60e01b815260040160405180910390fd5b6137246136ed565b5f516020614f4f5f395f51905f52805460ff191681557f5db9ee0a495bf2e6ff9c91a7834c1ba4fdd244a5e8aa4e537bd38aeae4b073aa335b6040516001600160a01b039091168152602001611898565b61377d61363a565b156134505760405163d4d97d0160e01b815260040160405180910390fd5b6040516358717e0d60e01b81526004810184905260248101839052604481018290525f9073__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__906358717e0d90606401612071565b6003545f9061181e9084908490613dac565b5f818152601960205260408120541561381a57505f9081526019602052604090205490565b6116e182613a54565b919050565b5f7fc50959b2b0264fed58f3489f13cdf8345df0911245cc2b741070787ee7aceaa2828051906020012060405160200161386c929190918252602082015260400190565b604051602081830303815290604052805190602001209050919050565b5f73__$b869c6e4b8f03e2cd3fa34181d9d41f33f$__63766718086040518163ffffffff1660e01b8152600401602060405180830381865af41580156138d1573d5f5f3e3d5ffd5b505050506040513d601f19601f820116820180604052508101906116f0919061493b565b5f818152601660205260408120805461390d90614ca6565b15159392505050565b5f807ff0c57e16840df040f15088dc2f81fe391c3923bec73e23a9662efc9c229c6a006116e1565b613450613dd9565b61394e613dd9565b613450613dfe565b61395e613dd9565b61396786613cda565b8451600955602094850151600a558351600b5592840151600c558151600d80546001600160a01b03199081166001600160a01b039384161790915583860151600e80548316918416919091179055604080850151600f8054841691851691909117905560609094015160108054831691841691909117905582516011558286015160125591909201516013558251601480549092169216919091179055015160155550565b613a146135d3565b5f516020614f4f5f395f51905f52805460ff191660011781557f62e78cea01bee320cd4e420270b5ea74000d11b0c9f74754ebdbfc544b05a2583361375d565b604080517fe7cbc1eb0e9b3f8688b0bc91a8278f7d2867f14f2a10dc3f0d9fcfdc32dada1260208201529081018290525f9060600161386c565b5f5f613a99836138f5565b5f8481526018602090815260408083205460199092529091205491925060ff1690151582158015613ac8575081155b8015613ad2575080155b15613af35760405163bdd4a69960e01b8152600481018690526024016119c1565b8215613b0357505f949350505050565b8015613b1457506002949350505050565b506001949350505050565b5f81815260166020526040812080546060929190613b3c90614ca6565b80601f0160208091040260200160405190810160405280929190818152602001828054613b6890614ca6565b8015613bb35780601f10613b8a57610100808354040283529160200191613bb3565b820191905f5260205f20905b815481529060010190602001808311613b9657829003601f168201915b5050505050905080515f036116e15760405163b615632f60e01b8152600481018490526024016119c1565b80471015613c085760405163cf47918160e01b8152476004820152602481018290526044016119c1565b5f5f836001600160a01b0316836040515f6040518083038185875af1925050503d805f8114613c52576040519150601f19603f3d011682016040523d82523d5f602084013e613c57565b606091505b509150915081611a8557611a8581613e06565b6001600160a01b038116613c91576040516368de5c4b60e01b815260040160405180910390fd5b602280546001600160a01b0319166001600160a01b0383169081179091556040517f7e7ee4175d63f671fac3401d5f401ed18d1f48a586e756f404d5696fc77a7058905f90a250565b80516001600160a01b0316613d025760405163c9f9ba1560e01b815260040160405180910390fd5b8051600180546001600160a01b039283166001600160a01b0319918216179091556020830151600280549184169183169190911790556040830151600355606083015160048054919093169116179055608081015160055560a081015160065560c081015160075560e00151600855565b613d7d8282612837565b611f975760405163e2517d3f60e01b81526001600160a01b0382166004820152602481018390526044016119c1565b82820283158482048414178202613dca5763ad251c275f526004601cfd5b81810615159190040192915050565b613de1613e2e565b61345057604051631afcd79f60e31b815260040160405180910390fd5b613650613dd9565b805115613e1557805160208201fd5b60405163d6bda27560e01b815260040160405180910390fd5b5f613e37613916565b54600160401b900460ff16919050565b5f5f60408385031215613e58575f5ffd5b50508035926020909101359150565b5f60208284031215613e77575f5ffd5b81356001600160e01b03198116811461181e575f5ffd5b5f60208284031215613e9e575f5ffd5b5035919050565b634e487b7160e01b5f52604160045260245ffd5b604080519081016001600160401b0381118282101715613edb57613edb613ea5565b60405290565b60405161010081016001600160401b0381118282101715613edb57613edb613ea5565b604051601f8201601f191681016001600160401b0381118282101715613f2c57613f2c613ea5565b604052919050565b6001600160a01b0381168114611e66575f5ffd5b803561382381613f34565b5f60408284031215613f63575f5ffd5b613f6b613eb9565b90508135613f7881613f34565b815260209182013591810191909152919050565b5f60408284031215613f9c575f5ffd5b61181e8383613f53565b5f60408284031215613fb6575f5ffd5b613fbe613eb9565b823581526020928301359281019290925250919050565b5f60408284031215613fe5575f5ffd5b61181e8383613fa6565b5f5f5f60608486031215614001575f5ffd5b505081359360208301359350604090920135919050565b8151815260208083015190820152604081016116e1565b5f5f60408385031215614040575f5ffd5b82359150602083013561405281613f34565b809150509250929050565b5f5f5f5f60808587031215614070575f5ffd5b843561407b81613f34565b966020860135965060408601359560600135945092505050565b5f5f83601f8401126140a5575f5ffd5b5081356001600160401b038111156140bb575f5ffd5b6020830191508360208260051b85010111156116aa575f5ffd5b5f5f5f5f604085870312156140e8575f5ffd5b84356001600160401b038111156140fd575f5ffd5b61410987828801614095565b90955093505060208501356001600160401b03811115614127575f5ffd5b61413387828801614095565b95989497509550505050565b5f81518084528060208401602086015e5f602082860101526020601f19601f83011685010191505092915050565b5f602082016020835280845180835260408501915060408160051b8601019250602086015f5b828110156141c457603f198786030184526141af85835161413f565b94506020938401939190910190600101614193565b50929695505050505050565b5f5f5f5f5f5f5f5f6080898b0312156141e7575f5ffd5b88356001600160401b038111156141fc575f5ffd5b6142088b828c01614095565b90995097505060208901356001600160401b03811115614226575f5ffd5b6142328b828c01614095565b90975095505060408901356001600160401b03811115614250575f5ffd5b61425c8b828c01614095565b90955093505060608901356001600160401b0381111561427a575f5ffd5b6142868b828c01614095565b999c989b5096995094979396929594505050565b602080825282518282018190525f918401906040840190835b818110156142d15783518352602093840193909201916001016142b3565b509095945050505050565b5f5f604083850312156142ed575f5ffd5b82356142f881613f34565b946020939093013593505050565b5f5f60408385031215614317575f5ffd5b823561432281613f34565b9150602083013560088110614052575f5ffd5b5f5f60408385031215614346575f5ffd5b823561435181613f34565b9150602083013561405281613f34565b5f60208284031215614371575f5ffd5b813561181e81613f34565b5f6060828403121561438c575f5ffd5b604051606081016001600160401b03811182821017156143ae576143ae613ea5565b60409081528335825260208085013590830152928301359281019290925250919050565b5f606082840312156143e2575f5ffd5b61181e838361437c565b5f602082840312156143fc575f5ffd5b81356001600160401b03811115614411575f5ffd5b8201601f81018413614421575f5ffd5b80356001600160401b0381111561443a5761443a613ea5565b61444d601f8201601f1916602001613f04565b818152856020838501011115614461575f5ffd5b816020840160208301375f91810160200191909152949350505050565b602081525f61181e602083018461413f565b5f5f5f606084860312156144a2575f5ffd5b83356144ad81613f34565b95602085013595506040909401359392505050565b5f5f5f5f5f5f5f5f5f60a08a8c0312156144da575f5ffd5b89356144e581613f34565b985060208a01356001600160401b038111156144ff575f5ffd5b61450b8c828d01614095565b90995097505060408a01356001600160401b03811115614529575f5ffd5b6145358c828d01614095565b90975095505060608a01356001600160401b03811115614553575f5ffd5b61455f8c828d01614095565b90955093505060808a01356001600160401b0381111561457d575f5ffd5b6145898c828d01614095565b915080935050809150509295985092959850929598565b803563ffffffff81168114613823575f5ffd5b5f5f604083850312156145c4575f5ffd5b6145cd836145a0565b91506145db602084016145a0565b90509250929050565b5f61010082840312156145f5575f5ffd5b6145fd613ee1565b9050813561460a81613f34565b815261461860208301613f48565b60208201526040828101359082015261463360608301613f48565b60608201526080828101359082015260a0808301359082015260c0808301359082015260e09182013591810191909152919050565b5f60808284031215614678575f5ffd5b604051608081016001600160401b038111828210171561469a5761469a613ea5565b60405290508082356146ab81613f34565b815260208301356146bb81613f34565b602082015260408301356146ce81613f34565b604082015260608301356146e181613f34565b6060919091015292915050565b5f5f5f5f5f5f6102a08789031215614704575f5ffd5b61470e88886145e4565b955061471e886101008901613fa6565b945061472e886101408901613fa6565b935061473e886101808901614668565b925061474e88610200890161437c565b915061475e886102608901613f53565b90509295509295509295565b634e487b7160e01b5f52602160045260245ffd5b60208101600383106147925761479261476a565b91905290565b5f608082840312156147a8575f5ffd5b61181e8383614668565b5f5f5f5f5f60a086880312156147c6575f5ffd5b85356147d181613f34565b97602087013597506040870135966060810135965060800135945092505050565b5f5f5f5f5f5f5f6080888a031215614808575f5ffd5b873561481381613f34565b965060208801356001600160401b0381111561482d575f5ffd5b6148398a828b01614095565b90975095505060408801356001600160401b03811115614857575f5ffd5b6148638a828b01614095565b90955093505060608801356001600160401b03811115614881575f5ffd5b61488d8a828b01614095565b989b979a50959850939692959293505050565b5f61010082840312156148b1575f5ffd5b61181e83836145e4565b5f5f5f5f5f606086880312156148cf575f5ffd5b85356148da81613f34565b945060208601356001600160401b038111156148f4575f5ffd5b61490088828901614095565b90955093505060408601356001600160401b0381111561491e575f5ffd5b61492a88828901614095565b969995985093965092949392505050565b5f6020828403121561494b575f5ffd5b5051919050565b5f5f5f60608486031215614964575f5ffd5b5050815160208301516040909301519094929350919050565b5f6020828403121561498d575f5ffd5b8151801515811461181e575f5ffd5b5f5f604083850312156149ad575f5ffd5b505080516020909101519092909150565b634e487b7160e01b5f52603260045260245ffd5b634e487b7160e01b5f52601160045260245ffd5b808201808211156116e1576116e16149d2565b5f5f8335601e19843603018112614a0e575f5ffd5b8301803591506001600160401b03821115614a27575f5ffd5b6020019150368190038213156116aa575f5ffd5b818382375f9101908152919050565b8183525f6001600160fb1b03831115614a61575f5ffd5b8260051b80836020870137939093016020019392505050565b60a081525f614a8d60a083018b8d614a4a565b8281036020840152614aa0818a8c614a4a565b90508281036040840152614ab581888a614a4a565b90508281036060840152614aca818688614a4a565b9150508260808301529a9950505050505050505050565b5f6001600160401b03821115614af957614af9613ea5565b5060051b60200190565b5f60208284031215614b13575f5ffd5b81516001600160401b03811115614b28575f5ffd5b8201601f81018413614b38575f5ffd5b8051614b4b614b4682614ae1565b613f04565b8082825260208201915060208360051b850101925086831115614b6c575f5ffd5b6020840193505b82841015614b8e578351825260209384019390910190614b73565b9695505050505050565b60208101600883106147925761479261476a565b81835281816020850137505f828201602090810191909152601f909101601f19169091010190565b5f8383855260208501945060208460051b820101835f5b86811015614c6157838303601f190188525f80833536899003601e19018112614c12575f5ffd5b88016020810192503590506001600160401b03811115614c30575f5ffd5b803603821315614c3e575f5ffd5b614c49858284614bac565b60209a8b019a90955093909301925050600101614beb565b50909695505050505050565b606081525f614c80606083018789614bd4565b8281036020840152614c93818688614a4a565b9150508260408301529695505050505050565b600181811c90821680614cba57607f821691505b602082108103614cd857634e487b7160e01b5f52602260045260245ffd5b50919050565b6001600160a01b038b16815260c0602082018190525f90614d029083018b8d614a4a565b8281036040840152614d15818a8c614a4a565b90508281036060840152614d2a81888a614a4a565b90508281036080840152614d3f818688614a4a565b9150508260a08301529b9a5050505050505050505050565b5f60208284031215614d67575f5ffd5b815161181e81613f34565b6001600160a01b038916815260a0602082018190525f90614d96908301898b614bd4565b8281036040840152614da981888a614a4a565b83810360608501528581529050602080820190600587901b830101875f36829003601e19015b89821015614e3f57858403601f190185528235818112614ded575f5ffd5b8b016020810190356001600160401b03811115614e08575f5ffd5b8060051b3603821315614e19575f5ffd5b614e24868284614bd4565b95505050602083019250602085019450600182019150614dcf565b5050508093505050508260808301529998505050505050505050565b6001600160a01b03871681526080602082018190525f90614e7f9083018789614bd4565b8281036040840152614e92818688614a4a565b915050826060830152979650505050505050565b6001600160a01b038a16815260a0602082018190525f90614eca9083018a8c614a4a565b8281036040840152614edd81898b614a4a565b90508281036060840152614ef2818789614a4a565b90508281036080840152614f07818587614a4a565b9c9b505050505050505050505050565b80820281158282048414176116e1576116e16149d256fe02dd7bc7dec4dceedda775e58dd541e08a116c6c53815c0bd028192f7b626800cd5ed15c6e187e77e9aee88184c21f4f2182ab5827cb3b7e07fbedcd63f033009b779b17422d0df92223018b32b4d1fa46e071723d6817e2486d003becc55f00a164736f6c634300081d000a" } diff --git a/packages/contracts/vendored/README.md b/packages/contracts/vendored/README.md index edff859..6405999 100644 --- a/packages/contracts/vendored/README.md +++ b/packages/contracts/vendored/README.md @@ -1,16 +1,16 @@ # Vendored contract artifacts -Compiled artifacts the devnet deployer needs but `@0xintuition/contracts-v2` -does not (yet) export: +Compiled artifacts retained by the devnet deployer. Some fill package export +gaps; others are deployer-specific builds or legacy pins awaiting cleanup: | Artifact | Source | Why vendored | | --- | --- | --- | | `TransparentUpgradeableProxy` | `@openzeppelin/contracts@5.4.0` | Every protocol contract is deployed behind one (`_disableInitializers()` in the implementations makes proxies mandatory); the package ships no proxy bytecode. | | `TimelockController` | `@openzeppelin/contracts@5.4.0` | Owns the proxy admins (upgrades) and parameter changes in the canonical deploy. | | `UpgradeableBeacon` | `@openzeppelin/contracts@5.4.0` | AtomWallet instances are beacon proxies. | -| `AtomWarden` | `@0xintuition/contracts-v2` `src/protocol/wallet/AtomWarden.sol` | Missing from the package's `/abis` + `/bytecodes` exports. | -| `WrappedTrust` | `@0xintuition/contracts-v2` `src/WrappedTrust.sol` | Missing from the package's exports; the devnet uses it as the TRUST token. | -| `MultiVaultSizeFit` | `@0xintuition/contracts-v2` `src/protocol/MultiVault.sol` | `optimizer_runs=200` build (runtime 24,033 B) for EIP-170 chains — Intuition Sepolia enforces the cap, so the package's production `optimizer_runs=10000` bytecode (27,666 B runtime) cannot deploy there. | +| `AtomWarden` | `@0xintuition/contracts-v2` `src/protocol/wallet/AtomWarden.sol` | Legacy vendored pin; 1.1 now exports it, so consumers can migrate separately. | +| `WrappedTrust` | `@0xintuition/contracts-v2` `src/WrappedTrust.sol` | Legacy vendored pin; 1.1 now exports it, so consumers can migrate separately. | +| `MultiVaultSizeFit` | `@0xintuition/contracts-v2` `src/protocol/MultiVault.sol` | `optimizer_runs=200` build (runtime 20,379 B) for EIP-170 chains. Its `MultiVaultLib` placeholder must be linked before deployment. | Compiler settings mirror `intuition-contracts-v2` `foundry.toml`: solc `0.8.29`, `optimizer_runs = 10_000`, `evm_version = "cancun"`, @@ -32,6 +32,7 @@ source, which is **BUSL-1.1** — these compiled artifacts inherit that license (both this repo and the contracts are 0xIntuition projects; the artifacts are vendored here solely to deploy the protocol to development chains). -**Upstream plan**: these files disappear once `@0xintuition/contracts-v2` -exports `AtomWarden`/`WrappedTrust` and the OZ infra bytecodes (tracked for -`1.0.0-alpha.1`; see `docs/local-devnet.md` follow-ups). +`@0xintuition/contracts-v2@1.1.0-alpha.0` now exports `AtomWarden` and +`WrappedTrust`; migrating those two consumers away from their vendored copies +can happen separately. The OpenZeppelin infrastructure and size-fit +MultiVault artifacts remain deployer-specific outputs. diff --git a/packages/contracts/vendored/WrappedTrust.json b/packages/contracts/vendored/WrappedTrust.json index 636e521..269057a 100644 --- a/packages/contracts/vendored/WrappedTrust.json +++ b/packages/contracts/vendored/WrappedTrust.json @@ -1,6 +1,6 @@ { "contractName": "WrappedTrust", - "source": "@0xintuition/contracts-v2@1.0.0-alpha.0 src/WrappedTrust.sol", + "source": "@0xintuition/contracts-v2@1.1.0-alpha.0 src/WrappedTrust.sol", "compiler": { "solc": "0.8.29", "optimizerRuns": 10000, diff --git a/packages/database-kg/drizzle/0003_silky_valkyrie.sql b/packages/database-kg/drizzle/0003_silky_valkyrie.sql new file mode 100644 index 0000000..90e3473 --- /dev/null +++ b/packages/database-kg/drizzle/0003_silky_valkyrie.sql @@ -0,0 +1,6 @@ +ALTER TABLE "kg"."nodes" ADD COLUMN "iid" text;--> statement-breakpoint +ALTER TABLE "kg"."nodes" ADD CONSTRAINT "chk_nodes_raw_type_iid" CHECK ("kg"."nodes"."raw_type" IN ('string', 'json', 'http_uri', 'ipfs_uri', 'iid')) NOT VALID;--> statement-breakpoint +ALTER TABLE "kg"."nodes" VALIDATE CONSTRAINT "chk_nodes_raw_type_iid";--> statement-breakpoint +ALTER TABLE "kg"."nodes" DROP CONSTRAINT "chk_nodes_raw_type";--> statement-breakpoint +ALTER TABLE "kg"."nodes" RENAME CONSTRAINT "chk_nodes_raw_type_iid" TO "chk_nodes_raw_type";--> statement-breakpoint +CREATE INDEX "idx_nodes_iid" ON "kg"."nodes" USING btree ("iid") WHERE "kg"."nodes"."iid" IS NOT NULL; diff --git a/packages/database-kg/drizzle/0004_add-node-contexts.sql b/packages/database-kg/drizzle/0004_add-node-contexts.sql new file mode 100644 index 0000000..bfc137f --- /dev/null +++ b/packages/database-kg/drizzle/0004_add-node-contexts.sql @@ -0,0 +1,25 @@ +CREATE TABLE "kg"."node_contexts" ( + "node_id" text NOT NULL, + "event_sequence" bigint NOT NULL, + "block_number" bigint NOT NULL, + "block_timestamp" timestamp with time zone NOT NULL, + "block_hash" text NOT NULL, + "transaction_hash" text NOT NULL, + "log_index" integer NOT NULL, + "ordinal" integer NOT NULL, + "registrant" text NOT NULL, + "uri_hex" text NOT NULL, + "uri_text" text, + "created_at" timestamp with time zone DEFAULT now() NOT NULL, + CONSTRAINT "node_contexts_pkey" PRIMARY KEY("node_id","transaction_hash","log_index","ordinal"), + CONSTRAINT "chk_node_contexts_event_sequence" CHECK ("kg"."node_contexts"."event_sequence" >= 0), + CONSTRAINT "chk_node_contexts_block_number" CHECK ("kg"."node_contexts"."block_number" >= 0), + CONSTRAINT "chk_node_contexts_log_index" CHECK ("kg"."node_contexts"."log_index" >= 0), + CONSTRAINT "chk_node_contexts_ordinal" CHECK ("kg"."node_contexts"."ordinal" >= 0), + CONSTRAINT "chk_node_contexts_uri_hex" CHECK ("kg"."node_contexts"."uri_hex" ~ '^0x([0-9a-f]{2})*$') +); +--> statement-breakpoint +ALTER TABLE "kg"."node_contexts" ADD CONSTRAINT "node_contexts_node_id_nodes_id_fk" FOREIGN KEY ("node_id") REFERENCES "kg"."nodes"("id") ON DELETE no action ON UPDATE no action;--> statement-breakpoint +CREATE UNIQUE INDEX "idx_node_contexts_event_ordinal" ON "kg"."node_contexts" USING btree ("event_sequence","ordinal");--> statement-breakpoint +CREATE INDEX "idx_node_contexts_node_sequence" ON "kg"."node_contexts" USING btree ("node_id","event_sequence","ordinal");--> statement-breakpoint +CREATE INDEX "idx_node_contexts_transaction" ON "kg"."node_contexts" USING btree ("transaction_hash","log_index"); \ No newline at end of file diff --git a/packages/database-kg/drizzle/meta/0003_snapshot.json b/packages/database-kg/drizzle/meta/0003_snapshot.json new file mode 100644 index 0000000..1cbca82 --- /dev/null +++ b/packages/database-kg/drizzle/meta/0003_snapshot.json @@ -0,0 +1,2568 @@ +{ + "id": "19a504e1-f7eb-423c-a027-4500b072152d", + "prevId": "69169a30-5c41-4ac3-b626-143fde2f15ed", + "version": "7", + "dialect": "postgresql", + "tables": { + "kg.account_stats": { + "name": "account_stats", + "schema": "kg", + "columns": { + "account_id": { + "name": "account_id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "created_node_count": { + "name": "created_node_count", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": "0" + }, + "created_triple_count": { + "name": "created_triple_count", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": "0" + }, + "deposit_count": { + "name": "deposit_count", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": "0" + }, + "withdrawal_count": { + "name": "withdrawal_count", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": "0" + }, + "last_deposit_at": { + "name": "last_deposit_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_withdrawal_at": { + "name": "last_withdrawal_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "account_stats_account_id_accounts_id_fk": { + "name": "account_stats_account_id_accounts_id_fk", + "tableFrom": "account_stats", + "tableTo": "accounts", + "schemaTo": "kg", + "columnsFrom": [ + "account_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.accounts": { + "name": "accounts", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "last_seen_at": { + "name": "last_seen_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "idx_accounts_last_seen_at": { + "name": "idx_accounts_last_seen_at", + "columns": [ + { + "expression": "last_seen_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_accounts_deleted_at": { + "name": "idx_accounts_deleted_at", + "columns": [ + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.adjacency": { + "name": "adjacency", + "schema": "kg", + "columns": { + "source_id": { + "name": "source_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source_type": { + "name": "source_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "direction": { + "name": "direction", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_id": { + "name": "predicate_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_type": { + "name": "predicate_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "neighbor_id": { + "name": "neighbor_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "neighbor_type": { + "name": "neighbor_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "triple_id": { + "name": "triple_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "weight": { + "name": "weight", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "market_weight": { + "name": "market_weight", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "social_weight": { + "name": "social_weight", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "idx_adjacency_source_ref": { + "name": "idx_adjacency_source_ref", + "columns": [ + { + "expression": "source_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "source_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_adjacency_predicate_ref": { + "name": "idx_adjacency_predicate_ref", + "columns": [ + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_adjacency_neighbor_ref": { + "name": "idx_adjacency_neighbor_ref", + "columns": [ + { + "expression": "neighbor_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "neighbor_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "adjacency_triple_id_triples_id_fk": { + "name": "adjacency_triple_id_triples_id_fk", + "tableFrom": "adjacency", + "tableTo": "triples", + "schemaTo": "kg", + "columnsFrom": [ + "triple_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "adjacency_pkey": { + "name": "adjacency_pkey", + "columns": [ + "source_id", + "source_type", + "direction", + "predicate_id", + "predicate_type", + "neighbor_id", + "neighbor_type", + "triple_id" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_adjacency_source_type": { + "name": "chk_adjacency_source_type", + "value": "\"kg\".\"adjacency\".\"source_type\" IN ('node', 'triple')" + }, + "chk_adjacency_predicate_type": { + "name": "chk_adjacency_predicate_type", + "value": "\"kg\".\"adjacency\".\"predicate_type\" IN ('node', 'triple')" + }, + "chk_adjacency_neighbor_type": { + "name": "chk_adjacency_neighbor_type", + "value": "\"kg\".\"adjacency\".\"neighbor_type\" IN ('node', 'triple')" + } + }, + "isRLSEnabled": false + }, + "kg.api_keys": { + "name": "api_keys", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "key_hash": { + "name": "key_hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "account_id": { + "name": "account_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "can_write": { + "name": "can_write", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "rate_limit_rpm": { + "name": "rate_limit_rpm", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "revoked_at": { + "name": "revoked_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_used_at": { + "name": "last_used_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "idx_api_keys_key_hash": { + "name": "idx_api_keys_key_hash", + "columns": [ + { + "expression": "key_hash", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_api_keys_account_id": { + "name": "idx_api_keys_account_id", + "columns": [ + { + "expression": "account_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "api_keys_account_id_accounts_id_fk": { + "name": "api_keys_account_id_accounts_id_fk", + "tableFrom": "api_keys", + "tableTo": "accounts", + "schemaTo": "kg", + "columnsFrom": [ + "account_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.artifacts": { + "name": "artifacts", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "node_id": { + "name": "node_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "artifact_kind": { + "name": "artifact_kind", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "artifact_version": { + "name": "artifact_version", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source_uri": { + "name": "source_uri", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_hash": { + "name": "source_hash", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "data": { + "name": "data", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "extracted": { + "name": "extracted", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "error": { + "name": "error", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_by_account_id": { + "name": "created_by_account_id", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "idx_artifacts_node_id": { + "name": "idx_artifacts_node_id", + "columns": [ + { + "expression": "node_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_artifacts_created_by_account_id": { + "name": "idx_artifacts_created_by_account_id", + "columns": [ + { + "expression": "created_by_account_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_artifacts_kind_version_status": { + "name": "idx_artifacts_kind_version_status", + "columns": [ + { + "expression": "artifact_kind", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "artifact_version", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_artifacts_kind_source_hash": { + "name": "idx_artifacts_kind_source_hash", + "columns": [ + { + "expression": "artifact_kind", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "source_hash", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "artifacts_node_id_nodes_id_fk": { + "name": "artifacts_node_id_nodes_id_fk", + "tableFrom": "artifacts", + "tableTo": "nodes", + "schemaTo": "kg", + "columnsFrom": [ + "node_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + }, + "artifacts_created_by_account_id_accounts_id_fk": { + "name": "artifacts_created_by_account_id_accounts_id_fk", + "tableFrom": "artifacts", + "tableTo": "accounts", + "schemaTo": "kg", + "columnsFrom": [ + "created_by_account_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.events": { + "name": "events", + "schema": "kg", + "columns": { + "event_time": { + "name": "event_time", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "id": { + "name": "id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "actor_id": { + "name": "actor_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "entity_kind": { + "name": "entity_kind", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "entity_id": { + "name": "entity_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "event_type": { + "name": "event_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "classification_type": { + "name": "classification_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "is_onchain": { + "name": "is_onchain", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "block_number": { + "name": "block_number", + "type": "bigint", + "primaryKey": false, + "notNull": false + }, + "tx_hash": { + "name": "tx_hash", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "payload": { + "name": "payload", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "idx_kg_events_entity": { + "name": "idx_kg_events_entity", + "columns": [ + { + "expression": "entity_kind", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entity_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "event_time", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_kg_events_actor": { + "name": "idx_kg_events_actor", + "columns": [ + { + "expression": "actor_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "event_time", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_kg_events_type": { + "name": "idx_kg_events_type", + "columns": [ + { + "expression": "event_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "event_time", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": { + "kg_events_pkey": { + "name": "kg_events_pkey", + "columns": [ + "event_time", + "id" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_kg_events_entity_kind": { + "name": "chk_kg_events_entity_kind", + "value": "\"kg\".\"events\".\"entity_kind\" IN ('node', 'triple', 'predicate', 'artifact')" + } + }, + "isRLSEnabled": false + }, + "kg.node_urls": { + "name": "node_urls", + "schema": "kg", + "columns": { + "node_id": { + "name": "node_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "url": { + "name": "url", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "domain": { + "name": "domain", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source": { + "name": "source", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "artifact_id": { + "name": "artifact_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "is_primary": { + "name": "is_primary", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "idx_node_urls_node_domain": { + "name": "idx_node_urls_node_domain", + "columns": [ + { + "expression": "node_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "domain", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_node_urls_domain": { + "name": "idx_node_urls_domain", + "columns": [ + { + "expression": "domain", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_node_urls_one_primary_per_node": { + "name": "idx_node_urls_one_primary_per_node", + "columns": [ + { + "expression": "node_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"kg\".\"node_urls\".\"is_primary\" = true", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "node_urls_node_id_nodes_id_fk": { + "name": "node_urls_node_id_nodes_id_fk", + "tableFrom": "node_urls", + "tableTo": "nodes", + "schemaTo": "kg", + "columnsFrom": [ + "node_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "node_urls_artifact_id_artifacts_id_fk": { + "name": "node_urls_artifact_id_artifacts_id_fk", + "tableFrom": "node_urls", + "tableTo": "artifacts", + "schemaTo": "kg", + "columnsFrom": [ + "artifact_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "node_urls_pkey": { + "name": "node_urls_pkey", + "columns": [ + "node_id", + "url" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_node_urls_url_nonempty": { + "name": "chk_node_urls_url_nonempty", + "value": "\"kg\".\"node_urls\".\"url\" <> ''" + }, + "chk_node_urls_domain_nonempty": { + "name": "chk_node_urls_domain_nonempty", + "value": "\"kg\".\"node_urls\".\"domain\" <> ''" + } + }, + "isRLSEnabled": false + }, + "kg.node_stats": { + "name": "node_stats", + "schema": "kg", + "columns": { + "node_id": { + "name": "node_id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "in_degree": { + "name": "in_degree", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "out_degree": { + "name": "out_degree", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "neighbor_kind_counts": { + "name": "neighbor_kind_counts", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "predicate_counts": { + "name": "predicate_counts", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "node_stats_node_id_nodes_id_fk": { + "name": "node_stats_node_id_nodes_id_fk", + "tableFrom": "node_stats", + "tableTo": "nodes", + "schemaTo": "kg", + "columnsFrom": [ + "node_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.nodes": { + "name": "nodes", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "is_onchain": { + "name": "is_onchain", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "visibility": { + "name": "visibility", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'public'" + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "raw_type": { + "name": "raw_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "data": { + "name": "data", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "data_hex": { + "name": "data_hex", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "iid": { + "name": "iid", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "data_resolved": { + "name": "data_resolved", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "parse_attempts": { + "name": "parse_attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "parse_status": { + "name": "parse_status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "parse_started_at": { + "name": "parse_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "parse_lease_expires_at": { + "name": "parse_lease_expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "parsed_at": { + "name": "parsed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "parse_error": { + "name": "parse_error", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "parse_result": { + "name": "parse_result", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "classification_attempts": { + "name": "classification_attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "classification_status": { + "name": "classification_status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "classification_started_at": { + "name": "classification_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "classification_lease_expires_at": { + "name": "classification_lease_expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "classified_at": { + "name": "classified_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "classification_error": { + "name": "classification_error", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "classification_result": { + "name": "classification_result", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "classification_type": { + "name": "classification_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'Unknown'" + }, + "enrichment_attempts": { + "name": "enrichment_attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "enrichment_status": { + "name": "enrichment_status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "enrichment_started_at": { + "name": "enrichment_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "enrichment_lease_expires_at": { + "name": "enrichment_lease_expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "enriched_at": { + "name": "enriched_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "enrichment_error": { + "name": "enrichment_error", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "processing_meta": { + "name": "processing_meta", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "search_text": { + "name": "search_text", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "''" + } + }, + "indexes": { + "idx_nodes_status_visibility_created_at": { + "name": "idx_nodes_status_visibility_created_at", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "visibility", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_visibility": { + "name": "idx_nodes_visibility", + "columns": [ + { + "expression": "visibility", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_raw_type_data_hex": { + "name": "idx_nodes_raw_type_data_hex", + "columns": [ + { + "expression": "raw_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "data_hex", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_classification_type": { + "name": "idx_nodes_classification_type", + "columns": [ + { + "expression": "classification_type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_created_by_created_at": { + "name": "idx_nodes_created_by_created_at", + "columns": [ + { + "expression": "created_by", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_data_hex": { + "name": "idx_nodes_data_hex", + "columns": [ + { + "expression": "data_hex", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_iid": { + "name": "idx_nodes_iid", + "columns": [ + { + "expression": "iid", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"kg\".\"nodes\".\"iid\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_parse_recovery": { + "name": "idx_nodes_parse_recovery", + "columns": [ + { + "expression": "parse_status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "parse_lease_expires_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_classification_recovery": { + "name": "idx_nodes_classification_recovery", + "columns": [ + { + "expression": "classification_status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "classification_lease_expires_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_enrichment_recovery": { + "name": "idx_nodes_enrichment_recovery", + "columns": [ + { + "expression": "enrichment_status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "enrichment_lease_expires_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_processing_statuses": { + "name": "idx_nodes_processing_statuses", + "columns": [ + { + "expression": "parse_status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "classification_status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "enrichment_status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "nodes_created_by_accounts_id_fk": { + "name": "nodes_created_by_accounts_id_fk", + "tableFrom": "nodes", + "tableTo": "accounts", + "schemaTo": "kg", + "columnsFrom": [ + "created_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_nodes_visibility": { + "name": "chk_nodes_visibility", + "value": "\"kg\".\"nodes\".\"visibility\" IN ('public', 'unlisted')" + }, + "chk_nodes_status": { + "name": "chk_nodes_status", + "value": "\"kg\".\"nodes\".\"status\" IN ('active', 'draft')" + }, + "chk_nodes_raw_type": { + "name": "chk_nodes_raw_type", + "value": "\"kg\".\"nodes\".\"raw_type\" IN ('string', 'json', 'http_uri', 'ipfs_uri', 'iid')" + }, + "chk_nodes_parse_status": { + "name": "chk_nodes_parse_status", + "value": "\"kg\".\"nodes\".\"parse_status\" IN ('pending', 'processing', 'completed', 'failed', 'skipped')" + }, + "chk_nodes_classification_status": { + "name": "chk_nodes_classification_status", + "value": "\"kg\".\"nodes\".\"classification_status\" IN ('pending', 'processing', 'completed', 'failed', 'skipped')" + }, + "chk_nodes_enrichment_status": { + "name": "chk_nodes_enrichment_status", + "value": "\"kg\".\"nodes\".\"enrichment_status\" IN ('pending', 'processing', 'completed', 'failed', 'skipped')" + } + }, + "isRLSEnabled": false + }, + "kg.predicate_stats": { + "name": "predicate_stats", + "schema": "kg", + "columns": { + "predicate_id": { + "name": "predicate_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_type": { + "name": "predicate_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "triple_count": { + "name": "triple_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "distinct_subject_count": { + "name": "distinct_subject_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "distinct_object_count": { + "name": "distinct_object_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "avg_out_degree": { + "name": "avg_out_degree", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "avg_in_degree": { + "name": "avg_in_degree", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "selectivity_score": { + "name": "selectivity_score", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": { + "predicate_stats_pkey": { + "name": "predicate_stats_pkey", + "columns": [ + "predicate_type", + "predicate_id" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_predicate_stats_predicate_type": { + "name": "chk_predicate_stats_predicate_type", + "value": "\"kg\".\"predicate_stats\".\"predicate_type\" IN ('node', 'triple')" + } + }, + "isRLSEnabled": false + }, + "kg.predicates": { + "name": "predicates", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "slug": { + "name": "slug", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "label": { + "name": "label", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "inverse_predicate_id": { + "name": "inverse_predicate_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "is_transitive": { + "name": "is_transitive", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_symmetric": { + "name": "is_symmetric", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_hierarchical": { + "name": "is_hierarchical", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_social": { + "name": "is_social", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_market": { + "name": "is_market", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "predicates_slug_unique": { + "name": "predicates_slug_unique", + "nullsNotDistinct": false, + "columns": [ + "slug" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.triple_pattern_stats": { + "name": "triple_pattern_stats", + "schema": "kg", + "columns": { + "subject_kind": { + "name": "subject_kind", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_id": { + "name": "predicate_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_type": { + "name": "predicate_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "object_kind": { + "name": "object_kind", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "triple_count": { + "name": "triple_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "distinct_subject_count": { + "name": "distinct_subject_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "distinct_object_count": { + "name": "distinct_object_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "selectivity_score": { + "name": "selectivity_score", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": { + "triple_pattern_stats_pkey": { + "name": "triple_pattern_stats_pkey", + "columns": [ + "subject_kind", + "predicate_type", + "predicate_id", + "object_kind" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_triple_pattern_stats_predicate_type": { + "name": "chk_triple_pattern_stats_predicate_type", + "value": "\"kg\".\"triple_pattern_stats\".\"predicate_type\" IN ('node', 'triple')" + } + }, + "isRLSEnabled": false + }, + "kg.triples": { + "name": "triples", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "is_onchain": { + "name": "is_onchain", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "visibility": { + "name": "visibility", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'public'" + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "subject_id": { + "name": "subject_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "subject_type": { + "name": "subject_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "predicate_id": { + "name": "predicate_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_type": { + "name": "predicate_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "object_id": { + "name": "object_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "object_type": { + "name": "object_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "is_counter_triple": { + "name": "is_counter_triple", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "sibling_triple_id": { + "name": "sibling_triple_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "edge_kind": { + "name": "edge_kind", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'claim'" + }, + "source": { + "name": "source", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_uri": { + "name": "source_uri", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "confidence": { + "name": "confidence", + "type": "numeric(6, 5)", + "primaryKey": false, + "notNull": false + }, + "inferred": { + "name": "inferred", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "provenance": { + "name": "provenance", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + } + }, + "indexes": { + "idx_triples_spo": { + "name": "idx_triples_spo", + "columns": [ + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_sop": { + "name": "idx_triples_sop", + "columns": [ + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_pso": { + "name": "idx_triples_pso", + "columns": [ + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_pos": { + "name": "idx_triples_pos", + "columns": [ + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_osp": { + "name": "idx_triples_osp", + "columns": [ + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_ops": { + "name": "idx_triples_ops", + "columns": [ + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_subject_ref": { + "name": "idx_triples_subject_ref", + "columns": [ + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_predicate_ref": { + "name": "idx_triples_predicate_ref", + "columns": [ + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_object_ref": { + "name": "idx_triples_object_ref", + "columns": [ + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_status_visibility_created_at": { + "name": "idx_triples_status_visibility_created_at", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "visibility", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_sibling_triple_id": { + "name": "idx_triples_sibling_triple_id", + "columns": [ + { + "expression": "sibling_triple_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_counter_triple": { + "name": "idx_triples_counter_triple", + "columns": [ + { + "expression": "is_counter_triple", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_created_by_created_at": { + "name": "idx_triples_created_by_created_at", + "columns": [ + { + "expression": "created_by", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_edge_kind_status_created_at": { + "name": "idx_triples_edge_kind_status_created_at", + "columns": [ + { + "expression": "edge_kind", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_source_uri": { + "name": "idx_triples_source_uri", + "columns": [ + { + "expression": "source_uri", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_confidence_desc": { + "name": "idx_triples_confidence_desc", + "columns": [ + { + "expression": "confidence", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "triples_created_by_accounts_id_fk": { + "name": "triples_created_by_accounts_id_fk", + "tableFrom": "triples", + "tableTo": "accounts", + "schemaTo": "kg", + "columnsFrom": [ + "created_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "triples_sibling_triple_id_triples_id_fk": { + "name": "triples_sibling_triple_id_triples_id_fk", + "tableFrom": "triples", + "tableTo": "triples", + "schemaTo": "kg", + "columnsFrom": [ + "sibling_triple_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_triples_visibility": { + "name": "chk_triples_visibility", + "value": "\"kg\".\"triples\".\"visibility\" IN ('public', 'unlisted')" + }, + "chk_triples_status": { + "name": "chk_triples_status", + "value": "\"kg\".\"triples\".\"status\" IN ('active', 'draft')" + }, + "chk_triples_subject_type": { + "name": "chk_triples_subject_type", + "value": "\"kg\".\"triples\".\"subject_type\" IN ('node', 'triple')" + }, + "chk_triples_predicate_type": { + "name": "chk_triples_predicate_type", + "value": "\"kg\".\"triples\".\"predicate_type\" IN ('node', 'triple')" + }, + "chk_triples_object_type": { + "name": "chk_triples_object_type", + "value": "\"kg\".\"triples\".\"object_type\" IN ('node', 'triple')" + }, + "chk_triples_counter_sibling_required": { + "name": "chk_triples_counter_sibling_required", + "value": "\"kg\".\"triples\".\"is_counter_triple\" = false OR \"kg\".\"triples\".\"sibling_triple_id\" IS NOT NULL" + }, + "chk_triples_sibling_not_self": { + "name": "chk_triples_sibling_not_self", + "value": "\"kg\".\"triples\".\"sibling_triple_id\" IS NULL OR \"kg\".\"triples\".\"sibling_triple_id\" <> \"kg\".\"triples\".\"id\"" + }, + "chk_triples_confidence_range": { + "name": "chk_triples_confidence_range", + "value": "\"kg\".\"triples\".\"confidence\" IS NULL OR (\"kg\".\"triples\".\"confidence\" >= 0 AND \"kg\".\"triples\".\"confidence\" <= 1)" + } + }, + "isRLSEnabled": false + } + }, + "enums": {}, + "schemas": { + "kg": "kg" + }, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} \ No newline at end of file diff --git a/packages/database-kg/drizzle/meta/0004_snapshot.json b/packages/database-kg/drizzle/meta/0004_snapshot.json new file mode 100644 index 0000000..2d24556 --- /dev/null +++ b/packages/database-kg/drizzle/meta/0004_snapshot.json @@ -0,0 +1,2770 @@ +{ + "id": "d627a291-ba7f-43eb-98b3-57c3eff66bd3", + "prevId": "19a504e1-f7eb-423c-a027-4500b072152d", + "version": "7", + "dialect": "postgresql", + "tables": { + "kg.account_stats": { + "name": "account_stats", + "schema": "kg", + "columns": { + "account_id": { + "name": "account_id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "created_node_count": { + "name": "created_node_count", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": "0" + }, + "created_triple_count": { + "name": "created_triple_count", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": "0" + }, + "deposit_count": { + "name": "deposit_count", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": "0" + }, + "withdrawal_count": { + "name": "withdrawal_count", + "type": "bigint", + "primaryKey": false, + "notNull": true, + "default": "0" + }, + "last_deposit_at": { + "name": "last_deposit_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_withdrawal_at": { + "name": "last_withdrawal_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": { + "account_stats_account_id_accounts_id_fk": { + "name": "account_stats_account_id_accounts_id_fk", + "tableFrom": "account_stats", + "tableTo": "accounts", + "schemaTo": "kg", + "columnsFrom": [ + "account_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.accounts": { + "name": "accounts", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "last_seen_at": { + "name": "last_seen_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "deleted_at": { + "name": "deleted_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "idx_accounts_last_seen_at": { + "name": "idx_accounts_last_seen_at", + "columns": [ + { + "expression": "last_seen_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_accounts_deleted_at": { + "name": "idx_accounts_deleted_at", + "columns": [ + { + "expression": "deleted_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.adjacency": { + "name": "adjacency", + "schema": "kg", + "columns": { + "source_id": { + "name": "source_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source_type": { + "name": "source_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "direction": { + "name": "direction", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_id": { + "name": "predicate_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_type": { + "name": "predicate_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "neighbor_id": { + "name": "neighbor_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "neighbor_type": { + "name": "neighbor_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "triple_id": { + "name": "triple_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "weight": { + "name": "weight", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "market_weight": { + "name": "market_weight", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "social_weight": { + "name": "social_weight", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "idx_adjacency_source_ref": { + "name": "idx_adjacency_source_ref", + "columns": [ + { + "expression": "source_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "source_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_adjacency_predicate_ref": { + "name": "idx_adjacency_predicate_ref", + "columns": [ + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_adjacency_neighbor_ref": { + "name": "idx_adjacency_neighbor_ref", + "columns": [ + { + "expression": "neighbor_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "neighbor_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "adjacency_triple_id_triples_id_fk": { + "name": "adjacency_triple_id_triples_id_fk", + "tableFrom": "adjacency", + "tableTo": "triples", + "schemaTo": "kg", + "columnsFrom": [ + "triple_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "adjacency_pkey": { + "name": "adjacency_pkey", + "columns": [ + "source_id", + "source_type", + "direction", + "predicate_id", + "predicate_type", + "neighbor_id", + "neighbor_type", + "triple_id" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_adjacency_source_type": { + "name": "chk_adjacency_source_type", + "value": "\"kg\".\"adjacency\".\"source_type\" IN ('node', 'triple')" + }, + "chk_adjacency_predicate_type": { + "name": "chk_adjacency_predicate_type", + "value": "\"kg\".\"adjacency\".\"predicate_type\" IN ('node', 'triple')" + }, + "chk_adjacency_neighbor_type": { + "name": "chk_adjacency_neighbor_type", + "value": "\"kg\".\"adjacency\".\"neighbor_type\" IN ('node', 'triple')" + } + }, + "isRLSEnabled": false + }, + "kg.api_keys": { + "name": "api_keys", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "key_hash": { + "name": "key_hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "name": { + "name": "name", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "account_id": { + "name": "account_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "can_write": { + "name": "can_write", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": true + }, + "rate_limit_rpm": { + "name": "rate_limit_rpm", + "type": "integer", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "revoked_at": { + "name": "revoked_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "last_used_at": { + "name": "last_used_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "idx_api_keys_key_hash": { + "name": "idx_api_keys_key_hash", + "columns": [ + { + "expression": "key_hash", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_api_keys_account_id": { + "name": "idx_api_keys_account_id", + "columns": [ + { + "expression": "account_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "api_keys_account_id_accounts_id_fk": { + "name": "api_keys_account_id_accounts_id_fk", + "tableFrom": "api_keys", + "tableTo": "accounts", + "schemaTo": "kg", + "columnsFrom": [ + "account_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.artifacts": { + "name": "artifacts", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "node_id": { + "name": "node_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "artifact_kind": { + "name": "artifact_kind", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "artifact_version": { + "name": "artifact_version", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source_uri": { + "name": "source_uri", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_hash": { + "name": "source_hash", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "data": { + "name": "data", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "extracted": { + "name": "extracted", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "error": { + "name": "error", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "created_by_account_id": { + "name": "created_by_account_id", + "type": "text", + "primaryKey": false, + "notNull": false + } + }, + "indexes": { + "idx_artifacts_node_id": { + "name": "idx_artifacts_node_id", + "columns": [ + { + "expression": "node_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_artifacts_created_by_account_id": { + "name": "idx_artifacts_created_by_account_id", + "columns": [ + { + "expression": "created_by_account_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_artifacts_kind_version_status": { + "name": "idx_artifacts_kind_version_status", + "columns": [ + { + "expression": "artifact_kind", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "artifact_version", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_artifacts_kind_source_hash": { + "name": "idx_artifacts_kind_source_hash", + "columns": [ + { + "expression": "artifact_kind", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "source_hash", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "artifacts_node_id_nodes_id_fk": { + "name": "artifacts_node_id_nodes_id_fk", + "tableFrom": "artifacts", + "tableTo": "nodes", + "schemaTo": "kg", + "columnsFrom": [ + "node_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + }, + "artifacts_created_by_account_id_accounts_id_fk": { + "name": "artifacts_created_by_account_id_accounts_id_fk", + "tableFrom": "artifacts", + "tableTo": "accounts", + "schemaTo": "kg", + "columnsFrom": [ + "created_by_account_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.events": { + "name": "events", + "schema": "kg", + "columns": { + "event_time": { + "name": "event_time", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "id": { + "name": "id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "actor_id": { + "name": "actor_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "entity_kind": { + "name": "entity_kind", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "entity_id": { + "name": "entity_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "event_type": { + "name": "event_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "classification_type": { + "name": "classification_type", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "is_onchain": { + "name": "is_onchain", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "block_number": { + "name": "block_number", + "type": "bigint", + "primaryKey": false, + "notNull": false + }, + "tx_hash": { + "name": "tx_hash", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "payload": { + "name": "payload", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "idx_kg_events_entity": { + "name": "idx_kg_events_entity", + "columns": [ + { + "expression": "entity_kind", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "entity_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "event_time", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_kg_events_actor": { + "name": "idx_kg_events_actor", + "columns": [ + { + "expression": "actor_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "event_time", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_kg_events_type": { + "name": "idx_kg_events_type", + "columns": [ + { + "expression": "event_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "event_time", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": {}, + "compositePrimaryKeys": { + "kg_events_pkey": { + "name": "kg_events_pkey", + "columns": [ + "event_time", + "id" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_kg_events_entity_kind": { + "name": "chk_kg_events_entity_kind", + "value": "\"kg\".\"events\".\"entity_kind\" IN ('node', 'triple', 'predicate', 'artifact')" + } + }, + "isRLSEnabled": false + }, + "kg.node_contexts": { + "name": "node_contexts", + "schema": "kg", + "columns": { + "node_id": { + "name": "node_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "event_sequence": { + "name": "event_sequence", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "block_number": { + "name": "block_number", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "block_timestamp": { + "name": "block_timestamp", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true + }, + "block_hash": { + "name": "block_hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "transaction_hash": { + "name": "transaction_hash", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "log_index": { + "name": "log_index", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "ordinal": { + "name": "ordinal", + "type": "integer", + "primaryKey": false, + "notNull": true + }, + "registrant": { + "name": "registrant", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "uri_hex": { + "name": "uri_hex", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "uri_text": { + "name": "uri_text", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "idx_node_contexts_event_ordinal": { + "name": "idx_node_contexts_event_ordinal", + "columns": [ + { + "expression": "event_sequence", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "ordinal", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_node_contexts_node_sequence": { + "name": "idx_node_contexts_node_sequence", + "columns": [ + { + "expression": "node_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "event_sequence", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "ordinal", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_node_contexts_transaction": { + "name": "idx_node_contexts_transaction", + "columns": [ + { + "expression": "transaction_hash", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "log_index", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "node_contexts_node_id_nodes_id_fk": { + "name": "node_contexts_node_id_nodes_id_fk", + "tableFrom": "node_contexts", + "tableTo": "nodes", + "schemaTo": "kg", + "columnsFrom": [ + "node_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "node_contexts_pkey": { + "name": "node_contexts_pkey", + "columns": [ + "node_id", + "transaction_hash", + "log_index", + "ordinal" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_node_contexts_event_sequence": { + "name": "chk_node_contexts_event_sequence", + "value": "\"kg\".\"node_contexts\".\"event_sequence\" >= 0" + }, + "chk_node_contexts_block_number": { + "name": "chk_node_contexts_block_number", + "value": "\"kg\".\"node_contexts\".\"block_number\" >= 0" + }, + "chk_node_contexts_log_index": { + "name": "chk_node_contexts_log_index", + "value": "\"kg\".\"node_contexts\".\"log_index\" >= 0" + }, + "chk_node_contexts_ordinal": { + "name": "chk_node_contexts_ordinal", + "value": "\"kg\".\"node_contexts\".\"ordinal\" >= 0" + }, + "chk_node_contexts_uri_hex": { + "name": "chk_node_contexts_uri_hex", + "value": "\"kg\".\"node_contexts\".\"uri_hex\" ~ '^0x([0-9a-f]{2})*$'" + } + }, + "isRLSEnabled": false + }, + "kg.node_urls": { + "name": "node_urls", + "schema": "kg", + "columns": { + "node_id": { + "name": "node_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "url": { + "name": "url", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "domain": { + "name": "domain", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "source": { + "name": "source", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "artifact_id": { + "name": "artifact_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "is_primary": { + "name": "is_primary", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": { + "idx_node_urls_node_domain": { + "name": "idx_node_urls_node_domain", + "columns": [ + { + "expression": "node_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "domain", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_node_urls_domain": { + "name": "idx_node_urls_domain", + "columns": [ + { + "expression": "domain", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_node_urls_one_primary_per_node": { + "name": "idx_node_urls_one_primary_per_node", + "columns": [ + { + "expression": "node_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": true, + "where": "\"kg\".\"node_urls\".\"is_primary\" = true", + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "node_urls_node_id_nodes_id_fk": { + "name": "node_urls_node_id_nodes_id_fk", + "tableFrom": "node_urls", + "tableTo": "nodes", + "schemaTo": "kg", + "columnsFrom": [ + "node_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "cascade", + "onUpdate": "no action" + }, + "node_urls_artifact_id_artifacts_id_fk": { + "name": "node_urls_artifact_id_artifacts_id_fk", + "tableFrom": "node_urls", + "tableTo": "artifacts", + "schemaTo": "kg", + "columnsFrom": [ + "artifact_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": { + "node_urls_pkey": { + "name": "node_urls_pkey", + "columns": [ + "node_id", + "url" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_node_urls_url_nonempty": { + "name": "chk_node_urls_url_nonempty", + "value": "\"kg\".\"node_urls\".\"url\" <> ''" + }, + "chk_node_urls_domain_nonempty": { + "name": "chk_node_urls_domain_nonempty", + "value": "\"kg\".\"node_urls\".\"domain\" <> ''" + } + }, + "isRLSEnabled": false + }, + "kg.node_stats": { + "name": "node_stats", + "schema": "kg", + "columns": { + "node_id": { + "name": "node_id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "in_degree": { + "name": "in_degree", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "out_degree": { + "name": "out_degree", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "neighbor_kind_counts": { + "name": "neighbor_kind_counts", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "predicate_counts": { + "name": "predicate_counts", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": { + "node_stats_node_id_nodes_id_fk": { + "name": "node_stats_node_id_nodes_id_fk", + "tableFrom": "node_stats", + "tableTo": "nodes", + "schemaTo": "kg", + "columnsFrom": [ + "node_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "no action", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.nodes": { + "name": "nodes", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "is_onchain": { + "name": "is_onchain", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "visibility": { + "name": "visibility", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'public'" + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "raw_type": { + "name": "raw_type", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "data": { + "name": "data", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "data_hex": { + "name": "data_hex", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "iid": { + "name": "iid", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "data_resolved": { + "name": "data_resolved", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "parse_attempts": { + "name": "parse_attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "parse_status": { + "name": "parse_status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "parse_started_at": { + "name": "parse_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "parse_lease_expires_at": { + "name": "parse_lease_expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "parsed_at": { + "name": "parsed_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "parse_error": { + "name": "parse_error", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "parse_result": { + "name": "parse_result", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "classification_attempts": { + "name": "classification_attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "classification_status": { + "name": "classification_status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "classification_started_at": { + "name": "classification_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "classification_lease_expires_at": { + "name": "classification_lease_expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "classified_at": { + "name": "classified_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "classification_error": { + "name": "classification_error", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "classification_result": { + "name": "classification_result", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "classification_type": { + "name": "classification_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'Unknown'" + }, + "enrichment_attempts": { + "name": "enrichment_attempts", + "type": "integer", + "primaryKey": false, + "notNull": true, + "default": 0 + }, + "enrichment_status": { + "name": "enrichment_status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'pending'" + }, + "enrichment_started_at": { + "name": "enrichment_started_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "enrichment_lease_expires_at": { + "name": "enrichment_lease_expires_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "enriched_at": { + "name": "enriched_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + }, + "enrichment_error": { + "name": "enrichment_error", + "type": "jsonb", + "primaryKey": false, + "notNull": false + }, + "processing_meta": { + "name": "processing_meta", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "search_text": { + "name": "search_text", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "''" + } + }, + "indexes": { + "idx_nodes_status_visibility_created_at": { + "name": "idx_nodes_status_visibility_created_at", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "visibility", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_visibility": { + "name": "idx_nodes_visibility", + "columns": [ + { + "expression": "visibility", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_raw_type_data_hex": { + "name": "idx_nodes_raw_type_data_hex", + "columns": [ + { + "expression": "raw_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "data_hex", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_classification_type": { + "name": "idx_nodes_classification_type", + "columns": [ + { + "expression": "classification_type", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_created_by_created_at": { + "name": "idx_nodes_created_by_created_at", + "columns": [ + { + "expression": "created_by", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_data_hex": { + "name": "idx_nodes_data_hex", + "columns": [ + { + "expression": "data_hex", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_iid": { + "name": "idx_nodes_iid", + "columns": [ + { + "expression": "iid", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "where": "\"kg\".\"nodes\".\"iid\" IS NOT NULL", + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_parse_recovery": { + "name": "idx_nodes_parse_recovery", + "columns": [ + { + "expression": "parse_status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "parse_lease_expires_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_classification_recovery": { + "name": "idx_nodes_classification_recovery", + "columns": [ + { + "expression": "classification_status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "classification_lease_expires_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_enrichment_recovery": { + "name": "idx_nodes_enrichment_recovery", + "columns": [ + { + "expression": "enrichment_status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "enrichment_lease_expires_at", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_nodes_processing_statuses": { + "name": "idx_nodes_processing_statuses", + "columns": [ + { + "expression": "parse_status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "classification_status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "enrichment_status", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "nodes_created_by_accounts_id_fk": { + "name": "nodes_created_by_accounts_id_fk", + "tableFrom": "nodes", + "tableTo": "accounts", + "schemaTo": "kg", + "columnsFrom": [ + "created_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_nodes_visibility": { + "name": "chk_nodes_visibility", + "value": "\"kg\".\"nodes\".\"visibility\" IN ('public', 'unlisted')" + }, + "chk_nodes_status": { + "name": "chk_nodes_status", + "value": "\"kg\".\"nodes\".\"status\" IN ('active', 'draft')" + }, + "chk_nodes_raw_type": { + "name": "chk_nodes_raw_type", + "value": "\"kg\".\"nodes\".\"raw_type\" IN ('string', 'json', 'http_uri', 'ipfs_uri', 'iid')" + }, + "chk_nodes_parse_status": { + "name": "chk_nodes_parse_status", + "value": "\"kg\".\"nodes\".\"parse_status\" IN ('pending', 'processing', 'completed', 'failed', 'skipped')" + }, + "chk_nodes_classification_status": { + "name": "chk_nodes_classification_status", + "value": "\"kg\".\"nodes\".\"classification_status\" IN ('pending', 'processing', 'completed', 'failed', 'skipped')" + }, + "chk_nodes_enrichment_status": { + "name": "chk_nodes_enrichment_status", + "value": "\"kg\".\"nodes\".\"enrichment_status\" IN ('pending', 'processing', 'completed', 'failed', 'skipped')" + } + }, + "isRLSEnabled": false + }, + "kg.predicate_stats": { + "name": "predicate_stats", + "schema": "kg", + "columns": { + "predicate_id": { + "name": "predicate_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_type": { + "name": "predicate_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "triple_count": { + "name": "triple_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "distinct_subject_count": { + "name": "distinct_subject_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "distinct_object_count": { + "name": "distinct_object_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "avg_out_degree": { + "name": "avg_out_degree", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "avg_in_degree": { + "name": "avg_in_degree", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "selectivity_score": { + "name": "selectivity_score", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": { + "predicate_stats_pkey": { + "name": "predicate_stats_pkey", + "columns": [ + "predicate_type", + "predicate_id" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_predicate_stats_predicate_type": { + "name": "chk_predicate_stats_predicate_type", + "value": "\"kg\".\"predicate_stats\".\"predicate_type\" IN ('node', 'triple')" + } + }, + "isRLSEnabled": false + }, + "kg.predicates": { + "name": "predicates", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "slug": { + "name": "slug", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "label": { + "name": "label", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "description": { + "name": "description", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "inverse_predicate_id": { + "name": "inverse_predicate_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "is_transitive": { + "name": "is_transitive", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_symmetric": { + "name": "is_symmetric", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_hierarchical": { + "name": "is_hierarchical", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_social": { + "name": "is_social", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "is_market": { + "name": "is_market", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": {}, + "uniqueConstraints": { + "predicates_slug_unique": { + "name": "predicates_slug_unique", + "nullsNotDistinct": false, + "columns": [ + "slug" + ] + } + }, + "policies": {}, + "checkConstraints": {}, + "isRLSEnabled": false + }, + "kg.triple_pattern_stats": { + "name": "triple_pattern_stats", + "schema": "kg", + "columns": { + "subject_kind": { + "name": "subject_kind", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_id": { + "name": "predicate_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_type": { + "name": "predicate_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "object_kind": { + "name": "object_kind", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "triple_count": { + "name": "triple_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "distinct_subject_count": { + "name": "distinct_subject_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "distinct_object_count": { + "name": "distinct_object_count", + "type": "bigint", + "primaryKey": false, + "notNull": true + }, + "selectivity_score": { + "name": "selectivity_score", + "type": "numeric", + "primaryKey": false, + "notNull": false + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": false + } + }, + "indexes": {}, + "foreignKeys": {}, + "compositePrimaryKeys": { + "triple_pattern_stats_pkey": { + "name": "triple_pattern_stats_pkey", + "columns": [ + "subject_kind", + "predicate_type", + "predicate_id", + "object_kind" + ] + } + }, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_triple_pattern_stats_predicate_type": { + "name": "chk_triple_pattern_stats_predicate_type", + "value": "\"kg\".\"triple_pattern_stats\".\"predicate_type\" IN ('node', 'triple')" + } + }, + "isRLSEnabled": false + }, + "kg.triples": { + "name": "triples", + "schema": "kg", + "columns": { + "id": { + "name": "id", + "type": "text", + "primaryKey": true, + "notNull": true + }, + "created_at": { + "name": "created_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "updated_at": { + "name": "updated_at", + "type": "timestamp with time zone", + "primaryKey": false, + "notNull": true, + "default": "now()" + }, + "is_onchain": { + "name": "is_onchain", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "status": { + "name": "status", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'active'" + }, + "visibility": { + "name": "visibility", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'public'" + }, + "created_by": { + "name": "created_by", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "subject_id": { + "name": "subject_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "subject_type": { + "name": "subject_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "predicate_id": { + "name": "predicate_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "predicate_type": { + "name": "predicate_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "object_id": { + "name": "object_id", + "type": "text", + "primaryKey": false, + "notNull": true + }, + "object_type": { + "name": "object_type", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'node'" + }, + "is_counter_triple": { + "name": "is_counter_triple", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "sibling_triple_id": { + "name": "sibling_triple_id", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "edge_kind": { + "name": "edge_kind", + "type": "text", + "primaryKey": false, + "notNull": true, + "default": "'claim'" + }, + "source": { + "name": "source", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "source_uri": { + "name": "source_uri", + "type": "text", + "primaryKey": false, + "notNull": false + }, + "confidence": { + "name": "confidence", + "type": "numeric(6, 5)", + "primaryKey": false, + "notNull": false + }, + "inferred": { + "name": "inferred", + "type": "boolean", + "primaryKey": false, + "notNull": true, + "default": false + }, + "provenance": { + "name": "provenance", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + }, + "metadata": { + "name": "metadata", + "type": "jsonb", + "primaryKey": false, + "notNull": true, + "default": "'{}'::jsonb" + } + }, + "indexes": { + "idx_triples_spo": { + "name": "idx_triples_spo", + "columns": [ + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_sop": { + "name": "idx_triples_sop", + "columns": [ + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_pso": { + "name": "idx_triples_pso", + "columns": [ + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_pos": { + "name": "idx_triples_pos", + "columns": [ + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_osp": { + "name": "idx_triples_osp", + "columns": [ + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_ops": { + "name": "idx_triples_ops", + "columns": [ + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_subject_ref": { + "name": "idx_triples_subject_ref", + "columns": [ + { + "expression": "subject_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "subject_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_predicate_ref": { + "name": "idx_triples_predicate_ref", + "columns": [ + { + "expression": "predicate_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "predicate_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_object_ref": { + "name": "idx_triples_object_ref", + "columns": [ + { + "expression": "object_type", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "object_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_status_visibility_created_at": { + "name": "idx_triples_status_visibility_created_at", + "columns": [ + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "visibility", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_sibling_triple_id": { + "name": "idx_triples_sibling_triple_id", + "columns": [ + { + "expression": "sibling_triple_id", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_counter_triple": { + "name": "idx_triples_counter_triple", + "columns": [ + { + "expression": "is_counter_triple", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_created_by_created_at": { + "name": "idx_triples_created_by_created_at", + "columns": [ + { + "expression": "created_by", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_edge_kind_status_created_at": { + "name": "idx_triples_edge_kind_status_created_at", + "columns": [ + { + "expression": "edge_kind", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "status", + "isExpression": false, + "asc": true, + "nulls": "last" + }, + { + "expression": "created_at", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_source_uri": { + "name": "idx_triples_source_uri", + "columns": [ + { + "expression": "source_uri", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + }, + "idx_triples_confidence_desc": { + "name": "idx_triples_confidence_desc", + "columns": [ + { + "expression": "confidence", + "isExpression": false, + "asc": true, + "nulls": "last" + } + ], + "isUnique": false, + "concurrently": false, + "method": "btree", + "with": {} + } + }, + "foreignKeys": { + "triples_created_by_accounts_id_fk": { + "name": "triples_created_by_accounts_id_fk", + "tableFrom": "triples", + "tableTo": "accounts", + "schemaTo": "kg", + "columnsFrom": [ + "created_by" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + }, + "triples_sibling_triple_id_triples_id_fk": { + "name": "triples_sibling_triple_id_triples_id_fk", + "tableFrom": "triples", + "tableTo": "triples", + "schemaTo": "kg", + "columnsFrom": [ + "sibling_triple_id" + ], + "columnsTo": [ + "id" + ], + "onDelete": "set null", + "onUpdate": "no action" + } + }, + "compositePrimaryKeys": {}, + "uniqueConstraints": {}, + "policies": {}, + "checkConstraints": { + "chk_triples_visibility": { + "name": "chk_triples_visibility", + "value": "\"kg\".\"triples\".\"visibility\" IN ('public', 'unlisted')" + }, + "chk_triples_status": { + "name": "chk_triples_status", + "value": "\"kg\".\"triples\".\"status\" IN ('active', 'draft')" + }, + "chk_triples_subject_type": { + "name": "chk_triples_subject_type", + "value": "\"kg\".\"triples\".\"subject_type\" IN ('node', 'triple')" + }, + "chk_triples_predicate_type": { + "name": "chk_triples_predicate_type", + "value": "\"kg\".\"triples\".\"predicate_type\" IN ('node', 'triple')" + }, + "chk_triples_object_type": { + "name": "chk_triples_object_type", + "value": "\"kg\".\"triples\".\"object_type\" IN ('node', 'triple')" + }, + "chk_triples_counter_sibling_required": { + "name": "chk_triples_counter_sibling_required", + "value": "\"kg\".\"triples\".\"is_counter_triple\" = false OR \"kg\".\"triples\".\"sibling_triple_id\" IS NOT NULL" + }, + "chk_triples_sibling_not_self": { + "name": "chk_triples_sibling_not_self", + "value": "\"kg\".\"triples\".\"sibling_triple_id\" IS NULL OR \"kg\".\"triples\".\"sibling_triple_id\" <> \"kg\".\"triples\".\"id\"" + }, + "chk_triples_confidence_range": { + "name": "chk_triples_confidence_range", + "value": "\"kg\".\"triples\".\"confidence\" IS NULL OR (\"kg\".\"triples\".\"confidence\" >= 0 AND \"kg\".\"triples\".\"confidence\" <= 1)" + } + }, + "isRLSEnabled": false + } + }, + "enums": {}, + "schemas": { + "kg": "kg" + }, + "sequences": {}, + "roles": {}, + "policies": {}, + "views": {}, + "_meta": { + "columns": {}, + "schemas": {}, + "tables": {} + } +} \ No newline at end of file diff --git a/packages/database-kg/drizzle/meta/_journal.json b/packages/database-kg/drizzle/meta/_journal.json index 45819a1..2eb38f0 100644 --- a/packages/database-kg/drizzle/meta/_journal.json +++ b/packages/database-kg/drizzle/meta/_journal.json @@ -22,6 +22,20 @@ "when": 1783020952014, "tag": "0002_api_key_rate_limit", "breakpoints": true + }, + { + "idx": 3, + "version": "7", + "when": 1786389108899, + "tag": "0003_silky_valkyrie", + "breakpoints": true + }, + { + "idx": 4, + "version": "7", + "when": 1786391719465, + "tag": "0004_add-node-contexts", + "breakpoints": true } ] } \ No newline at end of file diff --git a/packages/database-kg/package.json b/packages/database-kg/package.json index ebde422..3ade2a2 100644 --- a/packages/database-kg/package.json +++ b/packages/database-kg/package.json @@ -19,6 +19,7 @@ "db:generate": "bun with-env drizzle-kit generate", "db:migrate": "bun with-env bun src/migrate.ts", "db:push": "bun with-env drizzle-kit push", + "test": "bun test src", "ci": "biome check", "lint": "biome lint", "format": "biome format", @@ -26,6 +27,7 @@ "check:write": "biome check --write" }, "dependencies": { + "@0xintuition/ids": "0.1.0-alpha.0", "drizzle-orm": "0.45.1", "postgres": "^3.4.7", "viem": "^2.23.2" diff --git a/packages/database-kg/src/actions/ids.test.ts b/packages/database-kg/src/actions/ids.test.ts index a78a24b..abb85fc 100644 --- a/packages/database-kg/src/actions/ids.test.ts +++ b/packages/database-kg/src/actions/ids.test.ts @@ -1,6 +1,11 @@ import { describe, expect, test } from 'bun:test'; import { kgAtomId, kgTripleId } from './ids'; +const SUBJECT_ID = '0x05bb6d28ed5ca3c5206f33f5818da27b3b0bbf6401cd40f082e8db7fcf481787'; +const PREDICATE_ID = '0xdb3dc8c92d6141c4e0c9b453b00fc1f237624ef8373b6ae9972d09557d8aaa8d'; +const OBJECT_ID = '0x39afce29ac0e4be2400fa0421b537f63ad2d78d7f8b4be4ff839a162ff3e5ffc'; +const TRIPLE_ID = '0x57946a02776dbd4eec339ecf5cdf6e0005b8de381fb3d9a2bf303da083bf5166'; + describe('deterministic protocol term ids', () => { test('kgTripleId matches the @0xintuition/ids known answer', () => { // Parity lock: this exact vector is documented in @0xintuition/ids. @@ -9,24 +14,52 @@ describe('deterministic protocol term ids', () => { kgTripleId({ subject: { type: 'node', - id: '0x05bb6d28ed5ca3c5206f33f5818da27b3b0bbf6401cd40f082e8db7fcf481787', + id: SUBJECT_ID, }, predicate: { type: 'node', - id: '0xdb3dc8c92d6141c4e0c9b453b00fc1f237624ef8373b6ae9972d09557d8aaa8d', + id: PREDICATE_ID, }, object: { type: 'node', - id: '0x39afce29ac0e4be2400fa0421b537f63ad2d78d7f8b4be4ff839a162ff3e5ffc', + id: OBJECT_ID, }, }) - ).toBe('0x57946a02776dbd4eec339ecf5cdf6e0005b8de381fb3d9a2bf303da083bf5166'); + ).toBe(TRIPLE_ID); }); - test('kgAtomId is deterministic and 32 bytes', () => { - const a = kgAtomId('https://example.com'); - expect(a).toBe(kgAtomId('https://example.com')); - expect(a).toMatch(/^0x[0-9a-f]{64}$/); - expect(kgAtomId('something else')).not.toBe(a); + test('preserves Core prefix normalization before public triple derivation', () => { + expect( + kgTripleId({ + subject: { type: 'node', id: ` atom:${SUBJECT_ID} ` }, + predicate: { type: 'node', id: `triple:${PREDICATE_ID}` }, + object: { type: 'node', id: OBJECT_ID }, + }) + ).toBe(TRIPLE_ID); + }); + + test('locks public atom derivation for UTF-8, raw hex, IID, and legacy JSON inputs', () => { + const fixtures = [ + { + input: 'hello', + expected: '0xa0e157e5fa1b17d3b54ec73622ce3317296920a06502661617613d59f58e947e', + }, + { + input: '0x68656c6c6f', + expected: '0xa0e157e5fa1b17d3b54ec73622ce3317296920a06502661617613d59f58e947e', + }, + { + input: 'int:isrc:USRC17607839', + expected: '0xd3368a8190d3afd5fb05abd01db8141c09a77c3f290bb7a0ef2ffb2b5acab8d8', + }, + { + input: '{"@context":"https://schema.org","@type":"MusicRecording","name":"One Last Time"}', + expected: '0x9500b299bfb982c6c11574351aef0b12ffa82f1856c141a2d090b071a9714349', + }, + ] as const; + + for (const fixture of fixtures) { + expect(kgAtomId(fixture.input)).toBe(fixture.expected); + } }); }); diff --git a/packages/database-kg/src/actions/ids.ts b/packages/database-kg/src/actions/ids.ts index c9e9f8d..608c591 100644 --- a/packages/database-kg/src/actions/ids.ts +++ b/packages/database-kg/src/actions/ids.ts @@ -1,33 +1,16 @@ -import { encodePacked, type Hex, isHex, keccak256, toHex } from 'viem'; +import { calculateAtomId, calculateTripleId } from '@0xintuition/ids'; import type { TripleInput } from './types'; const PROTOCOL_TERM_ID_RE = /^0x[0-9a-fA-F]{64}$/; -/** - * Protocol-defined salt for atom ID derivation: `keccak256(toHex('ATOM_SALT'))`. - */ -const ATOM_SALT: Hex = keccak256(toHex('ATOM_SALT')); - -/** - * Protocol-defined salt for triple ID derivation: `keccak256(toHex('TRIPLE_SALT'))`. - * Value: `0x23ad11f0a1505378b82984192ad0461e6a012820fc5bf2e4ba16513f8e430552`. - */ -const TRIPLE_SALT: Hex = keccak256(toHex('TRIPLE_SALT')); - /** * Compute a deterministic atom ID from raw atom data. * - * The algorithm mirrors the on-chain derivation: - * 1. Convert `atomData` to hex if it is a plain string. - * 2. Hash the hex data with keccak256. - * 3. Pack `[ATOM_SALT, keccak256(data)]` and hash again. - * - * The result is deterministic: identical atom data always produces the same - * ID regardless of caller or timestamp. + * Core intentionally preserves the public helper's boundary: Viem-recognized + * `0x...` values are raw bytes and every other string is UTF-8 atom data. */ export function kgAtomId(atomData: string): string { - const data: Hex = isHex(atomData) ? atomData : toHex(atomData); - return keccak256(encodePacked(['bytes32', 'bytes'], [ATOM_SALT, keccak256(data)])); + return calculateAtomId(atomData); } /** @@ -43,16 +26,10 @@ export function kgAtomId(atomData: string): string { * → triple 0x57946a02776dbd4eec339ecf5cdf6e0005b8de381fb3d9a2bf303da083bf5166 */ export function kgTripleId(input: TripleInput): string { - return keccak256( - encodePacked( - ['bytes32', 'bytes32', 'bytes32', 'bytes32'], - [ - TRIPLE_SALT, - normalizeProtocolTermId(input.subject.id, 'subject.id'), - normalizeProtocolTermId(input.predicate.id, 'predicate.id'), - normalizeProtocolTermId(input.object.id, 'object.id'), - ] - ) + return calculateTripleId( + normalizeProtocolTermId(input.subject.id, 'subject.id'), + normalizeProtocolTermId(input.predicate.id, 'predicate.id'), + normalizeProtocolTermId(input.object.id, 'object.id') ); } diff --git a/packages/database-kg/src/actions/iid-reconciliation.test.ts b/packages/database-kg/src/actions/iid-reconciliation.test.ts new file mode 100644 index 0000000..2ef14eb --- /dev/null +++ b/packages/database-kg/src/actions/iid-reconciliation.test.ts @@ -0,0 +1,53 @@ +import { describe, expect, test } from 'bun:test'; +import { PgDialect } from 'drizzle-orm/pg-core'; + +import { listIidReconciliationCandidates } from './iid-reconciliation'; +import type { KgActionDb } from './types'; + +describe('IID reconciliation candidate reads', () => { + test('uses a stable ID cursor and bounded IID-or-prefix selection', async () => { + let whereSql: { getSQL(): unknown } | undefined; + let orderSql: { getSQL(): unknown } | undefined; + let limitValue: number | undefined; + const db = { + select() { + return { + from() { + return { + where(value: { getSQL(): unknown }) { + whereSql = value; + return { + orderBy(order: { getSQL(): unknown }) { + orderSql = order; + return { + limit(limit: number) { + limitValue = limit; + return Promise.resolve([]); + }, + }; + }, + }; + }, + }; + }, + }; + }, + } as unknown as KgActionDb; + + await listIidReconciliationCandidates(db, { limit: 100, after: '0xabc' }); + const dialect = new PgDialect(); + const predicate = dialect.sqlToQuery(whereSql?.getSQL() as never); + expect(predicate.sql).toContain('"kg"."nodes"."iid" is not null'); + expect(predicate.sql).toContain('"kg"."nodes"."data" ilike $1'); + expect(predicate.sql).toContain('"kg"."nodes"."id" > $2'); + expect(predicate.params).toEqual(['int:%', '0xabc']); + expect(dialect.sqlToQuery(orderSql?.getSQL() as never).sql).toBe('"kg"."nodes"."id" asc'); + expect(limitValue).toBe(100); + }); + + test('rejects unbounded pages', async () => { + await expect( + listIidReconciliationCandidates({} as KgActionDb, { limit: 1_001 }) + ).rejects.toThrow('between 1 and 1000'); + }); +}); diff --git a/packages/database-kg/src/actions/iid-reconciliation.ts b/packages/database-kg/src/actions/iid-reconciliation.ts new file mode 100644 index 0000000..5438daf --- /dev/null +++ b/packages/database-kg/src/actions/iid-reconciliation.ts @@ -0,0 +1,47 @@ +import { and, asc, gt, ilike, isNotNull, or } from 'drizzle-orm'; + +import { nodes } from '../schema'; +import { invalidInput } from './errors'; +import type { KgActionDb } from './types'; + +export type IidReconciliationCandidate = { + id: string; + data: string | null; + iid: string | null; + parseStatus: string; + classificationStatus: string; + enrichmentStatus: string; +}; + +/** + * Lists a deterministic, bounded page of nodes that are either already + * promoted to an IID cluster or look like historical `int:` input. The public + * IID adapter remains authoritative; this query never classifies by prefix. + */ +export async function listIidReconciliationCandidates( + db: KgActionDb, + input: { limit: number; after?: string } +): Promise { + if (!Number.isInteger(input.limit) || input.limit < 1 || input.limit > 1_000) { + throw invalidInput('IID reconciliation limit must be an integer between 1 and 1000.'); + } + + return db + .select({ + id: nodes.id, + data: nodes.data, + iid: nodes.iid, + parseStatus: nodes.parseStatus, + classificationStatus: nodes.classificationStatus, + enrichmentStatus: nodes.enrichmentStatus, + }) + .from(nodes) + .where( + and( + or(isNotNull(nodes.iid), ilike(nodes.data, 'int:%')), + ...(input.after ? [gt(nodes.id, input.after)] : []) + ) + ) + .orderBy(asc(nodes.id)) + .limit(input.limit); +} diff --git a/packages/database-kg/src/actions/index.ts b/packages/database-kg/src/actions/index.ts index b7c40de..1eb96ca 100644 --- a/packages/database-kg/src/actions/index.ts +++ b/packages/database-kg/src/actions/index.ts @@ -9,6 +9,7 @@ export * from './artifacts'; export * from './errors'; export * from './ids'; +export * from './iid-reconciliation'; export * from './nodes'; export * from './processing'; export * from './triples'; diff --git a/packages/database-kg/src/actions/node-reads.test.ts b/packages/database-kg/src/actions/node-reads.test.ts new file mode 100644 index 0000000..3978777 --- /dev/null +++ b/packages/database-kg/src/actions/node-reads.test.ts @@ -0,0 +1,90 @@ +import { describe, expect, test } from 'bun:test'; +import { PgDialect } from 'drizzle-orm/pg-core'; + +import { listNodeContexts, listPublicNodesByIid } from './nodes'; +import type { KgActionDb } from './types'; + +type Capture = { + selection?: Record; + where?: { getSQL(): unknown }; + orderBy?: Array<{ getSQL(): unknown }>; + limit?: number; + offset?: number; +}; + +function captureSelect(rows: unknown[] = []) { + const capture: Capture = {}; + const db = { + select(selection: Record) { + capture.selection = selection; + return { + from() { + return { + where(where: Capture['where']) { + capture.where = where; + return { + orderBy(...orderBy: NonNullable) { + capture.orderBy = orderBy; + return { + limit(limit: number) { + capture.limit = limit; + return { + offset(offset: number) { + capture.offset = offset; + return { + async execute() { + return rows; + }, + }; + }, + }; + }, + async execute() { + return rows; + }, + }; + }, + }; + }, + }; + }, + }; + }, + } as unknown as KgActionDb; + return { capture, db }; +} + +function sqlOf(fragment: { getSQL(): unknown } | undefined) { + if (!fragment) throw new Error('missing captured SQL fragment'); + return new PgDialect().sqlToQuery(fragment.getSQL() as never); +} + +describe('semantic atom reader query shapes', () => { + test('reads context by node and orders by event sequence then ordinal', async () => { + const { capture, db } = captureSelect(); + await listNodeContexts(db, '0xatom'); + + expect(sqlOf(capture.where)).toMatchObject({ + sql: '"kg"."node_contexts"."node_id" = $1', + params: ['0xatom'], + }); + expect(capture.orderBy?.map((entry) => sqlOf(entry).sql)).toEqual([ + '"kg"."node_contexts"."event_sequence" asc', + '"kg"."node_contexts"."ordinal" asc', + ]); + }); + + test('uses exact IID equality with public filters and bounded pagination', async () => { + const { capture, db } = captureSelect(); + await listPublicNodesByIid(db, 'int:isrc:USQX91300108', { limit: 25, offset: 50 }); + + const predicate = sqlOf(capture.where); + expect(predicate.sql).toBe( + '("kg"."nodes"."iid" = $1 and "kg"."nodes"."status" = $2 and "kg"."nodes"."visibility" = $3)' + ); + expect(predicate.params).toEqual(['int:isrc:USQX91300108', 'active', 'public']); + expect(capture.limit).toBe(25); + expect(capture.offset).toBe(50); + expect(Object.keys(capture.selection ?? {})).not.toContain('context'); + }); +}); diff --git a/packages/database-kg/src/actions/nodes.test.ts b/packages/database-kg/src/actions/nodes.test.ts new file mode 100644 index 0000000..941f494 --- /dev/null +++ b/packages/database-kg/src/actions/nodes.test.ts @@ -0,0 +1,54 @@ +import { describe, expect, test } from 'bun:test'; + +import { ensureNodeWithCreation } from './nodes'; +import type { EnsureNodeInput, KgActionDb } from './types'; + +function captureNodeInsert() { + let inserted: Record | undefined; + const db = { + insert() { + return { + values(value: Record) { + inserted = value; + return { + onConflictDoNothing() { + return { + async returning() { + return [{ id: value.id }]; + }, + }; + }, + }; + }, + }; + }, + } as unknown as KgActionDb; + + return { db, getInserted: () => inserted }; +} + +function nodeInput(overrides: Partial = {}): EnsureNodeInput { + return { + id: `0x${'44'.repeat(32)}`, + rawType: 'string', + classificationType: 'Unknown', + data: 'opaque atom input', + ...overrides, + }; +} + +describe('node search defaults', () => { + test('does not use opaque input or a raw IID as implicit search text', async () => { + for (const input of [nodeInput(), nodeInput({ rawType: 'iid', data: 'opaque IID value' })]) { + const capture = captureNodeInsert(); + await ensureNodeWithCreation(capture.db, input); + expect(capture.getInserted()?.searchText).toBe(''); + } + }); + + test('preserves an explicit search projection', async () => { + const capture = captureNodeInsert(); + await ensureNodeWithCreation(capture.db, nodeInput({ searchText: 'Resolved display value' })); + expect(capture.getInserted()?.searchText).toBe('Resolved display value'); + }); +}); diff --git a/packages/database-kg/src/actions/nodes.ts b/packages/database-kg/src/actions/nodes.ts index 2c1bf20..abfd490 100644 --- a/packages/database-kg/src/actions/nodes.ts +++ b/packages/database-kg/src/actions/nodes.ts @@ -1,4 +1,6 @@ -import { accounts, nodes } from '../schema'; +import { and, asc, desc, eq } from 'drizzle-orm'; + +import { accounts, nodeContexts, nodes } from '../schema'; import { invalidInput } from './errors'; import { kgAtomId, normalizeProtocolTermId } from './ids'; import type { EnsureNodeInput, KgActionDb } from './types'; @@ -52,7 +54,10 @@ export async function ensureNodeWithCreation( data: input.data, dataHex: input.dataHex, dataResolved: input.dataResolved ?? {}, - searchText: input.searchText ?? input.data ?? input.id, + // Search text is a presentation projection, not a fallback copy of the + // atom's opaque/content-addressed input. Callers that understand the + // value must opt in; parse/enrichment workers promote it later. + searchText: input.searchText ?? '', createdBy: input.createdBy, }) .onConflictDoNothing() @@ -66,6 +71,63 @@ export async function ensureNode(db: KgActionDb, input: EnsureNodeInput): Promis return nodeId; } +/** Read immutable on-chain context in contract event/array order. */ +export async function listNodeContexts(db: KgActionDb, nodeId: string) { + return db + .select({ + nodeId: nodeContexts.nodeId, + eventSequence: nodeContexts.eventSequence, + blockNumber: nodeContexts.blockNumber, + blockTimestamp: nodeContexts.blockTimestamp, + blockHash: nodeContexts.blockHash, + transactionHash: nodeContexts.transactionHash, + logIndex: nodeContexts.logIndex, + ordinal: nodeContexts.ordinal, + registrant: nodeContexts.registrant, + uriHex: nodeContexts.uriHex, + uriText: nodeContexts.uriText, + }) + .from(nodeContexts) + .where(eq(nodeContexts.nodeId, nodeId)) + .orderBy(asc(nodeContexts.eventSequence), asc(nodeContexts.ordinal)) + .execute(); +} + +/** + * Exact same-identity cluster read. The equality predicate is backed by + * `idx_nodes_iid`; no prefix parsing, normalization, or fuzzy search occurs. + */ +export async function listPublicNodesByIid( + db: KgActionDb, + iid: string, + options: { limit: number; offset: number } +) { + return db + .select({ + id: nodes.id, + createdAt: nodes.createdAt, + isOnchain: nodes.isOnchain, + rawType: nodes.rawType, + data: nodes.data, + iid: nodes.iid, + dataResolved: nodes.dataResolved, + parseResult: nodes.parseResult, + classificationType: nodes.classificationType, + parseStatus: nodes.parseStatus, + classificationStatus: nodes.classificationStatus, + classificationResult: nodes.classificationResult, + enrichmentStatus: nodes.enrichmentStatus, + enrichmentError: nodes.enrichmentError, + enrichedAt: nodes.enrichedAt, + }) + .from(nodes) + .where(and(eq(nodes.iid, iid), eq(nodes.status, 'active'), eq(nodes.visibility, 'public'))) + .orderBy(desc(nodes.createdAt), desc(nodes.id)) + .limit(options.limit) + .offset(options.offset) + .execute(); +} + function createNodeId(input: EnsureNodeInput): string { const atomData = input.dataHex ?? input.data; if (!atomData) { diff --git a/packages/database-kg/src/actions/processing.test.ts b/packages/database-kg/src/actions/processing.test.ts new file mode 100644 index 0000000..cf7d33a --- /dev/null +++ b/packages/database-kg/src/actions/processing.test.ts @@ -0,0 +1,177 @@ +import { describe, expect, test } from 'bun:test'; + +import { + completeNodeEnrichmentStageWithArtifacts, + completeNodeProcessingStage, +} from './processing'; +import type { KgActionDb } from './types'; + +type CapturedPatch = Record; + +function capturingDb() { + let patch: CapturedPatch | undefined; + const db = { + update() { + return { + set(value: CapturedPatch) { + patch = value; + return { + where() { + return { + async returning() { + return [{ id: `0x${'11'.repeat(32)}` }]; + }, + }; + }, + }; + }, + }; + }, + } as unknown as KgActionDb; + + return { db, getPatch: () => patch }; +} + +describe('node processing IID promotion', () => { + test('writes raw type and canonical IID in the guarded parse completion patch', async () => { + const capture = capturingDb(); + const iid = 'int:isrc:USQX91300108'; + + await completeNodeProcessingStage(capture.db, { + stage: 'parse', + nodeId: `0x${'11'.repeat(32)}`, + runId: 'parse-run', + data: { kind: 'iid' }, + promotedFields: { rawType: 'iid', iid }, + }); + + expect(capture.getPatch()).toMatchObject({ + parseStatus: 'completed', + parseResult: { kind: 'iid' }, + rawType: 'iid', + iid, + }); + }); + + test('does not clear IID fields when a legacy parse omits promotions', async () => { + const capture = capturingDb(); + + await completeNodeProcessingStage(capture.db, { + stage: 'parse', + nodeId: `0x${'22'.repeat(32)}`, + runId: 'legacy-parse-run', + data: { kind: 'plain_string' }, + }); + + const patch = capture.getPatch(); + expect(patch).toBeDefined(); + expect(Object.hasOwn(patch ?? {}, 'rawType')).toBe(false); + expect(Object.hasOwn(patch ?? {}, 'iid')).toBe(false); + }); +}); + +describe('enrichment completion transaction', () => { + test('persists artifacts, diagnostics, and promoted fields in one guarded transaction', async () => { + const nodeId = `0x${'33'.repeat(32)}`; + const events: Array<{ kind: string; value?: unknown }> = []; + let updateCount = 0; + const tx = { + update() { + updateCount += 1; + return { + set(value: CapturedPatch) { + events.push({ kind: updateCount === 1 ? 'lock' : 'complete', value }); + return { + where() { + return { + async returning() { + return [{ id: nodeId }]; + }, + }; + }, + }; + }, + }; + }, + select() { + return { + from() { + return { + where() { + return { + async limit() { + return []; + }, + }; + }, + }; + }, + }; + }, + insert() { + return { + values(value: unknown) { + events.push({ kind: 'artifact', value }); + return { + async onConflictDoUpdate() {}, + }; + }, + }; + }, + } as unknown as KgActionDb; + const db = { + async transaction(run: (transaction: KgActionDb) => Promise): Promise { + events.push({ kind: 'begin' }); + const result = await run(tx); + events.push({ kind: 'commit' }); + return result; + }, + } as unknown as KgActionDb; + const errors = [ + { + pluginId: 'secondary-provider', + code: 'upstream_error', + message: 'partial failure', + retriable: true, + }, + ]; + const skipped = [{ pluginId: 'unused-provider', reason: 'not applicable' }]; + + await completeNodeEnrichmentStageWithArtifacts(db, { + nodeId, + runId: 'enrichment-run', + artifactVersion: 'v1', + artifacts: [ + { + artifactKind: 'opengraph', + data: { title: 'Resolved title' }, + meta: { provider: 'fixture' }, + }, + ], + errors, + skipped, + promotedFields: { + dataResolved: { name: 'Resolved title' }, + searchText: 'Resolved title', + }, + }); + + expect(events.map((event) => event.kind)).toEqual([ + 'begin', + 'lock', + 'artifact', + 'complete', + 'commit', + ]); + const artifact = events.find((event) => event.kind === 'artifact')?.value as { + data?: { errors?: unknown; skipped?: unknown }; + }; + expect(artifact.data?.errors).toEqual(errors); + expect(artifact.data?.skipped).toEqual(skipped); + expect(events.find((event) => event.kind === 'complete')?.value).toMatchObject({ + enrichmentStatus: 'completed', + dataResolved: { name: 'Resolved title' }, + searchText: 'Resolved title', + }); + }); +}); diff --git a/packages/database-kg/src/actions/processing.ts b/packages/database-kg/src/actions/processing.ts index 6c00d0c..433d9e4 100644 --- a/packages/database-kg/src/actions/processing.ts +++ b/packages/database-kg/src/actions/processing.ts @@ -4,7 +4,7 @@ import { nodes } from '../schema'; import { createArtifacts, hashArtifactPayload } from './artifacts'; import { invalidInput, notFound } from './errors'; import { normalizeProtocolTermId } from './ids'; -import { inKgTransaction, type KgActionDb } from './types'; +import { inKgTransaction, type KgActionDb, type KgNodeRawType } from './types'; export type NodeProcessingStage = 'parse' | 'classification' | 'enrichment'; @@ -28,6 +28,10 @@ export type NodeProcessingPromotedFields = { dataResolved?: unknown; searchText?: string; classificationType?: string; + /** Refined storage lane; the parse worker may promote an indexed string to IID. */ + rawType?: KgNodeRawType; + /** Canonical IID cluster key. Invalid or non-IID inputs must leave this unset. */ + iid?: string; }; export type NodeProcessingPrerequisite = { @@ -254,6 +258,12 @@ export async function completeNodeProcessingStage( if (input.promotedFields?.classificationType !== undefined) { patch.classificationType = input.promotedFields.classificationType; } + if (input.promotedFields?.rawType !== undefined) { + patch.rawType = input.promotedFields.rawType; + } + if (input.promotedFields?.iid !== undefined) { + patch.iid = input.promotedFields.iid; + } const [node] = await db .update(nodes) @@ -763,6 +773,7 @@ export async function completeNodeEnrichmentStageWithArtifacts( targetUrl?: string | null; traceId?: string | null; artifacts: NodeEnrichmentArtifactInput[]; + promotedFields?: NodeProcessingPromotedFields; timings?: unknown; errors?: unknown; skipped?: unknown; @@ -779,6 +790,7 @@ export async function completeNodeEnrichmentStageWithArtifacts( stage: 'enrichment', nodeId: input.nodeId, runId: input.runId, + promotedFields: input.promotedFields, }); return { node, artifactIds }; diff --git a/packages/database-kg/src/actions/types.ts b/packages/database-kg/src/actions/types.ts index 662f807..5e09414 100644 --- a/packages/database-kg/src/actions/types.ts +++ b/packages/database-kg/src/actions/types.ts @@ -9,7 +9,7 @@ export type KgActionRef = { id: string; }; -export type KgNodeRawType = 'string' | 'json' | 'http_uri' | 'ipfs_uri'; +export type KgNodeRawType = 'string' | 'json' | 'http_uri' | 'ipfs_uri' | 'iid'; export type EnsureNodeInput = { id?: string; diff --git a/packages/database-kg/src/schemas/kg/index.ts b/packages/database-kg/src/schemas/kg/index.ts index 8c1d42e..0f440cb 100644 --- a/packages/database-kg/src/schemas/kg/index.ts +++ b/packages/database-kg/src/schemas/kg/index.ts @@ -3,6 +3,7 @@ export * from './adjacency'; export * from './api_keys'; export * from './artifacts'; export * from './events'; +export * from './node_contexts'; export * from './node_urls'; export * from './nodes'; export * from './predicates'; diff --git a/packages/database-kg/src/schemas/kg/node_contexts.test.ts b/packages/database-kg/src/schemas/kg/node_contexts.test.ts new file mode 100644 index 0000000..2c0ae97 --- /dev/null +++ b/packages/database-kg/src/schemas/kg/node_contexts.test.ts @@ -0,0 +1,43 @@ +import { describe, expect, test } from 'bun:test'; +import { getTableConfig, PgDialect } from 'drizzle-orm/pg-core'; + +import { nodeContexts } from './node_contexts'; + +describe('node contexts schema', () => { + test('uses immutable event-provenance and ordinal identity', () => { + const config = getTableConfig(nodeContexts); + const primaryKey = config.primaryKeys.find((key) => key.name === 'node_contexts_pkey'); + + expect(primaryKey?.columns.map((column) => column.name)).toEqual([ + 'node_id', + 'transaction_hash', + 'log_index', + 'ordinal', + ]); + expect(config.uniqueConstraints).toHaveLength(0); + expect( + config.indexes.find((index) => index.config.name === 'idx_node_contexts_event_ordinal') + ?.config.unique + ).toBe(true); + }); + + test('keeps canonical hex required and decoded text optional', () => { + const config = getTableConfig(nodeContexts); + const uriHex = config.columns.find((column) => column.name === 'uri_hex'); + const uriText = config.columns.find((column) => column.name === 'uri_text'); + const hexCheck = config.checks.find((check) => check.name === 'chk_node_contexts_uri_hex'); + const sql = hexCheck ? new PgDialect().sqlToQuery(hexCheck.value).sql : ''; + + expect(uriHex?.notNull).toBe(true); + expect(uriText?.notNull).toBe(false); + expect(sql).toContain('^0x([0-9a-f]{2})*$'); + }); + + test('references the canonical node id', () => { + const config = getTableConfig(nodeContexts); + const foreignKey = config.foreignKeys.at(0)?.reference(); + + expect(foreignKey?.columns.map((column) => column.name)).toEqual(['node_id']); + expect(foreignKey?.foreignColumns.map((column) => column.name)).toEqual(['id']); + }); +}); diff --git a/packages/database-kg/src/schemas/kg/node_contexts.ts b/packages/database-kg/src/schemas/kg/node_contexts.ts new file mode 100644 index 0000000..133214e --- /dev/null +++ b/packages/database-kg/src/schemas/kg/node_contexts.ts @@ -0,0 +1,62 @@ +import { relations, sql } from 'drizzle-orm'; +import { + bigint, + check, + index, + integer, + primaryKey, + text, + timestamp, + uniqueIndex, +} from 'drizzle-orm/pg-core'; + +import { nodes } from './nodes'; +import { kgSchema } from './schema'; + +/** + * Immutable, on-chain context bytes registered for an atom. + * + * URI values remain opaque here: `uri_hex` is the canonical byte-preserving + * representation and `uri_text` is only a best-effort UTF-8 rendering. Neither + * this table nor its writer fetches, normalizes, or assigns trust to a URI. + */ +export const nodeContexts = kgSchema.table( + 'node_contexts', + { + nodeId: text('node_id') + .notNull() + .references(() => nodes.id), + eventSequence: bigint('event_sequence', { mode: 'bigint' }).notNull(), + blockNumber: bigint('block_number', { mode: 'bigint' }).notNull(), + blockTimestamp: timestamp('block_timestamp', { withTimezone: true }).notNull(), + blockHash: text('block_hash').notNull(), + transactionHash: text('transaction_hash').notNull(), + logIndex: integer('log_index').notNull(), + ordinal: integer('ordinal').notNull(), + registrant: text('registrant').notNull(), + uriHex: text('uri_hex').notNull(), + uriText: text('uri_text'), + createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(), + }, + (t) => [ + primaryKey({ + columns: [t.nodeId, t.transactionHash, t.logIndex, t.ordinal], + name: 'node_contexts_pkey', + }), + uniqueIndex('idx_node_contexts_event_ordinal').on(t.eventSequence, t.ordinal), + index('idx_node_contexts_node_sequence').on(t.nodeId, t.eventSequence, t.ordinal), + index('idx_node_contexts_transaction').on(t.transactionHash, t.logIndex), + check('chk_node_contexts_event_sequence', sql`${t.eventSequence} >= 0`), + check('chk_node_contexts_block_number', sql`${t.blockNumber} >= 0`), + check('chk_node_contexts_log_index', sql`${t.logIndex} >= 0`), + check('chk_node_contexts_ordinal', sql`${t.ordinal} >= 0`), + check('chk_node_contexts_uri_hex', sql`${t.uriHex} ~ '^0x([0-9a-f]{2})*$'`), + ] +); + +export const nodeContextsRelations = relations(nodeContexts, ({ one }) => ({ + node: one(nodes, { + fields: [nodeContexts.nodeId], + references: [nodes.id], + }), +})); diff --git a/packages/database-kg/src/schemas/kg/nodes.test.ts b/packages/database-kg/src/schemas/kg/nodes.test.ts new file mode 100644 index 0000000..9fa04be --- /dev/null +++ b/packages/database-kg/src/schemas/kg/nodes.test.ts @@ -0,0 +1,33 @@ +import { describe, expect, test } from 'bun:test'; +import { getTableConfig, PgDialect } from 'drizzle-orm/pg-core'; + +import { nodes } from './nodes'; + +describe('nodes IID schema runway', () => { + test('stores a nullable, non-unique canonical IID cluster key', () => { + const config = getTableConfig(nodes); + const iid = config.columns.find((column) => column.name === 'iid'); + const index = config.indexes.find((candidate) => candidate.config.name === 'idx_nodes_iid'); + + expect(iid).toBeDefined(); + expect(iid?.notNull).toBe(false); + expect(iid?.isUnique).toBe(false); + expect(index?.config.unique).toBe(false); + expect( + index?.config.columns.map((column) => ('name' in column ? column.name : undefined)) + ).toEqual(['iid']); + expect(index?.config.where).toBeDefined(); + }); + + test('accepts IID as an additive raw type', () => { + const config = getTableConfig(nodes); + const constraint = config.checks.find((candidate) => candidate.name === 'chk_nodes_raw_type'); + const sql = constraint ? new PgDialect().sqlToQuery(constraint.value).sql : ''; + + expect(sql).toContain("'string'"); + expect(sql).toContain("'json'"); + expect(sql).toContain("'http_uri'"); + expect(sql).toContain("'ipfs_uri'"); + expect(sql).toContain("'iid'"); + }); +}); diff --git a/packages/database-kg/src/schemas/kg/nodes.ts b/packages/database-kg/src/schemas/kg/nodes.ts index 3554c43..466248f 100644 --- a/packages/database-kg/src/schemas/kg/nodes.ts +++ b/packages/database-kg/src/schemas/kg/nodes.ts @@ -23,9 +23,13 @@ export const nodes = kgSchema.table( status: text('status').notNull().default('active'), // Draft means nodes can't be added to triples, stacks, posts, etc... visibility: text('visibility').notNull().default('public'), // Unlisted means it's been flagged by the moderators. createdBy: text('created_by').references(() => accounts.id, { onDelete: 'set null' }), - rawType: text('raw_type').notNull(), // string | json | json-ld | http_uri | ipfs_uri + rawType: text('raw_type').notNull(), // string | json | http_uri | ipfs_uri | iid data: text('data'), dataHex: text('data_hex'), + // Canonical Intuition Identifier shared by an anchor and any richer + // representations of the same identity. Nullable for legacy/non-IID atoms + // and deliberately non-unique because multiple atoms may form one cluster. + iid: text('iid'), dataResolved: jsonb('data_resolved').notNull().default({}), // Parsing - first worker stage for raw node data parseAttempts: integer('parse_attempts').notNull().default(0), @@ -68,6 +72,7 @@ export const nodes = kgSchema.table( index('idx_nodes_classification_type').on(t.classificationType), index('idx_nodes_created_by_created_at').on(t.createdBy, t.createdAt), index('idx_nodes_data_hex').on(t.dataHex), + index('idx_nodes_iid').on(t.iid).where(sql`${t.iid} IS NOT NULL`), index('idx_nodes_parse_recovery').on(t.parseStatus, t.parseLeaseExpiresAt, t.createdAt), index('idx_nodes_classification_recovery').on( t.classificationStatus, @@ -86,7 +91,10 @@ export const nodes = kgSchema.table( ), check('chk_nodes_visibility', sql`${t.visibility} IN ('public', 'unlisted')`), check('chk_nodes_status', sql`${t.status} IN ('active', 'draft')`), - check('chk_nodes_raw_type', sql`${t.rawType} IN ('string', 'json', 'http_uri', 'ipfs_uri')`), + check( + 'chk_nodes_raw_type', + sql`${t.rawType} IN ('string', 'json', 'http_uri', 'ipfs_uri', 'iid')` + ), check( 'chk_nodes_parse_status', sql`${t.parseStatus} IN ('pending', 'processing', 'completed', 'failed', 'skipped')` diff --git a/packages/database-timescale/src/schema.ts b/packages/database-timescale/src/schema.ts index c32e404..748db17 100644 --- a/packages/database-timescale/src/schema.ts +++ b/packages/database-timescale/src/schema.ts @@ -3,6 +3,7 @@ import type { InferSelectViewModel } from 'drizzle-orm'; export * from './schemas/timescale'; export * from './timescale-wrappers'; +import type { atomContextRegisteredEvents } from './schemas/timescale/events'; import type { signal } from './schemas/timescale/signals'; import type { stats } from './schemas/timescale/stats'; import type { vault } from './schemas/timescale/vaults'; @@ -27,6 +28,7 @@ import type { export type VaultRow = typeof vault.$inferSelect; export type StatsRow = typeof stats.$inferSelect; +export type AtomContextRegisteredEventsRow = typeof atomContextRegisteredEvents.$inferSelect; export type AccountPnlSnapshotRow = typeof account_pnl_snapshot.$inferSelect; export type DepositedEventsRow = typeof deposited_events.$inferSelect; diff --git a/packages/database-timescale/src/schemas/timescale/events.ts b/packages/database-timescale/src/schemas/timescale/events.ts index 874310a..ac9ee77 100644 --- a/packages/database-timescale/src/schemas/timescale/events.ts +++ b/packages/database-timescale/src/schemas/timescale/events.ts @@ -170,6 +170,38 @@ export const atomCreatedEvents = pgTable( }) ); +export const atomContextRegisteredEvents = pgTable( + 'atom_context_registered_events', + { + blockNumber: bigint('block_number', { mode: 'bigint' }).notNull(), + blockTimestamp: timestamp('block_timestamp', { withTimezone: true }).notNull(), + blockHash: text('block_hash').notNull(), + transactionHash: text('transaction_hash').notNull(), + logIndex: integer('log_index').notNull(), + registrant: text('registrant').notNull(), + termId: numeric('term_id').notNull(), + termIdHex: text('term_id_hex').notNull(), + uris: jsonb('uris').$type().notNull(), + sequenceNumber: bigint('sequence_number', { mode: 'bigint' }).notNull(), + }, + (table) => ({ + atomContextRegisteredEventsPkey: primaryKey({ + columns: [table.transactionHash, table.logIndex], + }), + idxAtomContextRegisteredTerm: index('idx_atom_context_registered_term').on( + table.termId, + table.sequenceNumber + ), + idxAtomContextRegisteredTermHex: index('idx_atom_context_registered_term_hex').on( + table.termIdHex, + table.sequenceNumber + ), + uxAtomContextRegisteredSeq: uniqueIndex('ux_atom_context_registered_seq').on( + table.sequenceNumber + ), + }) +); + export const tripleCreatedEvents = pgTable( 'triple_created_events', { diff --git a/packages/database-timescale/src/schemas/timescale/manifest.json b/packages/database-timescale/src/schemas/timescale/manifest.json index 1bef995..22c2926 100644 --- a/packages/database-timescale/src/schemas/timescale/manifest.json +++ b/packages/database-timescale/src/schemas/timescale/manifest.json @@ -240,6 +240,62 @@ "name": "active_vault_position", "primaryKey": ["term_id", "curve_id", "account_id"] }, + { + "columns": [ + { + "name": "block_number", + "notNull": true, + "type": "bigint" + }, + { + "name": "block_timestamp", + "notNull": true, + "type": "timestamptz" + }, + { + "name": "block_hash", + "notNull": true, + "type": "text" + }, + { + "name": "transaction_hash", + "notNull": true, + "type": "text" + }, + { + "name": "log_index", + "notNull": true, + "type": "integer" + }, + { + "name": "registrant", + "notNull": true, + "type": "text" + }, + { + "name": "term_id", + "notNull": true, + "type": "numeric" + }, + { + "name": "term_id_hex", + "notNull": true, + "type": "text" + }, + { + "name": "uris", + "notNull": true, + "type": "jsonb" + }, + { + "name": "sequence_number", + "notNull": true, + "type": "bigint" + } + ], + "name": "atom_context_registered_events", + "primaryKey": ["transaction_hash", "log_index"] + }, { "columns": [ { diff --git a/packages/database-timescale/src/timescale-generation/layout.ts b/packages/database-timescale/src/timescale-generation/layout.ts index 3794087..8d1d93b 100644 --- a/packages/database-timescale/src/timescale-generation/layout.ts +++ b/packages/database-timescale/src/timescale-generation/layout.ts @@ -9,6 +9,7 @@ export const timescaleFileGroups = [ 'redemption_fact', 'fee_transfer_fact', 'atom_created_events', + 'atom_context_registered_events', 'triple_created_events', 'deposited_events', 'redeemed_events', @@ -60,3 +61,12 @@ export const timescaleFileGroups = [ ] as const; export type TimescaleFileGroup = (typeof timescaleFileGroups)[number]; + +/** JSONB columns whose application-level shape is part of the public schema. */ +export const timescaleJsonColumnTypes: Readonly>>> = + { + atom_context_registered_events: { + // Contract order, duplicates, and opaque 0x byte strings are preserved. + uris: 'string[]', + }, + }; diff --git a/packages/database-timescale/src/timescale-generation/render.ts b/packages/database-timescale/src/timescale-generation/render.ts index b65e134..2f5ec89 100644 --- a/packages/database-timescale/src/timescale-generation/render.ts +++ b/packages/database-timescale/src/timescale-generation/render.ts @@ -1,4 +1,4 @@ -import { timescaleFileGroups } from './layout'; +import { timescaleFileGroups, timescaleJsonColumnTypes } from './layout'; import type { ColumnDefault, ColumnDefinition, @@ -83,7 +83,10 @@ function renderTable(table: TableDefinition): string { const hasCallback = table.primaryKey.length > 1 || table.indexes.length > 0 || table.uniqueConstraints.length > 0; const renderedColumns = table.columns - .map((column) => `\t\t${toCamelCase(column.name)}: ${renderColumn(column, table.primaryKey)},`) + .map( + (column) => + `\t\t${toCamelCase(column.name)}: ${renderColumn(column, table.name, table.primaryKey)},` + ) .join('\n'); if (!hasCallback) { @@ -124,8 +127,8 @@ function renderTable(table: TableDefinition): string { return `export const ${toCamelCase(table.name)} = pgTable(\n\t'${table.name}',\n\t{\n${renderedColumns}\n\t},\n\t(table) => ({\n${callbackEntries.join('\n')}\n\t})\n);`; } -function renderColumn(column: ColumnDefinition, primaryKey: string[]): string { - const builder = renderColumnBuilder(column); +function renderColumn(column: ColumnDefinition, tableName: string, primaryKey: string[]): string { + const builder = renderColumnBuilder(column, tableName); const chainedCalls: string[] = []; const isSingleColumnPrimaryKey = primaryKey.length === 1 && primaryKey[0] === column.name; @@ -150,7 +153,7 @@ function renderColumn(column: ColumnDefinition, primaryKey: string[]): string { return chainedCalls.reduce((value, chain) => `${value}.${chain}`, builder); } -function renderColumnBuilder(column: ColumnDefinition): string { +function renderColumnBuilder(column: ColumnDefinition, tableName: string): string { switch (column.type) { case 'bigint': return `bigint('${column.name}', { mode: 'bigint' })`; @@ -161,7 +164,9 @@ function renderColumnBuilder(column: ColumnDefinition): string { case 'integer': return `integer('${column.name}')`; case 'jsonb': - return `jsonb('${column.name}')`; + return timescaleJsonColumnTypes[tableName]?.[column.name] + ? `jsonb('${column.name}').$type<${timescaleJsonColumnTypes[tableName]?.[column.name]}>()` + : `jsonb('${column.name}')`; case 'numeric': if (column.precision) { const scale = column.scale ?? 0; diff --git a/packages/database-timescale/tests/atom-context-migration.test.ts b/packages/database-timescale/tests/atom-context-migration.test.ts new file mode 100644 index 0000000..7562478 --- /dev/null +++ b/packages/database-timescale/tests/atom-context-migration.test.ts @@ -0,0 +1,55 @@ +import { describe, expect, it } from 'bun:test'; +import { readFile } from 'node:fs/promises'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { parseTimescaleMigrations } from '../src/timescale-generation/parser'; + +const testsDirectory = path.dirname(fileURLToPath(import.meta.url)); +const migrationPath = path.resolve( + testsDirectory, + '../../../migrations/timescale/050_add_atom_context_registered.sql' +); + +describe('AtomContextRegistered Timescale migration', () => { + it('creates the regular typed table with canonical identity and ordering columns', async () => { + const sql = await readFile(migrationPath, 'utf8'); + const { compatInventory, tables } = parseTimescaleMigrations([ + { fileName: '050_add_atom_context_registered.sql', sql }, + ]); + const table = tables.get('atom_context_registered_events'); + + expect(table?.primaryKey).toEqual(['transaction_hash', 'log_index']); + expect(table?.columns.map(({ name, notNull, type }) => ({ name, notNull, type }))).toEqual( + expect.arrayContaining([ + { name: 'block_number', notNull: true, type: 'bigint' }, + { name: 'block_timestamp', notNull: true, type: 'timestamptz' }, + { name: 'block_hash', notNull: true, type: 'text' }, + { name: 'transaction_hash', notNull: true, type: 'text' }, + { name: 'log_index', notNull: true, type: 'integer' }, + { name: 'registrant', notNull: true, type: 'text' }, + { name: 'term_id', notNull: true, type: 'numeric' }, + { name: 'term_id_hex', notNull: true, type: 'text' }, + { name: 'uris', notNull: true, type: 'jsonb' }, + { name: 'sequence_number', notNull: true, type: 'bigint' }, + ]) + ); + expect(table?.indexes).toEqual( + expect.arrayContaining([ + expect.objectContaining({ columns: ['sequence_number'], unique: true }), + expect.objectContaining({ columns: ['term_id', 'sequence_number'] }), + expect.objectContaining({ columns: ['term_id_hex', 'sequence_number'] }), + ]) + ); + expect(compatInventory.hypertables).not.toContain('atom_context_registered_events'); + expect(sql).toContain("CHECK (jsonb_typeof(uris) = 'array')"); + }); + + it('removes the closed-world event type whitelist without adding a replacement', async () => { + const sql = await readFile(migrationPath, 'utf8'); + + expect(sql).toContain('DROP CONSTRAINT IF EXISTS event_store_event_type_check'); + expect(sql).not.toContain('event_store_event_type_check_v2'); + expect(sql).not.toContain('ADD CONSTRAINT event_store_event_type_check'); + expect(sql).toContain('future additions'); + }); +}); diff --git a/packages/database-timescale/tests/atom-context-schema.test.ts b/packages/database-timescale/tests/atom-context-schema.test.ts new file mode 100644 index 0000000..536acb1 --- /dev/null +++ b/packages/database-timescale/tests/atom-context-schema.test.ts @@ -0,0 +1,41 @@ +import { describe, expect, it } from 'bun:test'; +import { readFile } from 'node:fs/promises'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +type Manifest = { + tables: Array<{ + name: string; + primaryKey: string[]; + columns: Array<{ name: string; type: string; notNull: boolean }>; + }>; +}; + +const schemaDirectory = path.resolve( + path.dirname(fileURLToPath(import.meta.url)), + '../src/schemas/timescale' +); + +describe('AtomContextRegistered generated schema', () => { + it('includes the typed table in the checked-in manifest', async () => { + const manifest = JSON.parse( + await readFile(path.join(schemaDirectory, 'manifest.json'), 'utf8') + ) as Manifest; + const table = manifest.tables.find(({ name }) => name === 'atom_context_registered_events'); + + expect(table?.primaryKey).toEqual(['transaction_hash', 'log_index']); + expect(table?.columns).toEqual( + expect.arrayContaining([ + { name: 'term_id_hex', type: 'text', notNull: true }, + { name: 'uris', type: 'jsonb', notNull: true }, + { name: 'sequence_number', type: 'bigint', notNull: true }, + ]) + ); + }); + + it('types uris as an ordered array rather than a scalar JSON value', async () => { + const eventsSchema = await readFile(path.join(schemaDirectory, 'events.ts'), 'utf8'); + + expect(eventsSchema).toContain("uris: jsonb('uris').$type().notNull()"); + }); +}); diff --git a/packages/database-timescale/tests/typecheck/schema-actions-fixture.ts b/packages/database-timescale/tests/typecheck/schema-actions-fixture.ts index 6673d84..b7aa7de 100644 --- a/packages/database-timescale/tests/typecheck/schema-actions-fixture.ts +++ b/packages/database-timescale/tests/typecheck/schema-actions-fixture.ts @@ -14,6 +14,7 @@ import { import { createTimescaleConnection } from '../../src/client'; import { type AccountPnlSnapshotRow, + type AtomContextRegisteredEventsRow, account_pnl_snapshot, type DepositedEventsRow, deposited_events, @@ -49,6 +50,10 @@ import { term_market_cap_history, } from '../../src/schema'; +declare const atomContextRows: AtomContextRegisteredEventsRow[]; +const firstOpaqueUri: string | undefined = atomContextRows[0]?.uris[0]; +void firstOpaqueUri; + const positionChangeInput: ListPositionChangeRowsInput = { accountId: '0xabc', limit: 5, diff --git a/scripts/smoke-index.sh b/scripts/smoke-index.sh index 475ea8e..70cdbd8 100755 --- a/scripts/smoke-index.sh +++ b/scripts/smoke-index.sh @@ -14,6 +14,8 @@ Environment: API_URL API base URL (default: discovered from Compose) SMOKE_INDEX_TIMEOUT_SECONDS Indexing/projection timeout (default: 240) SMOKE_BUILD=0 Reuse existing Docker images instead of rebuilding + SMOKE_EXPECT_CONTEXT_EVENT_COUNT + Optional exact AtomContextRegistered raw-event count KEEP_SMOKE_STACK=1 Leave containers and volumes running after the test USAGE } @@ -33,6 +35,7 @@ esac PROJECT_NAME=${SMOKE_INDEX_PROJECT_NAME:-intuition-core-smoke-index} API_URL=${API_URL:-} TIMEOUT_SECONDS=${SMOKE_INDEX_TIMEOUT_SECONDS:-240} +EXPECTED_CONTEXT_EVENT_COUNT=${SMOKE_EXPECT_CONTEXT_EVENT_COUNT:-} # Public, keyless Intuition testnet window used for deterministic smoke runs. DEFAULT_INTUITION_RPC_URL=https://testnet.rpc.intuition.systems/http @@ -92,6 +95,14 @@ validate_positive_integer() { [ "$value" -gt 0 ] || fail "$name must be greater than 0" } +validate_non_negative_integer() { + name=$1 + value=$2 + case "$value" in + "" | *[!0-9]*) fail "$name must be a non-negative integer" ;; + esac +} + json_get() { path=$1 bun -e ' @@ -244,6 +255,9 @@ validate_positive_integer SMOKE_INDEX_TIMEOUT_SECONDS "$TIMEOUT_SECONDS" validate_positive_integer CHAIN_ID "$CHAIN_ID" validate_positive_integer MULTIVAULT_START_BLOCK "$MULTIVAULT_START_BLOCK" validate_positive_integer MULTIVAULT_END_BLOCK "$MULTIVAULT_END_BLOCK" +if [ -n "$EXPECTED_CONTEXT_EVENT_COUNT" ]; then + validate_non_negative_integer SMOKE_EXPECT_CONTEXT_EVENT_COUNT "$EXPECTED_CONTEXT_EVENT_COUNT" +fi printf 'Starting Docker Compose project %s with indexing profile\n' "$PROJECT_NAME" compose --profile indexing down -v --remove-orphans >/dev/null 2>&1 || true @@ -276,5 +290,11 @@ stats_body=$WORK_DIR/stats.json api_get /api/stats "$stats_body" atom_count=$(json_file_get "$stats_body" data.atoms) -printf 'Index smoke test passed: events=%s checkpoints=%s core_entities_checkpoint=%s/%s atoms=%s window=%s-%s\n' \ - "$event_count" "$checkpoint_count" "$core_entities_checkpoint" "$core_entities_target_sequence" "$atom_count" "$MULTIVAULT_START_BLOCK" "$MULTIVAULT_END_BLOCK" +context_event_count=$(timescale_sql "SELECT count(*) FROM event_store WHERE is_canonical = true AND event_type = 'AtomContextRegistered';") +if [ -n "$EXPECTED_CONTEXT_EVENT_COUNT" ] && + [ "$context_event_count" -ne "$EXPECTED_CONTEXT_EVENT_COUNT" ]; then + fail "expected $EXPECTED_CONTEXT_EVENT_COUNT AtomContextRegistered events, got $context_event_count" +fi + +printf 'Index smoke test passed: events=%s context_events=%s checkpoints=%s core_entities_checkpoint=%s/%s atoms=%s window=%s-%s\n' \ + "$event_count" "$context_event_count" "$checkpoint_count" "$core_entities_checkpoint" "$core_entities_target_sequence" "$atom_count" "$MULTIVAULT_START_BLOCK" "$MULTIVAULT_END_BLOCK" diff --git a/scripts/smoke-test.sh b/scripts/smoke-test.sh index 4a9751f..da7da37 100755 --- a/scripts/smoke-test.sh +++ b/scripts/smoke-test.sh @@ -14,6 +14,8 @@ Environment: API_URL API base URL (default: discovered from Compose) SMOKE_TIMEOUT_SECONDS Health/worker timeout (default: 180) SMOKE_BUILD=0 Reuse existing Docker images instead of rebuilding + SMOKE_EXPECT_SEMANTIC_READS=1 + Assert the default-off additive atom read envelope KEEP_SMOKE_STACK=1 Leave containers and volumes running after the test USAGE } @@ -33,6 +35,7 @@ esac PROJECT_NAME=${SMOKE_PROJECT_NAME:-intuition-core-smoke} API_URL=${API_URL:-} TIMEOUT_SECONDS=${SMOKE_TIMEOUT_SECONDS:-180} +EXPECT_SEMANTIC_READS=${SMOKE_EXPECT_SEMANTIC_READS:-0} PREDICATE_ID=0x0840db4575bf6bdb49b66c21dc40cb4cbb5e1b26bd239d7f56b126c14e452c07 SMOKE_ACCOUNT=0x0000000000000000000000000000000000000001 AUTH_HEADER_FILE= @@ -80,6 +83,15 @@ validate_positive_integer() { [ "$value" -gt 0 ] || fail "$name must be greater than 0" } +validate_boolean_integer() { + name=$1 + value=$2 + case "$value" in + 0 | 1) ;; + *) fail "$name must be 0 or 1" ;; + esac +} + json_get() { path=$1 bun -e ' @@ -180,10 +192,24 @@ wait_for_atom_processing() { fail "atom $atom_id did not finish processing: classificationType=$classification_type parse=$parse_status classification=$classification_status enrichment=$enrichment_status" } +verify_semantic_atom_view() { + output=$1 + raw_type=$(json_file_get "$output" data.raw.type) + classification_type=$(json_file_get "$output" data.classification.type) + resolution_status=$(json_file_get "$output" data.resolution.status) + [ "$raw_type" = "http_uri" ] || fail "expected semantic raw type http_uri, got $raw_type" + [ -n "$classification_type" ] || fail "expected semantic classification type" + case "$resolution_status" in + pending | processing | resolved | retryable | terminal | skipped | failed) ;; + *) fail "expected a recognized semantic resolution status, got $resolution_status" ;; + esac +} + need bun need curl need docker validate_positive_integer SMOKE_TIMEOUT_SECONDS "$TIMEOUT_SECONDS" +validate_boolean_integer SMOKE_EXPECT_SEMANTIC_READS "$EXPECT_SEMANTIC_READS" printf 'Starting Docker Compose project %s\n' "$PROJECT_NAME" compose --profile indexing down -v --remove-orphans >/dev/null 2>&1 || true @@ -223,6 +249,10 @@ api_post /api/atoms '{"input":"Intuition Core smoke test object"}' "$object_body object_id=$(json_file_get "$object_body" data.id) wait_for_atom_processing "$subject_id" "$atom_body" +if [ "$EXPECT_SEMANTIC_READS" = "1" ]; then + printf 'Verifying additive semantic atom read envelope\n' + verify_semantic_atom_view "$atom_body" +fi printf 'Creating triple\n' api_post /api/triples \ diff --git a/services/api/src/app.ts b/services/api/src/app.ts index 73c84e4..1c7bbb6 100644 --- a/services/api/src/app.ts +++ b/services/api/src/app.ts @@ -13,12 +13,21 @@ import { ensureNodeWithCreation, ensureTripleWithCreation, type KgNodeRawType, + listNodeContexts, + listPublicNodesByIid, } from '@0xintuition/database-kg/actions'; import { and, desc, eq, getTableColumns, ilike, or, sql } from 'drizzle-orm'; import { alias } from 'drizzle-orm/pg-core'; import { Hono } from 'hono'; import { getConnInfo } from 'hono/bun'; import { cors } from 'hono/cors'; +import { + type ExpandedAtomTermSource, + exposeAtomDetailView, + exposeAtomListView, + exposeExpandedAtomTerm, + presentPersistedAtomContext, +} from './atom-view'; import { type ApiKeyIdentity, bearerToken, resolveApiKey } from './auth'; import type { ApiConfig } from './config'; import { createRateLimiter } from './rate-limit'; @@ -111,7 +120,6 @@ export function createApp(config: ApiConfig) { const connection = createKgConnection({ connectionString: config.databaseKgUrl }); const db: KgDb = connection.db; let schemaMetadataPromise: Promise | null = null; - const app = new Hono(); app.use( @@ -287,11 +295,16 @@ export function createApp(config: ApiConfig) { isOnchain: nodes.isOnchain, rawType: nodes.rawType, data: nodes.data, + iid: nodes.iid, dataResolved: nodes.dataResolved, + parseResult: nodes.parseResult, classificationType: nodes.classificationType, parseStatus: nodes.parseStatus, classificationStatus: nodes.classificationStatus, + classificationResult: nodes.classificationResult, enrichmentStatus: nodes.enrichmentStatus, + enrichmentError: nodes.enrichmentError, + enrichedAt: nodes.enrichedAt, }) .from(nodes) .where(and(...filters)) @@ -299,7 +312,24 @@ export function createApp(config: ApiConfig) { .limit(limit) .offset(offset); - return c.json({ data: rows, pagination: { limit, offset, count: rows.length } }); + return c.json({ + data: rows.map((row) => exposeAtomListView(row, config.atomSemanticReadsEnabled)), + pagination: { limit, offset, count: rows.length }, + }); + }); + + app.get('/api/iids/:iid/atoms', async (c) => { + const iid = c.req.param('iid'); + if (!iid.trim()) { + return c.json({ error: 'invalid_iid', message: 'iid must not be empty' }, 400); + } + const { limit, offset } = parsePagination(c.req.query()); + const rows = await listPublicNodesByIid(db, iid, { limit, offset }); + + return c.json({ + data: rows.map((row) => exposeAtomListView(row, config.atomSemanticReadsEnabled)), + pagination: { limit, offset, count: rows.length }, + }); }); app.get('/api/atoms/:id', async (c) => { @@ -316,11 +346,21 @@ export function createApp(config: ApiConfig) { // Graph-degree stats are maintained by the adjacency projections; absent // until the node participates in a triple. - const [stats] = await db.select().from(nodeStats).where(eq(nodeStats.nodeId, id)).limit(1); + const [statsRows, contextRows] = await Promise.all([ + db.select().from(nodeStats).where(eq(nodeStats.nodeId, id)).limit(1), + config.atomSemanticReadsEnabled ? listNodeContexts(db, id) : Promise.resolve(undefined), + ]); + const stats = statsRows[0]; + const detailSource = { + ...row, + ...(contextRows + ? { context: contextRows.map((context) => presentPersistedAtomContext(context)) } + : {}), + }; return c.json({ data: { - ...row, + ...exposeAtomDetailView(detailSource, config.atomSemanticReadsEnabled), stats: stats ? { inDegree: Number(stats.inDegree), @@ -383,18 +423,21 @@ export function createApp(config: ApiConfig) { subject: { id: subjectNodes.id, data: subjectNodes.data, + dataResolved: subjectNodes.dataResolved, classificationType: subjectNodes.classificationType, rawType: subjectNodes.rawType, }, predicate: { id: predicateNodes.id, data: predicateNodes.data, + dataResolved: predicateNodes.dataResolved, classificationType: predicateNodes.classificationType, rawType: predicateNodes.rawType, }, object: { id: objectNodes.id, data: objectNodes.data, + dataResolved: objectNodes.dataResolved, classificationType: objectNodes.classificationType, rawType: objectNodes.rawType, }, @@ -431,6 +474,25 @@ export function createApp(config: ApiConfig) { const wantsExpandedTerms = (query: Record) => query.expand === 'terms'; + const presentExpandedTerms = < + T extends { + subject: ExpandedAtomTermSource | null; + predicate: ExpandedAtomTermSource | null; + object: ExpandedAtomTermSource | null; + }, + >( + row: T + ) => ({ + ...row, + subject: row.subject + ? exposeExpandedAtomTerm(row.subject, config.atomSemanticReadsEnabled) + : null, + predicate: row.predicate + ? exposeExpandedAtomTerm(row.predicate, config.atomSemanticReadsEnabled) + : null, + object: row.object ? exposeExpandedAtomTerm(row.object, config.atomSemanticReadsEnabled) : null, + }); + // All triples touching an atom, in any position — served by the hexastore. app.get('/api/atoms/:id/triples', async (c) => { const id = c.req.param('id'); @@ -449,7 +511,7 @@ export function createApp(config: ApiConfig) { .limit(limit) .offset(offset); return c.json({ - data: rows, + data: rows.map(presentExpandedTerms), pagination: { limit, offset, count: rows.length }, }); } @@ -541,7 +603,7 @@ export function createApp(config: ApiConfig) { .limit(limit) .offset(offset); return c.json({ - data: rows, + data: rows.map(presentExpandedTerms), pagination: { limit, offset, count: rows.length }, }); } @@ -567,7 +629,7 @@ export function createApp(config: ApiConfig) { if (!row) { return c.json({ error: 'not_found' }, 404); } - return c.json({ data: row }); + return c.json({ data: presentExpandedTerms(row) }); } const [row] = await db diff --git a/services/api/src/atom-view.ts b/services/api/src/atom-view.ts new file mode 100644 index 0000000..7259bc7 --- /dev/null +++ b/services/api/src/atom-view.ts @@ -0,0 +1,442 @@ +export type AtomRawView = { + type: string; + data: string | null; + dataHex?: string | null; +}; + +export type AtomIdentityView = { + raw: string | null; + canonical: string | null; + profile?: 'p0' | 'p1' | 'p2'; + scheme?: string; + value?: string; + class?: 'A' | 'B' | 'C'; + typing?: 'unambiguous' | 'polymorphic'; + anchorIneligibilityReason?: 'class-c' | 'polymorphic-scheme'; + valid?: boolean; + anchorEligible?: boolean; + provenance?: { + producer: string; + version: string; + specificationVersion?: string; + }; +}; + +export type AtomClassificationView = { + type: string; + status?: string; + source?: string; +}; + +export type AtomContextView = { + eventSequence?: string; + ordinal: number; + uri: string | null; + source?: string; + raw?: string; + registrant?: string; + transactionHash?: string; + logIndex?: number; +}; + +export type PersistedAtomContextSource = { + eventSequence: bigint; + ordinal: number; + uriText: string | null; + uriHex: string; + registrant: string; + transactionHash: string; + logIndex: number; +}; + +export type AtomResolutionView = { + status: string; + updatedAt?: Date | string | null; +}; + +export type AtomDisplayView = { + name?: string; + description?: string; + image?: string; +}; + +export type AtomViewFields = { + raw: AtomRawView; + identity?: AtomIdentityView; + classification: AtomClassificationView; + context?: AtomContextView[]; + resolution?: AtomResolutionView; + display?: AtomDisplayView; +}; + +export type AtomViewSource = { + rawType: string; + data: string | null; + dataHex?: string | null; + iid?: string | null; + dataResolved?: unknown; + parseResult?: unknown; + classificationType: string; + classificationStatus?: string; + classificationResult?: unknown; + enrichmentStatus?: string; + enrichmentError?: unknown; + enrichedAt?: Date | string | null; + context?: readonly AtomContextView[]; +}; + +const LIST_INTERNAL_EVIDENCE = [ + 'parseResult', + 'classificationResult', + 'enrichmentError', + 'enrichedAt', +] as const; + +/** + * List queries select additional evidence to build the semantic envelope. Do + * not expose those database fields until the additive response is enabled. + */ +export function exposeAtomListView(atom: T, enabled: boolean) { + if (enabled) { + return omitFields(presentAtom(atom), LIST_INTERNAL_EVIDENCE); + } + return omitFields(atom, ['iid', ...LIST_INTERNAL_EVIDENCE] as const); +} + +/** + * Detail queries historically exposed the full node row. Persisted `iid` and + * joined context were added later and remain hidden on the legacy path. + */ +export function exposeAtomDetailView(atom: T, enabled: boolean) { + return enabled ? presentAtom(atom) : omitFields(atom, ['iid', 'context'] as const); +} + +/** Map storage evidence without decoding, normalizing, sorting, or deduplicating it. */ +export function presentPersistedAtomContext(context: PersistedAtomContextSource): AtomContextView { + return { + eventSequence: context.eventSequence.toString(), + ordinal: context.ordinal, + uri: context.uriText, + source: 'onchain', + raw: context.uriHex, + registrant: context.registrant, + transactionHash: context.transactionHash, + logIndex: context.logIndex, + }; +} + +/** Return the byte-for-byte legacy object shape while exposure is disabled. */ +export function exposeAtomView(atom: T, enabled: false): T; +export function exposeAtomView( + atom: T, + enabled: true +): T & AtomViewFields; +export function exposeAtomView( + atom: T, + enabled: boolean +): T | (T & AtomViewFields); +export function exposeAtomView( + atom: T, + enabled: boolean +): T | (T & AtomViewFields) { + return enabled ? presentAtom(atom) : atom; +} + +export type ExpandedAtomTermSource = AtomViewSource & { id: string }; + +/** + * Expanded triple terms historically contain exactly four fields. The query + * also selects resolved data for the optional display view, but that internal + * input must not leak while exposure is disabled. + */ +export function exposeExpandedAtomTerm( + term: T, + enabled: boolean +) { + if (enabled) { + return presentAtom(term); + } + + return { + id: term.id, + data: term.data, + classificationType: term.classificationType, + rawType: term.rawType, + }; +} + +/** + * Add the stable atom presentation envelope without removing or renaming any + * database-shaped fields. Optional sections are omitted until their evidence + * exists; in particular, an absent context reader must not look like an atom + * with a proven empty context list. + */ +export function presentAtom(atom: T): T & AtomViewFields { + const identity = resolveIdentity(atom); + const display = resolveDisplay(atom.dataResolved); + const classificationSource = readString(toRecord(atom.classificationResult)?.source); + + return { + ...atom, + raw: { + type: atom.rawType, + data: atom.data, + ...(atom.dataHex !== undefined ? { dataHex: atom.dataHex } : {}), + }, + ...(identity ? { identity } : {}), + classification: { + type: atom.classificationType, + ...(atom.classificationStatus ? { status: atom.classificationStatus } : {}), + ...(classificationSource ? { source: classificationSource } : {}), + }, + ...(atom.context !== undefined ? { context: [...atom.context] } : {}), + ...(atom.enrichmentStatus + ? { + resolution: { + status: normalizeResolutionStatus(atom.enrichmentStatus, atom.enrichmentError), + ...(atom.enrichedAt !== undefined ? { updatedAt: atom.enrichedAt } : {}), + }, + } + : {}), + ...(display ? { display } : {}), + }; +} + +function resolveIdentity(atom: AtomViewSource): AtomIdentityView | undefined { + const parseResult = toRecord(atom.parseResult); + if (parseResult?.kind === 'iid') { + const nestedIdentity = resolveNestedIdentity(parseResult.identity); + if (nestedIdentity) { + return nestedIdentity; + } + } + + const persistedIid = atom.rawType === 'iid' ? readString(atom.iid) : undefined; + if (persistedIid) { + return { + raw: atom.data, + canonical: persistedIid, + }; + } + + if (parseResult?.kind !== 'iid') { + return undefined; + } + + const canonical = + readString(parseResult.canonicalIid) ?? readString(parseResult.canonicalId) ?? null; + const raw = atom.data ?? readString(parseResult.normalizedInput) ?? null; + const profile = readIdentityProfile(parseResult.profile); + const scheme = readString(parseResult.scheme); + const value = readString(parseResult.value); + const identityClass = readIdentityClass(parseResult.class); + const typing = readIdentityTyping(parseResult.typing); + const anchorIneligibilityReason = readAnchorIneligibilityReason( + parseResult.anchorIneligibilityReason + ); + const anchorEligible = parseResult.anchorEligible; + const provenance = resolveIdentityProvenance(parseResult.provenance); + + return { + raw, + canonical, + ...(profile ? { profile } : {}), + ...(scheme ? { scheme } : {}), + ...(value ? { value } : {}), + ...(identityClass ? { class: identityClass } : {}), + ...(typing ? { typing } : {}), + ...(anchorIneligibilityReason ? { anchorIneligibilityReason } : {}), + ...(typeof parseResult.valid === 'boolean' ? { valid: parseResult.valid } : {}), + ...(typeof anchorEligible === 'boolean' ? { anchorEligible } : {}), + ...(provenance ? { provenance } : {}), + }; +} + +function resolveNestedIdentity(value: unknown): AtomIdentityView | undefined { + const identity = toRecord(value); + if (!identity) { + return undefined; + } + + const raw = readString(identity.raw); + const canonical = readString(identity.canonical); + const scheme = readString(identity.scheme); + const identityValue = readString(identity.value); + if ( + !raw || + !canonical || + !scheme || + !identityValue || + typeof identity.anchorEligible !== 'boolean' + ) { + return undefined; + } + + const profile = readIdentityProfile(identity.profile); + const identityClass = readIdentityClass(identity.class); + const typing = readIdentityTyping(identity.typing); + const anchorIneligibilityReason = readAnchorIneligibilityReason( + identity.anchorIneligibilityReason + ); + const provenance = resolveIdentityProvenance(identity.provenance); + return { + raw, + canonical, + ...(profile ? { profile } : {}), + scheme, + value: identityValue, + ...(identityClass ? { class: identityClass } : {}), + ...(typing ? { typing } : {}), + ...(anchorIneligibilityReason ? { anchorIneligibilityReason } : {}), + ...(typeof identity.valid === 'boolean' ? { valid: identity.valid } : {}), + anchorEligible: identity.anchorEligible, + ...(provenance ? { provenance } : {}), + }; +} + +function resolveIdentityProvenance( + value: unknown +): NonNullable | undefined { + const provenance = toRecord(value); + const producer = readString(provenance?.producer); + const version = readString(provenance?.version); + if (!producer || !version) { + return undefined; + } + + const specificationVersion = readString(provenance?.specificationVersion); + return { + producer, + version, + ...(specificationVersion ? { specificationVersion } : {}), + }; +} + +function readIdentityProfile(value: unknown): AtomIdentityView['profile'] { + return value === 'p0' || value === 'p1' || value === 'p2' ? value : undefined; +} + +function readIdentityClass(value: unknown): AtomIdentityView['class'] { + return value === 'A' || value === 'B' || value === 'C' ? value : undefined; +} + +function readIdentityTyping(value: unknown): AtomIdentityView['typing'] { + return value === 'unambiguous' || value === 'polymorphic' ? value : undefined; +} + +function readAnchorIneligibilityReason( + value: unknown +): AtomIdentityView['anchorIneligibilityReason'] { + return value === 'class-c' || value === 'polymorphic-scheme' ? value : undefined; +} + +function resolveDisplay(value: unknown): AtomDisplayView | undefined { + const root = toRecord(value); + if (!root) { + return undefined; + } + + const candidates = [root, toRecord(root.resolvedAtom), toRecord(root.data)].filter( + (value): value is Record => value !== undefined + ); + const name = firstString(candidates, ['name', 'title', 'displayName', 'headline']); + const description = firstString(candidates, ['description', 'summary']); + const image = firstImage(candidates); + + if (!name && !description && !image) { + return undefined; + } + + return { + ...(name ? { name } : {}), + ...(description ? { description } : {}), + ...(image ? { image } : {}), + }; +} + +function normalizeResolutionStatus(status: string, error: unknown): string { + if (status === 'completed') { + return 'resolved'; + } + if (status === 'failed') { + const retriable = toRecord(error)?.retriable; + if (retriable === true) { + return 'retryable'; + } + if (retriable === false) { + return 'terminal'; + } + } + return status; +} + +function firstString( + records: ReadonlyArray>, + fields: readonly string[] +): string | undefined { + for (const record of records) { + for (const field of fields) { + const value = readString(record[field]); + if (value) { + return value; + } + } + } + return undefined; +} + +function firstImage(records: ReadonlyArray>): string | undefined { + const fields = [ + 'image', + 'imageUrl', + 'image_url', + 'thumbnailUrl', + 'thumbnail', + 'avatarUrl', + 'profileImageUrl', + 'logo', + ] as const; + + for (const record of records) { + for (const field of fields) { + const value = record[field]; + const direct = readString(value); + if (direct) { + return direct; + } + const nested = toRecord(value); + const nestedUrl = readString(nested?.url) ?? readString(nested?.contentUrl); + if (nestedUrl) { + return nestedUrl; + } + } + } + return undefined; +} + +function toRecord(value: unknown): Record | undefined { + return typeof value === 'object' && value !== null && !Array.isArray(value) + ? (value as Record) + : undefined; +} + +function omitFields( + value: T, + keys: K +): Omit { + const result = { ...value }; + for (const key of keys) { + delete (result as Record)[key]; + } + return result; +} + +function readString(value: unknown): string | undefined { + if (typeof value === 'string' && value.trim()) { + return value.trim(); + } + if (Array.isArray(value)) { + return value.map(readString).find((entry): entry is string => entry !== undefined); + } + return undefined; +} diff --git a/services/api/src/config.ts b/services/api/src/config.ts index 1ccee49..edf77ae 100644 --- a/services/api/src/config.ts +++ b/services/api/src/config.ts @@ -23,6 +23,11 @@ export type ApiConfig = { * clients can mint a fresh bucket per request and bypass limits entirely. */ trustProxy: boolean; + /** + * Expose the additive semantic atom envelope on read endpoints. Default-off + * until identity/context ingestion and consumer compatibility are verified. + */ + atomSemanticReadsEnabled: boolean; }; function parseAuthMode(raw: string | undefined): ApiAuthMode { @@ -33,6 +38,17 @@ function parseAuthMode(raw: string | undefined): ApiAuthMode { throw new Error(`API_AUTH must be one of: open, public-read, gated (got "${raw}")`); } +export function parseBooleanFlag(raw: string | undefined, name: string): boolean { + const normalized = (raw ?? 'false').trim().toLowerCase(); + if (normalized === 'true' || normalized === '1') { + return true; + } + if (normalized === 'false' || normalized === '0' || normalized === '') { + return false; + } + throw new Error(`${name} must be true, false, 1, or 0 (got "${raw}")`); +} + export function loadConfig(env: Record = process.env): ApiConfig { const databaseKgUrl = env.DATABASE_KG_URL?.trim(); if (!databaseKgUrl) { @@ -49,5 +65,9 @@ export function loadConfig(env: Record = process.env authMode: parseAuthMode(env.API_AUTH), rateLimitRpm: Number.parseInt(env.API_RATE_LIMIT_RPM ?? '120', 10), trustProxy: (env.API_TRUST_PROXY ?? '').trim() === '1', + atomSemanticReadsEnabled: parseBooleanFlag( + env.API_ATOM_SEMANTIC_READS_ENABLED, + 'API_ATOM_SEMANTIC_READS_ENABLED' + ), }; } diff --git a/services/api/tests/atom-view.test.ts b/services/api/tests/atom-view.test.ts new file mode 100644 index 0000000..75c1e35 --- /dev/null +++ b/services/api/tests/atom-view.test.ts @@ -0,0 +1,353 @@ +import { describe, expect, test } from 'bun:test'; +import { + exposeAtomDetailView, + exposeAtomListView, + exposeAtomView, + exposeExpandedAtomTerm, + presentAtom, + presentPersistedAtomContext, +} from '../src/atom-view'; + +type GoldenFixture = { + identityCases: Array<{ + id: string; + input: string; + expectedIntent: { + minimumProfile?: string; + schemeTyping?: string; + scheme?: string; + value?: string; + classification: string | null; + }; + }>; + legacyJsonAtom: { + rawType: string; + data: string; + expectedIntent: { classification: string }; + }; + contextCases: Array<{ + id: string; + entries: Array<{ + ordinal: number; + uri: string | null; + raw?: string; + source: string; + expectedLinkable: boolean; + }>; + }>; + resolutionCases: Array<{ + id: string; + source: { + id: string; + rawType: string; + data: string; + dataResolved: unknown; + parseResult?: unknown; + classificationType: string; + classificationStatus: string; + classificationResult?: unknown; + enrichmentStatus: string; + enrichmentError?: unknown; + enrichedAt?: string; + }; + expected: { + resolutionStatus: string; + displayName: string | null; + displayImage?: string; + }; + }>; +}; + +const golden = (await Bun.file( + new URL('../../../tests/fixtures/atom-semantic-read-model.v1.json', import.meta.url) +).json()) as GoldenFixture; + +function byId(cases: T[], id: string): T { + const found = cases.find((entry) => entry.id === id); + if (!found) { + throw new Error(`missing golden fixture case: ${id}`); + } + return found; +} + +describe('presentAtom', () => { + test('maps persisted context without decoding or dropping ordering provenance', () => { + expect( + presentPersistedAtomContext({ + eventSequence: 42n, + ordinal: 3, + uriText: null, + uriHex: '0xff00', + registrant: '0xregistrant', + transactionHash: '0xtx', + logIndex: 7, + }) + ).toEqual({ + eventSequence: '42', + ordinal: 3, + uri: null, + source: 'onchain', + raw: '0xff00', + registrant: '0xregistrant', + transactionHash: '0xtx', + logIndex: 7, + }); + }); + + test('keeps semantic list evidence dark and exposes nested identity when enabled', () => { + const source = { + id: '0xlist', + createdAt: '2026-08-10T00:00:00.000Z', + isOnchain: true, + rawType: 'iid', + data: 'int:isrc:USQX91300108', + iid: 'int:isrc:USQX91300108', + dataResolved: { name: 'One Last Time' }, + parseResult: { + kind: 'iid', + identity: { + raw: 'int:isrc:USQX91300108', + canonical: 'int:isrc:preferred-nested', + scheme: 'isrc', + value: 'USQX91300108', + profile: 'p0', + anchorEligible: true, + }, + }, + classificationType: 'MusicRecording', + parseStatus: 'completed', + classificationStatus: 'completed', + classificationResult: { source: 'iid-registry' }, + enrichmentStatus: 'completed', + enrichmentError: null, + enrichedAt: '2026-08-10T00:01:00.000Z', + }; + + const legacy = exposeAtomListView(source, false); + expect(legacy).toEqual({ + id: source.id, + createdAt: source.createdAt, + isOnchain: source.isOnchain, + rawType: source.rawType, + data: source.data, + dataResolved: source.dataResolved, + classificationType: source.classificationType, + parseStatus: source.parseStatus, + classificationStatus: source.classificationStatus, + enrichmentStatus: source.enrichmentStatus, + }); + expect('iid' in legacy).toBe(false); + expect('parseResult' in legacy).toBe(false); + + const semantic = exposeAtomListView(source, true); + expect(semantic.identity).toMatchObject({ + canonical: 'int:isrc:preferred-nested', + scheme: 'isrc', + value: 'USQX91300108', + }); + expect(semantic.iid).toBe('int:isrc:USQX91300108'); + expect('parseResult' in semantic).toBe(false); + expect('classificationResult' in semantic).toBe(false); + expect('enrichmentError' in semantic).toBe(false); + expect('enrichedAt' in semantic).toBe(false); + expect(semantic.classification).toEqual({ + type: 'MusicRecording', + status: 'completed', + source: 'iid-registry', + }); + }); + + test('strips only the new persisted IID from legacy detail and uses it when enabled', () => { + const source = { + id: '0xdetail', + rawType: 'iid', + data: 'int:isrc:raw-input', + iid: 'int:isrc:persisted-canonical', + parseResult: { + kind: 'iid', + identity: { + raw: 'int:isrc:raw-input', + canonical: null, + scheme: 'isrc', + value: 'raw-input', + anchorEligible: true, + }, + }, + classificationType: 'MusicRecording', + classificationResult: { source: 'iid-registry' }, + enrichmentStatus: 'pending', + context: [{ ordinal: 0, uri: null, raw: '0xff00', source: 'onchain' }], + }; + + const legacy = exposeAtomDetailView(source, false); + expect(legacy).toEqual({ + id: source.id, + rawType: source.rawType, + data: source.data, + parseResult: source.parseResult, + classificationType: source.classificationType, + classificationResult: source.classificationResult, + enrichmentStatus: source.enrichmentStatus, + }); + expect('iid' in legacy).toBe(false); + expect('context' in legacy).toBe(false); + + const semantic = exposeAtomDetailView(source, true); + expect(semantic.identity).toEqual({ + raw: source.data, + canonical: source.iid, + }); + expect(semantic.iid).toBe(source.iid); + }); + + test('does not treat a persisted IID as identity evidence for a non-IID atom', () => { + const view = presentAtom({ + rawType: 'string', + data: 'int:isrc:lookalike', + iid: 'int:isrc:must-not-leak-into-identity', + classificationType: 'Unknown', + }); + + expect(view.identity).toBeUndefined(); + }); + + test('leaves the legacy response object unchanged while exposure is disabled', () => { + const legacy = golden.legacyJsonAtom; + const source = { + id: '0xatom', + rawType: legacy.rawType, + data: legacy.data, + dataResolved: { name: 'resolved but dark' }, + classificationType: legacy.expectedIntent.classification, + }; + + expect(exposeAtomView(source, false)).toBe(source); + expect(exposeAtomView(source, false)).toEqual(source); + expect('display' in exposeAtomView(source, false)).toBe(false); + }); + + test('does not leak display-only query fields into legacy expanded terms', () => { + const source = byId(golden.resolutionCases, 'resolved-isrc').source; + + expect(exposeExpandedAtomTerm(source, false)).toEqual({ + id: source.id, + rawType: 'iid', + data: 'int:isrc:USQX91300108', + classificationType: 'MusicRecording', + }); + expect(exposeExpandedAtomTerm(source, true)).toMatchObject({ + display: { name: 'One Last Time' }, + raw: { type: 'iid', data: 'int:isrc:USQX91300108' }, + }); + }); + + test('adds an envelope while preserving legacy fields', () => { + const fixture = byId(golden.resolutionCases, 'resolved-isrc'); + const view = presentAtom(fixture.source); + + expect(view).toMatchObject({ + ...fixture.source, + raw: { type: fixture.source.rawType, data: fixture.source.data }, + classification: { type: 'MusicRecording', status: 'completed', source: 'iid-registry' }, + resolution: { + status: fixture.expected.resolutionStatus, + updatedAt: fixture.source.enrichedAt, + }, + display: { + name: fixture.expected.displayName, + image: fixture.expected.displayImage, + }, + }); + }); + + test('projects IID identity from the parser result without an IID package dependency', () => { + const identity = byId(golden.identityCases, 'p0-isrc-recording'); + const view = presentAtom({ + rawType: 'iid', + data: identity.input, + dataResolved: {}, + parseResult: { + kind: 'iid', + normalizedInput: identity.input, + identity: { + raw: identity.input, + canonical: identity.input, + profile: identity.expectedIntent.minimumProfile, + scheme: identity.expectedIntent.scheme, + value: identity.expectedIntent.value, + anchorEligible: true, + }, + }, + classificationType: identity.expectedIntent.classification ?? 'Unknown', + classificationStatus: 'completed', + enrichmentStatus: 'pending', + }); + + expect(view.identity).toEqual({ + raw: identity.input, + canonical: identity.input, + profile: identity.expectedIntent.minimumProfile, + scheme: identity.expectedIntent.scheme, + value: identity.expectedIntent.value, + anchorEligible: true, + }); + expect(view.resolution).toEqual({ status: 'pending' }); + }); + + test('preserves all Core producer identity metadata', () => { + const source = byId(golden.resolutionCases, 'unresolved-isrc').source; + const identity = (source.parseResult as { identity: Record }).identity; + + expect(presentAtom(source).identity).toEqual(identity); + }); + + test('retains compatibility with the legacy flat IID parse result', () => { + const source = byId(golden.resolutionCases, 'resolved-isrc').source; + const identity = presentAtom(source).identity; + + expect(identity).toMatchObject({ + raw: source.data, + scheme: 'isrc', + value: 'USQX91300108', + profile: 'p0', + }); + }); + + test('does not fabricate identity, context, resolution, or display evidence', () => { + const view = presentAtom({ + rawType: 'string', + data: 'plain atom', + classificationType: 'Unknown', + }); + + expect(view.raw).toEqual({ type: 'string', data: 'plain atom' }); + expect(view.classification).toEqual({ type: 'Unknown' }); + expect('identity' in view).toBe(false); + expect('context' in view).toBe(false); + expect('resolution' in view).toBe(false); + expect('display' in view).toBe(false); + }); + + test('preserves supplied context order and opaque values', () => { + const context = byId(golden.contextCases, 'ordered-duplicate-and-unsafe-context').entries.map( + ({ expectedLinkable: _expectedLinkable, ...entry }) => entry + ); + const view = presentAtom({ + rawType: 'string', + data: 'atom', + classificationType: 'Unknown', + context, + }); + + expect(view.context).toEqual(context); + expect(view.context).not.toBe(context); + }); + + test('exposes every modeled resolution state from the golden corpus', () => { + for (const fixture of golden.resolutionCases) { + const view = presentAtom(fixture.source); + expect(view.resolution?.status, fixture.id).toBe(fixture.expected.resolutionStatus); + expect(view.display?.name ?? null, fixture.id).toBe(fixture.expected.displayName); + } + }); +}); diff --git a/services/api/tests/config.test.ts b/services/api/tests/config.test.ts new file mode 100644 index 0000000..445e77e --- /dev/null +++ b/services/api/tests/config.test.ts @@ -0,0 +1,21 @@ +import { describe, expect, test } from 'bun:test'; +import { loadConfig, parseBooleanFlag } from '../src/config'; + +describe('semantic atom read configuration', () => { + test('is default-off', () => { + expect(loadConfig({ DATABASE_KG_URL: 'postgres://test' }).atomSemanticReadsEnabled).toBe(false); + }); + + test('accepts explicit boolean values', () => { + for (const value of ['true', 'TRUE', '1']) { + expect(parseBooleanFlag(value, 'FLAG')).toBe(true); + } + for (const value of ['false', 'FALSE', '0', '', undefined]) { + expect(parseBooleanFlag(value, 'FLAG')).toBe(false); + } + }); + + test('rejects ambiguous values instead of accidentally enabling exposure', () => { + expect(() => parseBooleanFlag('yes', 'FLAG')).toThrow('FLAG must be true, false, 1, or 0'); + }); +}); diff --git a/services/workers/src/core/classification.ts b/services/workers/src/core/classification.ts index 9d273d0..29a4192 100644 --- a/services/workers/src/core/classification.ts +++ b/services/workers/src/core/classification.ts @@ -5,6 +5,12 @@ import { createTypeProfilesPlugin, type JsonLdTypeDefinition, } from '@0xintuition/atom-classification'; +import type { + IdentityClassificationDecision, + IdentityProviderPlan, + NormalizedAtomIdentity, +} from './identity-contract'; +import type { IidSemanticResolution } from './iid-registry'; import type { CompactParseResult } from './parse'; import { resolveFallbackUrl, @@ -26,6 +32,9 @@ export type WorkerClassificationResult = { knownType?: boolean; targetUrl?: string; targetSource?: ClassificationTargetSource; + identity?: NormalizedAtomIdentity; + identityDecision?: IdentityClassificationDecision; + providerPlan?: IdentityProviderPlan; }; export type ClassificationPlan = { @@ -34,6 +43,7 @@ export type ClassificationPlan = { targetUrl: string | undefined; targetSource: ClassificationTargetSource | undefined; usesStructuredDocument: boolean; + identity?: NormalizedAtomIdentity; }; export function deriveClassificationPlan(input: { @@ -72,6 +82,7 @@ export function deriveClassificationPlan(input: { targetUrl, targetSource, usesStructuredDocument: true, + ...(parseResult?.identity ? { identity: parseResult.identity } : {}), }; } @@ -86,6 +97,7 @@ export function deriveClassificationPlan(input: { targetUrl: fallbackTarget.url, targetSource: fallbackTarget.source, usesStructuredDocument: false, + ...(parseResult?.identity ? { identity: parseResult.identity } : {}), }; } @@ -121,6 +133,24 @@ export function deriveClassificationResultFromRuntime(input: { }; } +export function deriveIidClassificationResult(input: { + identity: NormalizedAtomIdentity; + resolution: IidSemanticResolution; +}): WorkerClassificationResult { + const decision = input.resolution.identityDecision; + + return { + status: decision.status === 'classified' ? 'recognized' : 'not_applicable', + source: 'iid-registry', + ...(decision.schemaType ? { schemaType: decision.schemaType } : {}), + ...(decision.category ? { category: decision.category } : {}), + knownType: decision.status === 'classified', + identity: input.identity, + identityDecision: decision, + providerPlan: input.resolution.providerPlan, + }; +} + export function resolveClassificationType(result: WorkerClassificationResult): string { return result.schemaType ?? result.category ?? 'Unknown'; } diff --git a/services/workers/src/core/enrichment.test.ts b/services/workers/src/core/enrichment.test.ts index c96a89d..447e323 100644 --- a/services/workers/src/core/enrichment.test.ts +++ b/services/workers/src/core/enrichment.test.ts @@ -1,7 +1,10 @@ import { describe, expect, test } from 'bun:test'; import { buildClassifiedInputFromPlan, + buildEnrichmentCompletionPromotedFields, + buildIidProviderExecutionPlan, deriveEnrichmentPlan, + evaluateEnrichmentCompletion, evaluateEnrichmentProcessingScope, getArtifactTypeAllowListForEnrichmentPlan, } from './enrichment'; @@ -83,6 +86,14 @@ describe('KG enrichment core', () => { url: 'https://open.spotify.com/track/123', }, }); + expect(buildEnrichmentCompletionPromotedFields(plan)).toMatchObject({ + dataResolved: { + '@type': 'MusicRecording', + name: 'Fixture Track', + description: 'Track description', + }, + searchText: 'Fixture Track Track description', + }); }); test('normalizes classification category when building enrichment input', () => { @@ -181,6 +192,214 @@ describe('KG enrichment core', () => { expect(input).toBeNull(); }); + test('turns persisted identity handoffs into an explicit identifier enrichment request', () => { + const identity = { + raw: 'opaque raw identity', + canonical: 'opaque canonical identity', + scheme: 'fixture-scheme', + value: 'fixture-value', + profile: 'p0' as const, + anchorEligible: true, + provenance: { producer: 'adapter', version: 'test' }, + }; + const providerPlan = { + status: 'planned' as const, + targets: [ + { + provider: 'openlibrary', + capabilities: ['metadata'], + identifierHints: [{ kind: 'isbn', value: '9780684832722' }], + }, + ], + provenance: { producer: 'registry-adapter', version: 'test' }, + }; + const plan = deriveEnrichmentPlan({ + rawInput: null, + parseResult: { kind: 'iid', normalizedInput: identity.raw, identity }, + classificationResult: { + status: 'recognized', + source: 'future-adapter', + identity, + providerPlan, + }, + }); + + expect(plan.identity).toEqual(identity); + expect(plan.providerPlan).toEqual(providerPlan); + expect(buildClassifiedInputFromPlan(plan)).toMatchObject({ + hints: { identifiers: { isbn: '9780684832722' } }, + }); + expect(buildIidProviderExecutionPlan({ plan, registeredPluginIds: ['openlibrary'] })).toEqual({ + status: 'ready', + plugins: ['openlibrary'], + identifiers: { isbn: '9780684832722' }, + }); + expect(buildEnrichmentCompletionPromotedFields(plan)).toBeUndefined(); + expect( + buildEnrichmentCompletionPromotedFields(plan, [ + { + artifact_type: 'openlibrary', + data: { + title: 'The Sovereign Individual', + authors: ['James Dale Davidson', 'William Rees-Mogg'], + coverUrl: 'https://covers.example/gatsby.jpg', + }, + meta: { + pluginId: 'openlibrary', + provider: 'openlibrary', + fetchedAt: '2026-08-12T00:00:00.000Z', + sourceUrl: 'https://openlibrary.org/books/OL7721520M', + }, + }, + ]) + ).toMatchObject({ + dataResolved: { + name: 'The Sovereign Individual', + image: 'https://covers.example/gatsby.jpg', + resolution: { provider: 'openlibrary', identity: identity.canonical }, + }, + searchText: 'The Sovereign Individual James Dale Davidson William Rees-Mogg', + }); + }); + + test('distinguishes retryable plugin drift from terminal unknown provider drift', () => { + const basePlan = { + targetUrl: undefined, + structuredDocument: undefined, + identity: { + raw: 'int:isbn:9780684832722', + canonical: 'int:isbn:9780684832722', + scheme: 'isbn', + value: '9780684832722', + anchorEligible: true, + provenance: { producer: 'adapter', version: 'test' }, + }, + classificationResult: { status: 'recognized' as const, source: 'iid-registry' }, + }; + const target = { + capabilities: ['metadata'], + identifierHints: [{ kind: 'isbn', value: '9780684832722' }], + }; + + expect( + buildIidProviderExecutionPlan({ + plan: { + ...basePlan, + providerPlan: { + status: 'planned', + targets: [{ ...target, provider: 'openlibrary' }], + provenance: { producer: 'registry', version: 'test' }, + }, + }, + registeredPluginIds: [], + }) + ).toMatchObject({ status: 'blocked', retriable: true }); + expect( + buildIidProviderExecutionPlan({ + plan: { + ...basePlan, + providerPlan: { + status: 'planned', + targets: [{ ...target, provider: 'not-a-provider' }], + provenance: { producer: 'registry', version: 'test' }, + }, + }, + registeredPluginIds: [], + }) + ).toMatchObject({ status: 'blocked', retriable: false }); + }); + + test('classifies zero-artifact outcomes without reporting terminal misses as resolved', () => { + const retryableError = { + pluginId: 'fixture-provider', + code: 'rate_limited' as const, + message: 'retry later', + retriable: true, + }; + const skipped = [{ pluginId: 'not-applicable-provider', reason: 'not applicable' }]; + + expect( + evaluateEnrichmentCompletion({ artifacts: [], errors: [retryableError], skipped }) + ).toMatchObject({ + kind: 'retryable_failure', + diagnostics: { errors: [retryableError], skipped }, + }); + expect( + evaluateEnrichmentCompletion({ + artifacts: [], + errors: [{ ...retryableError, retriable: false }], + skipped, + }) + ).toMatchObject({ + kind: 'terminal_unresolved', + diagnostics: { errors: [{ ...retryableError, retriable: false }], skipped }, + }); + expect(evaluateEnrichmentCompletion({ artifacts: [], errors: [], skipped })).toMatchObject({ + kind: 'terminal_unresolved', + diagnostics: { errors: [], skipped }, + }); + }); + + test('bounds retained diagnostics for zero-artifact outcomes', () => { + const result = evaluateEnrichmentCompletion({ + artifacts: [], + errors: Array.from({ length: 30 }, (_, index) => ({ + pluginId: `provider-${index}`, + code: 'validation_error' as const, + message: 'x'.repeat(1_500), + retriable: false, + })), + skipped: Array.from({ length: 30 }, (_, index) => ({ + pluginId: `skipped-${index}`, + reason: 'y'.repeat(500), + })), + }); + + expect(result.kind).toBe('terminal_unresolved'); + if (result.kind !== 'terminal_unresolved') { + throw new Error('expected terminal unresolved result'); + } + expect(result.diagnostics.errors).toHaveLength(25); + expect(result.diagnostics.skipped).toHaveLength(25); + expect(result.diagnostics.errors[0]?.message).toHaveLength(1_000); + expect(result.diagnostics.skipped[0]?.reason).toHaveLength(256); + expect(result.diagnostics).toMatchObject({ + totalErrors: 30, + totalSkipped: 30, + errorsTruncated: true, + skippedTruncated: true, + }); + }); + + test('completes partial enrichment while retaining explicit errors and skips', () => { + const result = { + artifacts: [ + { + artifact_type: 'opengraph', + data: { title: 'Partial result' }, + meta: { + pluginId: 'opengraph', + provider: 'fixture', + fetchedAt: '2026-08-10T00:00:00.000Z', + }, + }, + ], + errors: [ + { + pluginId: 'fixture-provider', + code: 'upstream_error' as const, + message: 'retry later', + retriable: true, + }, + ], + skipped: [{ pluginId: 'another-provider', reason: 'not applicable' }], + }; + + expect(evaluateEnrichmentCompletion(result)).toEqual({ kind: 'complete' }); + expect(result.errors).toHaveLength(1); + expect(result.skipped).toHaveLength(1); + }); + test('keeps full processing scope behavior unchanged', () => { const plan = deriveEnrichmentPlan({ rawInput: 'https://example.com', diff --git a/services/workers/src/core/enrichment.ts b/services/workers/src/core/enrichment.ts index 80b9f1e..70b526e 100644 --- a/services/workers/src/core/enrichment.ts +++ b/services/workers/src/core/enrichment.ts @@ -1,10 +1,20 @@ -import type { ClassifiedAtomInput } from '@0xintuition/atom-enrichment'; +import { + type ClassifiedAtomInput, + createIdentifierProviderPlan, + type EnrichmentRunResult, + type IdentifierProviderPlanEntry, +} from '@0xintuition/atom-enrichment'; import { getProcessingScopeDomains, type ProcessingDomain, type ProcessingScopePreset, } from '../shared/processing-scope'; import type { WorkerClassificationResult } from './classification'; +import type { + IdentityClassificationDecision, + IdentityProviderPlan, + NormalizedAtomIdentity, +} from './identity-contract'; import type { CompactParseResult } from './parse'; import { resolveFallbackUrl, resolveStructuredDocumentTarget } from './structured-targets'; @@ -38,8 +48,29 @@ export type EnrichmentPlan = { classificationResult: WorkerClassificationResult; targetUrl: string | undefined; structuredDocument: CompactParseResult['structuredDocument']; + identity?: NormalizedAtomIdentity; + identityDecision?: IdentityClassificationDecision; + providerPlan?: IdentityProviderPlan; +}; + +export type EnrichmentCompletionPromotedFields = { + dataResolved: Record; + searchText: string; }; +export type IidProviderExecutionPlan = + | { + status: 'ready'; + plugins: string[]; + identifiers: Record; + } + | { + status: 'blocked'; + retriable: boolean; + reason: string; + entries: IdentifierProviderPlanEntry[]; + }; + export type ScopedEnrichmentDecision = | { shouldEnrich: true; @@ -52,6 +83,77 @@ export type ScopedEnrichmentDecision = matchedDomains: ProcessingDomain[]; }; +export type EnrichmentCompletionDisposition = + | { kind: 'complete' } + | { + kind: 'retryable_failure'; + diagnostics: EnrichmentDiagnostics; + } + | { + kind: 'terminal_unresolved'; + diagnostics: EnrichmentDiagnostics; + }; + +export type EnrichmentDiagnostics = { + errors: EnrichmentRunResult['errors']; + skipped: EnrichmentRunResult['skipped']; + totalErrors: number; + totalSkipped: number; + errorsTruncated: boolean; + skippedTruncated: boolean; +}; + +const MAX_ENRICHMENT_DIAGNOSTICS_PER_KIND = 25; +const MAX_ENRICHMENT_ERROR_MESSAGE_LENGTH = 1_000; +const MAX_ENRICHMENT_SKIP_REASON_LENGTH = 256; + +/** + * A partial run is useful and completes. A run with no artifacts retries only + * when every explicit provider failure is marked retryable by the enrichment + * engine. A zero-artifact terminal error or all-skipped run is terminally + * unresolved; it must never be presented as a successful resolution. + */ +export function evaluateEnrichmentCompletion( + result: Pick +): EnrichmentCompletionDisposition { + if ( + result.artifacts.length === 0 && + result.errors.length > 0 && + result.errors.every((error) => error.retriable) + ) { + return { kind: 'retryable_failure', diagnostics: boundEnrichmentDiagnostics(result) }; + } + + if (result.artifacts.length === 0) { + return { kind: 'terminal_unresolved', diagnostics: boundEnrichmentDiagnostics(result) }; + } + + return { kind: 'complete' }; +} + +/** + * Keep terminal/retry evidence useful without allowing provider fan-out or + * messages to create an unbounded processing-error row. + */ +export function boundEnrichmentDiagnostics( + result: Pick +): EnrichmentDiagnostics { + return { + errors: result.errors.slice(0, MAX_ENRICHMENT_DIAGNOSTICS_PER_KIND).map((error) => ({ + ...error, + message: error.message.slice(0, MAX_ENRICHMENT_ERROR_MESSAGE_LENGTH), + })), + skipped: result.skipped.slice(0, MAX_ENRICHMENT_DIAGNOSTICS_PER_KIND).map((entry) => ({ + ...entry, + reason: entry.reason.slice(0, MAX_ENRICHMENT_SKIP_REASON_LENGTH), + })), + totalErrors: result.errors.length, + totalSkipped: result.skipped.length, + errorsTruncated: result.errors.length > MAX_ENRICHMENT_DIAGNOSTICS_PER_KIND, + skippedTruncated: result.skipped.length > MAX_ENRICHMENT_DIAGNOSTICS_PER_KIND, + }; +} + export function deriveEnrichmentPlan(input: { parseResult: CompactParseResult | null; classificationResult: WorkerClassificationResult; @@ -63,15 +165,64 @@ export function deriveEnrichmentPlan(input: { structuredTarget.url ?? resolveFallbackUrl(input.parseResult, input.rawInput); + const identity = input.classificationResult.identity ?? input.parseResult?.identity; return { classificationResult: input.classificationResult, targetUrl, structuredDocument: input.parseResult?.structuredDocument, + ...(identity ? { identity } : {}), + ...(input.classificationResult.identityDecision + ? { identityDecision: input.classificationResult.identityDecision } + : {}), + ...(input.classificationResult.providerPlan + ? { providerPlan: input.classificationResult.providerPlan } + : {}), }; } +/** + * Keeps the existing structured-document projection in the enrichment + * completion transaction. Identity-derived projections remain deliberately + * unplugged until the public resolver package provides resolved presentation + * data and provenance. + */ +export function buildEnrichmentCompletionPromotedFields( + plan: EnrichmentPlan, + artifacts: EnrichmentRunResult['artifacts'] = [] +): EnrichmentCompletionPromotedFields | undefined { + if (plan.identity) { + return buildIdentityPromotedFields(plan, artifacts); + } + + if (plan.structuredDocument?.topLevelType !== 'object') { + return undefined; + } + + const dataResolved = toRecordMaybe(plan.structuredDocument.data); + if (!dataResolved) { + return undefined; + } + + const name = resolveDisplayText(dataResolved.name); + const description = resolveDisplayText(dataResolved.description); + const searchText = [name, description] + .filter((value): value is string => value !== undefined) + .join(' ') + .slice(0, 20_000); + if (!searchText) { + return undefined; + } + + return { dataResolved, searchText }; +} + export function buildClassifiedInputFromPlan(plan: EnrichmentPlan): ClassifiedAtomInput | null { - if (!plan.targetUrl && plan.structuredDocument?.topLevelType !== 'object') { + const identifiers = collectIdentityIdentifierHints(plan); + if ( + !plan.targetUrl && + plan.structuredDocument?.topLevelType !== 'object' && + Object.keys(identifiers).length === 0 + ) { return null; } @@ -85,6 +236,7 @@ export function buildClassifiedInputFromPlan(plan: EnrichmentPlan): ClassifiedAt ...(name ? { name } : {}), ...(description ? { description } : {}), ...(plan.targetUrl ? { url: plan.targetUrl } : {}), + ...(Object.keys(identifiers).length > 0 ? { identifiers } : {}), }; return { @@ -106,6 +258,145 @@ export function buildClassifiedInputFromPlan(plan: EnrichmentPlan): ClassifiedAt }; } +/** + * Converts the persisted public-registry plan into the exact plugin request + * understood by Core's enrichment runtime. Any registry/runtime drift remains + * explicit: unknown providers are terminal and missing deployed plugins are + * retryable. The worker never silently falls back to running every plugin. + */ +export function buildIidProviderExecutionPlan(input: { + plan: EnrichmentPlan; + registeredPluginIds: Iterable; +}): IidProviderExecutionPlan | undefined { + if (!input.plan.identity) { + return undefined; + } + + const providerPlan = input.plan.providerPlan; + if (!providerPlan || providerPlan.status !== 'planned' || providerPlan.targets.length === 0) { + return { + status: 'blocked', + retriable: false, + reason: 'The IID registry did not provide an executable provider plan.', + entries: [], + }; + } + + const execution = createIdentifierProviderPlan({ + providers: providerPlan.targets.map((target) => target.provider), + identifiers: collectIdentityIdentifierHints(input.plan), + registeredPluginIds: input.registeredPluginIds, + }); + const blockers = execution.entries.filter((entry) => entry.status !== 'scheduled'); + if (blockers.length > 0) { + const hasTerminalBlocker = blockers.some((entry) => entry.disposition === 'terminal'); + return { + status: 'blocked', + retriable: !hasTerminalBlocker, + reason: hasTerminalBlocker + ? 'The IID provider plan contains an unsupported provider capability.' + : 'The IID provider plan requires a plugin that is not registered in this runtime.', + entries: blockers, + }; + } + + return { + status: 'ready', + plugins: execution.plugins, + identifiers: execution.identifiers, + }; +} + +function collectIdentityIdentifierHints(plan: EnrichmentPlan): Record { + const identifiers: Record = {}; + for (const target of plan.providerPlan?.targets ?? []) { + for (const hint of target.identifierHints) { + const existing = identifiers[hint.kind]; + if (existing !== undefined && existing !== hint.value) { + throw new Error( + `IID provider plan supplied conflicting values for identifier hint "${hint.kind}".` + ); + } + identifiers[hint.kind] = hint.value; + } + } + return identifiers; +} + +function buildIdentityPromotedFields( + plan: EnrichmentPlan, + artifacts: EnrichmentRunResult['artifacts'] +): EnrichmentCompletionPromotedFields | undefined { + const primary = artifacts[0]; + if (!primary) { + return undefined; + } + + const data = toRecordMaybe(primary.data) ?? {}; + const name = resolveDisplayText(data.name) ?? resolveDisplayText(data.title); + const description = resolveDisplayText(data.description) ?? resolveDisplayText(data.summary); + const image = + resolveHttpUrl(data.image) ?? + resolveHttpUrl(data.imageUrl) ?? + resolveHttpUrl(data.coverUrl) ?? + resolveHttpUrl(data.thumbnailUrl) ?? + resolveHttpUrl(data.logoUrl); + const provider = resolveDisplayText(primary.meta.provider); + const sourceUrl = resolveDisplayText(primary.meta.sourceUrl); + const searchText = [ + name, + description, + resolveDisplayText(data.artistCredit), + resolveDisplayText(data.publisher), + resolveDisplayList(data.authors), + ] + .filter((value): value is string => value !== undefined) + .join(' ') + .slice(0, 20_000); + + return { + dataResolved: { + ...(name ? { name } : {}), + ...(description ? { description } : {}), + ...(image ? { image } : {}), + resolvedAtom: data, + resolution: { + artifactType: primary.artifact_type, + ...(provider ? { provider } : {}), + ...(sourceUrl ? { sourceUrl } : {}), + identity: plan.identity?.canonical, + providerPlanProvenance: plan.providerPlan?.provenance, + }, + }, + searchText, + }; +} + +function resolveDisplayList(value: unknown): string | undefined { + if (!Array.isArray(value)) { + return undefined; + } + const displayValues = value + .map((entry) => resolveDisplayText(entry)) + .filter((entry): entry is string => entry !== undefined); + return displayValues.length > 0 ? displayValues.join(' ') : undefined; +} + +function resolveHttpUrl(value: unknown): string | undefined { + const text = resolveString(value); + if (!text) { + return undefined; + } + try { + const parsed = new URL(text); + return parsed.protocol === 'http:' || parsed.protocol === 'https:' + ? parsed.toString() + : undefined; + } catch { + return undefined; + } +} + function resolveAtomType(value: string | undefined): ClassifiedAtomInput['atomType'] { const normalized = value?.toLowerCase(); switch (normalized) { diff --git a/services/workers/src/core/identity-contract.test.ts b/services/workers/src/core/identity-contract.test.ts new file mode 100644 index 0000000..58480cc --- /dev/null +++ b/services/workers/src/core/identity-contract.test.ts @@ -0,0 +1,94 @@ +import { describe, expect, test } from 'bun:test'; +import { + isIdentityClassificationDecision, + isIdentityProviderPlan, + isNormalizedAtomIdentity, +} from './identity-contract'; + +const provenance = { + producer: 'public-package-adapter', + version: '0.1.0-test', + specificationVersion: 'fixture-v1', +}; + +describe('identity handoff contracts', () => { + test('accepts normalized identity without interpreting its scheme or value', () => { + expect( + isNormalizedAtomIdentity({ + raw: 'opaque raw identity', + canonical: 'opaque canonical identity', + scheme: 'fixture-scheme', + value: 'value:with:colons', + profile: 'p0', + class: 'A', + typing: 'unambiguous', + anchorEligible: true, + provenance, + }) + ).toBe(true); + expect( + isNormalizedAtomIdentity({ + raw: 'same identity without representation context', + canonical: 'opaque canonical identity', + scheme: 'fixture-scheme', + value: 'value:with:colons', + class: 'C', + typing: 'polymorphic', + anchorIneligibilityReason: 'polymorphic-scheme', + anchorEligible: false, + provenance, + }) + ).toBe(true); + }); + + test('rejects incomplete normalized identity records', () => { + expect(isNormalizedAtomIdentity({ canonical: 'missing-fields' })).toBe(false); + expect( + isNormalizedAtomIdentity({ + raw: 'raw', + canonical: 'canonical', + scheme: 'scheme', + value: 'value', + profile: 'future-profile', + anchorEligible: true, + provenance, + }) + ).toBe(false); + expect( + isNormalizedAtomIdentity({ + raw: 'raw', + canonical: 'canonical', + scheme: 'scheme', + value: 'value', + class: 'unknown-class', + anchorEligible: false, + provenance, + }) + ).toBe(false); + }); + + test('accepts classification decisions and ordered provider plans', () => { + expect( + isIdentityClassificationDecision({ + status: 'classified', + classificationSlug: 'music-recording', + schemaType: 'MusicRecording', + provenance, + }) + ).toBe(true); + + expect( + isIdentityProviderPlan({ + status: 'planned', + targets: [ + { + provider: 'music-provider', + capabilities: ['recording-metadata'], + identifierHints: [{ kind: 'recording-code', value: 'fixture-value' }], + }, + ], + provenance, + }) + ).toBe(true); + }); +}); diff --git a/services/workers/src/core/identity-contract.ts b/services/workers/src/core/identity-contract.ts new file mode 100644 index 0000000..904ce66 --- /dev/null +++ b/services/workers/src/core/identity-contract.ts @@ -0,0 +1,148 @@ +/** + * Core-owned persistence and worker-handoff contracts for normalized identity. + * + * These DTOs intentionally contain no IID grammar, scheme registry, semantic + * mappings, or provider selection logic. Future adapters map public package + * results into these stable Core records before they cross worker/database + * boundaries. + */ + +export type SemanticContractProvenance = { + producer: string; + version: string; + specificationVersion?: string; +}; + +export type NormalizedAtomIdentity = { + raw: string; + canonical: string; + scheme: string; + value: string; + /** Atom representation profile, when the producing adapter has that context. */ + profile?: 'p0' | 'p1' | 'p2'; + /** Public inspection metadata; optional so older persisted records remain valid. */ + class?: 'A' | 'B' | 'C'; + typing?: 'unambiguous' | 'polymorphic'; + anchorIneligibilityReason?: 'class-c' | 'polymorphic-scheme'; + anchorEligible: boolean; + provenance: SemanticContractProvenance; +}; + +export type IdentityClassificationDecision = { + status: 'classified' | 'unmapped' | 'ambiguous'; + classificationSlug?: string; + schemaType?: string; + category?: string; + provenance: SemanticContractProvenance; +}; + +export type IdentityProviderHint = { + kind: string; + value: string; +}; + +export type IdentityProviderTarget = { + provider: string; + capabilities: readonly string[]; + identifierHints: readonly IdentityProviderHint[]; +}; + +export type IdentityProviderPlan = { + status: 'planned' | 'unsupported'; + targets: readonly IdentityProviderTarget[]; + provenance: SemanticContractProvenance; +}; + +export function isNormalizedAtomIdentity(value: unknown): value is NormalizedAtomIdentity { + if (!isRecord(value)) return false; + return ( + isNonEmptyString(value.raw) && + isNonEmptyString(value.canonical) && + isNonEmptyString(value.scheme) && + isNonEmptyString(value.value) && + isOptionalIdentityProfile(value.profile) && + isOptionalIdentityClass(value.class) && + isOptionalIdentityTyping(value.typing) && + isOptionalAnchorIneligibilityReason(value.anchorIneligibilityReason) && + typeof value.anchorEligible === 'boolean' && + isSemanticContractProvenance(value.provenance) + ); +} + +export function isIdentityClassificationDecision( + value: unknown +): value is IdentityClassificationDecision { + if (!isRecord(value) || !isClassificationDecisionStatus(value.status)) return false; + if (!isOptionalNonEmptyString(value.classificationSlug)) return false; + if (!isOptionalNonEmptyString(value.schemaType)) return false; + if (!isOptionalNonEmptyString(value.category)) return false; + return isSemanticContractProvenance(value.provenance); +} + +export function isIdentityProviderPlan(value: unknown): value is IdentityProviderPlan { + if (!isRecord(value) || (value.status !== 'planned' && value.status !== 'unsupported')) { + return false; + } + if (!Array.isArray(value.targets) || !value.targets.every(isIdentityProviderTarget)) { + return false; + } + return isSemanticContractProvenance(value.provenance); +} + +function isIdentityProviderTarget(value: unknown): value is IdentityProviderTarget { + if (!isRecord(value) || !isNonEmptyString(value.provider)) return false; + if (!Array.isArray(value.capabilities) || !value.capabilities.every(isNonEmptyString)) + return false; + return ( + Array.isArray(value.identifierHints) && value.identifierHints.every(isIdentityProviderHint) + ); +} + +function isIdentityProviderHint(value: unknown): value is IdentityProviderHint { + return isRecord(value) && isNonEmptyString(value.kind) && isNonEmptyString(value.value); +} + +export function isSemanticContractProvenance(value: unknown): value is SemanticContractProvenance { + return ( + isRecord(value) && + isNonEmptyString(value.producer) && + isNonEmptyString(value.version) && + isOptionalNonEmptyString(value.specificationVersion) + ); +} + +function isClassificationDecisionStatus( + value: unknown +): value is IdentityClassificationDecision['status'] { + return value === 'classified' || value === 'unmapped' || value === 'ambiguous'; +} + +function isOptionalIdentityProfile(value: unknown): value is NormalizedAtomIdentity['profile'] { + return value === undefined || value === 'p0' || value === 'p1' || value === 'p2'; +} + +function isOptionalIdentityClass(value: unknown): value is NormalizedAtomIdentity['class'] { + return value === undefined || value === 'A' || value === 'B' || value === 'C'; +} + +function isOptionalIdentityTyping(value: unknown): value is NormalizedAtomIdentity['typing'] { + return value === undefined || value === 'unambiguous' || value === 'polymorphic'; +} + +function isOptionalAnchorIneligibilityReason( + value: unknown +): value is NormalizedAtomIdentity['anchorIneligibilityReason'] { + return value === undefined || value === 'class-c' || value === 'polymorphic-scheme'; +} + +function isOptionalNonEmptyString(value: unknown): boolean { + return value === undefined || isNonEmptyString(value); +} + +function isNonEmptyString(value: unknown): value is string { + return typeof value === 'string' && value.trim().length > 0; +} + +function isRecord(value: unknown): value is Record { + return value !== null && typeof value === 'object' && !Array.isArray(value); +} diff --git a/services/workers/src/core/iid-inspection.test.ts b/services/workers/src/core/iid-inspection.test.ts new file mode 100644 index 0000000..8332073 --- /dev/null +++ b/services/workers/src/core/iid-inspection.test.ts @@ -0,0 +1,173 @@ +import { describe, expect, test } from 'bun:test'; +import type { ParseResult } from '@0xintuition/atom-parser/types'; +import { WorkerConfigurationError } from '../shared/errors'; +import { + createIidInspectionAdapter, + type PublicIidInspection, + parseAtomWithIidRead, +} from './iid-inspection'; + +const PACKAGE_VERSION = '0.1.0-alpha.0'; +const SPECIFICATION_VERSION = '@0xintuition/iid-spec@0.1.0-alpha.0'; + +describe('IID inspection adapter', () => { + test('maps a public valid inspection without reconstructing colon-bearing values', async () => { + const rawInput = 'int:tmdb:movie:603'; + const adapter = createIidInspectionAdapter({ + inspectIntuitionId: () => ({ + valid: true, + iid: rawInput, + scheme: 'tmdb', + value: 'movie:603', + class: 'C', + typing: 'polymorphic', + anchorEligible: false, + anchorIneligibilityReason: 'class-c', + }), + packageVersion: PACKAGE_VERSION, + specificationVersion: SPECIFICATION_VERSION, + }); + let legacyCalls = 0; + + const outcome = await parseAtomWithIidRead({ + rawInput, + iidReadEnabled: true, + adapter, + parseLegacy: () => { + legacyCalls += 1; + return legacyResult(rawInput); + }, + }); + + expect(legacyCalls).toBe(0); + expect(outcome).toEqual({ + result: { + kind: 'iid', + normalizedInput: rawInput, + canonicalId: rawInput, + identity: { + raw: rawInput, + canonical: rawInput, + scheme: 'tmdb', + value: 'movie:603', + class: 'C', + typing: 'polymorphic', + anchorEligible: false, + anchorIneligibilityReason: 'class-c', + provenance: { + producer: '@0xintuition/iid/inspectIntuitionId', + version: PACKAGE_VERSION, + specificationVersion: SPECIFICATION_VERSION, + }, + }, + }, + }); + expect(outcome.result.identity).not.toHaveProperty('profile'); + }); + + test('preserves the legacy path for every invalid reason and ignores repairs', async () => { + const cases: PublicIidInspection[] = [ + { valid: false, reason: 'malformed' }, + { valid: false, reason: 'unknown-scheme', scheme: 'src', value: '123' }, + { + valid: false, + reason: 'noncanonical', + scheme: 'isbn', + value: '0-684-83272-0', + canonical: 'int:isbn:9780684832722', + }, + ]; + + for (const inspection of cases) { + const rawInput = inspection.valid ? '' : `historical-${inspection.reason}`; + const adapter = createIidInspectionAdapter({ + inspectIntuitionId: () => inspection, + packageVersion: PACKAGE_VERSION, + }); + const outcome = await parseAtomWithIidRead({ + rawInput, + iidReadEnabled: true, + adapter, + parseLegacy: () => legacyResult(rawInput), + }); + + expect(outcome.fallbackReason).toBe(inspection.valid ? undefined : inspection.reason); + expect(outcome.result).toMatchObject({ + kind: 'plain_string', + normalizedInput: rawInput, + canonicalId: rawInput, + iidFallback: { + reason: inspection.valid ? undefined : inspection.reason, + provenance: { + producer: '@0xintuition/iid/inspectIntuitionId', + version: PACKAGE_VERSION, + }, + }, + }); + expect(outcome.result.identity).toBeUndefined(); + } + }); + + test('keeps disabled behavior byte-compatible and does not call public inspection', async () => { + const rawInput = 'int:isrc:USRC17607839'; + let inspectionCalls = 0; + const adapter = createIidInspectionAdapter({ + inspectIntuitionId: () => { + inspectionCalls += 1; + return validIsrcInspection(rawInput); + }, + packageVersion: PACKAGE_VERSION, + }); + + const outcome = await parseAtomWithIidRead({ + rawInput, + iidReadEnabled: false, + adapter, + parseLegacy: () => legacyResult(rawInput), + }); + + expect(inspectionCalls).toBe(0); + expect(outcome).toEqual({ + result: { + kind: 'plain_string', + normalizedInput: rawInput, + canonicalId: rawInput, + hints: { trimmed: rawInput }, + }, + }); + }); + + test('fails closed when the read flag is enabled without a public adapter', async () => { + expect( + parseAtomWithIidRead({ + rawInput: 'int:isrc:USRC17607839', + iidReadEnabled: true, + parseLegacy: () => legacyResult('int:isrc:USRC17607839'), + }) + ).rejects.toBeInstanceOf(WorkerConfigurationError); + }); +}); + +function validIsrcInspection(iid: string): PublicIidInspection { + return { + valid: true, + iid, + scheme: 'isrc', + value: 'USRC17607839', + class: 'A', + typing: 'unambiguous', + anchorEligible: true, + }; +} + +function legacyResult(input: string): ParseResult { + return { + kind: 'plain_string', + input, + normalizedInput: input, + warnings: [], + structuredDocument: undefined, + original: input, + trimmed: input, + }; +} diff --git a/services/workers/src/core/iid-inspection.ts b/services/workers/src/core/iid-inspection.ts new file mode 100644 index 0000000..b958da9 --- /dev/null +++ b/services/workers/src/core/iid-inspection.ts @@ -0,0 +1,128 @@ +import type { ParseResult } from '@0xintuition/atom-parser/types'; +import { WorkerConfigurationError } from '../shared/errors'; +import type { SemanticContractProvenance } from './identity-contract'; +import { type CompactParseResult, toCompactParseResult } from './parse'; + +export type PublicIidInspection = + | { + readonly valid: true; + readonly iid: string; + readonly scheme: string; + readonly value: string; + readonly class: 'A' | 'B' | 'C'; + readonly typing: 'unambiguous' | 'polymorphic'; + readonly anchorEligible: boolean; + readonly anchorIneligibilityReason?: 'class-c' | 'polymorphic-scheme'; + } + | { + readonly valid: false; + readonly reason: 'malformed' | 'unknown-scheme' | 'noncanonical'; + readonly scheme?: string; + readonly value?: string; + readonly canonical?: string; + }; + +export type IidFallbackReason = Extract['reason']; + +export type IidInspectionAdapter = { + inspect(rawInput: string): PublicIidInspection; + provenance: SemanticContractProvenance; +}; + +export type IidParseOutcome = { + result: CompactParseResult; + fallbackReason?: IidFallbackReason; +}; + +/** + * Binds Core to the public inspection function without copying IID grammar or + * committing an unpublished package dependency. The composition root must pass + * the exact consumed package/corpus versions when the packages become eligible. + */ +export function createIidInspectionAdapter(input: { + inspectIntuitionId: (rawInput: string) => PublicIidInspection; + packageVersion: string; + specificationVersion?: string; +}): IidInspectionAdapter { + return { + inspect: input.inspectIntuitionId, + provenance: { + producer: '@0xintuition/iid/inspectIntuitionId', + version: requireVersion(input.packageVersion, '@0xintuition/iid'), + ...(input.specificationVersion + ? { + specificationVersion: requireVersion( + input.specificationVersion, + '@0xintuition/iid-spec' + ), + } + : {}), + }, + }; +} + +/** + * Runs public IID inspection before generic parsing only when the real read flag + * is enabled. Invalid historical values take the unchanged legacy parser path; + * even a supplied canonical repair is deliberately ignored here. + */ +export async function parseAtomWithIidRead(input: { + rawInput: string; + iidReadEnabled: boolean; + adapter?: IidInspectionAdapter; + parseLegacy: () => ParseResult | Promise; +}): Promise { + if (!input.iidReadEnabled) { + return { result: toCompactParseResult(await input.parseLegacy()) }; + } + + if (!input.adapter) { + throw new WorkerConfigurationError( + 'WORKERS_IID_READ_ENABLED requires an @0xintuition/iid inspection adapter.' + ); + } + + const inspection = input.adapter.inspect(input.rawInput); + if (!inspection.valid) { + const result = toCompactParseResult(await input.parseLegacy()); + return { + result: { + ...result, + iidFallback: { + reason: inspection.reason, + provenance: input.adapter.provenance, + }, + }, + fallbackReason: inspection.reason, + }; + } + + return { + result: { + kind: 'iid', + normalizedInput: inspection.iid, + canonicalId: inspection.iid, + identity: { + raw: input.rawInput, + canonical: inspection.iid, + scheme: inspection.scheme, + value: inspection.value, + class: inspection.class, + typing: inspection.typing, + anchorEligible: inspection.anchorEligible, + ...(inspection.anchorIneligibilityReason + ? { anchorIneligibilityReason: inspection.anchorIneligibilityReason } + : {}), + provenance: input.adapter.provenance, + }, + }, + }; +} + +function requireVersion(value: string, packageName: string): string { + const version = value.trim(); + if (!version) { + throw new WorkerConfigurationError(`${packageName} adapter requires an exact version.`); + } + return version; +} diff --git a/services/workers/src/core/iid-registry.test.ts b/services/workers/src/core/iid-registry.test.ts new file mode 100644 index 0000000..fec439b --- /dev/null +++ b/services/workers/src/core/iid-registry.test.ts @@ -0,0 +1,140 @@ +import { describe, expect, test } from 'bun:test'; +import { deriveIidClassificationResult } from './classification'; +import type { NormalizedAtomIdentity } from './identity-contract'; +import { createIidRegistryAdapter } from './iid-registry'; + +const PACKAGE_VERSION = '0.1.0-alpha.0'; + +describe('IID registry adapter', () => { + test('preserves public classification, provider order, hints, and package provenance', () => { + const calls: string[] = []; + const adapter = createIidRegistryAdapter({ + classificationForIid: (iid) => { + calls.push(`classification:${iid}`); + return { + slug: 'music-recording', + schemaType: 'MusicRecording', + displayName: 'Music Recording', + category: 'Media', + }; + }, + providersForIid: (iid) => { + calls.push(`providers:${iid}`); + return ['musicbrainz', 'spotify']; + }, + identifierHintsForIid: (iid) => { + calls.push(`hints:${iid}`); + return { isrc: 'USRC17607839', recordingCode: 'USRC17607839' }; + }, + packageVersion: PACKAGE_VERSION, + }); + const identity = identityFixture({ + canonical: 'int:isrc:USRC17607839', + scheme: 'isrc', + value: 'USRC17607839', + }); + + const resolution = adapter.resolve(identity); + + expect(calls).toEqual([ + 'classification:int:isrc:USRC17607839', + 'providers:int:isrc:USRC17607839', + 'hints:int:isrc:USRC17607839', + ]); + expect(resolution).toEqual({ + identityDecision: { + status: 'classified', + classificationSlug: 'music-recording', + schemaType: 'MusicRecording', + category: 'Media', + provenance: { + producer: '@0xintuition/iid-registry/classificationForIid', + version: PACKAGE_VERSION, + }, + }, + providerPlan: { + status: 'planned', + targets: [ + { + provider: 'musicbrainz', + capabilities: ['musicbrainz'], + identifierHints: [ + { kind: 'isrc', value: 'USRC17607839' }, + { kind: 'recordingCode', value: 'USRC17607839' }, + ], + }, + { + provider: 'spotify', + capabilities: ['spotify'], + identifierHints: [ + { kind: 'isrc', value: 'USRC17607839' }, + { kind: 'recordingCode', value: 'USRC17607839' }, + ], + }, + ], + provenance: { + producer: '@0xintuition/iid-registry/providersForIid+identifierHintsForIid', + version: PACKAGE_VERSION, + }, + }, + }); + expect(deriveIidClassificationResult({ identity, resolution })).toMatchObject({ + status: 'recognized', + source: 'iid-registry', + schemaType: 'MusicRecording', + category: 'Media', + knownType: true, + identity, + identityDecision: { status: 'classified' }, + providerPlan: { status: 'planned' }, + }); + }); + + test('distinguishes polymorphic ambiguity from ratified-unmapped identity', () => { + const adapter = createIidRegistryAdapter({ + classificationForIid: () => undefined, + providersForIid: () => [], + identifierHintsForIid: () => ({}), + packageVersion: PACKAGE_VERSION, + }); + const ambiguous = adapter.resolve( + identityFixture({ + canonical: 'int:wd:Q42', + scheme: 'wd', + value: 'Q42', + typing: 'polymorphic', + anchorEligible: false, + }) + ); + const unmapped = adapter.resolve( + identityFixture({ + canonical: 'int:iswc:T-034.524.680-1', + scheme: 'iswc', + value: 'T-034.524.680-1', + }) + ); + + expect(ambiguous.identityDecision.status).toBe('ambiguous'); + expect(unmapped.identityDecision.status).toBe('unmapped'); + expect(ambiguous.providerPlan).toMatchObject({ status: 'unsupported', targets: [] }); + expect(unmapped.providerPlan).toMatchObject({ status: 'unsupported', targets: [] }); + }); +}); + +function identityFixture(overrides: Partial): NormalizedAtomIdentity { + const canonical = overrides.canonical ?? 'int:isrc:USRC17607839'; + return { + raw: canonical, + canonical, + scheme: 'isrc', + value: 'USRC17607839', + class: 'A', + typing: 'unambiguous', + anchorEligible: true, + provenance: { + producer: '@0xintuition/iid/inspectIntuitionId', + version: PACKAGE_VERSION, + }, + ...overrides, + }; +} diff --git a/services/workers/src/core/iid-registry.ts b/services/workers/src/core/iid-registry.ts new file mode 100644 index 0000000..6718a88 --- /dev/null +++ b/services/workers/src/core/iid-registry.ts @@ -0,0 +1,97 @@ +import { WorkerConfigurationError } from '../shared/errors'; +import type { + IdentityClassificationDecision, + IdentityProviderHint, + IdentityProviderPlan, + NormalizedAtomIdentity, + SemanticContractProvenance, +} from './identity-contract'; + +export type PublicIidClassification = { + readonly slug: string; + readonly schemaType: string; + readonly displayName: string; + readonly category: string; +}; + +export type IidSemanticResolution = { + identityDecision: IdentityClassificationDecision; + providerPlan: IdentityProviderPlan; +}; + +export type IidRegistryAdapter = { + resolve(identity: NormalizedAtomIdentity): IidSemanticResolution; +}; + +/** + * Maps only public iid-registry calls into Core DTOs. Scheme typing comes from + * the already-inspected identity; every classification, provider order, and + * identifier hint remains owned by the registry package. + */ +export function createIidRegistryAdapter(input: { + classificationForIid: (iid: string) => PublicIidClassification | undefined; + providersForIid: (iid: string) => readonly string[]; + identifierHintsForIid: (iid: string) => Record; + packageVersion: string; +}): IidRegistryAdapter { + const packageVersion = requireVersion(input.packageVersion); + const classificationProvenance: SemanticContractProvenance = { + producer: '@0xintuition/iid-registry/classificationForIid', + version: packageVersion, + }; + const providerProvenance: SemanticContractProvenance = { + producer: '@0xintuition/iid-registry/providersForIid+identifierHintsForIid', + version: packageVersion, + }; + + return { + resolve(identity) { + const classification = input.classificationForIid(identity.canonical); + const providers = input.providersForIid(identity.canonical); + const identifierHints = toIdentifierHints(input.identifierHintsForIid(identity.canonical)); + + return { + identityDecision: classification + ? { + status: 'classified', + classificationSlug: classification.slug, + schemaType: classification.schemaType, + category: classification.category, + provenance: classificationProvenance, + } + : { + status: identity.typing === 'polymorphic' ? 'ambiguous' : 'unmapped', + provenance: classificationProvenance, + }, + providerPlan: { + status: providers.length > 0 ? 'planned' : 'unsupported', + targets: providers.map((provider) => ({ + provider, + // Registry provider slugs are capability identifiers. Keep their + // ordering and spelling intact for the execution adapter in C10. + capabilities: [provider], + identifierHints, + })), + provenance: providerProvenance, + }, + }; + }, + }; +} + +function toIdentifierHints(hints: Record): readonly IdentityProviderHint[] { + return Object.entries(hints) + .filter(([kind, value]) => kind.trim().length > 0 && value.trim().length > 0) + .sort(([left], [right]) => left.localeCompare(right)) + .map(([kind, value]) => ({ kind, value })); +} + +function requireVersion(value: string): string { + const version = value.trim(); + if (!version) { + throw new WorkerConfigurationError( + '@0xintuition/iid-registry adapter requires an exact version.' + ); + } + return version; +} diff --git a/services/workers/src/core/iid-runtime.test.ts b/services/workers/src/core/iid-runtime.test.ts new file mode 100644 index 0000000..7ec15b7 --- /dev/null +++ b/services/workers/src/core/iid-runtime.test.ts @@ -0,0 +1,94 @@ +import { describe, expect, test } from 'bun:test'; +import { WorkerConfigurationError } from '../shared/errors'; +import { composeIidWorkerAdapters } from './iid-runtime'; + +describe('IID worker runtime composition', () => { + test('binds public module functions to exact installed manifest versions', () => { + const adapters = composeIidWorkerAdapters({ + iid: { + inspectIntuitionId: (iid) => ({ + valid: true, + iid, + scheme: 'isrc', + value: 'USUM71703861', + class: 'A', + typing: 'unambiguous', + anchorEligible: true, + }), + }, + iidManifest: { name: '@0xintuition/iid', version: '0.1.0-alpha.0' }, + iidRegistry: { + classificationForIid: () => ({ + slug: 'music-recording', + schemaType: 'MusicRecording', + displayName: 'Music Recording', + category: 'Media', + }), + providersForIid: () => ['musicbrainz', 'spotify'], + identifierHintsForIid: () => ({ isrc: 'USUM71703861' }), + }, + iidRegistryManifest: { + name: '@0xintuition/iid-registry', + version: '0.1.0-alpha.0', + }, + iidSpecificationVersion: '@0xintuition/iid-spec@0.1.0-alpha.0', + }); + + const inspection = adapters.iidInspection.inspect('int:isrc:USUM71703861'); + expect(inspection.valid).toBe(true); + expect(adapters.iidInspection.provenance).toEqual({ + producer: '@0xintuition/iid/inspectIntuitionId', + version: '0.1.0-alpha.0', + specificationVersion: '@0xintuition/iid-spec@0.1.0-alpha.0', + }); + const resolution = adapters.iidRegistry.resolve({ + raw: 'int:isrc:USUM71703861', + canonical: 'int:isrc:USUM71703861', + scheme: 'isrc', + value: 'USUM71703861', + class: 'A', + typing: 'unambiguous', + anchorEligible: true, + provenance: adapters.iidInspection.provenance, + }); + expect(resolution).toMatchObject({ + identityDecision: { + status: 'classified', + classificationSlug: 'music-recording', + provenance: { version: '0.1.0-alpha.0' }, + }, + providerPlan: { + status: 'planned', + targets: [{ provider: 'musicbrainz' }, { provider: 'spotify' }], + provenance: { version: '0.1.0-alpha.0' }, + }, + }); + }); + + test('rejects substituted manifests and non-exact runtime versions', () => { + const compose = (overrides: { iidName?: string; iidVersion?: string; registryName?: string }) => + composeIidWorkerAdapters({ + iid: { inspectIntuitionId: () => ({ valid: false, reason: 'malformed' }) }, + iidManifest: { + name: overrides.iidName ?? '@0xintuition/iid', + version: overrides.iidVersion ?? '0.1.0-alpha.0', + }, + iidRegistry: { + classificationForIid: () => undefined, + providersForIid: () => [], + identifierHintsForIid: () => ({}), + }, + iidRegistryManifest: { + name: overrides.registryName ?? '@0xintuition/iid-registry', + version: '0.1.0-alpha.0', + }, + iidSpecificationVersion: '@0xintuition/iid-spec@0.1.0-alpha.0', + }); + + expect(() => compose({ iidName: '@attacker/iid' })).toThrow(WorkerConfigurationError); + expect(() => compose({ registryName: '@attacker/registry' })).toThrow(WorkerConfigurationError); + expect(() => compose({ iidVersion: '*' })).toThrow(WorkerConfigurationError); + expect(() => compose({ iidVersion: 'workspace:*' })).toThrow(WorkerConfigurationError); + expect(() => compose({ iidVersion: '^0.1.0' })).toThrow(WorkerConfigurationError); + }); +}); diff --git a/services/workers/src/core/iid-runtime.ts b/services/workers/src/core/iid-runtime.ts new file mode 100644 index 0000000..d9a2bd8 --- /dev/null +++ b/services/workers/src/core/iid-runtime.ts @@ -0,0 +1,91 @@ +import { WorkerConfigurationError } from '../shared/errors'; +import { + createIidInspectionAdapter, + type IidInspectionAdapter, + type PublicIidInspection, +} from './iid-inspection'; +import { + createIidRegistryAdapter, + type IidRegistryAdapter, + type PublicIidClassification, +} from './iid-registry'; + +const EXACT_SEMVER = + /^(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)(?:-[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?(?:\+[0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*)?$/; + +export type PublicIidRuntimeModule = { + inspectIntuitionId(rawInput: string): PublicIidInspection; +}; + +export type PublicIidRegistryRuntimeModule = { + classificationForIid(iid: string): PublicIidClassification | undefined; + providersForIid(iid: string): readonly string[]; + identifierHintsForIid(iid: string): Record; +}; + +export type PublicPackageManifest = { + name: string; + version: string; +}; + +export type IidWorkerAdapters = { + iidInspection: IidInspectionAdapter; + iidRegistry: IidRegistryAdapter; +}; + +/** + * Production composition boundary for the two public IID packages. + * + * The caller must statically import these modules from exact, lockfile-backed + * package dependencies. This factory intentionally does not resolve arbitrary + * paths, URLs, NODE_PATH entries, or package names at runtime: those mechanisms + * would bypass Core's minimum-release-age and deterministic-install policy. + */ +export function composeIidWorkerAdapters(input: { + iid: PublicIidRuntimeModule; + iidManifest: PublicPackageManifest; + iidRegistry: PublicIidRegistryRuntimeModule; + iidRegistryManifest: PublicPackageManifest; + iidSpecificationVersion: string; +}): IidWorkerAdapters { + assertPackageManifest(input.iidManifest, '@0xintuition/iid'); + assertPackageManifest(input.iidRegistryManifest, '@0xintuition/iid-registry'); + + return { + iidInspection: createIidInspectionAdapter({ + inspectIntuitionId: input.iid.inspectIntuitionId, + packageVersion: input.iidManifest.version, + specificationVersion: requireExactVersion( + input.iidSpecificationVersion, + '@0xintuition/iid-spec', + '@0xintuition/iid-spec@' + ), + }), + iidRegistry: createIidRegistryAdapter({ + classificationForIid: input.iidRegistry.classificationForIid, + providersForIid: input.iidRegistry.providersForIid, + identifierHintsForIid: input.iidRegistry.identifierHintsForIid, + packageVersion: input.iidRegistryManifest.version, + }), + }; +} + +function assertPackageManifest(manifest: PublicPackageManifest, expectedName: string): void { + if (manifest.name !== expectedName) { + throw new WorkerConfigurationError( + `Expected ${expectedName} package manifest, received ${manifest.name || ''}.` + ); + } + requireExactVersion(manifest.version, expectedName); +} + +function requireExactVersion(value: string, packageName: string, prefix = ''): string { + const version = value.trim(); + const semver = prefix ? version.slice(prefix.length) : version; + if ((prefix && !version.startsWith(prefix)) || !EXACT_SEMVER.test(semver)) { + throw new WorkerConfigurationError( + `${packageName} runtime composition requires an exact installed version.` + ); + } + return version; +} diff --git a/services/workers/src/core/parse.test.ts b/services/workers/src/core/parse.test.ts new file mode 100644 index 0000000..de7eb75 --- /dev/null +++ b/services/workers/src/core/parse.test.ts @@ -0,0 +1,42 @@ +import { describe, expect, test } from 'bun:test'; +import { type CompactParseResult, resolveParseSearchText } from './parse'; + +describe('parse search projection', () => { + test('restores legacy strings after authoritative parsing but keeps valid IID identities opaque', () => { + expect(resolveParseSearchText({ kind: 'plain_string', normalizedInput: 'legacy value' })).toBe( + 'legacy value' + ); + expect( + resolveParseSearchText({ kind: 'iid', normalizedInput: 'opaque canonical identity' }) + ).toBe(''); + }); + + test('uses display fields for structured documents without indexing raw JSON', () => { + const result: CompactParseResult = { + kind: 'json', + normalizedInput: '{"opaque":"payload"}', + structuredDocument: { + source: 'inline_json', + format: 'jsonld', + topLevelType: 'object', + urlCandidates: [], + data: { name: 'Display name', description: ['Display description'] }, + }, + }; + + expect(resolveParseSearchText(result)).toBe('Display name Display description'); + expect(resolveParseSearchText({ kind: 'json', normalizedInput: '{"opaque":"payload"}' })).toBe( + '' + ); + }); + + test('retains canonical identifiers for understood non-opaque parser kinds', () => { + expect( + resolveParseSearchText({ + kind: 'url', + normalizedInput: 'https://example.com/path', + canonicalId: 'https://example.com/path', + }) + ).toBe('https://example.com/path'); + }); +}); diff --git a/services/workers/src/core/parse.ts b/services/workers/src/core/parse.ts index 69186aa..705fb78 100644 --- a/services/workers/src/core/parse.ts +++ b/services/workers/src/core/parse.ts @@ -1,9 +1,17 @@ import type { ParseResult } from '@0xintuition/atom-parser/types'; +import type { NormalizedAtomIdentity, SemanticContractProvenance } from './identity-contract'; export type CompactParseResult = { - kind: ParseResult['kind']; + // IID is a Core persistence kind produced only by the public inspection + // adapter; the legacy atom parser remains limited to ParseResult['kind']. + kind: ParseResult['kind'] | 'iid'; normalizedInput: string; canonicalId?: string; + identity?: NormalizedAtomIdentity; + iidFallback?: { + reason: 'malformed' | 'unknown-scheme' | 'noncanonical'; + provenance: SemanticContractProvenance; + }; remote?: { finalUrl?: string; contentType?: string; @@ -155,3 +163,54 @@ export function toCompactParseResult(result: ParseResult): CompactParseResult { hints: result.kind === 'plain_string' ? { trimmed: result.trimmed } : undefined, }; } + +/** + * Builds the parse-stage search projection from understood parser output. + * + * A `plain_string` result is authoritative parser output, so retaining its + * normalized value preserves legacy search behavior. IID payloads require + * resolved presentation data and are never promoted verbatim. This helper does + * not inspect prefixes or attempt to recognize IID grammar. + */ +export function resolveParseSearchText(result: CompactParseResult): string { + if (result.kind === 'iid') { + return ''; + } + + const structuredData = toRecordMaybe(result.structuredDocument?.data); + const name = resolveDisplayString(structuredData?.name); + const description = resolveDisplayString(structuredData?.description); + const identityCandidates = + result.kind === 'json' ? [] : [result.canonicalId, result.normalizedInput]; + + return Array.from( + new Set( + [name, description, ...identityCandidates].filter( + (value): value is string => typeof value === 'string' && value.trim().length > 0 + ) + ) + ) + .join(' ') + .slice(0, 20_000); +} + +function resolveDisplayString(value: unknown): string | undefined { + if (typeof value === 'string' && value.trim().length > 0) { + return value.trim(); + } + + if (Array.isArray(value)) { + const first = value.find( + (entry): entry is string => typeof entry === 'string' && entry.trim().length > 0 + ); + return first?.trim(); + } + + return undefined; +} + +function toRecordMaybe(value: unknown): Record | undefined { + return value !== null && typeof value === 'object' && !Array.isArray(value) + ? (value as Record) + : undefined; +} diff --git a/services/workers/src/kg/atom-classification/index.ts b/services/workers/src/kg/atom-classification/index.ts index 013e410..b9fab9e 100644 --- a/services/workers/src/kg/atom-classification/index.ts +++ b/services/workers/src/kg/atom-classification/index.ts @@ -15,11 +15,17 @@ import { import { deriveClassificationPlan, deriveClassificationResultFromRuntime, + deriveIidClassificationResult, resolveClassificationType, } from '../../core/classification'; +import type { IidRegistryAdapter } from '../../core/iid-registry'; import type { CircuitBreaker } from '../../shared/circuit-breaker'; import type { WorkerConfig } from '../../shared/config'; -import { classifyWorkerError, toProcessingError } from '../../shared/errors'; +import { + classifyWorkerError, + toProcessingError, + WorkerConfigurationError, +} from '../../shared/errors'; import { createBoundedScheduler, RECONCILE_BATCH_SIZE_MULTIPLIER, @@ -47,7 +53,11 @@ export async function runKgClassificationWorker(input: { database: CircuitBreaker; runtime: CircuitBreaker; }; + iidRegistry?: IidRegistryAdapter; }): Promise { + const iidRegistry = input.config.iidReadEnabled + ? requireIidRegistry(input.iidRegistry) + : undefined; const classificationRuntime = createClassificationRuntime({ defaultPreset: input.config.defaultPreset, cacheProvider: input.config.cacheProvider, @@ -93,9 +103,16 @@ export async function runKgClassificationWorker(input: { const parseResult = toCompactParseResultMaybe(claimed.parseResult); const rawInput = claimed.data ?? claimed.dataHex; const plan = deriveClassificationPlan({ parseResult, rawInput }); - let classificationResult = plan.classificationResult; + const iidClassificationResult = + input.config.iidReadEnabled && parseResult?.kind === 'iid' && parseResult.identity + ? deriveIidClassificationResult({ + identity: parseResult.identity, + resolution: requireIidRegistry(iidRegistry).resolve(parseResult.identity), + }) + : undefined; + let classificationResult = iidClassificationResult ?? plan.classificationResult; - if (!plan.usesStructuredDocument) { + if (!iidClassificationResult && !plan.usesStructuredDocument) { if (!plan.runtimeInput) { // See atom-parsing for the rationale: classification-skip + // downstream-skip must commit atomically or prerequisite-driven @@ -284,3 +301,12 @@ export async function runKgClassificationWorker(input: { } } } + +function requireIidRegistry(adapter: IidRegistryAdapter | undefined): IidRegistryAdapter { + if (!adapter) { + throw new WorkerConfigurationError( + 'WORKERS_IID_READ_ENABLED requires an @0xintuition/iid-registry adapter for IID classification.' + ); + } + return adapter; +} diff --git a/services/workers/src/kg/atom-enrichment/index.ts b/services/workers/src/kg/atom-enrichment/index.ts index ead3a32..13e8125 100644 --- a/services/workers/src/kg/atom-enrichment/index.ts +++ b/services/workers/src/kg/atom-enrichment/index.ts @@ -12,7 +12,10 @@ import { } from '@0xintuition/database-kg/actions'; import { buildClassifiedInputFromPlan, + buildEnrichmentCompletionPromotedFields, + buildIidProviderExecutionPlan, deriveEnrichmentPlan, + evaluateEnrichmentCompletion, evaluateEnrichmentProcessingScope, } from '../../core/enrichment'; import type { CircuitBreaker } from '../../shared/circuit-breaker'; @@ -112,6 +115,46 @@ export async function runKgEnrichmentWorker(input: { classificationResult, rawInput: claimed.data ?? claimed.dataHex, }); + if (plan.identity && !input.config.iidResolutionEnabled) { + await input.circuits.database.execute(() => + markNodeProcessingStageSkipped(input.db, { + stage: 'enrichment', + nodeId: claimed.id, + runId, + reason: 'IID provider resolution is disabled by WORKERS_IID_RESOLUTION_ENABLED.', + }) + ); + input.metrics.increment('skipped', 'enrichment'); + return; + } + + const engine = enrichmentRuntime.createEngine(input.config.defaultPreset); + const iidProviderExecution = buildIidProviderExecutionPlan({ + plan, + registeredPluginIds: engine.listPlugins().map((plugin) => plugin.id), + }); + if (iidProviderExecution?.status === 'blocked') { + await input.circuits.database.execute(() => + failNodeProcessingStage(input.db, { + stage: 'enrichment', + nodeId: claimed.id, + runId, + error: { + code: iidProviderExecution.retriable + ? 'IID_PROVIDER_PLAN_UNAVAILABLE' + : 'IID_PROVIDER_PLAN_UNSUPPORTED', + message: iidProviderExecution.reason, + retriable: iidProviderExecution.retriable, + details: { entries: iidProviderExecution.entries }, + }, + }) + ); + input.metrics.increment( + iidProviderExecution.retriable ? 'retried' : 'failed', + 'enrichment' + ); + return; + } const scopeDecision = evaluateEnrichmentProcessingScope({ plan, scope: input.config.processingScope, @@ -147,15 +190,69 @@ export async function runKgEnrichmentWorker(input: { return; } - const engine = enrichmentRuntime.createEngine(input.config.defaultPreset); const enrichment = await input.circuits.runtime.execute(() => engine.enrich({ input: classifiedInput, runtime: 'server', + ...(iidProviderExecution?.status === 'ready' + ? { plugins: iidProviderExecution.plugins } + : {}), ...(scopeDecision.artifactTypes ? { artifactTypes: scopeDecision.artifactTypes } : {}), traceId: runId, }) ); + const completion = evaluateEnrichmentCompletion(enrichment); + if (completion.kind === 'retryable_failure') { + await input.circuits.database.execute(() => + failNodeProcessingStage(input.db, { + stage: 'enrichment', + nodeId: claimed.id, + runId, + error: { + code: 'ENRICHMENT_PROVIDERS_RETRYABLE', + message: + 'Enrichment produced no artifacts and every attempted provider failed retryably.', + retriable: true, + details: completion.diagnostics, + }, + }) + ); + input.metrics.increment('retried', 'enrichment'); + if (claimed.enrichmentAttempts >= input.config.maxAttempts) { + input.metrics.incrementDeadLetters({ worker: WORKER, stage: 'enrichment' }); + } + logger.warn('kg enrichment produced only retryable provider failures', { + durationMs: Date.now() - startedAt, + errorCodes: completion.diagnostics.errors.map((error) => error.code), + }); + return; + } + if (completion.kind === 'terminal_unresolved') { + await input.circuits.database.execute(() => + failNodeProcessingStage(input.db, { + stage: 'enrichment', + nodeId: claimed.id, + runId, + error: { + code: 'ENRICHMENT_PROVIDERS_TERMINAL', + message: + completion.diagnostics.totalErrors > 0 + ? 'Enrichment produced no artifacts and at least one provider failed terminally.' + : 'Enrichment produced no artifacts because every provider was skipped.', + retriable: false, + details: completion.diagnostics, + }, + }) + ); + input.metrics.increment('failed', 'enrichment'); + logger.warn('kg enrichment completed without a resolvable artifact', { + durationMs: Date.now() - startedAt, + errors: completion.diagnostics.totalErrors, + skipped: completion.diagnostics.totalSkipped, + }); + return; + } + const promotedFields = buildEnrichmentCompletionPromotedFields(plan, enrichment.artifacts); await input.circuits.database.execute(() => completeNodeEnrichmentStageWithArtifacts(input.db, { nodeId: claimed.id, @@ -172,6 +269,7 @@ export async function runKgEnrichmentWorker(input: { meta: artifact.meta, sourceUri: artifact.meta.sourceUrl ?? plan.targetUrl, })), + promotedFields, }) ); input.metrics.increment('completed', 'enrichment'); diff --git a/services/workers/src/kg/atom-parsing/index.ts b/services/workers/src/kg/atom-parsing/index.ts index dc7db8f..450b330 100644 --- a/services/workers/src/kg/atom-parsing/index.ts +++ b/services/workers/src/kg/atom-parsing/index.ts @@ -12,10 +12,15 @@ import { reapStuckProcessingNodes, releaseClaimedProcessingStageLeases, } from '@0xintuition/database-kg/actions'; -import { toCompactParseResult } from '../../core/parse'; +import { type IidInspectionAdapter, parseAtomWithIidRead } from '../../core/iid-inspection'; +import { resolveParseSearchText } from '../../core/parse'; import type { CircuitBreaker } from '../../shared/circuit-breaker'; import type { WorkerConfig } from '../../shared/config'; -import { classifyWorkerError, toProcessingError } from '../../shared/errors'; +import { + classifyWorkerError, + toProcessingError, + WorkerConfigurationError, +} from '../../shared/errors'; import { createBoundedScheduler, RECONCILE_BATCH_SIZE_MULTIPLIER, @@ -43,7 +48,14 @@ export async function runKgParsingWorker(input: { database: CircuitBreaker; runtime: CircuitBreaker; }; + iidInspection?: IidInspectionAdapter; }): Promise { + if (input.config.iidReadEnabled && !input.iidInspection) { + throw new WorkerConfigurationError( + 'WORKERS_IID_READ_ENABLED requires an @0xintuition/iid inspection adapter.' + ); + } + const processNode = async (nodeId: string) => { const node = await input.circuits.database.execute(() => getNodeForProcessing(input.db, nodeId) @@ -101,10 +113,14 @@ export async function runKgParsingWorker(input: { return; } - const result = await input.circuits.runtime.execute(() => - parseAtom(rawInput, input.config.parseOptions) - ); - const compact = toCompactParseResult(result); + const parseOutcome = await parseAtomWithIidRead({ + rawInput, + iidReadEnabled: input.config.iidReadEnabled, + ...(input.iidInspection ? { adapter: input.iidInspection } : {}), + parseLegacy: () => + input.circuits.runtime.execute(() => parseAtom(rawInput, input.config.parseOptions)), + }); + const compact = parseOutcome.result; await input.circuits.database.execute(() => completeNodeProcessingStage(input.db, { stage: 'parse', @@ -112,7 +128,10 @@ export async function runKgParsingWorker(input: { runId, data: compact, promotedFields: { - searchText: resolveSearchText(compact, rawInput), + searchText: resolveParseSearchText(compact), + ...(compact.identity + ? { rawType: 'iid' as const, iid: compact.identity.canonical } + : {}), ...(compact.structuredDocument?.data !== undefined ? { dataResolved: compact.structuredDocument.data } : {}), @@ -124,6 +143,7 @@ export async function runKgParsingWorker(input: { logger.info('kg parse completed', { durationMs: Date.now() - startedAt, kind: compact.kind, + ...(parseOutcome.fallbackReason ? { iidFallbackReason: parseOutcome.fallbackReason } : {}), }); } catch (error) { const classified = classifyWorkerError(error); @@ -266,37 +286,3 @@ export async function runKgParsingWorker(input: { function resolveParseInput(node: { data: string | null; dataHex: string | null }): string | null { return node.data ?? node.dataHex ?? null; } - -function resolveSearchText( - compact: ReturnType, - rawInput: string -): string { - const structuredData = - compact.structuredDocument?.data && - typeof compact.structuredDocument.data === 'object' && - !Array.isArray(compact.structuredDocument.data) - ? (compact.structuredDocument.data as Record) - : undefined; - const name = resolveString(structuredData?.name); - const description = resolveString(structuredData?.description); - - return [name, description, compact.canonicalId, compact.normalizedInput, rawInput] - .filter((value): value is string => Boolean(value?.trim())) - .join(' ') - .slice(0, 20_000); -} - -function resolveString(value: unknown): string | undefined { - if (typeof value === 'string' && value.trim().length > 0) { - return value.trim(); - } - - if (Array.isArray(value)) { - const first = value.find( - (entry): entry is string => typeof entry === 'string' && entry.trim().length > 0 - ); - return first?.trim(); - } - - return undefined; -} diff --git a/services/workers/src/kg/commands/index.test.ts b/services/workers/src/kg/commands/index.test.ts new file mode 100644 index 0000000..b3630a5 --- /dev/null +++ b/services/workers/src/kg/commands/index.test.ts @@ -0,0 +1,44 @@ +import { describe, expect, test } from 'bun:test'; + +import { parseIidReconciliationOptions } from './index'; + +describe('IID reconciliation command options', () => { + test('is a bounded dry run by default', () => { + expect(parseIidReconciliationOptions([])).toEqual({ + limit: 100, + after: undefined, + confirmed: false, + force: false, + parseVersion: 'unknown', + registryVersion: 'unknown', + resolverVersion: 'unknown', + }); + }); + + test('records resume and package provenance for an applied batch', () => { + expect( + parseIidReconciliationOptions([ + '--yes', + '--force', + '--limit=25', + '--after=0xabc', + '--parse-version=@0xintuition/iid@0.1.0-alpha.0', + '--registry-version=@0xintuition/iid-registry@0.1.0-alpha.0', + '--resolver-version=core-provider-adapter@1', + ]) + ).toEqual({ + limit: 25, + after: '0xabc', + confirmed: true, + force: true, + parseVersion: '@0xintuition/iid@0.1.0-alpha.0', + registryVersion: '@0xintuition/iid-registry@0.1.0-alpha.0', + resolverVersion: 'core-provider-adapter@1', + }); + }); + + test('rejects unbounded or empty options', () => { + expect(() => parseIidReconciliationOptions(['--limit=1001'])).toThrow('<= 1000'); + expect(() => parseIidReconciliationOptions(['--after='])).toThrow('non-empty'); + }); +}); diff --git a/services/workers/src/kg/commands/index.ts b/services/workers/src/kg/commands/index.ts index e4b56d6..cb3cdfe 100644 --- a/services/workers/src/kg/commands/index.ts +++ b/services/workers/src/kg/commands/index.ts @@ -1,5 +1,6 @@ import { type KgActionDb, + listIidReconciliationCandidates, listNodeProcessingDeadLetters, type NodeProcessingStage, type NodeProcessingStatus, @@ -16,7 +17,8 @@ type KgCommandName = | 'kg-dead-letter-enrichment' | 'kg-backfill-parse' | 'kg-backfill-classification' - | 'kg-backfill-enrichment'; + | 'kg-backfill-enrichment' + | 'kg-reconcile-iid'; const KG_COMMANDS: readonly KgCommandName[] = [ 'kg-requeue-parse', @@ -28,6 +30,7 @@ const KG_COMMANDS: readonly KgCommandName[] = [ 'kg-backfill-parse', 'kg-backfill-classification', 'kg-backfill-enrichment', + 'kg-reconcile-iid', ]; /** @@ -92,6 +95,9 @@ export async function runKgCommand(db: KgActionDb, args: string[]): Promise'}. Supported commands: ${KG_COMMANDS.join(', ')}.` @@ -99,6 +105,134 @@ export async function runKgCommand(db: KgActionDb, args: string[]): Promise { + const options = parseIidReconciliationOptions(rest, command); + const candidates = await listIidReconciliationCandidates(db, { + limit: options.limit, + ...(options.after ? { after: options.after } : {}), + }); + const nextAfter = candidates.at(-1)?.id; + const provenance = { + parseVersion: options.parseVersion, + registryVersion: options.registryVersion, + resolverVersion: options.resolverVersion, + }; + + if (options.confirmed) { + for (const candidate of candidates) { + await requeueNodeProcessingStage(db, { + stage: 'parse', + nodeId: candidate.id, + reason: `iid_reconciliation:${JSON.stringify(provenance)}`, + cascadeDownstream: true, + force: options.force, + }); + } + } + + console.log( + JSON.stringify({ + command, + mode: options.confirmed ? 'apply' : 'dry-run', + count: candidates.length, + limit: options.limit, + after: options.after ?? null, + nextAfter: nextAfter ?? null, + provenance, + candidates, + }) + ); +} + +export function parseIidReconciliationOptions( + rest: readonly string[], + command = 'kg-reconcile-iid' +): IidReconciliationOptions { + let limit = 100; + let after: string | undefined; + let confirmed = false; + let force = false; + let parseVersion = 'unknown'; + let registryVersion = 'unknown'; + let resolverVersion = 'unknown'; + + for (const arg of rest) { + if (arg === '--yes' || arg === '-y') { + confirmed = true; + continue; + } + if (arg === '--force') { + force = true; + continue; + } + if (arg.startsWith('--limit=')) { + limit = parsePositiveIntegerFlag(command, '--limit', arg.slice('--limit='.length), 1_000); + continue; + } + if (arg.startsWith('--after=')) { + after = requireFlagValue(command, '--after', arg.slice('--after='.length)); + continue; + } + if (arg.startsWith('--parse-version=')) { + parseVersion = requireFlagValue( + command, + '--parse-version', + arg.slice('--parse-version='.length) + ); + continue; + } + if (arg.startsWith('--registry-version=')) { + registryVersion = requireFlagValue( + command, + '--registry-version', + arg.slice('--registry-version='.length) + ); + continue; + } + if (arg.startsWith('--resolver-version=')) { + resolverVersion = requireFlagValue( + command, + '--resolver-version', + arg.slice('--resolver-version='.length) + ); + continue; + } + throw new Error(`${command}: unknown argument "${arg}".`); + } + + return { + limit, + after, + confirmed, + force, + parseVersion, + registryVersion, + resolverVersion, + }; +} + +function requireFlagValue(command: string, flag: string, value: string): string { + const trimmed = value.trim(); + if (!trimmed) { + throw new Error(`${command}: ${flag} requires a non-empty value.`); + } + return trimmed; +} + async function listDeadLetters( db: KgActionDb, stage: NodeProcessingStage, diff --git a/services/workers/src/kg/processing.test.ts b/services/workers/src/kg/processing.test.ts index 5ebf547..56c42d3 100644 --- a/services/workers/src/kg/processing.test.ts +++ b/services/workers/src/kg/processing.test.ts @@ -25,6 +25,45 @@ describe('KG processing helpers', () => { expect(toCompactParseResultMaybe([])).toBeNull(); }); + test('keeps legacy parse records valid and validates optional identity handoffs', () => { + const legacy = { kind: 'plain_string', normalizedInput: 'legacy value' } as const; + expect(toCompactParseResultMaybe(legacy)).toEqual(legacy); + + const withIdentity = { + kind: 'iid', + normalizedInput: 'opaque raw identity', + identity: { + raw: 'opaque raw identity', + canonical: 'opaque canonical identity', + scheme: 'fixture-scheme', + value: 'value:with:colons', + profile: 'p0', + anchorEligible: true, + provenance: { producer: 'adapter', version: 'test' }, + }, + } as const; + expect(toCompactParseResultMaybe(withIdentity)).toEqual(withIdentity); + expect(toCompactParseResultMaybe({ ...withIdentity, identity: { canonical: '' } })).toBeNull(); + + const withFallback = { + ...legacy, + iidFallback: { + reason: 'unknown-scheme', + provenance: { + producer: '@0xintuition/iid/inspectIntuitionId', + version: '0.1.0-alpha.0', + }, + }, + } as const; + expect(toCompactParseResultMaybe(withFallback)).toEqual(withFallback); + expect( + toCompactParseResultMaybe({ + ...withFallback, + iidFallback: { ...withFallback.iidFallback, reason: 'guessed-repair' }, + }) + ).toBeNull(); + }); + test('validates classification result shape', () => { expect( toClassificationResultMaybe({ @@ -37,4 +76,30 @@ describe('KG processing helpers', () => { expect(toClassificationResultMaybe({ status: 'done', source: 'inline_json' })).toBeNull(); expect(toClassificationResultMaybe('recognized')).toBeNull(); }); + + test('keeps legacy classification records valid and validates optional semantic handoffs', () => { + const legacy = { status: 'not_applicable', source: 'raw_input' } as const; + expect(toClassificationResultMaybe(legacy)).toEqual(legacy); + + const withHandoffs = { + status: 'recognized', + source: 'future-adapter', + identityDecision: { + status: 'unmapped', + provenance: { producer: 'registry-adapter', version: 'test' }, + }, + providerPlan: { + status: 'unsupported', + targets: [], + provenance: { producer: 'registry-adapter', version: 'test' }, + }, + } as const; + expect(toClassificationResultMaybe(withHandoffs)).toEqual(withHandoffs); + expect( + toClassificationResultMaybe({ + ...withHandoffs, + providerPlan: { status: 'planned', targets: 'not-an-array' }, + }) + ).toBeNull(); + }); }); diff --git a/services/workers/src/kg/processing.ts b/services/workers/src/kg/processing.ts index f32e6a5..5927a40 100644 --- a/services/workers/src/kg/processing.ts +++ b/services/workers/src/kg/processing.ts @@ -1,4 +1,10 @@ import type { WorkerClassificationResult } from '../core/classification'; +import { + isIdentityClassificationDecision, + isIdentityProviderPlan, + isNormalizedAtomIdentity, + isSemanticContractProvenance, +} from '../core/identity-contract'; import type { CompactParseResult } from '../core/parse'; export function getProcessingMetaString(meta: unknown, key: string): string { @@ -23,10 +29,29 @@ export function toCompactParseResultMaybe(value: unknown): CompactParseResult | if (!isNonEmptyString(maybe.kind) || !isNonEmptyString(maybe.normalizedInput)) { return null; } + if (maybe.identity !== undefined && !isNormalizedAtomIdentity(maybe.identity)) { + return null; + } + if (maybe.iidFallback !== undefined && !isIidFallback(maybe.iidFallback)) { + return null; + } return maybe as CompactParseResult; } +function isIidFallback(value: unknown): boolean { + if (!value || typeof value !== 'object' || Array.isArray(value)) { + return false; + } + const maybe = value as Record; + return ( + (maybe.reason === 'malformed' || + maybe.reason === 'unknown-scheme' || + maybe.reason === 'noncanonical') && + isSemanticContractProvenance(maybe.provenance) + ); +} + export function toClassificationResultMaybe(value: unknown): WorkerClassificationResult | null { if (!value || typeof value !== 'object' || Array.isArray(value)) { return null; @@ -36,6 +61,18 @@ export function toClassificationResultMaybe(value: unknown): WorkerClassificatio if (!isClassificationStatus(maybe.status) || !isNonEmptyString(maybe.source)) { return null; } + if (maybe.identity !== undefined && !isNormalizedAtomIdentity(maybe.identity)) { + return null; + } + if ( + maybe.identityDecision !== undefined && + !isIdentityClassificationDecision(maybe.identityDecision) + ) { + return null; + } + if (maybe.providerPlan !== undefined && !isIdentityProviderPlan(maybe.providerPlan)) { + return null; + } return maybe as WorkerClassificationResult; } diff --git a/services/workers/src/shared/config.test.ts b/services/workers/src/shared/config.test.ts index 6794431..999c983 100644 --- a/services/workers/src/shared/config.test.ts +++ b/services/workers/src/shared/config.test.ts @@ -6,6 +6,28 @@ describe('worker config', () => { expect(loadWorkerConfig({}).processingScope).toBe('full'); }); + test('keeps IID read and resolution paths disabled by default', () => { + const config = loadWorkerConfig({}); + expect(config.iidReadEnabled).toBe(false); + expect(config.iidResolutionEnabled).toBe(false); + }); + + test('allows IID read and resolution switches to be enabled independently', () => { + expect(loadWorkerConfig({ WORKERS_IID_READ_ENABLED: 'true' }).iidReadEnabled).toBe(true); + expect(loadWorkerConfig({ WORKERS_IID_RESOLUTION_ENABLED: 'true' }).iidResolutionEnabled).toBe( + true + ); + }); + + test('rejects malformed IID switch values', () => { + expect(() => loadWorkerConfig({ WORKERS_IID_READ_ENABLED: 'yes' })).toThrow( + /WORKERS_IID_READ_ENABLED/ + ); + expect(() => loadWorkerConfig({ WORKERS_IID_RESOLUTION_ENABLED: '1' })).toThrow( + /WORKERS_IID_RESOLUTION_ENABLED/ + ); + }); + test('accepts music and podcast processing scope presets', () => { expect( loadWorkerConfig({ diff --git a/services/workers/src/shared/config.ts b/services/workers/src/shared/config.ts index fadc736..77cb96d 100644 --- a/services/workers/src/shared/config.ts +++ b/services/workers/src/shared/config.ts @@ -30,6 +30,14 @@ const workerConfigSchema = z WORKERS_ENRICHMENT_VERSION: z.string().min(1).max(64).default('v1'), WORKERS_DEFAULT_PRESET: enrichmentPresetSchema.default('default'), WORKERS_PROCESSING_SCOPE: z.enum(PROCESSING_SCOPE_PRESETS).default('full'), + WORKERS_IID_READ_ENABLED: z + .enum(['true', 'false']) + .default('false') + .transform((value) => value === 'true'), + WORKERS_IID_RESOLUTION_ENABLED: z + .enum(['true', 'false']) + .default('false') + .transform((value) => value === 'true'), WORKERS_CACHE_PROVIDER: z.enum(['memory', 'none', 'upstash']).default('memory'), WORKERS_MEMORY_CACHE_MAX_ENTRIES: z.coerce.number().int().min(10).default(500), WORKERS_CLASSIFICATION_MEMORY_CACHE_MAX_ENTRIES: z.coerce.number().int().min(10).optional(), @@ -95,6 +103,8 @@ export type WorkerConfig = { enrichmentVersion: string; defaultPreset: z.infer; processingScope: ProcessingScopePreset; + iidReadEnabled: boolean; + iidResolutionEnabled: boolean; cacheProvider: 'memory' | 'none' | 'upstash'; memoryCacheMaxEntries: number; classificationMemoryCacheMaxEntries: number; @@ -145,6 +155,8 @@ export function loadWorkerConfig( enrichmentVersion: parsed.WORKERS_ENRICHMENT_VERSION, defaultPreset: parsed.WORKERS_DEFAULT_PRESET, processingScope: parsed.WORKERS_PROCESSING_SCOPE, + iidReadEnabled: parsed.WORKERS_IID_READ_ENABLED, + iidResolutionEnabled: parsed.WORKERS_IID_RESOLUTION_ENABLED, cacheProvider: parsed.WORKERS_CACHE_PROVIDER, memoryCacheMaxEntries: parsed.WORKERS_MEMORY_CACHE_MAX_ENTRIES, classificationMemoryCacheMaxEntries: diff --git a/tests/fixtures/README.md b/tests/fixtures/README.md new file mode 100644 index 0000000..8ed21c9 --- /dev/null +++ b/tests/fixtures/README.md @@ -0,0 +1,33 @@ +# Cross-stack semantic atom fixtures + +`atom-semantic-read-model.v1.json` is the package-neutral contract shared by +Core API and Explorer tests. It records identity inputs and their semantic +intent, not authoritative canonicalization output. Canonical bytes, normalized +identifiers, anchor eligibility, and atom IDs must eventually be verified +against packed public `@0xintuition/iid` and `@0xintuition/primitives` +artifacts. + +The fixture also covers the Core-owned read contract: legacy compatibility, +context ordering/duplicates/link safety, and unresolved/resolved/retryable/ +terminal presentation states. + +`minimumProfile` describes the minimum representation floor (`p0`/`p1`). +`schemeTyping` separately describes whether the supplied identity is +semantically unambiguous or requires polymorphic typing. These are intentionally +not collapsed into one `profile` label. + +The existing smoke paths remain unchanged by default. Optional checks are: + +```bash +# API semantic envelope over the existing local smoke atom +API_ATOM_SEMANTIC_READS_ENABLED=true \ +SMOKE_EXPECT_SEMANTIC_READS=1 \ +bun run smoke + +# On a caller-supplied v1.1 chain window, assert context event count without +# replacing the stable legacy testnet window. +MULTIVAULT_START_BLOCK=... \ +MULTIVAULT_END_BLOCK=... \ +SMOKE_EXPECT_CONTEXT_EVENT_COUNT=... \ +bun run smoke:index +``` diff --git a/tests/fixtures/atom-semantic-read-model.v1.json b/tests/fixtures/atom-semantic-read-model.v1.json new file mode 100644 index 0000000..89f06b8 --- /dev/null +++ b/tests/fixtures/atom-semantic-read-model.v1.json @@ -0,0 +1,233 @@ +{ + "schemaVersion": "1.0.0", + "fixtureKind": "atom-semantic-read-model", + "canonicalizationPolicy": "deferred-to-public-iid-package", + "identityCases": [ + { + "id": "p0-isrc-recording", + "input": "int:isrc:USQX91300108", + "expectedIntent": { + "validity": "valid", + "minimumProfile": "p0", + "schemeTyping": "unambiguous", + "scheme": "isrc", + "value": "USQX91300108", + "classification": "MusicRecording" + } + }, + { + "id": "typed-mbid-recording", + "input": "int:mbid:recording:d28cfc80-b16c-47c9-a370-f13f49732e75", + "expectedIntent": { + "validity": "valid", + "minimumProfile": "p0", + "schemeTyping": "unambiguous", + "scheme": "mbid", + "entityKind": "recording", + "classification": "MusicRecording" + } + }, + { + "id": "polymorphic-wikidata", + "input": "int:wd:Q42", + "expectedIntent": { + "validity": "valid", + "minimumProfile": "p1", + "schemeTyping": "polymorphic", + "scheme": "wd", + "value": "Q42", + "classification": null + } + }, + { + "id": "invalid-isrc-value", + "input": "int:isrc:notanisrc", + "expectedIntent": { + "validity": "invalid", + "scheme": "isrc", + "classification": null + } + }, + { + "id": "iid-lookalike", + "input": "int:isrc:X", + "expectedIntent": { + "validity": "invalid", + "scheme": "isrc", + "classification": null, + "mustNotBeTreatedAsHttpUrl": true + } + } + ], + "legacyJsonAtom": { + "id": "legacy-json-music-recording", + "rawType": "json", + "data": "{\"@context\":\"https://schema.org/\",\"@type\":\"MusicRecording\",\"name\":\"One Last Time\",\"sameAs\":[\"https://open.spotify.com/track/007jZK4eRSKsDBSjPUbh3H\"]}", + "expectedIntent": { + "classification": "MusicRecording", + "identity": null, + "legacyCompatible": true + } + }, + "contextCases": [ + { + "id": "ordered-duplicate-and-unsafe-context", + "entries": [ + { + "ordinal": 0, + "uri": "https://open.spotify.com/track/007jZK4eRSKsDBSjPUbh3H", + "source": "onchain", + "expectedLinkable": true + }, + { + "ordinal": 1, + "uri": "https://open.spotify.com/track/007jZK4eRSKsDBSjPUbh3H", + "source": "onchain", + "expectedLinkable": true + }, + { + "ordinal": 2, + "uri": "javascript:alert(1)", + "source": "onchain", + "expectedLinkable": false + }, + { + "ordinal": 3, + "uri": null, + "raw": "0xff00", + "source": "onchain", + "expectedLinkable": false, + "expectedDisplay": "0xff00" + }, + { + "ordinal": 4, + "uri": "https://user:secret@example.com/", + "source": "onchain", + "expectedLinkable": false + }, + { + "ordinal": 5, + "uri": "ipfs://bafybeigdyrzt5s", + "source": "onchain", + "expectedLinkable": false + }, + { + "ordinal": 6, + "uri": "not a URI", + "source": "onchain", + "expectedLinkable": false + } + ] + } + ], + "resolutionCases": [ + { + "id": "unresolved-isrc", + "source": { + "id": "0x1111111111111111111111111111111111111111111111111111111111111111", + "rawType": "iid", + "data": "int:isrc:USQX91300108", + "dataResolved": {}, + "parseResult": { + "kind": "iid", + "normalizedInput": "int:isrc:USQX91300108", + "identity": { + "raw": "int:isrc:USQX91300108", + "canonical": "int:isrc:USQX91300108", + "scheme": "isrc", + "value": "USQX91300108", + "class": "A", + "typing": "unambiguous", + "anchorEligible": true, + "provenance": { + "producer": "@0xintuition/iid/inspectIntuitionId", + "version": "0.1.0-alpha.0", + "specificationVersion": "@0xintuition/iid-spec@0.1.0-alpha.0" + } + } + }, + "classificationType": "MusicRecording", + "classificationStatus": "completed", + "enrichmentStatus": "pending" + }, + "expected": { + "resolutionStatus": "pending", + "displayName": null + } + }, + { + "id": "resolved-isrc", + "source": { + "id": "0x2222222222222222222222222222222222222222222222222222222222222222", + "rawType": "iid", + "data": "int:isrc:USQX91300108", + "dataResolved": { + "name": "One Last Time", + "description": "A resolved recording", + "image": "https://images.example/one-last-time.jpg" + }, + "parseResult": { + "kind": "iid", + "normalizedInput": "int:isrc:USQX91300108", + "profile": "p0", + "scheme": "isrc", + "value": "USQX91300108", + "valid": true, + "anchorEligible": true + }, + "classificationType": "MusicRecording", + "classificationStatus": "completed", + "classificationResult": { + "source": "iid-registry" + }, + "enrichmentStatus": "completed", + "enrichedAt": "2026-08-10T00:00:00.000Z" + }, + "expected": { + "resolutionStatus": "resolved", + "displayName": "One Last Time", + "displayImage": "https://images.example/one-last-time.jpg" + } + }, + { + "id": "retryable-provider-failure", + "source": { + "id": "0x3333333333333333333333333333333333333333333333333333333333333333", + "rawType": "iid", + "data": "int:isrc:USQX91300108", + "dataResolved": {}, + "classificationType": "MusicRecording", + "classificationStatus": "completed", + "enrichmentStatus": "failed", + "enrichmentError": { + "code": "provider_rate_limited", + "retriable": true + } + }, + "expected": { + "resolutionStatus": "retryable", + "displayName": null + } + }, + { + "id": "terminal-unsupported-identity", + "source": { + "id": "0x4444444444444444444444444444444444444444444444444444444444444444", + "rawType": "iid", + "data": "int:wd:Q42", + "dataResolved": {}, + "classificationType": "Unknown", + "classificationStatus": "completed", + "enrichmentStatus": "failed", + "enrichmentError": { + "code": "unsupported_identity", + "retriable": false + } + }, + "expected": { + "resolutionStatus": "terminal", + "displayName": null + } + } + ] +} diff --git a/tests/iid-semantic-enrichment.acceptance.test.ts b/tests/iid-semantic-enrichment.acceptance.test.ts new file mode 100644 index 0000000..b581631 --- /dev/null +++ b/tests/iid-semantic-enrichment.acceptance.test.ts @@ -0,0 +1,263 @@ +import { describe, expect, test } from 'bun:test'; +import { existsSync } from 'node:fs'; +import { join, resolve } from 'node:path'; +import { pathToFileURL } from 'node:url'; +import { + createEnrichmentEngine, + createIdentifierProviderPlan, + createMusicBrainzPlugin, + createOpenLibraryPlugin, + type EnrichmentPlugin, + type EnrichmentRequest, +} from '../packages/atom-enrichment/src'; +import type { FetchLike } from '../packages/atom-enrichment/src/plugins/providers'; +import { deriveIidClassificationResult } from '../services/workers/src/core/classification'; +import { evaluateEnrichmentCompletion } from '../services/workers/src/core/enrichment'; +import { + createIidInspectionAdapter, + type PublicIidInspection, + parseAtomWithIidRead, +} from '../services/workers/src/core/iid-inspection'; +import { + createIidRegistryAdapter, + type PublicIidClassification, +} from '../services/workers/src/core/iid-registry'; + +type PublicIidModule = { + inspectIntuitionId(input: string): PublicIidInspection; +}; + +type PublicRegistryModule = { + classificationForIid(iid: string): PublicIidClassification | undefined; + providersForIid(iid: string): readonly string[]; + identifierHintsForIid(iid: string): Record; +}; + +const packagesRepository = process.env.INTUITION_PACKAGES_REPO + ? resolve(process.env.INTUITION_PACKAGES_REPO) + : resolve(import.meta.dir, '../../packages'); +const iidEntry = join(packagesRepository, 'packages/iid/dist/index.js'); +const registryEntry = join(packagesRepository, 'packages/iid-registry/dist/index.js'); +const publicPackagesAvailable = existsSync(iidEntry) && existsSync(registryEntry); +const describeWithPublicPackages = publicPackagesAvailable ? describe : describe.skip; + +const PACKAGE_VERSION = '0.1.0-alpha.0'; +const NOW = '2026-08-12T12:00:00.000Z'; + +describeWithPublicPackages('canonical IID semantic enrichment acceptance', () => { + test('ISRC inspection preserves open-first provider order and executes MusicBrainz by ISRC', async () => { + let requestedUrl = ''; + const musicbrainz = createMusicBrainzPlugin({ + fetch: jsonFetch( + { + recordings: [ + { + id: '0c8c75d9-98d7-4c9f-a6d8-37ea8768cc91', + title: 'Shape of You', + isrcs: ['USUM71703861'], + 'artist-credit': [{ name: 'Ed Sheeran' }], + }, + ], + }, + 200, + (url) => { + requestedUrl = url; + } + ), + }); + const semantic = await inspectAndPlan('int:isrc:USUM71703861', [musicbrainz]); + + expect(semantic.identity.canonical).toBe('int:isrc:USUM71703861'); + expect(semantic.classification).toMatchObject({ + status: 'recognized', + source: 'iid-registry', + schemaType: 'MusicRecording', + }); + expect(semantic.registryProviders).toEqual(['musicbrainz', 'spotify', 'apple-music']); + expect(semantic.executionPlan.entries).toMatchObject([ + { providerSlug: 'musicbrainz', ordinal: 0, disposition: 'execute' }, + { providerSlug: 'spotify', ordinal: 1, disposition: 'retry' }, + { providerSlug: 'apple-music', ordinal: 2, disposition: 'retry' }, + ]); + expect(semantic.executionPlan.identifiers).toEqual({ isrc: 'USUM71703861' }); + + const result = await semantic.engine.enrich(semantic.request); + + expect(requestedUrl).toContain('query=isrc%3AUSUM71703861'); + expect(result.errors).toEqual([]); + expect(result.artifacts).toMatchObject([ + { + artifact_type: 'musicbrainz', + data: { + name: 'Shape of You', + isrcs: ['USUM71703861'], + }, + }, + ]); + expect(evaluateEnrichmentCompletion(result)).toEqual({ kind: 'complete' }); + }); + + test('ISBN inspection plans OpenLibrary and executes the canonical Books API lookup', async () => { + let requestedUrl = ''; + const openlibrary = createOpenLibraryPlugin({ + fetch: jsonFetch( + { + 'ISBN:9780684832722': { + key: '/books/OL7721520M', + title: 'The Sovereign Individual', + authors: [{ name: 'James Dale Davidson' }, { name: 'William Rees-Mogg' }], + publishers: [{ name: 'Scribner' }], + }, + }, + 200, + (url) => { + requestedUrl = url; + } + ), + }); + const semantic = await inspectAndPlan('int:isbn:9780684832722', [openlibrary]); + + expect(semantic.identity.canonical).toBe('int:isbn:9780684832722'); + expect(semantic.classification).toMatchObject({ + status: 'recognized', + source: 'iid-registry', + schemaType: 'Book', + }); + expect(semantic.registryProviders).toEqual(['openlibrary']); + expect(semantic.executionPlan).toMatchObject({ + complete: true, + identifiers: { isbn: '9780684832722' }, + plugins: ['openlibrary'], + entries: [{ providerSlug: 'openlibrary', ordinal: 0, disposition: 'execute' }], + }); + + const result = await semantic.engine.enrich(semantic.request); + + expect(requestedUrl).toContain('bibkeys=ISBN%3A9780684832722'); + expect(result.errors).toEqual([]); + expect(result.artifacts).toMatchObject([ + { + artifact_type: 'openlibrary', + data: { + isbn: '9780684832722', + olid: 'OL7721520M', + title: 'The Sovereign Individual', + }, + }, + ]); + expect(evaluateEnrichmentCompletion(result)).toEqual({ kind: 'complete' }); + }); + + test('distinguishes retryable provider failure from terminal no-match and bad capability', async () => { + const retryable = await inspectAndPlan('int:isbn:9780684832722', [ + createOpenLibraryPlugin({ fetch: jsonFetch({}, 503) }), + ]); + const retryResult = await retryable.engine.enrich(retryable.request); + expect(retryResult.errors).toMatchObject([ + { pluginId: 'openlibrary', code: 'upstream_error', retriable: true }, + ]); + expect(evaluateEnrichmentCompletion(retryResult).kind).toBe('retryable_failure'); + + const terminal = await inspectAndPlan('int:isbn:9780684832722', [ + createOpenLibraryPlugin({ fetch: jsonFetch({}, 404) }), + ]); + const terminalResult = await terminal.engine.enrich(terminal.request); + expect(terminalResult).toMatchObject({ artifacts: [], errors: [] }); + expect(evaluateEnrichmentCompletion(terminalResult).kind).toBe('terminal_unresolved'); + + const invalidCapability = createIdentifierProviderPlan({ + providers: ['openlibrary', 'future-provider', 'openlibrary'], + identifiers: { isbn: '9780684832722' }, + registeredPluginIds: ['openlibrary'], + }); + expect(invalidCapability.entries).toMatchObject([ + { disposition: 'execute' }, + { disposition: 'terminal', reason: 'unknown_provider_slug' }, + { disposition: 'terminal', reason: 'duplicate_provider_slug' }, + ]); + }); +}); + +async function inspectAndPlan(iid: string, plugins: EnrichmentPlugin[]) { + const publicIid = await importModule(iidEntry); + const publicRegistry = await importModule(registryEntry); + const inspectionAdapter = createIidInspectionAdapter({ + inspectIntuitionId: publicIid.inspectIntuitionId, + packageVersion: PACKAGE_VERSION, + }); + const parsed = await parseAtomWithIidRead({ + rawInput: iid, + iidReadEnabled: true, + adapter: inspectionAdapter, + parseLegacy: () => { + throw new Error('canonical acceptance input unexpectedly took the legacy parser path'); + }, + }); + if (!parsed.result.identity) { + throw new Error(`public IID inspection did not produce identity for ${iid}`); + } + + const registryAdapter = createIidRegistryAdapter({ + classificationForIid: publicRegistry.classificationForIid, + providersForIid: publicRegistry.providersForIid, + identifierHintsForIid: publicRegistry.identifierHintsForIid, + packageVersion: PACKAGE_VERSION, + }); + const resolution = registryAdapter.resolve(parsed.result.identity); + const classification = deriveIidClassificationResult({ + identity: parsed.result.identity, + resolution, + }); + const registryProviders = resolution.providerPlan.targets.map(({ provider }) => provider); + const identifiers = Object.fromEntries( + (resolution.providerPlan.targets[0]?.identifierHints ?? []).map(({ kind, value }) => [ + kind, + value, + ]) + ); + const engine = createEnrichmentEngine({ plugins, now: () => NOW }); + const executionPlan = createIdentifierProviderPlan({ + providers: registryProviders, + identifiers, + registeredPluginIds: engine.listPlugins().map(({ id }) => id), + }); + const request: EnrichmentRequest = { + input: { + atomType: classification.schemaType === 'MusicRecording' ? 'song' : 'thing', + jsonLd: { + '@context': 'https://schema.org', + '@type': classification.schemaType ?? 'Thing', + }, + hints: { identifiers: executionPlan.identifiers }, + source: { + classificationEngine: 'iid-semantic-acceptance', + classifiedAt: NOW, + }, + }, + runtime: 'server', + plugins: executionPlan.plugins, + }; + + return { + identity: parsed.result.identity, + classification, + registryProviders, + executionPlan, + engine, + request, + }; +} + +async function importModule(entry: string): Promise { + return (await import(pathToFileURL(entry).href)) as T; +} + +function jsonFetch(body: unknown, status = 200, inspect?: (url: string) => void): FetchLike { + return async (url) => { + inspect?.(url); + return new Response(JSON.stringify(body), { + status, + headers: { 'content-type': 'application/json' }, + }); + }; +} diff --git a/tests/semantic-identity-contract.test.ts b/tests/semantic-identity-contract.test.ts new file mode 100644 index 0000000..13da66e --- /dev/null +++ b/tests/semantic-identity-contract.test.ts @@ -0,0 +1,52 @@ +import { describe, expect, test } from 'bun:test'; +import { atomIdentitySchema } from '../apps/explorer/src/lib/api'; +import { presentAtom } from '../services/api/src/atom-view'; +import { isNormalizedAtomIdentity } from '../services/workers/src/core/identity-contract'; +import { toCompactParseResultMaybe } from '../services/workers/src/kg/processing'; + +type SemanticFixture = { + resolutionCases: Array<{ + id: string; + source: { + id: string; + rawType: string; + data: string; + dataResolved: unknown; + parseResult?: unknown; + classificationType: string; + classificationStatus: string; + enrichmentStatus: string; + }; + }>; +}; + +const fixture = (await Bun.file( + new URL('./fixtures/atom-semantic-read-model.v1.json', import.meta.url) +).json()) as SemanticFixture; + +describe('normalized identity cross-stack contract', () => { + test('survives persistence, API presentation, and Explorer validation losslessly', () => { + const golden = fixture.resolutionCases.find(({ id }) => id === 'unresolved-isrc'); + if (!golden) { + throw new Error('missing unresolved-isrc golden fixture'); + } + + // JSON round-trip models the JSONB persistence boundary without importing a + // database driver or duplicating any IID package semantics in Core. + const persisted = JSON.parse(JSON.stringify(golden.source)) as typeof golden.source; + const parseResult = toCompactParseResultMaybe(persisted.parseResult); + expect(parseResult).not.toBeNull(); + if (!parseResult) { + throw new Error('golden parse result did not survive persistence validation'); + } + expect(isNormalizedAtomIdentity(parseResult.identity)).toBe(true); + expect(parseResult.identity).not.toHaveProperty('profile'); + expect(parseResult.identity?.provenance.specificationVersion).toBe( + '@0xintuition/iid-spec@0.1.0-alpha.0' + ); + + const presented = presentAtom(persisted); + expect(presented.identity).toEqual(parseResult.identity); + expect(atomIdentitySchema.parse(presented.identity)).toEqual(parseResult.identity); + }); +});