SCITT/SCRAPI: refactor to SCRAPI lifecycle, per-participant registration, storage backend#136
SCITT/SCRAPI: refactor to SCRAPI lifecycle, per-participant registration, storage backend#136howethomas wants to merge 4 commits intomainfrom
Conversation
Replace the DataTrails-specific OIDC authentication and registration with generic SCRAPI calls to SCITTLEs (self-hosted transparency service). Register vcon_created and vcon_enhanced lifecycle events per draft-howe-vcon-lifecycle, storing COSE receipts as scitt_receipt analysis entries on each vCon. - Remove OIDC_Auth class and DataTrails-specific endpoints - Add register_statement() with sync (201) and async (303) SCRAPI handling - Add wait_for_entry_id() polling and get_receipt() for async flow - Store receipt metadata (entry_id, vcon_operation, vcon_hash) on vCon - Add 12 unit tests covering registration, polling, and link runner - Document SCITT Lifecycle Registration in README Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The conserver's __init__.py uses ``from links.scitt import ...`` which registers submodules under ``links.scitt.*`` in sys.modules, while pytest imports create a parallel ``server.links.scitt.*`` entry. Patching the wrong module object meant time_sleep, requests.get, and VconRedis mocks had no effect — tests hit real services and took 80s. Fixes: - Use ``links.scitt.register_signed_statement`` path for submodule attribute mocks (time_sleep, requests.get/post) - Use ``links.scitt.create_hashed_signed_statement`` for COSE mocks - Use ``server.links.scitt`` for __init__.py namespace names (VconRedis) - Replace http://scittles:8000 with non-routable RFC 6761 URL (http://scrapi.test.invalid:9999) as safety net - Test suite now runs in 0.5s instead of 80s Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each vCon party with a tel field now gets a separate SCITT entry with
subject=tel:{number}, enabling customer data portal queries like
WHERE subject = 'tel:+12026661834'. Falls back to vcon:// subject
when no parties have tel. Link version bumped to 0.3.0.
Changes:
- links/scitt: Loop over parties, register per-participant, store
receipt array instead of single receipt
- storage/scitt: New per-participant storage backend with same
party iteration pattern
- Handle both dict and Party object access for tel field
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- mock_vcon fixture: set parties = [{"tel": "+15551234567"}] so
(vcon.parties or []) is iterable and produces one subject
- test_run_uses_fallback_subject: set mock_vcon.parties = [] so
fallback subject vcon:// is used
- Add subject to expected receipt body in test_run_registers_and_stores_receipt
Made-with: Cursor
pavanputhra
left a comment
pavanputhra
left a comment
There was a problem hiding this comment.
A few observations while reading through — great cleanup overall, the move away from DataTrails is much simpler to reason about.
| # Operation complete — extract entry_id | ||
| data = response.json() | ||
| if "entryID" in data: | ||
| return data["entryID"] |
There was a problem hiding this comment.
Question: could we pin to one field name here?
The code handles both "entryID" (camelCase) and "entry_id" (snake_case) from the poll response, which is a thoughtful defensive approach. If the SCRAPI draft defines one canonical field name, it might be worth aligning to it and removing the other branch — just to make it clear to future readers which format the service actually returns.
| f"https://app.datatrails.ai/archivist/v1/publicscitt/entries/{entry_id}/receipt", | ||
| headers=headers, | ||
| f"{scrapi_url}/entries/{entry_id}", | ||
| headers={"Accept": "application/cose"}, |
There was a problem hiding this comment.
Possible issue with receipt endpoint path
Just wanted to double-check — is GET /entries/{entry_id} the correct endpoint to retrieve the COSE receipt bytes, or should it be GET /entries/{entry_id}/receipt? The 201 sync path gets the receipt directly from the POST response body (which looks right), so I wanted to make sure the async path is hitting the same data. Could be I'm misreading the SCITTLEs API — happy to be corrected!
| resp = Mock() | ||
| resp.status_code = 200 | ||
| resp.json.return_value = {"entry_id": "abs-entry"} | ||
| mock_get.return_value = resp |
There was a problem hiding this comment.
Minor: test mocks use different field names for entry_id
This might be worth a quick look — test_async_303_polls_and_fetches_receipt mocks the poll response with {"entryID": "entry-xyz"} (camelCase), while this test uses {"entry_id": "abs-entry"} (snake_case). Since both tests exercise the same wait_for_entry_id() code path, having them use different formats makes it a little tricky to know which one reflects the real SCRAPI service behaviour. Aligning them would also help clarify which branch in the production code is the expected path.
|
|
||
| # Store receipts as analysis entry on the vCon | ||
| if opts.get("store_receipt", True): | ||
| vcon.add_analysis( |
There was a problem hiding this comment.
@howethomas Should it got to attachments instead of analysis?
| logger.warning("scitt storage: party without tel in %s, skipping", vcon_id) | ||
|
|
||
| # Fall back to vcon:// subject if no parties have tel | ||
| if not party_tels: |
There was a problem hiding this comment.
This portion is repeated in both storage and link. We should refactor it as function and move to lib and call it from both places.
) Ports the storage/scitt module from PR #136 to the new common/storage/ path. Registers per-participant SCITT entries as a storage backend without writing receipts back to Redis (avoids races with parallel storage writers). Supports signing_key_pem (base64 env var) and signing_key_path, consistent with the scitt link. Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2 participants
Summary
Refactors the SCITT link from DataTrails to SCRAPI lifecycle registration, adds per-participant SCITT for portal queryability, fixes test mocks, and adds a SCITT storage backend. SCITT-only changes (no other consolidation stack).
Changes
1. Refactor SCITT link from DataTrails to SCRAPI lifecycle (7cf1acf)
client_id,client_secret,auth_url) andOIDC_Authclass.register_statement(scrapi_url, signed_statement)with sync (201) and async (303) handling,wait_for_entry_id()polling,get_receipt().scrapi_url,signing_key_path,issuer,key_id,vcon_operation,store_receipt.scitt_receiptanalysis entries on the vCon.server/links/scitt/tests/test_scitt.py.2. Fix SCITT test mocks leaking to live SCITTLEs (d6f2daa)
server.links.scitt.*while the conserver loadslinks.scitt.*.links.scitt.register_signed_statementandlinks.scitt.create_hashed_signed_statement; patchVconRedisonserver.links.scitt.http://scrapi.test.invalid:9999as safety net.3. Per-participant SCITT and SCITT storage (06c797d)
telgets a separate SCITT entry withsubject=tel:+number, enabling portal queries (e.g.WHERE subject = 'tel:+12026661834'). Falls back tovcon://{vcon_uuid}when no parties have tel. Receipts stored as array on vCon.server/storage/scitt/: Post-chain storage backend that registers per-participant SCITT entries. Options:scrapi_url,signing_key_path,issuer,key_id,operations(e.g.["vcon_enhanced"]). Does not write receipts back to the vCon (avoids races with parallel storage). Transparency service is authoritative for receipts.Made with Cursor