feat: RFC 9000 requirements manifest + MCP test report#17
Open
ElNiak wants to merge 220 commits into
Open
Conversation
- Add rfc9000_requirements.yaml with 101 extracted RFC 9000 requirements organized by level (MUST/SHOULD/MAY) and protocol layer - Add ivy-tools-test-report.md documenting live verification of all 19 MCP tools (22 PASS, 4 PARTIAL) - Update ivy-lsp submodule: MCP tool fixes (ISS-001/002/003/005/006/008) - Update panther-ivy-plugin submodule: auto-detect local ivy-lsp source
Reviewer's GuideAdds a structured RFC 9000 requirements manifest and an MCP test report for ivy-tools, plus updates ivy-lsp and panther-ivy-plugin submodules to improve MCP tool behavior, dependency handling, and local source detection. Sequence diagram for MCP QUIC requirements testing with RFC9000 manifestsequenceDiagram
actor Developer
participant IvyTools as ivy_tools
participant ProtocolTesting as protocol_testing_quic
participant Manifest as rfc9000_requirements_yaml
participant MCPTool as mcp_tool_quic
participant IvyLSP as ivy_lsp
Developer->>IvyTools: run_quic_requirements_tests()
IvyTools->>ProtocolTesting: start_test_run()
ProtocolTesting->>Manifest: load_requirements()
Manifest-->>ProtocolTesting: requirements_list(101)
loop for each_requirement
ProtocolTesting->>MCPTool: verify_requirement(requirement)
MCPTool->>IvyLSP: execute_quic_scenario(requirement_id)
IvyLSP-->>MCPTool: scenario_result(pass_or_partial)
MCPTool-->>ProtocolTesting: verification_result(status)
end
ProtocolTesting-->>IvyTools: aggregated_results(22_pass_4_partial)
IvyTools-->>Developer: write_report(ivy_tools_test_report_md)
ER diagram for RFC9000 requirements manifesterDiagram
RequirementManifest {
string rfc
string title
}
Requirement {
string requirement_id
string text
string section
string level
string layer
boolean testable
}
RequirementManifest ||--o{ Requirement : contains
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 2 issues, and left some high level feedback:
- The
rfc9000_requirements.yamlmanifest mixes machine-oriented content with human notes (e.g.,-> OK,-> Impossible to know, speculative questions) insidetextfields; consider moving reviewer commentary into separate metadata fields or comments so downstream tooling can reliably treattextas the canonical requirement body. - In
rfc9000_requirements.yaml, some fields are inconsistent in type/format (e.g.,sectionsometimes quoted vs. numeric, requirement IDs likerfc9000:5.2reused for different subsections, and layered text that appears truncated or merged), which could complicate automated consumers; normalizing section identifiers and ensuring each requirement entry is cleanly extracted would make the manifest more robust. - For the requirements marked with inline notes such as
-> not possible to testor-> impossible to know, you may want to encode that constraint in structured form (e.g.,testable: falseor atest_notesfield) rather than leavingtestable: trueso that test tooling and coverage reports more accurately reflect what can actually be verified.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The `rfc9000_requirements.yaml` manifest mixes machine-oriented content with human notes (e.g., `-> OK`, `-> Impossible to know`, speculative questions) inside `text` fields; consider moving reviewer commentary into separate metadata fields or comments so downstream tooling can reliably treat `text` as the canonical requirement body.
- In `rfc9000_requirements.yaml`, some fields are inconsistent in type/format (e.g., `section` sometimes quoted vs. numeric, requirement IDs like `rfc9000:5.2` reused for different subsections, and layered text that appears truncated or merged), which could complicate automated consumers; normalizing section identifiers and ensuring each requirement entry is cleanly extracted would make the manifest more robust.
- For the requirements marked with inline notes such as `-> not possible to test` or `-> impossible to know`, you may want to encode that constraint in structured form (e.g., `testable: false` or a `test_notes` field) rather than leaving `testable: true` so that test tooling and coverage reports more accurately reflect what can actually be verified.
## Individual Comments
### Comment 1
<location path="protocol-testing/quic/rfc9000_requirements.yaml" line_range="212-219" />
<code_context>
+ level: MAY
+ layer: address-validation
+ testable: true
+ rfc9000:8.2.1:
+ text: "Initiating Path Validation\n\nAn endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame\nto at least\
+ \ the smallest allowed maximum datagram size of 1200 bytes,\n**unless the anti-amplification limit for the path** does\
</code_context>
<issue_to_address>
**suggestion:** Remove embedded reviewer notes like "-> impossible to know ?" from the normative `text` field.
Mixing RFC language with inline reviewer commentary in `text` (e.g., `-> impossible to know ?`) can break tools that rely on this field containing only the requirement wording. Please move these annotations to a separate metadata field (such as `notes`) or to commit messages, and reserve `text` for the normative requirement only.
```suggestion
rfc9000:8.2.1:
text: "Initiating Path Validation\n\nAn endpoint MUST expand datagrams that contain a PATH_CHALLENGE frame\nto at least\
\ the smallest allowed maximum datagram size of 1200 bytes,\nunless the anti-amplification limit for the path does not\
\ permit\nsending a datagram of this size."
notes: "Previous inline reviewer note '-> impossible to know ?' was removed from the normative text and recorded here."
section: 8.2.1
level: MUST
layer: address-validation
testable: true
```
</issue_to_address>
### Comment 2
<location path="protocol-testing/quic/rfc9000_requirements.yaml" line_range="571" />
<code_context>
+ level: MUST NOT
+ layer: loss-recovery
+ testable: true
+ rfc9000:13.2.1.5:
+ text: "-> not possible to test\n\nIn order to assist loss detection at the sender, an endpoint SHOULD\t \t In order\
+ \ to assist loss detection at the sender, an endpoint SHOULD\nsend an ACK frame immediately on receiving an ack-eliciting\
</code_context>
<issue_to_address>
**issue (testing):** Align the `testable` flag with the "not possible to test" note to avoid contradictory metadata.
The note says `-> not possible to test` while `testable` is `true`, which is inconsistent and may mislead tooling that uses `testable` to determine expected automated checks. Please either set `testable: false` or update/remove the note so they convey the same intent.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
There was a problem hiding this comment.
Pull request overview
Adds a QUIC RFC 9000 requirements manifest and an MCP “ivy-tools” live test report, while updating ivy-lsp and panther-ivy-plugin submodules to pick up MCP fixes and local-source auto-detection.
Changes:
- Added
protocol-testing/quic/rfc9000_requirements.yamlcontaining 101 extracted RFC 9000 requirements. - Added
docs/ivy-tools-test-report.mddocumenting live verification results across 19 MCP tools. - Bumped
ivy-lspandpanther-ivy-pluginsubmodule commits.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 11 comments.
| File | Description |
|---|---|
| submodules/panther-ivy-plugin | Updates submodule pointer to a commit with local ivy-lsp source auto-detection. |
| submodules/ivy-lsp | Updates submodule pointer to a commit with MCP tool fixes and dependency updates. |
| protocol-testing/quic/rfc9000_requirements.yaml | Adds a new YAML requirements manifest intended to be machine-loadable by tooling. |
| docs/ivy-tools-test-report.md | Adds a new report documenting tool verification, fixes, and known limitations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Comment on lines
+28
to
+33
| text: "Issuing Connection IDs\n\nAn endpoint that initiates migration and requires non-zero-length\t An endpoint that\ | ||
| \ initiates migration and requires non-zero-length\nconnection IDs SHOULD ensure that the pool of connection IDs\t \ | ||
| \ connection IDs SHOULD ensure that the pool of connection IDs\navailable to its peer allows the peer to use a\ | ||
| \ new connection ID on\t available to its peer allows the peer to use a new connection ID on\nmigration, as the peer\ | ||
| \ will close the connection if the pool is\t migration, as the peer will be **unable** to respond if the pool is\n\ | ||
| exhausted." |
There was a problem hiding this comment.
The text fields appear to include extraction artifacts (duplicate sentences/columns, embedded “-> OK/Impossible to know” reviewer notes, odd backslash continuations, and inline commentary). If this manifest is consumed for traceability/coverage, these artifacts can break parsing/display and also make exact matching against RFC text unreliable. Suggestion: keep text strictly as the clean RFC requirement statement (prefer YAML block scalars | for readability), and move any commentary/feasibility notes into a separate field (e.g., notes: or rationale:) so tooling can treat requirement text as canonical.
Suggested change
| text: "Issuing Connection IDs\n\nAn endpoint that initiates migration and requires non-zero-length\t An endpoint that\ | |
| \ initiates migration and requires non-zero-length\nconnection IDs SHOULD ensure that the pool of connection IDs\t \ | |
| \ connection IDs SHOULD ensure that the pool of connection IDs\navailable to its peer allows the peer to use a\ | |
| \ new connection ID on\t available to its peer allows the peer to use a new connection ID on\nmigration, as the peer\ | |
| \ will close the connection if the pool is\t migration, as the peer will be **unable** to respond if the pool is\n\ | |
| exhausted." | |
| text: | | |
| Issuing Connection IDs | |
| An endpoint that initiates migration and requires non-zero-length | |
| connection IDs SHOULD ensure that the pool of connection IDs | |
| available to its peer allows the peer to use a new connection ID on | |
| migration, as the peer will close the connection if the pool is | |
| exhausted. |
Comment on lines
+89
to
+90
| text: "Endpoints MUST validate that received transport\t-> OK\nensures that that an attacker cannot influence the choice\ | ||
| \ of\t parameters match received connection ID values." |
There was a problem hiding this comment.
The text fields appear to include extraction artifacts (duplicate sentences/columns, embedded “-> OK/Impossible to know” reviewer notes, odd backslash continuations, and inline commentary). If this manifest is consumed for traceability/coverage, these artifacts can break parsing/display and also make exact matching against RFC text unreliable. Suggestion: keep text strictly as the clean RFC requirement statement (prefer YAML block scalars | for readability), and move any commentary/feasibility notes into a separate field (e.g., notes: or rationale:) so tooling can treat requirement text as canonical.
Comment on lines
+165
to
+168
| text: '->> is it that now 128 bits is minimum size ?? "would be" | ||
|
|
||
|
|
||
| Servers MUST ensure that replay of tokens is prevented or limited.' |
There was a problem hiding this comment.
The text fields appear to include extraction artifacts (duplicate sentences/columns, embedded “-> OK/Impossible to know” reviewer notes, odd backslash continuations, and inline commentary). If this manifest is consumed for traceability/coverage, these artifacts can break parsing/display and also make exact matching against RFC text unreliable. Suggestion: keep text strictly as the clean RFC requirement statement (prefer YAML block scalars | for readability), and move any commentary/feasibility notes into a separate field (e.g., notes: or rationale:) so tooling can treat requirement text as canonical.
Suggested change
| text: '->> is it that now 128 bits is minimum size ?? "would be" | |
| Servers MUST ensure that replay of tokens is prevented or limited.' | |
| text: | | |
| Servers MUST ensure that replay of tokens is prevented or limited. | |
| notes: 'is it that now 128 bits is minimum size ?? "would be"' |
Comment on lines
+254
to
+271
| text: '-> OK | ||
|
|
||
| An endpoint MUST NOT delay transmission of a | ||
|
|
||
| packet containing a PATH_RESPONSE frame unless constrained by | ||
|
|
||
| congestion control.' | ||
| section: 8.2.2 | ||
| level: MUST NOT | ||
| layer: address-validation | ||
| testable: true | ||
| rfc9000:8.2.2.3: | ||
| text: '-> Impossible to know | ||
|
|
||
|
|
||
| A PATH_RESPONSE frame MUST be sent on the network path where the | ||
|
|
||
| PATH_CHALLENGE frame was received.' |
There was a problem hiding this comment.
The text fields appear to include extraction artifacts (duplicate sentences/columns, embedded “-> OK/Impossible to know” reviewer notes, odd backslash continuations, and inline commentary). If this manifest is consumed for traceability/coverage, these artifacts can break parsing/display and also make exact matching against RFC text unreliable. Suggestion: keep text strictly as the clean RFC requirement statement (prefer YAML block scalars | for readability), and move any commentary/feasibility notes into a separate field (e.g., notes: or rationale:) so tooling can treat requirement text as canonical.
Suggested change
| text: '-> OK | |
| An endpoint MUST NOT delay transmission of a | |
| packet containing a PATH_RESPONSE frame unless constrained by | |
| congestion control.' | |
| section: 8.2.2 | |
| level: MUST NOT | |
| layer: address-validation | |
| testable: true | |
| rfc9000:8.2.2.3: | |
| text: '-> Impossible to know | |
| A PATH_RESPONSE frame MUST be sent on the network path where the | |
| PATH_CHALLENGE frame was received.' | |
| text: | | |
| An endpoint MUST NOT delay transmission of a packet containing a | |
| PATH_RESPONSE frame unless constrained by congestion control. | |
| notes: 'Reviewer feasibility: OK' | |
| section: 8.2.2 | |
| level: MUST NOT | |
| layer: address-validation | |
| testable: true | |
| rfc9000:8.2.2.3: | |
| text: | | |
| A PATH_RESPONSE frame MUST be sent on the network path where the | |
| PATH_CHALLENGE frame was received. | |
| notes: 'Reviewer feasibility: Impossible to know' |
Comment on lines
+45
to
+47
| rfc9000:5.2.2: | ||
| text: "An endpoint MUST generate a\t \t Initial, Retry, or Version Negotiation, MAY be discarded." | ||
| section: '5.2' |
There was a problem hiding this comment.
Requirement metadata looks inconsistent in a way that will make downstream grouping/navigation harder: (1) section sometimes doesn’t match the requirement id (e.g., rfc9000:5.2.2/5.2.3 have section: '5.2'), and (2) section values are inconsistently typed/quoted ('5.2' vs 5.2.1). Suggestion: normalize section to always be the precise subsection (e.g., 5.2.2) and always store it as a string for consistency across YAML parsers and tooling.
Comment on lines
+58
to
+61
| rfc9000:5.2.1: | ||
| text: "Client Packet Handling\n\nIf a client receives a packet that has an **unsupported version**, it\t \t If a client\ | ||
| \ receives a packet that uses a **different version** than it\nMUST discard that packet." | ||
| section: 5.2.1 |
There was a problem hiding this comment.
Requirement metadata looks inconsistent in a way that will make downstream grouping/navigation harder: (1) section sometimes doesn’t match the requirement id (e.g., rfc9000:5.2.2/5.2.3 have section: '5.2'), and (2) section values are inconsistently typed/quoted ('5.2' vs 5.2.1). Suggestion: normalize section to always be the precise subsection (e.g., 5.2.2) and always store it as a string for consistency across YAML parsers and tooling.
Comment on lines
+12
to
+19
| | Metric | Before | After | | ||
| |--------|--------|-------| | ||
| | Tools returning useful data | 8/19 | 19/19 | | ||
| | Visualization tools working | 0/7 | 7/7 | | ||
| | Semantic symbol queries working | 0/3 | 3/3 | | ||
| | Lint false positives (per file avg) | ~15 | 0 | | ||
| | RFC requirements loadable | 0 | 101 (after ISS-008 fix) | | ||
| | RequirementGraph nodes | 0 | 10,723 | |
There was a problem hiding this comment.
The tables use a leading empty column (|| ...) which renders as an extra blank column in Markdown. If that wasn’t intentional, switch to a standard table header row starting with a single | (e.g., | Metric | Before | After |) so the report renders cleanly and consistently across Markdown viewers.
| **Symptom**: MCP log shows `WARNING: PyYAML not installed; cannot load manifest .../rfc9000_requirements.yaml`. Both traceability/coverage tools return 0 requirements. `model_summary.totals.rfcTagsTotal = 0`. | ||
| **Root cause**: `pyyaml` not listed in `pyproject.toml` `[project.optional-dependencies] mcp` extra. The `uvx` virtual environment doesn't install it. | ||
| **Fix**: Added `"pyyaml>=6.0"` to the `mcp` optional dependency list in `submodules/ivy-lsp/pyproject.toml`. | ||
| **Status**: Fix applied — requires MCP server restart to verify. |
There was a problem hiding this comment.
The report is internally inconsistent about ISS-008 verification: it states the fix “requires MCP server restart to verify”, but later marks the traceability/coverage tools as PASS with 101 requirements loaded. Also ivy_model_summary() still calls out rfcTagsTotal=0 (ISS-008) even though the narrative says ISS-008 is fixed. Suggestion: align these sections to reflect the same post-fix state (e.g., either keep ISS-008 as “pending restart” and mark Step 4 as pending/partial, or update the symptom/status lines and the rfcTagsTotal note to match the verified run).
Suggested change
| **Status**: Fix applied — requires MCP server restart to verify. | |
| **Status**: FIX VERIFIED — MCP server restarted; traceability/coverage tools now load 101 requirements and `model_summary.totals.rfcTagsTotal` reflects the loaded RFC tags. |
Comment on lines
+118
to
+122
| ### Step 4: ISS-003+008 — ivy_traceability_matrix + ivy_requirement_coverage | ||
| | Tool | Result | Notes | | ||
| |------|--------|-------| | ||
| | `ivy_traceability_matrix()` | **PASS** | 101 total requirements loaded, 0 covered (ISS-004: bare tags) | | ||
| | `ivy_requirement_coverage()` | **PASS** | 101 total: MUST=45, MUST NOT=12, SHOULD=17, SHOULD NOT=3, MAY=24. 13 layers. | |
There was a problem hiding this comment.
The report is internally inconsistent about ISS-008 verification: it states the fix “requires MCP server restart to verify”, but later marks the traceability/coverage tools as PASS with 101 requirements loaded. Also ivy_model_summary() still calls out rfcTagsTotal=0 (ISS-008) even though the narrative says ISS-008 is fixed. Suggestion: align these sections to reflect the same post-fix state (e.g., either keep ISS-008 as “pending restart” and mark Step 4 as pending/partial, or update the symptom/status lines and the rfcTagsTotal note to match the verified run).
| ### Step 6: ISS-001 — Visualization tools (core 4) | ||
| | Tool | Result | Key Metrics | | ||
| |------|--------|-------------| | ||
| | `ivy_model_summary()` | **PASS** | 486 actions, 8,128 reqs, 2,109 svars, rfcTagsTotal=0 (ISS-008) | |
There was a problem hiding this comment.
The report is internally inconsistent about ISS-008 verification: it states the fix “requires MCP server restart to verify”, but later marks the traceability/coverage tools as PASS with 101 requirements loaded. Also ivy_model_summary() still calls out rfcTagsTotal=0 (ISS-008) even though the narrative says ISS-008 is fixed. Suggestion: align these sections to reflect the same post-fix state (e.g., either keep ISS-008 as “pending restart” and mark Step 4 as pending/partial, or update the symptom/status lines and the rfcTagsTotal note to match the verified run).
Suggested change
| | `ivy_model_summary()` | **PASS** | 486 actions, 8,128 reqs, 2,109 svars, rfcTagsTotal=0 (ISS-008) | | |
| | `ivy_model_summary()` | **PASS** | 486 actions, 8,128 reqs, 2,109 svars, rfcTagsTotal=101 (matches traceability tools) | |
- protocol-testing/patterns/: 13 reusable .ivy template files covering serdes (binary + JSON), variants, monitors, shims (UDP + TCP), modules, and entities (asymmetric + symmetric) patterns - pattern_catalog.yaml: Machine-readable registry with detection heuristics - Update ivy-lsp submodule (pattern detection engine + MCP tools) - Update panther-ivy-plugin submodule (command + skill + enhanced workflow)
ivy-lsp: semantic edge wiring, smart suggestions fix, new MCP tools (ivy_generate_manifest, ivy_scaffold_check, ivy_quality_gate), workspace scoping, per-action state var filtering. panther-ivy-plugin: latest plugin changes.
ivy-lsp: add ivy_diagnostics MCP tool, harden audit logging panther-ivy-plugin: smart LSP source detection, MCP config improvements
…submodules - Clean ~37 garbled requirement text fields (remove diff artifacts, tabs, editorial annotations like '-> OK', '-> impossible to know') - Set testable: false for 3 untestable requirements - Fix Section 14 layer: ecn -> datagram-size (5 requirements) - Update ivy-lsp: ValueError fix, cache thread-safety, truncated flag, path order - Update panther-ivy-plugin: JSON escaping, conditional --reinstall
- Create api/_shared.py so api/ package has self-contained imports (fixes broken `from ._shared import` in runner.py and discovery.py) - Remove misleading "TODO not used" from PantherIvyVersion docstring (class IS used as VERSION_CLASS and type annotation) - Update ivy-lsp submodule: mcp_server.py modularized into tools/ package - Update panther-ivy-plugin submodule: all 25 MCP tools wired, shell scripts deduplicated
- Remove shadowed adapt_environment_paths from IvyProtocolAwareMixin (MRO dead code) - Consolidate 3 protocol name methods into single get_protocol_name() from mixin - Merge duplicate output pattern lists into IvyOutputPatternMixin (fix artifacts/ prefix) - Remove dead build_tests() method (zero callers)
ivy-lsp: consolidate 25 MCP tools to 15, add counterexample parser, per-isolate verification caching, coverage regression detection. panther-ivy-plugin: consolidate agents (9→4), skills (14→6), commands (6→5), soften hooks, update CLAUDE.md.
…lers + plugin tests) - Add classify_endpoint_type() to _shared.py for directory-based test filtering - Modify _build_ivy_model_setup_commands() to selectively copy only relevant test subdirectory (server_tests/, client_tests/, mim_tests/) instead of all - Update submodules: ivy-lsp (goToImplementation + call hierarchy handlers), panther-ivy-plugin (55 integration tests + /nct-health command)
…ports - Replace _extract_test_directory_from_name() body with classify_endpoint_type() to fix mim/attacker test directory divergence between setup and compilation - Fix "strict subset" docstring claim (fallback copies ALL test files) - Improve classify_endpoint_type() docstring (document priority order, attacker mapping) - Add 3 edge case tests (empty string, mim+client precedence, attacker-only) - Remove unused Protocol imports from panther_ivy.py and ivy_command_mixin.py - Fix pattern_catalog.yaml: remove "variant this of" from entity markers (copy-paste), add note about monitor markers matching models not templates
Re-export classify_endpoint_type in api/_shared.py and rewrite api/compiler.py::_extract_test_directory to delegate to it. Adds attacker handling, correct mim>client>server precedence, and role-aware fallback. Updates tests to match.
Removed exact duplicates: rfc9000:5.2.3 (=5.2.2), rfc9000:17.1.3 (=17.1.2), rfc9000:17.2.4.4 (=17.2.4.3), rfc9000:19.11.3 (=19.11.2). Count: 101 → 97 requirements.
…misclassification Replace bare substring matching with position-aware segment matching. The function now splits test names by underscore and only checks segments before the first "test" segment for endpoint keywords, preventing scenario words like "mim" in "quic_client_test_0rtt_mim_replay" from overriding the actual endpoint "client". This fixes misclassification of 7+ real QUIC test files that contain "mim" in their scenario portion but are client or server tests.
…emove duplicate -print - Log warning when classify_endpoint_type falls back to role-based inference - Narrow except Exception to (AttributeError, TypeError) in _process_commands() - Remove duplicate -print from find-echo command that doubled entries in copied files list
- Fix 6 section-key mismatches in rfc9000_requirements.yaml where the section field was truncated relative to the key (e.g. key 5.2.2 had section 5.2). Affected keys: 5.2.2, 5.2.1.2, 7.3.2, 10.2.2, 14.2, 17.2.5.3.8 - Update .ivyworkspace to version 2 with scope_detection: auto - Update ivy-lsp submodule (gitignore fix for temp dir leakage) - Update panther-ivy-plugin submodule (LSP env config, version bump, hook scoping, agent/skill improvements)
14 tests covering: - Missing model path returns empty list with warning - Endpoint-type-aware subdirectory selection (server, client, mim, attacker) - Protocol name and $PYTHON_IVY_DIR references in generated commands - Prune flag for tests directory exclusion - Fallback warning logging when no endpoint keyword matches - No warning for known endpoint keywords - Different protocol names produce correct tests dir - If/else fallback branch presence in shell commands - Role-based inference via oppose_role when no keyword present
…commit Re-point submodule after accidental pointer regression in previous commit.
ivy-lsp: workspaceSymbol active_filepath passthrough, test_file scoping for coverage tools panther-ivy-plugin: nct-validate ground truth update (97 reqs), coverage scoping docs
…names, observability)
…w simplifications)
…line + AskUserQuestion recorder)
Picks up the panther-ivy-plugin work from session 2026-05-01:
* Refactor: rename build → scaffold across docs, hooks, evals, tests
(2f59080, 1a2cc55, bb73469).
* Refactor: consolidate helpers into hook_utils + tighten emit_hook_output
validation (b7fc853).
* Feat: add ruff PostToolUse formatter + SessionStart hook-path self-test
(4aa258a).
* Feat: record AskUserQuestion calls + migrate three test files +
fix ruff fix-count two-call regression (58ff31b).
The inner-submodule pytest baseline holds: 251 + 8 new = 259 passing,
6 pre-existing failures unchanged (test_g{1,2,3}_* reference removed
route-user-prompt.py; test_checkpoint_* reference removed
interaction-checkpoint.py).
…se hook_utils helpers)
…-placeholder validator)
Two fixes that together with the outer-worktree commit unblock Path α
Phase E. Each was hidden behind the previous one in the panther run pipeline.
Dockerfile.buildkit:59 — make the apt-mirror sed substitution idempotent.
The previous form `s|archive.ubuntu.com|fr.archive.ubuntu.com|g` matches
the inner `archive.ubuntu.com` substring inside an already-substituted
`fr.archive.ubuntu.com` and produces `fr.fr.archive.ubuntu.com`. The
panther_ivy stage inherits its sources.list from the panther_base_service
image which has already applied the same sed, so re-running it broke
apt-get update with `Could not resolve 'fr.fr.archive.ubuntu.com'`.
Anchoring the regex on `://` makes the substitution idempotent.
ivy_command_mixin.py:817-819 — extend the placeholder validator regex to
accept the secondary-endpoint bracket form `@{ivy_tester:ip[bgp_c]:hex}`
introduced by Path α. The network resolver already understood this form;
only the upstream validator was rejecting it as malformed.
Per Phase 3 of the bloat-audit master plan. Initial idle state for bgp, quic, apt, minip, coap. Generated by panther-ivy-plugin/scripts/migrate-bootstrap-project-md.py. Subsequent edits will be hook-driven (PostToolUse on ivy_workflow_state set/clear).
Picks up the 5-phase workspace + observability discipline work landed
2026-05-02 (commits 48c057d / eed4d18 / 611d4d3 / 114e3d4 / 7a91efb):
* T1/T2/T3 state-persistence message templates documented in
.claude/rules/output-style.md (Phase A).
* SessionStart workspace banner reader fix via shared
hook_utils.resolve_workspace_state_path (Phase B).
* notify-workspace-change.py surfaces mid-session ivy_workspace set/clear
as a T3 banner (Phase C).
* 8 file-writing hooks retrofitted to cite their journal/file paths
in systemMessage (Phase D).
* tests/test_observability_write_discipline.py AST + regex lint
backstops the discipline (Phase E).
Closes the three findings from the 2026-05-02 hooks audit:
SessionStart workspace bug, missing mid-session feedback, inconsistent
observability-write discipline across hooks.
…e-cache + 7 concurrent commits) Bumps submodules/panther-ivy-plugin from bf93e7b1 to 26f0505e. The new range bundles my targeted statusline-cache fix together with seven concurrent commits that landed during the same window; recording them together because they all already exist on the inner submodule branch and a partial bump is not feasible. Range bf93e7b1..26f0505e contents: 26f0505 feat(plugin/hooks): sync-statusline-cache.py mirrors active-workflow YAML at SessionStart <- this session's work 64c643e docs(plugin/contract): document session-activity flag and updated session_start semantics c37285f test(plugin/hooks): regression coverage for emit_dedup + PID-time error filter 1ce17f1 feat(plugin/hooks): mark session activity on specialist-agent dispatch b72c96d feat(plugin/hooks): add mark-mcp-activity.py hook for panther-ivy MCP tools 30b3fb8 feat(plugin/hooks): mark session activity on .ivy file edits ... (concurrent: session-activity-flag rollout cluster) Primary motivation for THIS bump: 26f0505 closes the statusline cache desync that left 'wf:workflow-triage' rendering for 5+ days while the on-disk active-workflow YAML had migrated to 'scaffold'. The remaining commits in the range are session-activity-flag and mark-mcp-activity work landed by parallel sessions; they all need to ship together because they share state and event-handler infrastructure.
…concurrent activity-gate work) Bumps submodules/panther-ivy-plugin from 26f0505e to b520327e. The new range bundles my Commit 2 docs canonicalization with two concurrent commits that landed in the same window. Range 26f0505e..b520327e contents: b520327 test(activity-gate): Task 10 — session-activity gate test suite <- concurrent; also performs C5 fixture canonicalization in test_workflow_state.py 37c6d8a docs(plugin): canonicalize legacy workflow-* names in rules + skills <- this session's Commit 2 7a1c844 test(plugin/hooks): tighten emit_dedup regression coverage from review <- concurrent The C5 deferred-fixture canonicalization documented in audit-complete-steady-hartmanis.md was attempted by this session as Commit 3 but the concurrent commit b520327 had already performed the identical sed-equivalent rewrite (workflow-build -> scaffold, workflow-verify -> refine, workflow-review -> review, workflow-triage -> triage, workflow-navigate -> navigate plus _KNOWN canonicalization). The session's change was therefore a no-op and is folded into b520327 rather than authored as a separate commit. Net statusline-task delta from this bump: documentation prose now uses the canonical workflow names (mcp-tool-reliability.md, plan-mode.md, triage-ops/SKILL.md, ivy-toolkit/references/tool-catalog.md), and the test_workflow_state.py fixture rewrite closes the deferred C5 work.
…ync-statusline-cache fixes + 5 new tests)
…no-op-write guard + ternary flatten)
…ion-isolation Phases 1-5) Lands the per-protocol + per-session statusline partitioning end-to-end: dff9185 Phase 1 — cache layer (active_group + overlay APIs + 29 tests) 35a3268 Phase 2 — hook writers (overlay routing + active_group threading + 4 tests) 3cef186 Phase 3 — renderer (protocol-partitioned cache, per-session overlay, session badge segment) bb90fa8 Phase 4 — one-shot legacy-cache migration + journaling-contract §12 1b357c7 Phase 5 follow-on — render-budget cap raised to 1500 ms for system-noise tolerance Also pulls in the parallel-session simplify pass: 0e5f8f1 refactor(plugin/hooks): simplify-pass cleanup of sync-statusline-cache.py User-visible behavior change: two Claude Code windows in the same panther_ivy/ workspace now see distinct statusline test_file segments (per-session overlay) and a per-window session badge. Cross-protocol crosstalk is fixed — running ivy_workspace(set, target=quic) in one window no longer rewrites the wf: segment in another window that's working on bgp. Tests: 422 pytest + 20 bash renderer tests pass. Full plan and verification at ~/.claude/plans/i-need-to-know-delegated-kahn.md.
…L PR1 Picks up 4 commits from the harness audit + grill plan (~/.claude/plans/the-plugin-users-elniak-documents-docume-floofy-kitten.md): 1cd11fb chore(plugin/readme): regenerate inventory + add parity test (L1) 7532f34 docs(plugin/critics): document parallel-vote dispatch (L6) 2910973 test(plugin): migrate workflow-* fixtures to canonical names (L11) 8181418 feat(plugin/hooks): references-cap-hit stderr log + load-order docs (L13) L1 also folds in 9 pre-staged files unrelated to the harness audit (check-indexing-ready.py, hook_utils.py, post-write-workflow-aware.py, statusline_cache.py, statusline test files, conftest.py) that were already in the index from concurrent statusline-overlay work; user accepted the absorption when the pattern surfaced mid-session. PR1 of 3 in the Tier-L bundle; L2 + L10 dropped per review-plan (H3 falsified, H12 falsified). PR2 (contract bug fixes) and PR3 (inline 14-layer reference) follow.
…ure) Lands Phase 1.5 of the ivy-lsp diagnostic IR migration: MCP-side ivy_diagnostics tool migrated to IvyDiagnostic IR; transitional ALLOWLIST in test_no_raw_dict_diagnostics.py emptied; audit-fence regex tightened to exclude pure (file, line) location anchors. ivy-lsp range: 808f764..9880af8 (25 commits, all on release/v0.12.0-package-redesign-and-workspace-features). PR #6 on ElNiak/ivy-lsp tracks the work. Not pushed — local-only bump per session 2026-05-04 user decision.
Picks up 5 commits across both submodules from the harness audit + grill plan (~/.claude/plans/the-plugin-users-elniak-documents-docume-floofy-kitten.md): ivy-lsp (3 commits): d789ad1 fix(workspace): error loudly when ivy_workspace gets roles + .ivy target (L4) 9ab3016 fix(workflow_state): add pending_dispatch to _VALID_EVENT_TYPES (L9) 456248c fix(diagnostics): narrow silent except in pattern checks (L14) panther-ivy-plugin (2 commits): 073d2e8 fix(orchestrator): validate target_workflow before consuming pending_dispatch (L8) 05c7153 test(plugin): cross-source parity for journal event types (L9) L3 + L12 closed as no-op (Phase 1.5 IR migration already happened; ALLOWLIST is empty; nothing to track or test). PR2 is the contract-bug slice; PR3 (inline 14-layer reference) follows next.
Picks up bc4464f (feat(plugin/specification-patterns): inline 14-layer
reference fallback (L5)) — the third and final PR of the Tier-L tactical
fixes from the harness audit + grill plan.
L5 closes the cold-start gap where the canonical 14-layer template was
only available via auto-memory; the inline table is the always-present
fallback for new contributors.
Tier-L bundle complete:
PR1 (1cd11fb..8181418, 4 commits) — README inventory + critic docs +
fixture migration + references-cap log
PR2 (073d2e8 + 05c7153 in plugin; d789ad1 + 9ab3016 + 456248c in
ivy-lsp) — contract bug fixes (workspace roles, target_workflow
validation, event-types parity, narrow except)
PR3 (bc4464f) — inline 14-layer reference
Tier M and X follow per the locked sequence in the plan; see §6 of
~/.claude/plans/the-plugin-users-elniak-documents-docume-floofy-kitten.md
Picks up aaa55bb fix(indexer): workspace-first stdlib resolution +
drift detector. Resolves the indexing breakage where 191 apt+bgp .ivy
files mis-fired with "wrong number of arguments to module
network_implementation" because the include resolver was reading a
stale 5-arg copy from a system install instead of the worktree's 6-arg
source.
Concrete impact for any session running `python -m ivy_lsp index --all`
from this worktree:
- bgp: 37 ast / 11 lexer / 11 failures -> 48 ast / 0 lexer / 0 failures
- apt: 145 ast / 180 lexer / 180 failures -> 289 ast / 36 lexer / 36 failures
(the 36 remaining apt failures are a separate layer-staging
issue with quic_time.ivy / random_value.ivy, unrelated to this
PR — tracked separately if it surfaces as a real symptom)
PR4 of the harness audit Tier-L bundle.
…ization Bumps panther-ivy-plugin to 0808c8b (Step-0 cleanup before the broader hooks/ refactor planned at /Users/elniak/.claude/plans/ow-to-refactor-and-lucky-rainbow.md). Inner commit lands the in-progress workflow-journal / active-workflow path-centralization work that was sitting uncommitted. See the panther-ivy-plugin commit body for the full change list and the known-degraded test baseline (8 failures, all addressed by PR1 of the refactor). Other unstaged state at this submodule level (ivy-lsp pointer, deleted observability log, modified workflow journal, runtime/cache directories) is intentionally NOT included — only the panther-ivy-plugin pointer bump rides along, per the Step-0 hooks-only scope.
Bumps panther-ivy-plugin to d0cb9d7. PR1 spans 6 inner commits: - 6a3a9fe: lib/ scaffolding + project_md_state/style_utils mv + ivy_path helper (T4) - aeac3d3: hook_utils.py split into lib/hook_utils/ (T1) - b85656d: workflow_state.py split into lib/workflow_state/ (T2) - 853b2f8: statusline_cache.py split into lib/statusline_cache/ (T3) - af46be5: lead intervention rewriting two surviving lib/-internal old-style imports - d0cb9d7: import sweep across 22 hooks + 16 tests + 4 outside scripts; git rm of the three old single-file modules (T5) Independent verification: pytest baseline preserved at 429 passed / 8 failed with the same 8 known-degraded test names from Step-0 (commit 0808c8b). PR1 introduced zero new failures. Per the plan at /Users/elniak/.claude/plans/ow-to-refactor-and-lucky-rainbow.md this is the first of four PRs; PR2 will move hooks into purpose folders and add the sys.path bootstrap pattern.
…override + I3 dead-code consolidation)
…cleanup before PR2)
…(test isolation fix)
…tor (purpose-folder moves + bootstrap + citation sweep)
…tor (refactor complete)
…ow-ups Picks up the three deferred follow-ups from PR4: - M1 (8049dd8): regression guard for the render.summary lazy __getattr__ recursion bug (one test in tests/test_render_summary.py). - M2 (486395a): rewrite three hook docstrings to fit the INDEX.md prefix-stripping regex (workspace/change-notify.py, journaling/contract-check.py, statusline/sync.py + INDEX.md regen). - M3 (cf3eed2): new AST-scan discipline test (tests/test_hook_bootstrap_discipline.py) verifying every hook script using `from lib.…` declares the canonical sys.path.insert prelude with depth-correct parents[N]. Plugin test suite: 445 → 447 passed.
…time fixes) Picks up panther-ivy-plugin 2a32d6b which closes the two PR1 Critical findings (workspace-common.sh + run-tests.sh shell-side lib.* imports) and the scaffold.py Important finding (dead warn-gate + raw stderr migration to push_warning) from the 2026-05-04 verification audit. Pytest 448/0 in the plugin at the new SHA.
…s sweep)
Picks up panther-ivy-plugin 9acd75e which closes audit items 3, 4, 5:
- Item 3: README sweep across 4 user-facing docs + restructure of
hooks/scripts/README.md to a family-README index + 4 new family READMEs
for lib/, observability/, posttooluse/, session/.
- Item 4: record/workflow-error.py:115 trigger-field rewrite to match
the post-PR2 path style.
- Item 5: 5 surfaces sweeping the stale render/summary.py reference to
the post-PR4 render/summary/main.py path.
Pytest 447/1 in the plugin at the new SHA. The 1 failure
(test_post_write_no_workflow_suggests_review) is environmental, not a
regression: documented in the plugin commit message.
See .claude/audit-2026-05-04-hooks-refactor-verification.md §1.2-1.4 for
the specific findings closed.
… schema-drift fix - ivy-lsp 563effa..21e5b89: drops caller/invocation_depth from ivy_workflow_state(action="set") signature, _handle_set, data dict; test_set_persists_full_data_to_file asserts the 3-field schema invariant via set-equality. - panther-ivy-plugin 9acd75e..1321551: drops caller/invocation_depth from statusline cache writer (sync.py), bash cache reader (cache.sh), workflow segment (workflow.sh); updates docs and 3 test fixture sites. Closes the deferred fix from ~/.claude/projects/<project>/memory/handoff-2026-05-04-active-workflow-schema-drift.md (option B). The "WorkflowContext dropped unknown fields" WARN that fired on every Stop / PostToolUse / UserPromptSubmit hook is now gone at its source: the MCP writer no longer produces the extra fields.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test Plan
Summary by Sourcery
Add a structured RFC 9000 requirements manifest and document end-to-end MCP tool verification, alongside submodule updates that improve ivy MCP tooling reliability and local development workflow.
New Features:
Bug Fixes:
Enhancements:
Documentation: