Skip to content

Commit 318f852

Browse files
maehwasoomaehwasoo
authored
Merge pull request #187 from maehwasoo/feat/issue-182-python-parity-verification
feat: verify Python parity for the current MCP read/write tool surface
2 parents 135c1a4 + 86aabc5 commit 318f852

File tree

6 files changed

+303
-0
lines changed

6 files changed

+303
-0
lines changed

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -209,6 +209,7 @@ Full reference: `docs/01-overview.md` and `docs/reference/mcp-tools.md`.
209209
- `docs/README.md`: documentation index
210210
- `docs/01-overview.md`: architecture + MCP tool surface
211211
- `docs/architecture/python-first-local-agent-backend.md`: transition baseline, service boundaries, API contract
212+
- `docs/architecture/python-mcp-parity.md`: parity target, migration gate, and current gaps for the MCP tool surface
212213
- `docs/ops/codex-cli.md`: Codex CLI setup
213214
- `docs/ops/local-dev.md`: local development
214215
- `docs/standards/vault/README.md`: vault model and rules

‎apps/api/tests/helpers.py‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -155,6 +155,7 @@ def seed_index_db(db_path: Path) -> None:
155155
"INSERT INTO note_tags(path, tag) VALUES (?, ?)",
156156
[
157157
("docs/03-plan.md", "architecture"),
158+
("docs/03-plan.md", "project"),
158159
("notes/random.md", "misc"),
159160
],
160161
)

‎apps/api/tests/test_parity_contract.py‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
from __future__ import annotations
2+
3+
import re
4+
from pathlib import Path
5+
from typing import get_args
6+
7+
from fastapi.testclient import TestClient
8+
from pytest import MonkeyPatch
9+
10+
from ailss_api.main import create_app
11+
from ailss_api.models import FailureCode
12+
from tests.helpers import (
13+
DEFAULT_AGENT_INPUT,
14+
DEFAULT_RETRIEVE_QUERY,
15+
build_settings_with_seed_data,
16+
set_fake_embedding,
17+
)
18+
19+
PARITY_DOC_PATH = (
20+
Path(__file__).resolve().parents[3] / "docs" / "architecture" / "python-mcp-parity.md"
21+
)
22+
23+
24+
def _extract_bulleted_code_items(markdown: str, heading: str) -> list[str]:
25+
lines = markdown.splitlines()
26+
heading_index = next(
27+
(index for index, line in enumerate(lines) if line.strip() == heading),
28+
-1,
29+
)
30+
if heading_index < 0:
31+
return []
32+
33+
values: list[str] = []
34+
for line in lines[heading_index + 1 :]:
35+
stripped = line.strip()
36+
if not stripped:
37+
if values:
38+
break
39+
continue
40+
if stripped.startswith("#"):
41+
break
42+
43+
match = re.match(r"^- `([^`]+)`$", stripped)
44+
if match is None:
45+
continue
46+
values.append(match.group(1))
47+
return values
48+
49+
50+
def test_parity_doc_lists_current_agent_failure_codes() -> None:
51+
doc = PARITY_DOC_PATH.read_text(encoding="utf-8")
52+
53+
documented = sorted(
54+
_extract_bulleted_code_items(
55+
doc,
56+
"## Required failure codes for Python-covered agent flows",
57+
)
58+
)
59+
actual = sorted(get_args(FailureCode))
60+
61+
assert documented == actual
62+
63+
64+
def test_retrieve_parity_respects_combined_scope_filters(
65+
tmp_path: Path, monkeypatch: MonkeyPatch
66+
) -> None:
67+
settings = build_settings_with_seed_data(tmp_path)
68+
set_fake_embedding(monkeypatch)
69+
client = TestClient(create_app(settings))
70+
71+
response = client.post(
72+
"/retrieve",
73+
json={
74+
"query": DEFAULT_RETRIEVE_QUERY,
75+
"top_k": 3,
76+
"path_prefix": "docs/",
77+
"tags_any": ["project"],
78+
"tags_all": ["architecture", "project"],
79+
},
80+
)
81+
82+
assert response.status_code == 200
83+
payload = response.json()
84+
assert [result["path"] for result in payload["results"]] == ["docs/03-plan.md"]
85+
assert set(payload["results"][0]["tags"]) == {"architecture", "project"}
86+
87+
88+
def test_agent_run_returns_missing_context_when_scoped_filters_remove_candidates(
89+
tmp_path: Path, monkeypatch: MonkeyPatch
90+
) -> None:
91+
settings = build_settings_with_seed_data(tmp_path)
92+
set_fake_embedding(monkeypatch)
93+
client = TestClient(create_app(settings))
94+
95+
response = client.post(
96+
"/agent/run",
97+
json={
98+
"input": DEFAULT_AGENT_INPUT,
99+
"context": {
100+
"path_prefix": "notes/",
101+
"tags_all": ["architecture"],
102+
"top_k": 1,
103+
},
104+
},
105+
)
106+
107+
assert response.status_code == 200
108+
payload = response.json()
109+
assert payload["outcome"] == "failed"
110+
assert payload["failure"]["code"] == "missing_context"
111+
112+
113+
def test_agent_run_rejects_write_request_with_apply_true(tmp_path: Path) -> None:
114+
settings = build_settings_with_seed_data(tmp_path)
115+
client = TestClient(create_app(settings))
116+
117+
response = client.post(
118+
"/agent/run",
119+
json={
120+
"input": "Write the summary into a new note.",
121+
"requested_write_action": "capture_note",
122+
"apply": True,
123+
},
124+
)
125+
126+
assert response.status_code == 200
127+
payload = response.json()
128+
assert payload["outcome"] == "failed"
129+
assert payload["failure"]["code"] == "write_not_allowed"
130+
assert payload["write_actions"] == [
131+
{
132+
"action": "capture_note",
133+
"allowed": False,
134+
"reason": "write_not_allowed",
135+
}
136+
]

‎docs/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ This folder is organized so you can read the AILSS system docs in order: **Conte
2424

2525
- Package structure: [architecture/packages.md](./architecture/packages.md)
2626
- Python-first local agent baseline: [architecture/python-first-local-agent-backend.md](./architecture/python-first-local-agent-backend.md)
27+
- Python parity for MCP tool surface: [architecture/python-mcp-parity.md](./architecture/python-mcp-parity.md)
2728
- Data & database: [architecture/data-db.md](./architecture/data-db.md)
2829

2930
## Ops

‎docs/architecture/python-mcp-parity.md‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
# Architecture: Python parity for the current MCP tool surface
2+
3+
This document defines the parity boundary tracked by issue #182.
4+
5+
It does not claim that the current Python backend already replaces the Node/MCP runtime.
6+
Instead, it records which MCP behaviors must match before any later migration removes the
7+
existing Node path, which differences are acceptable during the current baseline, and which
8+
tools remain explicit gaps.
9+
10+
## Terms
11+
12+
- Exact-match parity: behavior, safety constraints, and failure semantics must stay aligned
13+
with the current MCP contract before the Node path can be removed.
14+
- Acceptable baseline difference: a documented difference that is allowed while the Node/MCP
15+
path remains the source of truth for that capability.
16+
- Gap / Node-only: there is no public Python replacement yet; the current MCP tool must stay
17+
on the Node path.
18+
19+
## Current parity boundary
20+
21+
- Python-covered now:
22+
- `/retrieve` is the Python-side parity target for `get_context`-style retrieval behavior.
23+
- `/agent/run` is the Python-side parity target for grounded-answer behavior and explicit
24+
write refusal semantics.
25+
- Still Node-owned now:
26+
- MCP transport, indexer ownership, filesystem read tools, graph/navigation tools,
27+
metadata/list tools, diagnostics tools, and all explicit write tools.
28+
- Migration gate:
29+
- Do not remove the Node/TypeScript path until every `Gap / Node-only` row below has an
30+
exact-match Python replacement or an explicitly accepted new contract.
31+
32+
## Read tool parity matrix
33+
34+
| Tool | Current Python target | Status | Exact-match requirements before migration | Acceptable baseline differences / current gap |
35+
| ----------------------------- | ------------------------------------------------------- | --------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
36+
| `get_context` | `/retrieve` and the retrieval stage inside `/agent/run` | Partial parity | Preserve scoped candidate filtering (`path_prefix`, `tags_any`, `tags_all`), explicit readiness failures, grounded evidence, and no unrelated fallback when scope removes all candidates. | Response shape differs from MCP `structuredContent`; Python caps `top_k` at `20`; Python does not expose MCP-only fields such as `expand_top_k` or `applied_filters`. |
37+
| `expand_typed_links_outgoing` | None | Gap / Node-only | Add a Python graph expansion contract that preserves bounded traversal semantics before migration. | No Python graph traversal API exists yet. |
38+
| `resolve_note` | None | Gap / Node-only | Add a Python note-resolution contract before migration. | No Python resolution endpoint exists yet. |
39+
| `find_typed_links_incoming` | None | Gap / Node-only | Add a Python incoming-link query contract before migration. | No Python backref query exists yet. |
40+
| `list_typed_link_rels` | None | Gap / Node-only | Add a Python relation-listing contract before migration. | No Python typed-link relation listing exists yet. |
41+
| `read_note` | Internal note reads inside `/agent/run` only | Gap / Node-only | Add a public Python note-read contract with the current vault-boundary guarantees before migration. | The current Python backend only reads note excerpts internally; it does not expose the public pagination/change-token contract of `read_note`. |
42+
| `get_vault_tree` | None | Gap / Node-only | Add a Python vault-tree contract before migration. | No Python vault-tree API exists yet. |
43+
| `frontmatter_validate` | None | Gap / Node-only | Add a Python validator contract that preserves the current schema and typed-link diagnostic expectations before migration. | No Python frontmatter validation API exists yet. |
44+
| `find_broken_links` | None | Gap / Node-only | Add a Python broken-link diagnostic contract before migration. | No Python broken-link diagnostic API exists yet. |
45+
| `search_notes` | None | Gap / Node-only | Add a Python metadata-search contract before migration. | No Python metadata-search API exists yet. |
46+
| `list_tags` | None | Gap / Node-only | Add a Python tag-listing contract before migration. | No Python tag-listing API exists yet. |
47+
| `list_keywords` | None | Gap / Node-only | Add a Python keyword-listing contract before migration. | No Python keyword-listing API exists yet. |
48+
| `get_tool_failure_report` | None | Gap / Node-only | Add a Python diagnostics-report contract before migration. | No Python diagnostics-report API exists yet. |
49+
50+
## Write tool parity matrix
51+
52+
| Tool | Current Python target | Status | Exact-match requirements before migration | Acceptable baseline differences / current gap |
53+
| -------------------------- | ------------------------------ | --------------- | ---------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
54+
| `capture_note` | `/agent/run` refusal path only | Gap / Node-only | Preserve explicit `apply` gating and current vault-write safety before any Python write path replaces this tool. | The current Python baseline never creates notes. It only returns explicit refusal semantics. |
55+
| `canonicalize_typed_links` | `/agent/run` refusal path only | Gap / Node-only | Preserve explicit `apply` gating and safe deterministic mutation rules before migration. | The current Python baseline never rewrites typed links. It only returns explicit refusal semantics. |
56+
| `edit_note` | `/agent/run` refusal path only | Gap / Node-only | Preserve explicit `apply` gating, concurrency guards, and vault-boundary safety before migration. | The current Python baseline never edits notes. It only returns explicit refusal semantics. |
57+
| `improve_frontmatter` | `/agent/run` refusal path only | Gap / Node-only | Preserve explicit `apply` gating and current frontmatter safety rules before migration. | The current Python baseline never mutates frontmatter. It only returns explicit refusal semantics. |
58+
| `relocate_note` | `/agent/run` refusal path only | Gap / Node-only | Preserve explicit `apply` gating and current path safety rules before migration. | The current Python baseline never moves notes. It only returns explicit refusal semantics. |
59+
60+
## Safety invariants that must not regress
61+
62+
- Vault boundary: Python note reads must stay within `AILSS_VAULT_PATH`; path traversal or
63+
absolute-escape attempts must not read outside the configured vault.
64+
- Write gating: requested writes must fail explicitly unless the future Python path has an
65+
approved replacement for the current MCP write contract.
66+
- Failure visibility: missing or invalid local prerequisites must fail explicitly rather than
67+
silently degrading to unrelated retrieval results.
68+
- Grounding: Python-generated answers must keep inspectable citations; missing evidence is a
69+
failure, not a best-effort answer.
70+
71+
## Required failure codes for Python-covered agent flows
72+
73+
- `missing_context`
74+
- `ambiguous_note_resolution`
75+
- `grounding_failure`
76+
- `write_not_allowed`
77+
- `apply_not_requested`
78+
79+
## Verification currently in repo
80+
81+
- `packages/mcp/test/docs.mcpToolingConsistency.test.ts`
82+
- Keeps this parity matrix aligned with the live MCP tool surface.
83+
- `apps/api/tests/test_parity_contract.py`
84+
- Verifies the documented Python parity-critical behaviors that already exist in the
85+
baseline.
86+
- Existing Python route/unit coverage:
87+
- `apps/api/tests/test_retrieve.py`
88+
- `apps/api/tests/test_agent.py`

0 commit comments

Comments
 (0)