ideas: record the partner-doc source lane as held (proposed, droppable) - #58
Conversation
…w and reopen triggers Proposed, not asserted — see the PR description. This is the maintainer's call to make; the file is written in the shape ideas/stellar-org-source-lane.md established so it is easy to accept, amend, or drop without touching the eval evidence in the preceding commit. What it records about issue kalepail#18: - No partner-MCP federation. This declines the mechanism asked for in the issue thread, so it owes those commenters a reason: Alchemy's hosted MCP is 168 tools behind OAuth 2.1 including create_app/update_allowlist, and OpenZeppelin's advertises contract generation, not documentation retrieval. Federating either would let the partner decide what is callable, inverting ADR-0003. - Allowlisted first-party partner Markdown is credible, and held behind the four-phase gate. - A dated review (2026-10-31) and two separate reopen triggers — one for the Markdown lane, a narrower one for the MCP question, which reopens only on a read-only, unauthenticated, documentation-scoped partner MCP with a stable tool contract. Three things it is careful to say, because the easy version of this argument is wrong: - The hold is corroborated, not novel. stellarlight#448 reached the same "no" on the same llms.txt grounds three weeks earlier. - Ingestion is not retrieval. stellarlight#565 found that after stellar-docs#2573 was ingested, a bare q=Alchemy still returned zero results and the Indexers chunk ranked below top-15 for its own brand query. "The improvements pipeline already catches it" is too glib, and that live failure argues for fixing retrieval upstream before adding a source. - The residue is already measured upstream by stellarlight#561's monthly coverage-watch — whose verdict semantics were then corrected by #564, so its first run's 0/4 should not be quoted as standing fact. The issue's concrete Alchemy example healed from two directions at once: stellar-docs#2573 upstream, and stellarlight#446 downstream four days earlier.
kalepail
left a comment
There was a problem hiding this comment.
Merging as written. You were right that writing a decision record is a maintainer act, and right to unfuse it from the evidence — but the framing here is one I'd have written myself, so there's nothing to rewrite.
I checked every upstream citation rather than taking them on trust, because a decision record's whole value is that its references hold. All seven resolve, are merged, and say what you say they say: stellarlight #446, #447, #448, #561, #564, #565, and #657. That is not a given for a document with this citation density, and it's why I'm taking the analysis as-is.
The three things you were careful not to overclaim are the three that make this worth keeping as a file:
- "The hold is corroborated, not novel." Leading with the fact that #448 reached the same conclusion three weeks earlier, on the same
llms.txtgrounds, is the opposite of how a contributor usually pitches their own analysis. - "Ingestion is not retrieval." This is the load-bearing correction.
stellar-docs#2573merging and going live is exactly the kind of upstream fix our improvements pipeline exists to produce, and #565 showing that a bareq=Alchemystill returned zero results after ingestion is the finding that content reaching official docs is necessary and not sufficient. My own pipeline claim would have been too glib in the same way. - Not quoting the
coverage-watch0/4 as standing fact once #564 changed the verdict semantics underneath it.
The declines are correct on the merits. A 168-tool OAuth surface including create_app and update_allowlist is not a documentation source, and mounting it would invert ADR-0003 by letting the partner decide what is callable without a repo change — the reopen trigger you wrote for the MCP question (read-only, unauthenticated, documentation-scoped, stable tool contract) is the right shape, and "a partner adding more tools is evidence against, not for" is the sentence that keeps it honest.
One note for the record, since #18 is still the parent: the acute Alchemy instance closing does not close the issue's systemic point, and this file says so. That's the right disposition — held, dated, with triggers — rather than resolved.
Thanks for splitting these, and for the retraction discipline in #447 -> #448 that shows up as the negative-grep lesson in the provenance rules.
Refs #18. Proposed, not asserted — and explicitly droppable.
This is the half of my earlier #56 that I should not have fused with the evidence. Writing a decision record with a review date and reopen triggers is a maintainer act, not a contributor one, and coupling it to the eval work forced you to adjudicate the product question in order to accept a test corpus. The evidence now lives in its own PR, sharing no files with this one; either can merge without the other.
So: take this, amend it, ask me to rewrite it in a different voice, or close it and write your own. The analysis is offered; the framing is yours.
What it records
ideas/partner-doc-live-sources.md, in the shapeideas/stellar-org-source-lane.mdestablished.No partner-MCP federation. This declines the mechanism asked for in the issue thread itself — @oceans404's "ideally we'd tap into their docs mcps if they have them" and @mmazco's "most should have by now" — so it owes them a reason rather than a verdict. The reason is what those servers currently are: Alchemy's hosted MCP is 168 tools behind OAuth 2.1 including
create_app/update_allowlist; OpenZeppelin's advertises contract generation, not documentation retrieval. Federating either would let the partner decide what is callable, inverting ADR-0003 without a repo change.Allowlisted first-party partner Markdown is credible, and held behind the four-phase gate.
A dated review (2026-10-31) and two separate reopen triggers: one for the Markdown lane, and a narrower one for the MCP question that fires only on a read-only, unauthenticated, documentation-scoped partner MCP with a stable tool contract. Neither partner ships that today, and a partner adding more tools is evidence against, not for.
Three things it is careful not to overclaim
The easy version of this argument is wrong in ways an adversarial pass caught:
llms.txtgrounds three weeks earlier, and chose "the structured record plus a first-class pointer to the living source" over ingestion.stellar-docs#2573was ingested, a bareq=Alchemystill returned zero results, and the Indexers chunk documenting the Data API ranked below top-15 for its own brand query. That is a real, dated live failure of the kind this lane's own reopen trigger asks for — and it argues for fixing retrieval upstream before adding a source here.coverage-watch— whose verdict semantics were then corrected by #564, so its first run's "0 of 4 partners" should not be quoted as standing fact. Re-evaluation should read the current run, not re-derive the number.On #18's concrete example
It healed from two directions at once:
stellar/stellar-docs#2573upstream (merged, live-verified 2026-07-15, tracked assd-010), and stellarlight#446 downstream four days earlier, which corrected the ecosystemalchemyrecord to name both products and add theIndexertype.The acute instance is closed. The systemic point survives it — both fixes took humans noticing and shipping — but the residue is narrower than #18's framing implies, and sizing it against the current
coverage-watchrun is the honest first step at re-evaluation, before any adapter.If you'd rather not carry this as a file
Entirely reasonable. The alternative is that I post the body as a comment on #18 and you close the issue with whatever framing you prefer. Say the word and I'll close this PR.