Skip to content
This repository was archived by the owner on Jul 31, 2026. It is now read-only.

ideas: record the partner-doc source lane as held (proposed, droppable) - #58

Merged
kalepail merged 1 commit into
kalepail:mainfrom
Nearx-Labs:ideas/2026-07-31-partner-doc-source-lane
Jul 31, 2026
Merged

kalepail merged 1 commit into
kalepail:mainfrom
Nearx-Labs:ideas/2026-07-31-partner-doc-source-lane

Conversation

@pedro-pelicioni

Copy link
Copy Markdown
Contributor

Refs #18. Proposed, not asserted — and explicitly droppable.

This is the half of my earlier #56 that I should not have fused with the evidence. Writing a decision record with a review date and reopen triggers is a maintainer act, not a contributor one, and coupling it to the eval work forced you to adjudicate the product question in order to accept a test corpus. The evidence now lives in its own PR, sharing no files with this one; either can merge without the other.

So: take this, amend it, ask me to rewrite it in a different voice, or close it and write your own. The analysis is offered; the framing is yours.

What it records

ideas/partner-doc-live-sources.md, in the shape ideas/stellar-org-source-lane.md established.

No partner-MCP federation. This declines the mechanism asked for in the issue thread itself — @oceans404's "ideally we'd tap into their docs mcps if they have them" and @mmazco's "most should have by now" — so it owes them a reason rather than a verdict. The reason is what those servers currently are: Alchemy's hosted MCP is 168 tools behind OAuth 2.1 including create_app / update_allowlist; OpenZeppelin's advertises contract generation, not documentation retrieval. Federating either would let the partner decide what is callable, inverting ADR-0003 without a repo change.

Allowlisted first-party partner Markdown is credible, and held behind the four-phase gate.

A dated review (2026-10-31) and two separate reopen triggers: one for the Markdown lane, and a narrower one for the MCP question that fires only on a read-only, unauthenticated, documentation-scoped partner MCP with a stable tool contract. Neither partner ships that today, and a partner adding more tools is evidence against, not for.

Three things it is careful not to overclaim

The easy version of this argument is wrong in ways an adversarial pass caught:

  • The hold is corroborated, not novel. stellarlight#448 reached the same "no" on the same llms.txt grounds three weeks earlier, and chose "the structured record plus a first-class pointer to the living source" over ingestion.
  • Ingestion is not retrieval. My first draft said partner movement reaching official docs "is already caught by the improvements pipeline". stellarlight#565 shows that is too glib: after stellar-docs#2573 was ingested, a bare q=Alchemy still returned zero results, and the Indexers chunk documenting the Data API ranked below top-15 for its own brand query. That is a real, dated live failure of the kind this lane's own reopen trigger asks for — and it argues for fixing retrieval upstream before adding a source here.
  • The residue is already measured upstream, by stellarlight#561's monthly coverage-watch — whose verdict semantics were then corrected by #564, so its first run's "0 of 4 partners" should not be quoted as standing fact. Re-evaluation should read the current run, not re-derive the number.

On #18's concrete example

It healed from two directions at once: stellar/stellar-docs#2573 upstream (merged, live-verified 2026-07-15, tracked as sd-010), and stellarlight#446 downstream four days earlier, which corrected the ecosystem alchemy record to name both products and add the Indexer type.

The acute instance is closed. The systemic point survives it — both fixes took humans noticing and shipping — but the residue is narrower than #18's framing implies, and sizing it against the current coverage-watch run is the honest first step at re-evaluation, before any adapter.

If you'd rather not carry this as a file

Entirely reasonable. The alternative is that I post the body as a comment on #18 and you close the issue with whatever framing you prefer. Say the word and I'll close this PR.

…w and reopen triggers

Proposed, not asserted — see the PR description. This is the maintainer's call to make;
the file is written in the shape ideas/stellar-org-source-lane.md established so it is easy
to accept, amend, or drop without touching the eval evidence in the preceding commit.

What it records about issue kalepail#18:

- No partner-MCP federation. This declines the mechanism asked for in the issue thread, so
  it owes those commenters a reason: Alchemy's hosted MCP is 168 tools behind OAuth 2.1
  including create_app/update_allowlist, and OpenZeppelin's advertises contract generation,
  not documentation retrieval. Federating either would let the partner decide what is
  callable, inverting ADR-0003.
- Allowlisted first-party partner Markdown is credible, and held behind the four-phase gate.
- A dated review (2026-10-31) and two separate reopen triggers — one for the Markdown lane,
  a narrower one for the MCP question, which reopens only on a read-only, unauthenticated,
  documentation-scoped partner MCP with a stable tool contract.

Three things it is careful to say, because the easy version of this argument is wrong:

- The hold is corroborated, not novel. stellarlight#448 reached the same "no" on the same
  llms.txt grounds three weeks earlier.
- Ingestion is not retrieval. stellarlight#565 found that after stellar-docs#2573 was
  ingested, a bare q=Alchemy still returned zero results and the Indexers chunk ranked below
  top-15 for its own brand query. "The improvements pipeline already catches it" is too
  glib, and that live failure argues for fixing retrieval upstream before adding a source.
- The residue is already measured upstream by stellarlight#561's monthly coverage-watch —
  whose verdict semantics were then corrected by #564, so its first run's 0/4 should not be
  quoted as standing fact.

The issue's concrete Alchemy example healed from two directions at once: stellar-docs#2573
upstream, and stellarlight#446 downstream four days earlier.

@kalepail kalepail left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging as written. You were right that writing a decision record is a maintainer act, and right to unfuse it from the evidence — but the framing here is one I'd have written myself, so there's nothing to rewrite.

I checked every upstream citation rather than taking them on trust, because a decision record's whole value is that its references hold. All seven resolve, are merged, and say what you say they say: stellarlight #446, #447, #448, #561, #564, #565, and #657. That is not a given for a document with this citation density, and it's why I'm taking the analysis as-is.

The three things you were careful not to overclaim are the three that make this worth keeping as a file:

  • "The hold is corroborated, not novel." Leading with the fact that #448 reached the same conclusion three weeks earlier, on the same llms.txt grounds, is the opposite of how a contributor usually pitches their own analysis.
  • "Ingestion is not retrieval." This is the load-bearing correction. stellar-docs#2573 merging and going live is exactly the kind of upstream fix our improvements pipeline exists to produce, and #565 showing that a bare q=Alchemy still returned zero results after ingestion is the finding that content reaching official docs is necessary and not sufficient. My own pipeline claim would have been too glib in the same way.
  • Not quoting the coverage-watch 0/4 as standing fact once #564 changed the verdict semantics underneath it.

The declines are correct on the merits. A 168-tool OAuth surface including create_app and update_allowlist is not a documentation source, and mounting it would invert ADR-0003 by letting the partner decide what is callable without a repo change — the reopen trigger you wrote for the MCP question (read-only, unauthenticated, documentation-scoped, stable tool contract) is the right shape, and "a partner adding more tools is evidence against, not for" is the sentence that keeps it honest.

One note for the record, since #18 is still the parent: the acute Alchemy instance closing does not close the issue's systemic point, and this file says so. That's the right disposition — held, dated, with triggers — rather than resolved.

Thanks for splitting these, and for the retraction discipline in #447 -> #448 that shows up as the negative-grep lesson in the provenance rules.

@kalepail
kalepail merged commit 2a0bdbd into kalepail:main Jul 31, 2026
2 checks passed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants