feat: route skill feedback through Langfuse intake

[LysanderKie]

Summary

• prefer the authenticated submitFeedback tool on the Langfuse MCP server, then the generated public API through langfuse-cli
• require an exact payload preview and explicit permission; the consent step also offers a public GitHub discussion for users who want a trackable thread
• prohibit secrets, customer data, trace payloads, and unrelated context
• fall back to a prefilled GitHub discussion link (Ideas & Improvements) when intake is unavailable or unconfigured — user submits it themselves, no gh auth needed
• keep the CLI allowlist read-only: the feedback submit call intentionally goes through the normal permission prompt
• proactive trigger (offer once on wrong/outdated guidance) lives in the references-list entry and the reference frontmatter, per AGENTS.md routing rules
• bump all plugin manifests to 1.5.0 (minor: new capability per semver policy)

Dependency

Merge after langfuse/langfuse#14923 is deployed and its generated feedback operation is available.

Note: the companion stack was simplified during review (the docs-MCP relay and its token were dropped; rate limiting now uses the standard per-org RateLimitService). Neither change affects this PR: the submitFeedback MCP tool and the public API operation this skill references are unchanged, and correlation IDs are still returned.

Verification

• plugin manifests parse and versions are aligned
• git diff --check passes

Tests added: 0 — instruction-only behavior with no test harness in this repository.

[github-actions]

@claude review

[claude]

🔴 The two new allowed-tools entries — Bash(npx langfuse-cli api feedback *) and Bash(bunx langfuse-cli api feedback *) — pin the resource (feedback) but wildcard the action, so the submit/write action is auto-approved with no permission prompt, unlike every other entry which pins the action to a read-only verb. This violates AGENTS.md's explicit policy to auto-allow only 'read-only langfuse-specific commands,' and lets an agent (careless or prompt-injected) send feedback to Langfuse without the conversational approval step described in skill-feedback.md. Consider pinning the specific read-only discovery actions (e.g. __schema, --help) for feedback and leaving the submit action outside the allowlist so it still requires a permission prompt.

Extended reasoning...

The bug

Every pre-existing allowed-tools entry in skills/langfuse/SKILL.md follows the same shape: wildcard the resource, pin the action to a read-only verb — api __schema *, api * --help *, api * list *, api * get *. This is a deliberate design: the CLI models operations as <resource> <action>, and pinning the action to --help/list/get/__schema guarantees only non-mutating calls are ever auto-approved without a permission prompt.

This PR breaks that pattern. The two new entries — Bash(npx langfuse-cli api feedback *) and Bash(bunx langfuse-cli api feedback *) — invert it: they pin the resource (feedback) and wildcard the action. Since feedback is a resource whose entire purpose (per the same PRs references/skill-feedback.md, step 4: "discover the current public API operation ... then submit through the authenticated project interface") is to perform a write/submit call, wildcarding the action on this resource auto-approves that mutating, external network write with zero permission prompt.

Why nothing else prevents this

references/skill-feedback.md step 3 asks the model to "show the user every user-controlled field exactly as it will be submitted and ask for explicit permission" before submitting. That is a model-instruction safeguard, not something enforced by the tool-permission layer that allowed-tools controls. If the model skips that step — due to a bug, a compressed/summarized context that drops the instruction, or a prompt injection encountered while browsing docs or reading a trace payload — the actual Bash tool call to submit feedback will be auto-approved outright, because the CLI invocation matches the wildcarded feedback * pattern regardless of which action it is.

Repo policy this violates

AGENTS.md states explicitly: "Be cautious with allowed-tools. A tool not in the list still works — the user just grants permission the first time. Only auto-allow commands users would find a no-brainer (today: read-only langfuse-specific commands only). A risky auto-allow can make people hesitate to install the skill at all." The new entries are a direct violation of this: they auto-allow a write action, not a read-only one.

Step-by-step proof

1. User (or an agent responding to injected content while fetching docs/traces) decides "submit feedback" should happen.
2. Agent runs npx langfuse-cli api feedback submit --json {...} (the exact submit action/flags depend on the generated public API operation from feat(MCP, API): Add feedback submission via public API and MCP tool langfuse#14923, but any action name matches because the action segment is wildcarded).
3. The Bash permission layer checks allowed-tools and matches Bash(npx langfuse-cli api feedback *) — the command is auto-approved, no prompt shown to the user.
4. The feedback payload (potentially including whatever context the agent decided was relevant) is sent to the authenticated Langfuse project interface with no human in the loop, even though the reference doc assumes a permission prompt will occur first.

Suggested fix

Scope the feedback allowlist entries to only the read-only discovery actions that are genuinely no-brainers — e.g. Bash(npx langfuse-cli api feedback --help *), Bash(npx langfuse-cli api feedback __schema *) (schema/help discovery is harmless) — and leave the actual submit action out of allowed-tools entirely, so it continues to require the normal one-time permission grant, consistent with every other write path in this skill.

[claude]

🟡 All three plugin manifests (.claude-plugin, .codex-plugin, .cursor-plugin) bump 1.4.0 -> 1.4.1 (patch), but this PR replaces the entire skill-feedback mechanism (GitHub Discussions -> authenticated in-app/API intake, a new LANGFUSE_PROJECT_INTERFACE required_access token, two new allowed-tools entries, and a GitHub-issue fallback). Per AGENTS.md's semver policy this is a 'meaningful new capability'/'non-breaking behavior change', which maps to a minor bump (1.5.0), not patch.

Extended reasoning...

AGENTS.md (section Plugin Version Bumps) defines the semver policy for this repo explicitly: Patch is reserved for 'bug fixes in a skill, clarifications to skill instructions, small content corrections.' Minor is for 'adding a new skill, adding meaningful new capability to an existing skill, non-breaking behavior changes.' This is a repo-mandated rule that AGENTS.md itself lists as a PR-review checklist item ('Version bumps in lockstep... If published skill behavior changed, both manifests must be bumped'), not a subjective style preference.

This PR does far more than a clarification or content correction. It replaces the entire skill-feedback submission mechanism end-to-end: the old flow created a GitHub Discussion via gh api graphql; the new flow routes through an authenticated in-app submitFeedback MCP tool or the public Langfuse API (discovered via CLI schema), with a GitHub issue only as a fallback when authenticated intake is unavailable. Alongside that, the reference frontmatter gains a brand-new required_access token (LANGFUSE_PROJECT_INTERFACE) that did not exist on this file before, and SKILL.md gains two new allowed-tools entries (npx langfuse-cli api feedback * and the bunx equivalent) that expand the auto-permitted command surface for this skill.

None of that is a 'clarification' -- it is a wholesale, non-breaking behavior change to how a skill capability is delivered, which is precisely what the Minor bucket in AGENTS.md is written to capture. The 'err on the side of bumping patch if unsure' escape hatch (the line right after the Patch/Minor/Major definitions) does not rescue this: that clause exists for the ambiguous case of 'should I bump at all,' not for choosing between patch and minor once a bump is already happening and the change is unambiguously more than a clarification.

Step-by-step proof:

1. Diff skills/langfuse/references/skill-feedback.md: the submission backend changes from gh api graphql (GitHub Discussions) to submitFeedback MCP tool / public API via langfuse-cli, with GitHub issues demoted to fallback-only. This is an observable change in what happens when a user triggers 'submit feedback about this skill' -- a behavior change, not wording cleanup.
2. Diff skills/langfuse/SKILL.md frontmatter: two new allowed-tools entries are added (langfuse-cli api feedback * for both npx and bunx). Per AGENTS.md's own skill-authoring section, allowed-tools changes are meaningful enough to call out separately -- auto-allowed commands directly affect what runs without a permission prompt.
3. Diff skills/langfuse/references/skill-feedback.md frontmatter: required_access gains LANGFUSE_PROJECT_INTERFACE, a token that was not previously declared on this file, reflecting the new dependency on reaching the Langfuse project via CLI/API.
4. Cross-reference against AGENTS.md's Patch/Minor definitions: none of the above is a 'bug fix,' 'clarification,' or 'small content correction' (Patch); all of it is 'adding meaningful new capability to an existing skill' / 'non-breaking behavior changes' (Minor).
5. Yet all three manifests (.claude-plugin/plugin.json:4, .codex-plugin/plugin.json:3, .cursor-plugin/plugin.json:4) were bumped 1.4.0 -> 1.4.1, a patch increment, understating the scope of the change.

Fix: bump all three manifests to 1.5.0 instead of 1.4.1, keeping them in lockstep as AGENTS.md requires.

This has no functional/runtime impact -- the manifests still parse, and the lockstep requirement across all three files is satisfied (only the increment level is wrong) -- so it is a versioning-convention deviation rather than a behavioral defect.

[claude]

🟡 This PR deletes SKILL.md's '## Skill Feedback' section, which was the only instruction telling the agent when to proactively offer to submit skill feedback (wrong/outdated guidance, unexpected workflow results, missing coverage, user saying "this is wrong", etc.), and doesn't relocate those trigger criteria into references/skill-feedback.md or its frontmatter. The only remaining pointer is a passive use-case-list line with no activation guidance, so proactive offering — the feature's core purpose — will likely only happen when a user explicitly asks for it.

Extended reasoning...

Before this PR, skills/langfuse/SKILL.md had a dedicated ## Skill Feedback section that gave the agent concrete, always-loaded instructions for when to proactively offer to submit feedback about the skill: when the skill gave wrong or outdated instructions, when a workflow didn't produce the expected result, when the user wished the skill covered something it doesn't, or when the user said something like "this is wrong." It also carried an explicit exclusion (don't trigger for issues with Langfuse the product, only for issues with the skill itself), which prevented false positives on unrelated product complaints.

This PR deletes that entire section from SKILL.md and does not relocate the trigger criteria anywhere else. I checked references/skill-feedback.md in the diff and confirmed it now contains only submission mechanics (draft, sanitize, get explicit permission, submit via the authenticated MCP tool or CLI-discovered API operation, GitHub-issue fallback, report the receipt ID) — no guidance on when to proactively initiate the workflow. The frontmatter description for that file also lost its old activation clause ("Use when the user indicates the skill gave incorrect guidance, is missing information, or could be improved"), replaced with a description that only describes the submission mechanism.

The only surviving pointer is the line in SKILL.md's "Use case specific references" list: - submitting feedback about this skill: references/skill-feedback.md. That list is topical/passive — every other entry in it (e.g. "instrumenting an existing function/application: references/instrumentation.md") is written for a scenario where the agent already knows what it needs to do and is just looking up the how-to reference. It carries no proactive trigger heuristic, so it will only be consulted once the agent (or user) has already decided that feedback is warranted.

Concrete walk-through of the regression: Suppose a user runs through a Langfuse skill workflow, the CLI reference gives outdated flag names, and the workflow fails. Under the old SKILL.md, the agent had a standing instruction ("A workflow didn't produce the expected result" → offer to submit skill feedback) that would fire in this exact scenario without the user needing to ask. Post-PR, no instruction anywhere tells the agent this scenario warrants offering feedback — the agent would need the user to say "please file feedback about this skill" before the (now purely mechanical) skill-feedback.md reference would even be consulted. The feature still functions correctly once invoked, but the trigger that made it "proactive" is gone.

This looks like an unintentional side effect of the routing rewrite (the PR's actual intent, per its description, is changing the submission mechanism to authenticated Langfuse intake, not changing when to offer feedback) rather than a deliberate product decision. Since this is instruction-only content — nothing crashes, no incorrect output is produced, and the workflow still works correctly when a user explicitly requests it — this doesn't block merge, but it's worth restoring: either re-add a short trigger-conditions blurb to SKILL.md, or move the "when to trigger" criteria into skill-feedback.md's frontmatter description/body so the proactive-offer behavior isn't silently lost.

Suggested fix: Add back a condensed version of the deleted trigger list (or move it into the description: frontmatter of references/skill-feedback.md, similar to how it previously read "Use when the user indicates the skill gave incorrect guidance, is missing information, or could be improved") so the agent retains a standing instruction for proactively offering feedback, including the product-vs-skill exclusion.