Skip to content

[Feature]: Document how to author a skill, and complete the operator-facing skills docs #9434

Description

@SoumyaRaikwar

Requirement

As someone adding domain-specific trace-analysis knowledge to a Jaeger installation,
I want documentation on how to write a skill and how to serve one,
so that doing it well does not require reading mcptools source or the PR history.

Problem

The skills framework shipped with operator notes in
mcptools/README.md.
They are enough to mount a directory. They are not enough to write a skill worth mounting,
and they leave gaps on the operator side too.

Nothing tells an author:

  • What actually decides whether an agent opens a skill. The README says the line beside a
    link in the index is what matters, which is correct and easy to miss — and [Feature]: Agents skip skills because the telemetry tools appear sufficient #9336 records
    that even a good line is often not enough, because agents skip skills when a telemetry
    tool appears to have answered already. Someone writing their first skill needs that up
    front, not after their skill goes unread.
  • That Jaeger parses no frontmatter, so allowed-tools is documentation rather than a
    sandbox.
  • How large a skill should be, or when to split one.
  • How to tell whether a procedure actually decides anything. This is not hypothetical:
    detect-n-plus-one shipped with a criterion ("similar durations, within 2x of the
    median") that, measured in test(jaegerquery): Add labeled trace fixtures and E2E coverage for the built-in skills #9263, scored a parallel fan-out higher than a genuine N+1.
    The rule read like a criterion and discriminated nothing. Authors will write the same
    kind of rule unless something warns them.

And the operator half does not cover:

  • How to enable the MCP endpoint at all — the README starts from skills_dir, assuming
    ai.mcp is already configured.
  • Whether editing a skill needs a restart (it does not; skills_dir is read live).
  • What the startup and read-time failure messages mean.
  • The limits: os.OpenRoot containment, and the 512 KiB per-file cap with its truncation
    marker.
  • That write access to skills_dir is write access to the agent's instructions, which
    makes the directory part of the trusted configuration surface.

#8440 lists "a complete guide for users on How to Author Custom AI Skills for Jaeger" as
an expected outcome of the skills framework work, but has no issue tracking the
documentation itself. This is that issue.

Proposal

Two guides under mcptools/, where the existing operator notes and the pointers from
config.yaml and flags.go already live:

  • AUTHORING.md (new) — when a skill is the right answer at all; writing the index
    line as a trigger; what is realistically known about whether agents read skills, citing
    [Feature]: Agents skip skills because the telemetry tools appear sufficient #9336 rather than overstating it; frontmatter and what Jaeger does not do with it; a size
    target; writing procedures that decide rather than describe; and testing a skill against
    a trace where the naive answer is wrong.
  • README.md (extended) — keeps its current content and gains the operator gaps listed
    above.

Two things the guides should deliberately not claim: the ~500-line / ~5000-token budget is
a design target and should be labelled as one, since nothing has measured it; and the index
should be described as routing humans reliably and agents only sometimes, which is what
#9336 found.

Open questions

  1. Placement. In-repo beside the code, as proposed, or on the website
    (jaegertracing/documentation)? In-repo keeps the guides next to the code that has to
    stay true to them, and is where the existing pointers lead. A user-facing guide arguably
    belongs on the website, and the two are not exclusive.
  2. Framing. "The index line is the trigger" is accurate for main today, since no
    frontmatter is parsed. If the front-matter-derived catalog discussed on fix(mcp): Stop read_skill serving YAML front matter to the agent #9318 and [Feature]: Agents skip skills because the telemetry tools appear sufficient #9336
    lands, the emphasis shifts. Worth settling before the guide teaches one over the other.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions