Skip to content

Port PXI documentation eval to Python #12969

Description

@yfrigui2

Parent: #11598

Background

PXI's model-facing system prompt is now owned by the Phoenix server. The existing TypeScript documentation eval still lives in the frontend app evals and imports the frontend prompt artifact, which is being removed as part of the server-authoritative prompt work.

Current eval on origin/main, before removal: app/evals/documentation.eval.ts

Work

Port the documentation-link eval from TypeScript to Python so it evaluates the server-owned PXI prompt path instead of depending on frontend prompt state.

Acceptance Criteria

  • A Python eval replaces the current TypeScript app/evals/documentation.eval.ts behavior.
  • The eval uses the server-side PXI prompt assembly/source of truth, including user instructions as the dynamic section when needed.
  • The eval keeps checking that documentation answers use well-formed markdown links rooted at the canonical Phoenix docs domain.
  • Remove the obsolete TypeScript eval once the Python version covers the same behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    c/agentsrelated to the in-browser or terminal agent for phoenix datac/evalsc/serverServer implementationenhancementNew feature or requestpythonPull requests that update Python code

    Type

    No type
    No fields configured for issues without a type.

    Projects

    Status
    📘 Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions