[OPIK-5036] [DOCS] docs: add evaluation suites documentation page#5828
[OPIK-5036] [DOCS] docs: add evaluation suites documentation page#5828alexkuzmik wants to merge 3 commits intomainfrom
Conversation
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…-evaluation-suite-docs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
🌿 Preview your docs: https://opik-preview-838c7d17-7398-4584-8f8e-5b6a561e2043.docs.buildwithfern.com/docs/opik No broken links found 📌 Results for commit 816cd5e |
|
🔄 Test environment deployment process has started Phase 1: Deploying base version You can monitor the progress here. |
| --- | ||
| headline: Evaluation Suites | Opik Documentation | ||
| og:description: Create reusable test suites with natural-language assertions to evaluate | ||
| AI agents and LLM-powered applications across iterations. | ||
| og:site_name: Opik Documentation |
There was a problem hiding this comment.
evaluation_suites.mdx uses underscores instead of the kebab-case required by apps/opik-documentation/AGENTS.md, should we rename it to evaluation-suites.mdx?
evaluation_suites.mdx => evaluation-suites.mdx
Finding type: AI Coding Guidelines | Severity: 🟢 Low
Want Baz to fix this for you? Activate Fixer
Other fix methods
Prompt for AI Agents:
Before applying, verify this suggestion against the current code. In
apps/opik-documentation/documentation/fern/docs/evaluation/evaluation_suites.mdx around
lines 1-5, the filename uses underscores instead of the repo's required kebab-case.
Rename the file to
apps/opik-documentation/documentation/fern/docs/evaluation/evaluation-suites.mdx. After
renaming, search the repository for references to evaluation_suites.mdx and update them
to evaluation-suites.mdx (links, imports, or documentation indexes) so nothing breaks.
apps/opik-documentation/documentation/fern/docs/evaluation/evaluation_suites.mdx
Show resolved
Hide resolved
|
🌙 Nightly cleanup: The test environment for this PR ( |
4 similar comments
|
🌙 Nightly cleanup: The test environment for this PR ( |
|
🌙 Nightly cleanup: The test environment for this PR ( |
|
🌙 Nightly cleanup: The test environment for this PR ( |
|
🌙 Nightly cleanup: The test environment for this PR ( |
Details
Add a new documentation page for Evaluation Suites covering the full SDK workflow: creating suites with assertions and execution policies, adding test items (batch and single with item-level assertions/policies), defining task functions, running evaluations, and comparing prompt versions. Includes a runnable quickstart script and three screenshots (terminal output, experiment items, assertion detail).
evaluation_suite_quickstart.pyexample scriptdocs.ymlnavigationChange checklist
Issues
AI-WATERMARK
AI-WATERMARK: yes
Testing
Ran Fern dev server locally and verified the page renders correctly with all code blocks, screenshots, and MDX components (Tip, Note, Frame).
Documentation
This PR is the documentation change itself.