-
Notifications
You must be signed in to change notification settings - Fork 2
Issues
is:issue state:open
is:issue state:open
Issue creation is restricted in this repository
Search results
Add judge panel support: multi-judge evaluation with configurable aggregation
enhancementNew feature or requestNew feature or requesttier-1-criticalCritical priority - address in v0.2-v0.3Critical priority - address in v0.2-v0.3Status: Open.#186 In narmaku/eval-studio;feat: include rubric criteria in judge prompts for per-criterion evaluation
enhancementNew feature or requestNew feature or requestStatus: Open.#184 In narmaku/eval-studio;Add telemetry export to MLflow, OpenTelemetry, and Grafana
enhancementNew feature or requestNew feature or requesttier-2-highHigh priority - address in v0.3-v0.4High priority - address in v0.3-v0.4Status: Open.#132 In narmaku/eval-studio;Wire harness selection into evaluation flow with auto-provisioning
enhancementNew feature or requestNew feature or requesttier-1-criticalCritical priority - address in v0.2-v0.3Critical priority - address in v0.2-v0.3Status: Open.#131 In narmaku/eval-studio;Implement simulated AI-driven chat sessions (agent-as-user)
enhancementNew feature or requestNew feature or requestevaluation-modeNew evaluation mode/domainNew evaluation mode/domaintier-2-highHigh priority - address in v0.3-v0.4High priority - address in v0.3-v0.4Status: Open.#130 In narmaku/eval-studio;Add audit trail and changelog for all configuration changes
enhancementNew feature or requestNew feature or requesttier-2-highHigh priority - address in v0.3-v0.4High priority - address in v0.3-v0.4Status: Open.#129 In narmaku/eval-studio;Add project workspace with tagging and grouped configuration
enhancementNew feature or requestNew feature or requesttier-2-highHigh priority - address in v0.3-v0.4High priority - address in v0.3-v0.4uxUser experience and interfaceUser experience and interfaceStatus: Open.#128 In narmaku/eval-studio;Add CLI client and Python SDK for programmatic access
enhancementNew feature or requestNew feature or requesttier-1-criticalCritical priority - address in v0.2-v0.3Critical priority - address in v0.2-v0.3Status: Open.#127 In narmaku/eval-studio;Add headless API workflow for CI/CD pipeline integration
enhancementNew feature or requestNew feature or requesttier-1-criticalCritical priority - address in v0.2-v0.3Critical priority - address in v0.2-v0.3Status: Open.#126 In narmaku/eval-studio;Add evaluation run config fingerprint for trend tracking
enhancementNew feature or requestNew feature or requesttier-1-criticalCritical priority - address in v0.2-v0.3Critical priority - address in v0.2-v0.3Status: Open.#125 In narmaku/eval-studio;ls-eval adapter: seamlessly reuse eval-studio provider config as system.yaml
enhancementNew feature or requestNew feature or requestevaluation-modeNew evaluation mode/domainNew evaluation mode/domaintier-1-criticalCritical priority - address in v0.2-v0.3Critical priority - address in v0.2-v0.3Status: Open.#69 In narmaku/eval-studio;Add production feedback loop: traces → annotation → dataset curation
enhancementNew feature or requestNew feature or requestevaluation-modeNew evaluation mode/domainNew evaluation mode/domaintier-3-futureFuture priority - v0.4+Future priority - v0.4+Status: Open.#68 In narmaku/eval-studio;