Skip to content

Updated evals api docs with new endpoints #76

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
penguine-ip merged 4 commits into from
Mar 3, 2026
Merged

Updated evals api docs with new endpoints #76

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
838d204
fix evals api
A-Vamshi Feb 27, 2026
8a0654b
Merge branch 'main' of https://github.com/confident-ai/confident-docs
A-Vamshi Mar 2, 2026
c1abb77
added new endpoints
A-Vamshi Mar 2, 2026
4ad0e9a
Merge branch 'main' of https://github.com/confident-ai/confident-docs
A-Vamshi Mar 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
232 changes: 231 additions & 1 deletion fern/openapi-overrides.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,70 @@ paths:
type: string
example: "<PROJECT-API-KEY>"

/v1/test-runs:
get:
parameters:
- in: header
name: CONFIDENT_API_KEY
required: true
description: The API key of your Confident AI project.
schema:
type: string
example: "<PROJECT-API-KEY>"
- in: query
name: page
schema:
type: integer
minimum: 1
default: 1
description: Page number (must be a positive integer, default is 1).
- in: query
name: pageSize
schema:
type: integer
minimum: 1
maximum: 100
default: 25
description: Number of items per page (max 100, default is 25).
- in: query
name: status
schema:
type: string
enum: [COMPLETED, ERRORED, IN_PROGRESS, CANCELLED]
description: Filter test runs by status.
- in: query
name: multiTurn
schema:
type: string
enum: ["true", "false"]
description: Filter for conversational (multi-turn) test runs only ("true" or "false").
- in: query
name: sortBy
schema:
type: string
enum: ["createdAt", "runDuration"]
default: "createdAt"
description: Field to sort results by ("createdAt" or "runDuration").
- in: query
name: ascending
schema:
type: string
enum: ["asc", "desc"]
default: "desc"
description: Sort ascending (true) or descending (false).
- in: query
name: start
schema:
type: string
format: datetime
description: ISO 8601 start datetime filter (e.g. "2025-12-31T23:59:59Z").
- in: query
name: end
schema:
type: string
format: datetime
description: ISO 8601 end datetime filter (e.g. "2025-12-31T23:59:59Z").

/v1/test-runs/{testRunId}:
get:
parameters:
Expand Down Expand Up @@ -101,6 +165,17 @@ paths:
type: string
example: "<PROJECT-API-KEY>"

/v1/datasets:
get:
parameters:
- in: header
name: CONFIDENT_API_KEY
required: true
description: The API key of your Confident AI project.
schema:
type: string
example: "<PROJECT-API-KEY>"

/v1/datasets/{alias}:
get:
parameters:
Expand Down Expand Up @@ -234,7 +309,8 @@ paths:
name: ascending
required: false
schema:
type: boolean
type: string
enum: ["asc", "desc"]
description: This determines if the field specified in `sortBy` should be in ascending order. Defaults to `false`.

/v1/traces/{traceUuid}:
Expand Down Expand Up @@ -271,6 +347,151 @@ paths:
type: string
example: "<PROJECT-API-KEY>"

/v1/spans:
get:
parameters:
- in: header
name: CONFIDENT_API_KEY
required: true
description: The API key of your Confident AI project.
schema:
type: string
example: "<PROJECT-API-KEY>"
- in: query
name: page
required: false
schema:
type: integer
minimum: 1
description: This specifies the page number of the threads to return. Defaulted to 1.
- in: query
name: pageSize
required: false
schema:
type: integer
minimum: 1
description: This specifies the maximum number of threads per page. Defaulted to 25.
- in: query
name: type
description: Filter by the specific type of span.
schema:
type: string
enum: [SPAN, AGENT, TOOL, RETRIEVER, LLM]
- in: query
name: traceUuid
description: Filter spans that belong to a specific trace UUID.
schema:
type: string
format: uuid
- in: query
name: name
description: Filter spans by their exact name.
schema:
type: string
- in: query
name: hasError
description: Filter for spans that either failed (true) or succeeded (false).
schema:
type: boolean
- in: query
name: model
description: Filter LLM spans by the model used (e.g., 'gpt-4').
schema:
type: string
- in: query
name: promptAlias
required: false
schema:
type: string
description: This filters the spans by the prompt alias used.
- in: query
name: promptCommitHash
required: false
schema:
type: string
description: This filters the spans by the exact prompt commit hash used.
- in: query
name: promptVersion
required: false
schema:
type: string
description: This filters the spans by the prompt version used.
- in: query
name: promptLabel
required: false
schema:
type: string
description: This filters the spans by the prompt label used.
- in: query
name: embedder
description: Filter retriever spans by the embedder model used.
schema:
type: string
- in: query
name: topK
description: Filter retriever spans by the topK value.
schema:
type: integer
minimum: 0
- in: query
name: chunkSize
description: Filter retriever spans by the chunk size.
schema:
type: integer
minimum: 0
- in: query
name: environment
required: false
schema:
type: string
description: This filters the threads by the environment where the thread was created, and returns threads from all environments if not specified.
- in: query
name: start
required: false
schema:
type: string
format: date-time
description: This filters for threads created after the specified start datetime. Defaulted to 30 days ago.
- in: query
name: end
required: false
schema:
type: string
format: date-time
description: This filters for threads created before the specified end datetime. Defaulted to the current time.
- in: query
name: sortBy
required: false
schema:
type: string
enum: [createdAt, lastActivity]
description: This determines the field to sort by. Defaulted to `lastActivity`.
- in: query
name: ascending
required: false
schema:
type: string
enum: ["asc", "desc"]
description: This determines if the field specified in `sortBy` should be in ascending order. Defaults to `false`.

/v1/spans/{spanUuid}:
get:
parameters:
- in: header
name: CONFIDENT_API_KEY
required: true
description: The API key of your Confident AI project.
schema:
type: string
example: "<PROJECT-API-KEY>"
- in: path
name: spanUuid
required: true
schema:
type: string
description: This is the span UUID you wish to retrieve.
example: "SPAN-UUID"

/v1/evaluate/threads/{threadId}:
post:
parameters:
Expand Down Expand Up @@ -315,6 +536,15 @@ paths:
schema:
type: string
example: "<PROJECT-API-KEY>"
get:
parameters:
- in: header
name: CONFIDENT_API_KEY
required: true
description: The API key of your Confident AI project.
schema:
type: string
example: "<PROJECT-API-KEY>"

/v1/prompts/{alias}/labels/{label}:
get:
Expand Down
Loading
Loading