Skip to content

refactor validator to wrap Codex API #81

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

kelsey-wong
Copy link
Contributor

@kelsey-wong kelsey-wong commented May 14, 2025

Key Info

What changed?

Remove redundant validator logic from client library and instead wrap the Codex validator API. The backend now handles the following logic in the projects.validate endpoint:

  • Run all evals against TrustworthyRAG
  • Apply eval thresholds, if provided, to determine if the RAG app's generated response is good or bad
    • If the response is determined to be bad, then query Codex project for an expert answer that matches the query

What do you want the reviewer(s) to focus on?

Elias: Please confirm if the ML team needs/wants any changes to the client library functionality

Angela: General code review (test coverage, naming, format, etc.)


Checklist

  • Did you link the GitHub issue?
  • Did you follow deployment steps or bump the version if needed?
  • Did you add/update tests?
  • What QA did you do?
    • Tested...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant