Skip to content

Conversation

@kevinkim-ogp
Copy link
Contributor

@kevinkim-ogp kevinkim-ogp commented Nov 10, 2025

TL;DR

Added ai_response type to identify AI-generated variables and display appropriate warnings to users.

This change helps users identify AI-generated content and reminds them to verify the outputs before using them in critical workflows. By clearly marking AI responses and displaying appropriate warnings, we reduce the risk of users blindly trusting potentially incorrect AI-generated content.

What changed?

  • Added a new ai_response type to the TDataOutMetadatumType enum
  • Created getDataOutMetadata for the Pair action + updated AISAY's getDataOutMetadata
  • Added UI elements to warn users about AI-generated content:
    • Warning icon and message in the FlowStepTestController for AI steps
    • Special tag and tooltip for AI response variables in the VariablesList

How to test?

Create a flow with an AI step (either Pair or AISAY)

  • Run a test on the step and observe the warning message: "AI can make mistakes. Check your responses across multiple test cases."
  • Check that variables from AI steps are tagged as "AI response" with the tooltip: "This variable is AI-generated. Check for errors before using it."

Screenshots

Screenshot 2025-11-10 at 4.36.20 PM.png

Screenshot 2025-11-10 at 4.36.27 PM.png

Screenshot 2025-11-10 at 4.36.09 PM.png

Copy link
Contributor Author

kevinkim-ogp commented Nov 10, 2025

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@kevinkim-ogp kevinkim-ogp changed the title chore: frontend warnings for AI responses refactor: show warnings for AI responses Nov 10, 2025
@kevinkim-ogp kevinkim-ogp marked this pull request as ready for review November 10, 2025 08:50
@kevinkim-ogp kevinkim-ogp requested a review from a team as a code owner November 10, 2025 08:50
@kevinkim-ogp kevinkim-ogp mentioned this pull request Nov 18, 2025
4 tasks
@kevinkim-ogp kevinkim-ogp marked this pull request as draft November 27, 2025 01:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants