Skip to content

RAG Client DX #9125

@nerdoza

Description

@nerdoza

While working through the RAG client I found two oddities which had be stumped for hours and required digging into the code to understand what was happening. Both can be resolved with documentation improvements, but would really be best served by changing the implementation to be more intuitive.

  1. When creating the RAG context, only stored prompts are parsed for {context} and {query} variable replacement. The usefulness of custom prompts are largely negated without the ability to have the RAG context inserted into the prompt. They also only appear after the stored prompts.

  2. Stored system and user prompts are passed through the Python formatter function which tries to replace anything in curly brackets. This is meant to support the {context} and {query} replacement, but any curly bracket in the text gets caught by this function and an error is thrown. What makes this doubly confusing for the end user is that assistant and tool prompts, as well as all custom prompts don't get parsed, so two identical prompts can work as a custom prompt, and then fail as a stored prompt.

Again, documentation of both of these oddities would be helpful, but a change towards consistency would be the best solution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions