Errors Cost Tokens

[JamesonRGrieve]

Description

When an unable to process request comes back, that gets tokenized and counted against token usage.

    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "Unable to process request."
            },
            "finish_reason": "stop",
            "logprobs": null
        }
    ],
    "usage": {
        "prompt_tokens": 262,
        "completion_tokens": 5,
        "total_tokens": 267
    }

Operating System

• Linux
• Windows
• MacOS

Acknowledgements

• I am NOT trying to use localhost for providers running outside of the docker container.
• I am NOT trying to run AGiXT outside of docker, the only supported method to run it.
• Python 3.10.X is installed and the version in use on the host machine.
• The latest version of Docker is installed and running on the host machine.
• I am using the latest stable version of AGiXT.
• I have provided enough information for the maintainers to reproduce and diagnose the issue.
• I have searched the existing issues to make sure this bug has not been reported yet.