ChatBedrockConverse ValidationException on Non-Empty AI Message

[shahrukh-shaik]

Sample Code:

from langchain_aws import ChatBedrockConverse
model_id="us.anthropic.claude-3-5-haiku-20241022-v1:0"
llm1 = ChatBedrockConverse(model_id=model_id,disable_streaming=True)

llm1.invoke([HumanMessage(content=[{'type': 'text', 'text': 'For the earlier part, can you see if there are any updates'}], additional_kwargs={}, response_metadata={}, id='dbda63b2-4ed8-4d7c-ab51-219f82353cb6'),
 AIMessage(content=[{'type': 'text', 'text': '\n\n', 'index': 0}], additional_kwargs={}),
 HumanMessage(content="try again")])

Original Conversation

[HumanMessage(content=[{'type': 'text', 'text': 'For the earlier part, can you see if there are any updates'}], additional_kwargs={}, response_metadata={}, id='dbda63b2-4ed8-4d7c-ab51-219f82353cb6'),
 AIMessage(content=[{'type': 'text', 'text': "I'd be happy to check for any updates regarding your earlier query. Let me search for the most current information.", 'index': 0}], additional_kwargs={}, response_metadata={'stopReason': 'tool_use', 'metrics': {'latencyMs': 4917}, 'llm': 'us.anthropic.claude-3-7-sonnet-20250219-v1:0'}, name='general', id='run-12a3ef27-eaa6-4ea8-857e-1031b066560c', tool_calls=[{'name': 'get_current_date', 'args': {}, 'id': 'tooluse_8Ke4pZdcS2C9DklgqMNFvw', 'type': 'tool_call'}], usage_metadata={'input_tokens': 39897, 'output_tokens': 77, 'total_tokens': 39974}),
 ToolMessage(content='2025-03-13', name='get_current_date', id='006b1fc4-0162-42ae-a778-1e2eb9767e2b', tool_call_id='tooluse_8Ke4pZdcS2C9DklgqMNFvw'),
 AIMessage(content=[{'type': 'text', 'text': '\n\n', 'index': 0}], additional_kwargs={}, response_metadata={'stopReason': 'tool_use', 'metrics': {'latencyMs': 6728}, 'llm': 'us.anthropic.claude-3-7-sonnet-20250219-v1:0'}, name='general', id='run-ba56d3ef-7adc-404a-8029-f1e38a2802bc', tool_calls=[{'name': 'execute_redshift_sql_query', 'args': {'query': "SELECT * FROM mytable"}, 'id': 'tooluse_v7fPQJREQSGs_CW-TmYlgg', 'type': 'tool_call'}], usage_metadata={'input_tokens': 39991, 'output_tokens': 48, 'total_tokens': 40039})]

Error:

---------------------------------------------------------------------------
ValidationException                       Traceback (most recent call last)
Cell In[50], line 4
      1 from tools import AgentTools
      2 tool=AgentTools({})
----> 4 llm1.invoke([HumanMessage(content=[{'type': 'text', 'text': 'For the earlier part, can you see if there are any updates'}], additional_kwargs={}, response_metadata={}, id='dbda63b2-4ed8-4d7c-ab51-219f82353cb6'),
      5  AIMessage(content=[{'type': 'text', 'text': '\n\n', 'index': 0}], additional_kwargs={}),
      6  HumanMessage(content="try again")])

File ~/lang-chain/.env/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py:285, in BaseChatModel.invoke(self, input, config, stop, **kwargs)
    274 def invoke(
    275     self,
    276     input: LanguageModelInput,
   (...)
    280     **kwargs: Any,
    281 ) -> BaseMessage:
    282     config = ensure_config(config)
    283     return cast(
    284         ChatGeneration,
--> 285         self.generate_prompt(
    286             [self._convert_input(input)],
    287             stop=stop,
    288             callbacks=config.get("callbacks"),
    289             tags=config.get("tags"),
    290             metadata=config.get("metadata"),
    291             run_name=config.get("run_name"),
    292             run_id=config.pop("run_id", None),
    293             **kwargs,
    294         ).generations[0][0],
    295     ).message
...
   1030     error_class = self.exceptions.from_code(error_code)
-> 1031     raise error_class(parsed_response, operation_name)
   1032 else:
   1033     return parsed_response

ValidationException: An error occurred (ValidationException) when calling the Converse operation: The text field in the ContentBlock object at messages.1.content.0 is blank. Add text to the text field, and try again.

I see this is a valid scenario where AIMessage is clearly not empty, instead it is \n\n and with/without a tool call. In Either case this shall work as expected.

It even fails when the text is \n\t or \t or any I guess which is escapable.

[shahrukh-shaik]

Note: This AI message is not a fabricated one and in actual it gave that message as \n\n.

[blanketspy99]

Probably has to relook at the initial thread at #219.

@3coins

[3coins]

@blanketspy99
Can you submit full sample code here so we can reproduce. The original conversation you included here, seems to be using a tool call, but the code sample is not binding any tool, are you running the two conversations with two separate chat model instances?

[blanketspy99]

Sure, will share.

But I believe sample code should work too without any tool bind when you force ask the model to give an empty response or new line response. In my case it happened during the tool call invocation.

I don't see this issue is from langchain ChatBedrock or ChatBedrockConverse but rather from bedrock-runtime directly.

[shahrukh-shaik]

from langchain_aws import ChatBedrockConverse
from langchain_core.tools import tool
import pandas as pd

@tool
def execute_redshift_sql_query(query:str)-> dict:
    """
    Execute a SQL query on a Redshift database using the provided query.

    Args:
        query (str): The SQL query to be executed. Ensure that any table names
                    in the query are prefixed with their respective schema names
                    (e.g., 'schema_name.table_name') to avoid ambiguity and ensure
                    proper execution within the intended schema.

    Returns:
        dict: JSON formatted result of the query execution.
    """
    try:
        results=execute_redshift_query(query)
        df = pd.DataFrame(results["result"], columns=results["columns"])
        result = df.to_json(orient="records")
        return result
    except Exception as e:
        # print(e)
        raise e

model_id="us.anthropic.claude-3-5-haiku-20241022-v1:0"
llm = ChatBedrockConverse(model_id=model_id,disable_streaming=True).bind_tools([execute_redshift_sql_query])

llm.invoke([[HumanMessage(content=[{'type': 'text', 'text': 'For the earlier part, can you see if there are any updates'}], additional_kwargs={}, response_metadata={}, id='dbda63b2-4ed8-4d7c-ab51-219f82353cb6'),
 AIMessage(content=[{'type': 'text', 'text': "I'd be happy to check for any updates regarding your earlier query. Let me search for the most current information.", 'index': 0}], additional_kwargs={}, response_metadata={'stopReason': 'tool_use', 'metrics': {'latencyMs': 4917}, 'llm': 'us.anthropic.claude-3-7-sonnet-20250219-v1:0'}, name='general', id='run-12a3ef27-eaa6-4ea8-857e-1031b066560c', tool_calls=[{'name': 'get_current_date', 'args': {}, 'id': 'tooluse_8Ke4pZdcS2C9DklgqMNFvw', 'type': 'tool_call'}], usage_metadata={'input_tokens': 39897, 'output_tokens': 77, 'total_tokens': 39974}),
 ToolMessage(content='2025-03-13', name='get_current_date', id='006b1fc4-0162-42ae-a778-1e2eb9767e2b', tool_call_id='tooluse_8Ke4pZdcS2C9DklgqMNFvw'),
 AIMessage(content=[{'type': 'text', 'text': '\n\n', 'index': 0}], additional_kwargs={}, response_metadata={'stopReason': 'tool_use', 'metrics': {'latencyMs': 6728}, 'llm': 'us.anthropic.claude-3-7-sonnet-20250219-v1:0'}, name='general', id='run-ba56d3ef-7adc-404a-8029-f1e38a2802bc', tool_calls=[{'name': 'execute_redshift_sql_query', 'args': {'query': "SELECT * FROM mytable"}, 'id': 'tooluse_v7fPQJREQSGs_CW-TmYlgg', 'type': 'tool_call'}], usage_metadata={'input_tokens': 39991, 'output_tokens': 48, 'total_tokens': 40039})],
 HumanMessage(content="try again")])

@3coins , here is the code with bind_tools. Its more or less the same with/without tool call, if the AI responds with empty or \n or \t or \n\t or may be some other pattern. This can also happen when you ask the LLM forcefully to give empty response.

I still believe its not an issue with ChatBedrockConverse rather its with the bedrock-runtime.

Let me know your thoughts on this.

[3coins]

@shahrukh-shaik
I am still not clear how to reproduce your issue with the code provided here. There are many missing elements, for example, it seems like you have 2 tools being used here 1) get_current_date that returns the current date (this is not provided nor is it added to tools in bind_tools) and 2) execute_redshift_sql_query that executes a sql query. Another piece of missing information is how you are passing the schema for the database to the LLM, so it can formulate the user queries into an SQL query.

It seems like the problem lies between this request/response pair (I have formatted so it's clearly readable) after the current_date is passed to the LLM, and it moves on to return the relevant SQL query.

ToolMessage(
  content='2025-03-13', 
  name='get_current_date', 
  id='006b1fc4-0162-42ae-a778-1e2eb9767e2b', 
  tool_call_id='tooluse_8Ke4pZdcS2C9DklgqMNFvw'
)

AIMessage(
    content=[
        {
            'type': 'text',
            'text': '\n\n',
            'index': 0
        }
    ],
    additional_kwargs={},
    response_metadata={
        'stopReason': 'tool_use',
        'metrics': {
            'latencyMs': 6728
        },
        'llm': 'us.anthropic.claude-3-7-sonnet-20250219-v1:0'
    },
    name='general',
    id='run-ba56d3ef-7adc-404a-8029-f1e38a2802bc',
    tool_calls=[
        {
            'name': 'execute_redshift_sql_query',
            'args': {
                'query': "SELECT * FROM mytable"
            },
            'id': 'tooluse_v7fPQJREQSGs_CW-TmYlgg',
            'type': 'tool_call'
        }
    ],
    usage_metadata={
        'input_tokens': 39991,
        'output_tokens': 48,
        'total_tokens': 40039
    }
)

It's not clear whether there is a problem within ChatBedrockConverse where it ends up truncating the text content, or the converse API responds with the empty text (with "\n\n"), but in order for us to do anything here, we need a complete code sample that can reproduce the issue.

[shahrukh-shaik]

@3coins
I have reach out to AWS Support team and this seems to be issue with Bedrock converse/invoke api that it returns ValidationException when text block only contains whitespace (newline, space, tab, etc.).

They were further checking internally and has to confirm on the same as it is a genuine issue that can happen even though it can be a rare case.

Using AWS CLI

# Example 1
> aws bedrock-runtime converse --model-id amazon.titan-text-express-v1 --messages '[{"role": "user", "content": [{"text": "Describe the purpose of a \"hello world\" program in one line."}]}, {"role": "assistant","content": [{"text": "\n\n"}]},{"role": "user", "content": [{"text": "what?"}]}]'

An error occurred (ValidationException) when calling the Converse operation: The text field in the ContentBlock object at messages.1.content.0 is blank. Add text to the text field, and try again.

# Example 2: with AI response sample
> aws bedrock-runtime converse --model-id us.anthropic.claude-3-7-sonnet-20250219-v1:0 --messages '[{"role": "user", "content": [{"text": "without saying anything just give me 2 empty lines. You will be penalised if you say anything more. Answer should be only the literal plain value as '\\n\\n'. No comments or notes too"}]}]'
{
"output": {
"message": {
"role": "assistant",
"content": [
{
"text": "\n"
}
]
}
},
"stopReason": "end_turn",
"usage": {
"inputTokens": 48,
"outputTokens": 4,
"totalTokens": 52
},
"metrics": {
"latencyMs": 635
}
}

[shahrukh-shaik]

Here is the response from AWS Support

The engineer mentioned that a similar behavior can also be seen from Anthropic's 1P API {'type': 'invalid_request_error', 'message': 'messages: text content blocks must be non-empty'} suggesting that the error is from the LLMs and not Bedrock layer. They have also mentioned that It's difficult to provide generic workarounds, since it can impact the LLM generation. One potential way to avoid empty responses with tools is to add at the end of the tool description, something like 'When the tool is being used , use 'a tool' in returned text/let a user know tool is being used by saying 'a tool' in your response, instead of responding with an empty text".