Google VertexAI `InvalidArgument: 400 Request contains an invalid argument`

[khteh]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

    async for event in agent.astream(
        #{"messages": [{"role": "user", "content": messages}]},
        {"messages": [("user", messages)]},
        stream_mode="values",
        config=config, # This is needed by Checkpointer
    ):
        event["messages"][-1].pretty_print()

await email_parser_chain.ainvoke({"message": email})
await email_parser_chain.ainvoke({"message": [{"role": "system", "content": email}]}

Error Message and Stack Trace (if applicable)

2025-03-21 16:12:03 WARNING  Retrying langchain_google_vertexai.chat_models._acompletion_with_retry.<locals>._completion_with_retry_inner in 4.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..
2025-03-21 16:12:07 WARNING  Retrying langchain_google_vertexai.chat_models._acompletion_with_retry.<locals>._completion_with_retry_inner in 4.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..
2025-03-21 16:12:11 WARNING  Retrying langchain_google_vertexai.chat_models._acompletion_with_retry.<locals>._completion_with_retry_inner in 4.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..
2025-03-21 16:12:16 WARNING  Retrying langchain_google_vertexai.chat_models._acompletion_with_retry.<locals>._completion_with_retry_inner in 8.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..
2025-03-21 16:12:24 WARNING  Retrying langchain_google_vertexai.chat_models._acompletion_with_retry.<locals>._completion_with_retry_inner in 10.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..
Traceback (most recent call last):
  File "/home/khteh/.local/lib/python3.12/site-packages/google/api_core/grpc_helpers_async.py", line 77, in wait_for_connection
    await self._call.wait_for_connection()
  File "/home/khteh/.local/lib/python3.12/site-packages/grpc/aio/_call.py", line 659, in wait_for_connection
    await self._raise_for_status()
  File "/home/khteh/.local/lib/python3.12/site-packages/grpc/aio/_call.py", line 272, in _raise_for_status
    raise _create_rpc_error(
grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
	status = StatusCode.INVALID_ARGUMENT
	details = "Request contains an invalid argument."
	debug_error_string = "UNKNOWN:Error received from peer ipv4:74.125.68.95:443 {grpc_message:"Request contains an invalid argument.", grpc_status:3, created_time:"2025-03-21T16:12:34.945102813+08:00"}"
>

Description

Any call to Google VertexAI with ainvoke, astream, with_structured_output hits that error.

System Info

System Information
------------------
> OS:  Linux
> OS Version:  #19-Ubuntu SMP PREEMPT_DYNAMIC Wed Feb 12 21:43:43 UTC 2025
> Python Version:  3.12.7 (main, Feb  4 2025, 14:46:03) [GCC 14.2.0]

Package Information
-------------------
> langchain_core: 0.3.45
> langchain: 0.3.20
> langchain_community: 0.3.19
> langsmith: 0.3.15
> langchain_google_genai: 2.1.0
> langchain_google_vertexai: 2.0.9
> langchain_openai: 0.3.8
> langchain_text_splitters: 0.3.6
> langgraph_api: 0.0.28
> langgraph_cli: 0.1.75
> langgraph_license: Installed. No version info available.
> langgraph_sdk: 0.1.57
> langgraph_storage: Installed. No version info available.

Optional packages not installed
-------------------------------
> langserve

Other Dependencies
------------------
> aiohttp<4.0.0,>=3.8.3: Installed. No version info available.
> anthropic[vertexai]: Installed. No version info available.
> async-timeout<5.0.0,>=4.0.0;: Installed. No version info available.
> click: 8.1.8
> cryptography: 43.0.3
> dataclasses-json<0.7,>=0.5.7: Installed. No version info available.
> filetype: 1.2.0
> google-ai-generativelanguage: 0.6.16
> google-cloud-aiplatform: 1.84.0
> google-cloud-storage: 2.19.0
> httpx: 0.27.2
> httpx-sse: 0.4.0
> httpx-sse<1.0.0,>=0.4.0: Installed. No version info available.
> jsonpatch<2.0,>=1.33: Installed. No version info available.
> jsonschema-rs: 0.20.0
> langchain-anthropic;: Installed. No version info available.
> langchain-aws;: Installed. No version info available.
> langchain-cohere;: Installed. No version info available.
> langchain-community;: Installed. No version info available.
> langchain-core<1.0.0,>=0.3.34: Installed. No version info available.
> langchain-core<1.0.0,>=0.3.41: Installed. No version info available.
> langchain-core<1.0.0,>=0.3.42: Installed. No version info available.
> langchain-deepseek;: Installed. No version info available.
> langchain-fireworks;: Installed. No version info available.
> langchain-google-genai;: Installed. No version info available.
> langchain-google-vertexai;: Installed. No version info available.
> langchain-groq;: Installed. No version info available.
> langchain-huggingface;: Installed. No version info available.
> langchain-mistralai: Installed. No version info available.
> langchain-mistralai;: Installed. No version info available.
> langchain-ollama;: Installed. No version info available.
> langchain-openai;: Installed. No version info available.
> langchain-text-splitters<1.0.0,>=0.3.6: Installed. No version info available.
> langchain-together;: Installed. No version info available.
> langchain-xai;: Installed. No version info available.
> langchain<1.0.0,>=0.3.20: Installed. No version info available.
> langgraph: 0.3.11
> langgraph-checkpoint: 2.0.20
> langsmith-pyo3: Installed. No version info available.
> langsmith<0.4,>=0.1.125: Installed. No version info available.
> langsmith<0.4,>=0.1.17: Installed. No version info available.
> numpy<3,>=1.26.2: Installed. No version info available.
> openai-agents: Installed. No version info available.
> openai<2.0.0,>=1.58.1: Installed. No version info available.
> orjson: 3.10.15
> packaging: 24.2
> packaging<25,>=23.2: Installed. No version info available.
> pydantic: 2.9.2
> pydantic-settings<3.0.0,>=2.4.0: Installed. No version info available.
> pydantic<3.0.0,>=2.5.2;: Installed. No version info available.
> pydantic<3.0.0,>=2.7.4: Installed. No version info available.
> pydantic<3.0.0,>=2.7.4;: Installed. No version info available.
> pyjwt: 2.10.1
> pytest: 8.3.2
> python-dotenv: 1.0.1
> PyYAML>=5.3: Installed. No version info available.
> requests: 2.32.3
> requests-toolbelt: 1.0.0
> requests<3,>=2: Installed. No version info available.
> rich: 13.9.4
> SQLAlchemy<3,>=1.4: Installed. No version info available.
> sse-starlette: 2.1.3
> starlette: 0.46.1
> structlog: 25.1.0
> tenacity: 9.0.0
> tenacity!=8.4.0,<10,>=8.1.0: Installed. No version info available.
> tenacity!=8.4.0,<10.0.0,>=8.1.0: Installed. No version info available.
> tiktoken<1,>=0.7: Installed. No version info available.
> typing-extensions>=4.7: Installed. No version info available.
> uvicorn: 0.34.0
> watchfiles: 1.0.4
> zstandard: 0.23.0

[ccurme]

Can you provide a minimal reproducible example? ainvoke seems to work fine:

from langchain_google_vertexai import ChatVertexAI

llm = ChatVertexAI(model="gemini-1.5-flash-001")
await llm.ainvoke("hi")

[khteh]

I think you have to use / go through CompiledGraph to reproduce the error.

[khteh]

This is the minimum I can put together:

import os, vertexai, asyncio, logging
from dotenv import load_dotenv
from datetime import datetime
from langchain.chat_models import init_chat_model
from langgraph.prebuilt import ToolNode, tools_condition, create_react_agent, InjectedStore
from langgraph.checkpoint.memory import MemorySaver
from langgraph.store.memory import InMemoryStore
from langchain_core.runnables import RunnableConfig
load_dotenv()
from src.rag_agent.Tools import TOOLS
async def main():
    llm = init_chat_model("gemini-2.0-flash", model_provider="google_vertexai", streaming=True).bind_tools(TOOLS)
    agent = create_react_agent(llm, store=InMemoryStore(), tools=TOOLS, checkpointer=MemorySaver(), name="RAG ReAct Agent")
    config = RunnableConfig(run_name="RAG ReAct Agent", thread_id=datetime.now())
    messages = ["What is the standard method for Task Decomposition?", "Once you get the answer, look up common extensions of that method."]
    async for event in agent.astream(
        {"messages": [{"role": "user", "content": messages}]},
        stream_mode="values", # Use this to stream all values in the state after each step.
        config=config, # This is needed by Checkpointer
    ):
        event["messages"][-1].pretty_print()

if __name__ == "__main__":
    vertexai.init(project=os.environ.get("GOOGLE_CLOUD_PROJECT"), location=os.environ.get("GOOGLE_CLOUD_LOCATION"))
    asyncio.run(main())

Running it:

$ python whatever.py 
================================ Human Message =================================

['What is the standard method for Task Decomposition?', 'Once you get the answer, look up common extensions of that method.']
Retrying langchain_google_vertexai.chat_models._acompletion_with_retry.<locals>._completion_with_retry_inner in 4.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..
Retrying langchain_google_vertexai.chat_models._acompletion_with_retry.<locals>._completion_with_retry_inner in 4.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..
Retrying langchain_google_vertexai.chat_models._acompletion_with_retry.<locals>._completion_with_retry_inner in 4.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..
Retrying langchain_google_vertexai.chat_models._acompletion_with_retry.<locals>._completion_with_retry_inner in 8.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..
Retrying langchain_google_vertexai.chat_models._acompletion_with_retry.<locals>._completion_with_retry_inner in 10.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..
Traceback (most recent call last):
  File "/home/khteh/.local/lib/python3.12/site-packages/google/api_core/grpc_helpers_async.py", line 77, in wait_for_connection
    await self._call.wait_for_connection()
  File "/home/khteh/.local/lib/python3.12/site-packages/grpc/aio/_call.py", line 659, in wait_for_connection
    await self._raise_for_status()
  File "/home/khteh/.local/lib/python3.12/site-packages/grpc/aio/_call.py", line 272, in _raise_for_status
    raise _create_rpc_error(
grpc.aio._call.AioRpcError: <AioRpcError of RPC that terminated with:
	status = StatusCode.INVALID_ARGUMENT
	details = "Request contains an invalid argument."
	debug_error_string = "UNKNOWN:Error received from peer ipv4:74.125.24.95:443 {grpc_message:"Request contains an invalid argument.", grpc_status:3, created_time:"2025-03-22T19:48:06.353774327+08:00"}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/Python/rag-agent/whatever.py", line 25, in <module>
    asyncio.run(main())
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/usr/src/Python/rag-agent/whatever.py", line 16, in main
    async for event in agent.astream(
  File "/home/khteh/.local/lib/python3.12/site-packages/langgraph/pregel/__init__.py", line 2313, in astream
    async for _ in runner.atick(
  File "/home/khteh/.local/lib/python3.12/site-packages/langgraph/pregel/runner.py", line 444, in atick
    await arun_with_retry(
  File "/home/khteh/.local/lib/python3.12/site-packages/langgraph/pregel/retry.py", line 128, in arun_with_retry
    return await task.proc.ainvoke(task.input, config)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langgraph/utils/runnable.py", line 583, in ainvoke
    input = await step.ainvoke(input, config, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langgraph/utils/runnable.py", line 359, in ainvoke
    ret = await asyncio.create_task(coro, context=context)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langgraph/prebuilt/chat_agent_executor.py", line 705, in acall_model
    response = cast(AIMessage, await model_runnable.ainvoke(state, config))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 3071, in ainvoke
    input = await asyncio.create_task(part(), context=context)  # type: ignore
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_core/runnables/base.py", line 5377, in ainvoke
    return await self.bound.ainvoke(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 328, in ainvoke
    llm_result = await self.agenerate_prompt(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 853, in agenerate_prompt
    return await self.agenerate(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 813, in agenerate
    raise exceptions[0]
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 981, in _agenerate_with_cache
    result = await self._agenerate(
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_google_vertexai/chat_models.py", line 1527, in _agenerate
    return await agenerate_from_stream(stream_iter)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_core/language_models/chat_models.py", line 118, in agenerate_from_stream
    chunks = [chunk async for chunk in stream]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_google_vertexai/chat_models.py", line 1669, in _astream
    async for response_chunk in await response_iter:
                                ^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_google_vertexai/chat_models.py", line 645, in _acompletion_with_retry
    return await _completion_with_retry_inner(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line 189, in async_wrapped
    return await copy(fn, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line 111, in __call__
    do = await self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line 153, in iter
    result = await action(retry_state)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/tenacity/_utils.py", line 99, in inner
    return call(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 418, in exc_check
    raise retry_exc.reraise()
          ^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/tenacity/__init__.py", line 185, in reraise
    raise self.last_attempt.result()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/khteh/.local/lib/python3.12/site-packages/tenacity/asyncio/__init__.py", line 114, in __call__
    result = await fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/langchain_google_vertexai/chat_models.py", line 638, in _completion_with_retry_inner
    return await generation_method(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/khteh/.local/lib/python3.12/site-packages/google/api_core/grpc_helpers_async.py", line 178, in error_remapped_callable
    await call.wait_for_connection()
  File "/home/khteh/.local/lib/python3.12/site-packages/google/api_core/grpc_helpers_async.py", line 79, in wait_for_connection
    raise exceptions.from_grpc_error(rpc_error) from rpc_error
google.api_core.exceptions.InvalidArgument: 400 Request contains an invalid argument.
During task with name 'agent' and id '5efef15b-dd88-dcb3-ac66-90a5111b0d75'

Full-blown code: https://github.com/khteh/rag-agent/blob/master/src/rag_agent/RAGAgent.py