Conversation Role Alternation Error with Gemma-3-12b-it Model: litellm.BadRequestError

[henrywang0314]

Issue: Conversation Role Alternation Error with Gemma-3-12b-it Model

Description

When using the Gemma-3-12b-it model through hosted vLLM, the system encounters a conversation role alternation error. The error occurs during agent interactions, specifically when the conversation flow attempts to transfer between agents.

Error Message

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new
LiteLLM.Info: If you need to debug this error, use `litellm._turn_on_debug()'.

2025-06-13 17:06:35,578 - ERROR - fast_api.py:817 - Error in event_generator: litellm.BadRequestError: Hosted_vllmException - Conversation roles must alternate user/assistant/user/assistant/...
Traceback (most recent call last):
File "/root/anaconda3/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 801, in acompletion
headers, response = await self.make_openai_chat_completion_request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/lib/python3.12/site-packages/litellm/litellm_core_utils/logging_utils.py", line 135, in async_wrapper
result = await func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 436, in make_openai_chat_completion_request
raise e
File "/root/anaconda3/lib/python3.12/site-packages/litellm/llms/openai/openai.py", line 418, in make_openai_chat_completion_request
await openai_aclient.chat.completions.with_raw_response.create(
File "/root/anaconda3/lib/python3.12/site-packages/openai/_legacy_response.py", line 381, in wrapped
return cast(LegacyAPIResponse[R], await func(*args, **kwargs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/lib/python3.12/site-packages/openai/resources/chat/completions/completions.py", line 2028, in create
return await self._post(
^^^^^^^^^^^^^^^^^
File "/root/anaconda3/lib/python3.12/site-packages/openai/_base_client.py", line 1742, in post
return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/anaconda3/lib/python3.12/site-packages/openai/_base_client.py", line 1549, in request
raise self._make_status_error_from_response(err.response) from None
openai.BadRequestError: Error code: 400 - {'object': 'error', 'message': ' Conversation roles must alternate user/assistant/user/assistant/...', 'type': 'BadRequestError', 'param': None, 'code': 400}

Environment

• Model: google/gemma-3-12b-it
• Provider: hosted_vllm
• API Base URL: http://localhost:5021/v1
• Framework: Google ADK

Steps to Reproduce

1. Set up a multi-agent system with a coordinator agent and sub-agents
2. Configure the system to use Gemma-3-12b-it model through hosted vLLM
3. Initiate a conversation that requires agent transfer
4. The error occurs during the agent transfer process

Code Context

The error occurs in a system with the following agent structure:

booking_agent = LlmAgent(name="Booker", description="Handles flight and hotel bookings.")
info_agent = LlmAgent(name="Info", description="Provides general information and answers questions.")

root_agent = LlmAgent(
    name="Coordinator",
    model=model,
    instruction="You are an assistant. Delegate booking tasks to Booker and info requests to Info.",
    description="Main coordinator.",
    sub_agents=[booking_agent, info_agent]
)

Expected Behavior

The conversation should flow smoothly between agents, maintaining proper role alternation in the conversation history.

Actual Behavior

The system fails with a role alternation error, suggesting that the conversation history is not properly maintaining the user/assistant alternation pattern required by the model.

Impact

This issue prevents proper agent-to-agent communication and breaks the conversation flow in multi-agent systems using the Gemma-3-12b-it model.

[LiuYuWei]

Got the same problem about VLLM + Gemma 3 12B/27B Model.

[masahiro-hamada]

As a temporary solution, I used the following jinja-template which removed the error handling and the error went away.

{{ bos_token }} {%- if messages[0]['role'] == 'system' -%} {%- if messages[0]['content'] is string -%} {%- set first_user_prefix = messages[0]['content'] + '\n' -%} {%- else -%} {%- set first_user_prefix = messages[0]['content'][0]['text'] + '\n' -%} {%- endif -%} {%- set loop_messages = messages[1:] -%} {%- else -%} {%- set first_user_prefix = "" -%} {%- set loop_messages = messages -%} {%- endif -%} {%- for message in loop_messages -%} {%- if (message['role'] == 'assistant') -%} {%- set role = "model" -%} {%- else -%} {%- set role = message['role'] -%} {%- endif -%} {{ '<start_of_turn>' + role + '\n' + (first_user_prefix if loop.first else "") }} {%- if message['content'] is string -%} {{ message['content'] | trim }} {%- elif message['content'] is iterable -%} {%- for item in message['content'] -%} {%- if item['type'] == 'image' -%} {{ '<start_of_image>' }} {%- elif item['type'] == 'text' -%} {{ item['text'] | trim }} {%- endif -%} {%- endfor -%} {%- else -%} {{ raise_exception("Invalid content type") }} {%- endif -%} {{ '<end_of_turn>\n' }} {%- endfor -%} {%- if add_generation_prompt -%} {{'<start_of_turn>model\n'}} {%- endif -%}

[alfredcs]

The newly released medgemma-27b-it and medgemma-27b-text-it have the same issue.