[BUG] ResponseValidationError when areal gateway processing openclaw response

[0hujun]

Checklist

• The error occurs when using our provided Docker image.
• I can consistently reproduce the bug across multiple trials or random seeds.
• If the error causes experiment abortion, I've verified that this error is the root
  cause, not a secondary error caused by peer workers.

Detailed Information

Describe the bug

using areal repo file examples/openclaw/README.md[./examples/openclaw/README.md] to test openclaw rl, the proxy gateway is running fine, and openclaw can connect it, when chat with openclaw, got ResponseValidationError from areal.

Expected behavior

no ResponseValidationError

Full logs

Traceback (most recent call last):
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 416, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/fastapi/applications.py", line 1139, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/applications.py", line 107, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/routing.py", line 716, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/routing.py", line 736, in app
    await route.handle(scope, receive, send)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/routing.py", line 290, in handle
    await self.app(scope, receive, send)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/fastapi/routing.py", line 119, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/fastapi/routing.py", line 105, in app
    response = await f(request)
               ^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/fastapi/routing.py", line 407, in app
    content = await serialize_response(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/fastapi/routing.py", line 248, in serialize_response
    raise ResponseValidationError(
fastapi.exceptions.ResponseValidationError: 1 validation errors:
  {'type': 'model_attributes_type', 'loc': ('response',), 'msg': 'Input should be a valid dictionary or object to extract fields from',                       'input': <async_generator object AsyncCompletionsWithReward._create_stream at 0xfffe609c4d30>}

To Reproduce

Commit ID

https://github.com/inclusionAI/AReaL/tree/ascend-v1.0.1

Environment

areal branch ascend-v1.0.1
opeclaw 2026.3.8
ascend A2
docker ubuntu

Script

examples/openclaw/README.md

[0hujun]

i try debug, find the bug code is in the openai/client.py

return self._create_stream( completion_id=completion_id, current_time=current_time, output_text=output_text, tool_calls=tool_calls, response=response, )

[0hujun]

the _create_stream method return AsyncGenerator type may conflict with fastapi StreamingResponse

[garrett4wade]

Thank you for reporting the issue. AReaL's openai client is just trying to reproduce the functionality of the official one, but additionaly records the token IDs the log probabilities during inference. We'll try to fix it later.

[Zijun9]

I would like to try this