Skip to content

fix(legacy): preserve chat prompt roles - #1410

Draft
yaodong-shen wants to merge 1 commit into
Marker-Inc-Korea:mainfrom
yaodong-shen:fix/1234-preserve-chat-roles
Draft

fix(legacy): preserve chat prompt roles#1410
yaodong-shen wants to merge 1 commit into
Marker-Inc-Korea:mainfrom
yaodong-shen:fix/1234-preserve-chat-roles

Conversation

@yaodong-shen

Copy link
Copy Markdown

Summary

  • preserve structured chat messages during token-budget truncation for the legacy OpenAI, MiniMax, and VLLM API generators
  • send the complete role-preserving message list to the GPT-5 Responses API instead of flattening system/user turns and dropping assistant history
  • retain the existing MiniMax helper import surface while sharing the truncation implementation

Root cause

The OpenAI and VLLM API paths converted List[Dict] chat prompts to a single string before dispatch. That turned role markers into user content; GPT-5 additionally discarded prior assistant turns when forming a Responses request.

Validation

  • PYTHONPATH=/private/tmp/autorag-test-stubs uv run --no-sync pytest tests/autorag/nodes/generator/test_openai.py tests/autorag/nodes/generator/test_minimax.py tests/autorag/nodes/generator/test_vllm_api.py -q -k 'truncate_chat_prompt_preserves_message_roles or gpt_5_responses_preserve_full_chat_history or truncate_messages_preserves_roles or vllm_api_truncate_preserves_chat_messages'
    • 4 passed
  • uvx ruff check on all changed generator and test files
  • uvx ruff format --check on all changed generator and test files
  • git diff --check

The full legacy suite was not runnable locally because its macOS ARM development resolution includes an NVIDIA-only package and its OpenAI test module imports optional torch.cuda during collection. The focused regression tests use no model call, credentials, or GPU.

Closes #1234

@yaodong-shen
yaodong-shen force-pushed the fix/1234-preserve-chat-roles branch from e73a2a9 to f956727 Compare August 19, 2026 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Chat prompt truncation and gpt-5 routing drop role/turn structure

1 participant