VLLM deprecated GuidedDecodingParams -> StructuredOutputsParams

[ed-pai]

Reproduction

According to vllm-project/vllm#29326, GuidedDecodingParams were completely removed now in v0.12.0 (released this morning). This causes a failure in TRL.

(NOTE: v0.12.0 isn't supported in TRL, so just making the issue as I didn't see it filed yet).

from trl.trainer.grpo_trainer import RewardFunc

outputs:

Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/trl/trainer/grpo_trainer.py", line 95, in <module>
    from vllm.sampling_params import GuidedDecodingParams
ImportError: cannot import name 'GuidedDecodingParams' from 'vllm.sampling_params' (/usr/local/lib/python3.12/dist-packages/vllm/sampling_params.py)

System Info

trl 0.25.0
torch 2.9.0+cu129
transformers 4.57.1

Running on GH200 (ARM64)

trl env fails to run with:

/usr/local/lib/python3.12/dist-packages/trl/import_utils.py:91: UserWarning: TRL currently only supports vLLM version `0.10.2`. You have version 0.12.0 installed. We recommend to install this version to avoid compatibility issues.
  warnings.warn(
Traceback (most recent call last):
  File "/usr/local/lib/python3.12/dist-packages/trl/import_utils.py", line 156, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 999, in exec_module
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
  File "/usr/local/lib/python3.12/dist-packages/trl/trainer/grpo_trainer.py", line 95, in <module>
    from vllm.sampling_params import GuidedDecodingParams
ImportError: cannot import name 'GuidedDecodingParams' from 'vllm.sampling_params' (/usr/local/lib/python3.12/dist-packages/vllm/sampling_params.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/bin/trl", line 4, in <module>
    from trl.cli import main
  File "/usr/local/lib/python3.12/dist-packages/trl/cli.py", line 25, in <module>
    from .scripts.grpo import make_parser as make_grpo_parser
  File "/usr/local/lib/python3.12/dist-packages/trl/scripts/grpo.py", line 33, in <module>
    from trl import (
  File "<frozen importlib._bootstrap>", line 1412, in _handle_fromlist
  File "/usr/local/lib/python3.12/dist-packages/trl/import_utils.py", line 147, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/trl/import_utils.py", line 146, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/dist-packages/trl/import_utils.py", line 158, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import trl.trainer.grpo_trainer because of the following error (look up to see its traceback):
cannot import name 'GuidedDecodingParams' from 'vllm.sampling_params' (/usr/local/lib/python3.12/dist-packages/vllm/sampling_params.py)

Checklist

• I have checked that my issue isn't already filed (see open issues)
• I have included my system information
• Any code provided is minimal, complete, and reproducible (more on MREs)
• Any code provided is properly formatted in code blocks, (no screenshot, more on code blocks)
• Any traceback provided is complete

[qgallouedec]

Thanks for reporting, indeed, it one of the reasons TRL doesn't officially supports vLLM 0.12. I'd be happy to receive a PR for this if someone wants to tackle this issue

[ed-pai]

Just realized there is a PR #4117 that already covers this now. It doesn't address the VLLM versioning. Not sure how you'd want to handle that (e.g. pin to >=v0.12.0??)