-
Notifications
You must be signed in to change notification settings - Fork 3.4k
Description
I am using the newest verl code and the newest docker images:
docker pull verlai/verl:vllm017.latest
I refer the only veomini script ./verl/tests/special_e2e/run_ppo_trainer_veomni.sh
python3 -m verl.trainer.main_ppo
model_engine=veomni
algorithm.adv_estimator=grpo
data.train_files="$train_path"
data.val_files="$test_path"
data.train_batch_size=16
data.max_prompt_length=512
data.max_response_length=512
data.filter_overlong_prompts=True
data.truncation='error'
actor_rollout_ref.model.path=/mnt/data/HithinkOmniSSD/user_workspace/leizhengxing/verl/data/Qwen3-VL-4B-Instruct
actor_rollout_ref.actor.optim.lr=5e-7
actor_rollout_ref.model.use_remove_padding=True
actor_rollout_ref.model.enable_gradient_checkpointing=True
actor_rollout_ref.actor.veomni.param_offload=True
actor_rollout_ref.actor.veomni.optimizer_offload=True
actor_rollout_ref.actor.ppo_mini_batch_size=8
actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=1
actor_rollout_ref.actor.use_kl_loss=True
actor_rollout_ref.actor.kl_loss_coef=0.001
actor_rollout_ref.actor.kl_loss_type=low_var_kl
actor_rollout_ref.actor.entropy_coeff=0
actor_rollout_ref.actor.use_torch_compile=False
actor_rollout_ref.actor.veomni.fsdp_size="${FSDP_SIZE}"
actor_rollout_ref.actor.veomni.ulysses_parallel_size="${SP_SIZE}"
actor_rollout_ref.actor.veomni.expert_parallel_size="${EP_SIZE}"
actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=1
actor_rollout_ref.ref.veomni.param_offload=True
actor_rollout_ref.ref.use_torch_compile=False
actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=1
actor_rollout_ref.rollout.tensor_model_parallel_size=2
actor_rollout_ref.rollout.enable_chunked_prefill=False
actor_rollout_ref.rollout.name=vllm
actor_rollout_ref.rollout.gpu_memory_utilization=0.8
actor_rollout_ref.rollout.free_cache_engine=True
actor_rollout_ref.rollout.enforce_eager=True
actor_rollout_ref.rollout.free_cache_engine=True
actor_rollout_ref.rollout.n=2
actor_rollout_ref.ref.veomni.optimizer_offload=True
algorithm.kl_ctrl.kl_coef=0.001
trainer.use_legacy_worker_impl=disable
trainer.critic_warmup=0
trainer.logger=console
trainer.project_name='verl_veomni_test'
trainer.experiment_name="${VERL_EXP_NAME}"
trainer.n_gpus_per_node=4
trainer.val_before_train=False
trainer.nnodes=1
trainer.save_freq=-1
trainer.test_freq=-1
trainer.total_epochs=1
trainer.total_training_steps=1 $@
It will output the error:
File "/usr/local/lib/python3.12/dist-packages/hydra/_internal/instantiate/_instantiate2.py", line 347, in instantiate_node
return _call_target(target, partial, args, kwargs, full_key)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/hydra/_internal/instantiate/_instantiate2.py", line 97, in _call_target
raise InstantiationException(msg) from e
hydra.errors.InstantiationException: Error in call to target 'verl.workers.config.actor.FSDPActorConfig':
TypeError("FSDPActorConfig.init() got an unexpected keyword argument 'veomni'")
full_key: actor_rollout_ref.actor
I try to delete
model_engine=veomni \
But it still outputs the error
Key 'veomni' is not in struct
full_key: actor_rollout_ref.actor.veomni object_type=dict
Can you please help me fix the bug?