TypeError: can't assign a numpy.ndarray to a torch.FloatTensor

[wccccp]

le "/workspace/rl_exp/Skywork-OR1/verl/trainer/main_ppo.py", line 147, in
main()
File "/root/miniconda3/lib/python3.10/site-packages/hydra/main.py", line 94, in decorated_main
_run_hydra(
File "/root/miniconda3/lib/python3.10/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra
_run_app(
File "/root/miniconda3/lib/python3.10/site-packages/hydra/_internal/utils.py", line 457, in _run_app
run_and_report(
File "/root/miniconda3/lib/python3.10/site-packages/hydra/_internal/utils.py", line 223, in run_and_report
raise ex
File "/root/miniconda3/lib/python3.10/site-packages/hydra/_internal/utils.py", line 220, in run_and_report
return func()
File "/root/miniconda3/lib/python3.10/site-packages/hydra/_internal/utils.py", line 458, in
lambda: hydra.run(
File "/root/miniconda3/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 132, in run
_ = ret.return_value
File "/root/miniconda3/lib/python3.10/site-packages/hydra/core/utils.py", line 260, in return_value
raise self._return_value
File "/root/miniconda3/lib/python3.10/site-packages/hydra/core/utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
File "/workspace/rl_exp/Skywork-OR1/verl/trainer/main_ppo.py", line 26, in main
run_ppo(config, code_compute_score)
File "/workspace/rl_exp/Skywork-OR1/verl/trainer/main_ppo.py", line 46, in run_ppo
ray.get(main_task.remote(config, compute_score))
File "/root/miniconda3/lib/python3.10/site-packages/ray/_private/auto_init_hook.py", line 21, in auto_init_wrapper
return fn(*args, **kwargs)
File "/root/miniconda3/lib/python3.10/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
return func(*args, **kwargs)
File "/root/miniconda3/lib/python3.10/site-packages/ray/_private/worker.py", line 2822, in get
values, debugger_breakpoint = worker.get_objects(object_refs, timeout=timeout)
File "/root/miniconda3/lib/python3.10/site-packages/ray/_private/worker.py", line 930, in get_objects
raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(TypeError): ray::main_task() (pid=855066, ip=10.0.18.24)
File "/workspace/rl_exp/Skywork-OR1/verl/trainer/main_ppo.py", line 143, in main_task
trainer.fit()
File "/workspace/rl_exp/Skywork-OR1/verl/trainer/ppo/ray_trainer.py", line 1015, in fit
reward_tensor = self.reward_fn(batch)
File "/workspace/rl_exp/Skywork-OR1/verl/workers/reward_manager/yr_code.py", line 101, in call
reward_tensor[i, valid_response_length[i].item() - 1] = scores[i]
TypeError: can't assign a numpy.ndarray to a torch.FloatTensor

[hejujie]

Hi @wccccp,

Thanks for reporting this. This is an occasional issue that occurs during code execution.

We actually fixed this on an internal GitLab branch in April but unfortunately forgot to push the changes to GitHub.

I've now created a hotfix pull request with the fix: #46 (File changes can be seen here: https://github.com/SkyworkAI/Skywork-OR1/pull/46/files). You can cherry-pick the commit from this PR.

Just to let you know, we are currently on the Dragon Boat Festival holiday. Feel free to try this fix first. We will test its compatibility with the GitHub branch on June 4th.

Before we fully test this pr, here's a screenshot of the relevant commit from our internal GitLab for your reference:

Thanks!

[wccccp]

Hi @wccccp,

Thanks for reporting this. This is an occasional issue that occurs during code execution.

We actually fixed this on an internal GitLab branch in April but unfortunately forgot to push the changes to GitHub.

I've now created a hotfix pull request with the fix: #46 (File changes can be seen here: https://github.com/SkyworkAI/Skywork-OR1/pull/46/files). You can cherry-pick the commit from this PR.

Just to let you know, we are currently on the Dragon Boat Festival holiday. Feel free to try this fix first. We will test its compatibility with the GitHub branch on June 4th.

Before we fully test this pr, here's a screenshot of the relevant commit from our internal GitLab for your reference:

Thanks!

Thank you so much for the quick response and for creating the hotfix PR during the holiday — really appreciate it!