Skip to content

Teacher policy train NAN error #16

Open
@Ethonwinters

Description

@Ethonwinters

Hi, thank you for your amazing work!

when I run Teacher policy ,the following error was reported
My AMASS dataset is ACCAD.tar.bz2

Traceback (most recent call last):
File "/home/adminstrator/HOVER/scripts/rsl_rl/train_teacher_policy.py", line 122, in
main()
File "/home/adminstrator/HOVER/scripts/rsl_rl/train_teacher_policy.py", line 114, in main
ppo_runner.learn(num_learning_iterations=teacher_policy_cfg.runner.max_iterations, init_at_random_ep_len=True)
File "/home/adminstrator/HOVER/third_party/rsl_rl/rsl_rl/runners/on_policy_runner.py", line 122, in learn
actions = self.alg.act(obs, critic_obs)
File "/home/adminstrator/HOVER/third_party/rsl_rl/rsl_rl/algorithms/ppo.py", line 106, in act
self.transition.actions = self.actor_critic.act(obs).detach()
File "/home/adminstrator/HOVER/third_party/rsl_rl/rsl_rl/modules/actor_critic.py", line 132, in act
self.update_distribution(observations)
File "/home/adminstrator/HOVER/third_party/rsl_rl/rsl_rl/modules/actor_critic.py", line 129, in update_distribution
self.distribution = Normal(mean, mean * 0.0 + self.std)
File "/home/adminstrator/isaaclab/_isaac_sim/exts/omni.isaac.ml_archive/pip_prebundle/torch/distributions/normal.py", line 59, in init
super().init(batch_shape, validate_args=validate_args)
File "/home/adminstrator/isaaclab/_isaac_sim/exts/omni.isaac.ml_archive/pip_prebundle/torch/distributions/distribution.py", line 71, in init
raise ValueError(
ValueError: Expected parameter loc (Tensor of shape (1024, 19)) of distribution Normal(loc: torch.Size([1024, 19]), scale: torch.Size([1024, 19])) to satisfy the constraint Real(), but found invalid values:
tensor([[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
...,
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan],
[nan, nan, nan, ..., nan, nan, nan]], device='cuda:0')

Thanks for your answer, and thanks again for the amazing work!

ZHUO WANG

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions