How to deal with the effect of episode_length_s and roll_out on reward functions and robot behaviour?

[celestialdr4g0n]

In my robot reaching task using the skrl library, I’ve observed that modifying the episode_length_s (in *_cfg.py) or the roll_out (in *.yaml) drastically affects the impact of my reward function. Specifically, when using a negative reward for target orientation misalignment:
• Longer episodes or higher roll_out values: The robot overly focuses on adjusting the end effector’s orientation, which causes it to fail in reaching the object.
• Shorter episodes or lower roll_out values: The robot reaches the target but exhibits shaking at the end.

It’s also very challenging to continually recalibrate the reward scales with every adjustment of these hyperparameters. What strategies or modifications can I apply to balance these effects, ensuring that the robot both reaches the target reliably and maintains a stable orientation?

[RandomOakForest]

Thank you for posting this. The team will engage soon. @Toni-SM, any thoughts? Thanks.