Add a config option to clamp actor network output

[reeceomahoney]

When using the OnPolicyRunner for some envs and reward functions, the agent can often get stuck in a feedback loop where the value of the action explodes causing training to crash. Could be easily fixed by adding a torch.clamp in the update_distribution function of ActorCritic, along with a config option to set whether this is active or not. Would be happy to submit a pull request if you think this is a good idea.

[Mayankm96]

Thanks for the suggestion! I'm debating a bit whether we should handle this within rsl_rl or do it as a part of the environment itself.

One argument for rsl-rl could be that we can then propogate the gradient through the clamping operation (though not sure how that affects the training). If that doesn't work or make sense, then I think the environment should handle this operation (mainly for simplicity).

What do you think?

[reeceomahoney]

I am assuming you mean whether to add it to the OnPolicyRunner's env interaction or the actor critic directly? I think it might be easier to add it in the actor critic directly where the model is called as all other calls are downstream of this and then we wouldn't have to worry about duplicating this change for all of them?

Let me know if that sounds reasonable.