Feature branch

[RobinU434]

Thanks, looks good! Regarding the documentation: Yes, also welcome. But for the action space we have continuous and discrete version, so the documentation should contain both.

Ok the documentation I will add on the spot.

Regarding the continuous vs discrete action: you have both action spaces versions (self.action_space and self.discrete_action_space) and a variable which looks like to distinguish between the two action spaces but you do not use self.discrete_action_space and the memeberfunction discrete_to_continous_action. How should I proceed here?

• Add the option to use discrete actions
• Cleanup the discrete options
• Just ignore what happens in the code and only add the documentation for continuous actions

[georgmartius]

Yes, the environment is by default only for continuous actions. For convenience, we have provided helper functions to use it nevertheless, by manually calling the conversion function. Note, that people can also decide to use a different mapping from discrete to continuous. The reason why we are not having a wrapper environment is multifold:

1. there are two opponents, both don't need to use the same action encoding
2. during the competition, everything has to happen with the continuous action space
   So documentation, should only be for continuous.
   The following could be written for discrete actions:
   If you want to use discrete actions, we have a helper function discrete_to_continous_action(x) for conversion and discrete_action_space to define the action space for a single player.

[RobinU434]

Thanks for the reply,
I opened up another branch where opened up the possibility to change to discrete actions (for both actors) by setting a flag in the class signature:

HockeyEnv(continous=True)  # for continuous actions
HockeyEnv(continous=False)  # for discrete actions

You can read it up in the hockey env file. Please note that I spread out the environment over 3 files to be a bit more structured (dedicated param file and dedicated opponent file). Of course this does not has to stay this way.

As we discussed during the lecture it would be also possible to fulfill your wish to write a dedicated discrete Environment. Just let me know what would you like to do.

[RobinU434]

Additional feateure for this pull request:

Render API in Hockey Env deviates from gymnasium.
Your implementation takes in the render mode:

def render(self, render_mode):
   ....

official documentation states without any arguments in render but setting render mode in init(). This makes the environement compatible with RecordVideo wrapper:

def render(self):
   ....