[Proposal] Tutorials

[pseudo-rnd-thoughts]

Proposal

To encourage the use of Gymnasium and build up the RL community, I would propose that a large range of tutorials are created.

This is a list of tutorials that could be made

• Implementation of a custom environment
• Frozenlake training results with different map sizes
• Gymnasium vectorisation (gym.make_vec)
• Training agents for the blackjack environment, it has Tuple observation space
• DQN for atari implementation (doesn't need to be fast)
• Training a deep RL agent with pytorch from scratch
• Training a deep RL agent with jax from scratch
• How to use the action sample masking, with example from Taxi
• Car racing, comparing agents with continuous and discrete action spaces
• Exploring the impact of bipedal walker hardcore parameter on agent performance
• Experimenting with classic control reset options random state bounds
• Add environment or example using the Graph space

[kir0ul]

Will there be a required format for the tutorials?

[kir0ul]

Isn't the first tutorial on Implementation of a custom environment already done in: https://gymnasium.farama.org/content/environment_creation/ ?

[pseudo-rnd-thoughts]

We are using the sphinx-gallery project, therefore, we expect single python files with figures stored in a folder with the same name.
We are in the process of transferring our current tutorials to this style but pytorch use this style for their tutorials

[BalajiAI]

Since I'm a beginner, I'll take DQN for Atari tutorial.

[kir0ul]

I'm working on the Frozenlake one, still work in progress but I hope to have it ready soonish.

[siddarth-c]

I am working on "Training a deep RL agent with pytorch (with the new step API)".

[hahas94]

Could someone make an update on this proposal?
For instance, which tasks are done, which ones are work in progress, so that one may know what to work on.

[pseudo-rnd-thoughts]

Could someone make an update on this proposal? For instance, which tasks are done, which ones are work in progress, so that one may know what to work on.

I've have updated

[kir0ul]

@pseudo-rnd-thoughts also the Frozenlake training results with different map sizes tutorial is done.

[hahas94]

Thanks!

I am thinking to work on Training a deep RL agent with pytorch fromscratch. However, doesn't the REINFORCE implementation fit that requirement?

[pseudo-rnd-thoughts]

Good point @hahas94 I think we are always happy for more tutorials. You can work on another tutorial if you want

[hahas94]

Okay, if similar tutorials is a good thing I will stick to my aforementioned choice.
I am thinking of writing a tutorial on implementing the C51 agent in PyTorch.

[ApexArtist]

We need this. For practical use case. Need external environment tutorials not another memory environment