Skip to content

v2.6.0: New `LogEveryNTimesteps` callback and `has_attr` method, refactored hyperparameter optimization

Latest
Latest
Compare
Choose a tag to compare
@araffin araffin released this 24 Mar 15:01
· 1 commit to master since this release
v2.6.0
ea913a8
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

SB3 Contrib (more algorithms): https://github.com/Stable-Baselines-Team/stable-baselines3-contrib
RL Zoo3 (training framework): https://github.com/DLR-RM/rl-baselines3-zoo
Stable-Baselines Jax (SBX): https://github.com/araffin/sbx

To upgrade:

pip install stable_baselines3 sb3_contrib rl_zoo3 --upgrade

New Features:

  • Added has_attr method for VecEnv to check if an attribute exists
  • Added LogEveryNTimesteps callback to dump logs every N timesteps (note: you need to pass log_interval=None to avoid any interference)
  • Added Gymnasium v1.1 support

Bug fixes:

  • SubProcVecEnv will now exit gracefully (without big traceback) when using KeyboardInterrupt

SB3-Contrib

  • Renamed _dump_logs() to dump_logs()
  • Fixed issues with SubprocVecEnv and MaskablePPO by using vec_env.has_attr() (pickling issues, mask function not present)

RL Zoo

  • Refactored hyperparameter optimization. The Optuna Journal storage backend is now supported (recommended default) and you can easily load tuned hyperparameter via the new --trial-id argument of train.py.
  • Save the exact command line used to launch a training
  • Added support for special vectorized env (e.g. Brax, IsaacSim) by allowing to override the VecEnv class use to instantiate the env in the ExperimentManager
  • Allow to disable auto-logging by passing --log-interval -2 (useful when logging things manually)
  • Added Gymnasium v1.1 support
  • Fixed use of old HF api in get_hf_trained_models()

SBX (SB3 + Jax)

  • Updated PPO to support net_arch, and additional fixes
  • Fixed entropy coeff wrongly logged for SAC and derivatives.
  • Fixed PPO predict() for env that were not normalized (action spaces with limits != [-1, 1])
  • PPO now logs the standard deviation

Deprecations:

  • algo._dump_logs() is deprecated in favor of algo.dump_logs() and will be removed in SB3 v2.7.0

Others:

  • Updated black from v24 to v25
  • Improved error messages when checking Box space equality (loading VecNormalize)
  • Updated test to reflect how set_wrapper_attr should be used now

Documentation:

  • Clarify the use of Gym wrappers with make_vec_env in the section on Vectorized Environments (@pstahlhofen)
  • Updated callback doc for EveryNTimesteps
  • Added doc on how to set env attributes via VecEnv calls
  • Added ONNX export example for MultiInputPolicy (@darkopetrovic)

New Contributors

Full Changelog: v2.5.0...v2.6.0

Contributors

darkopetrovic and pstahlhofen
Assets 2
Loading