Skip to content

Fix ML environment registration bugs and improve meta-learning entry points#573

Open
marcusfechner wants to merge 4 commits intoFarama-Foundation:masterfrom
marcusfechner:fix_ml_env_registration_bugs
Open

Fix ML environment registration bugs and improve meta-learning entry points#573
marcusfechner wants to merge 4 commits intoFarama-Foundation:masterfrom
marcusfechner:fix_ml_env_registration_bugs

Conversation

@marcusfechner
Copy link
Copy Markdown

This PR fixes several bugs encountered when first using the Meta-Learning environments following the examples from the documentation. These issues can be confusing for newcomers to Meta-World, and I hope these fixes make the initial experience significantly smoother.

Summary

  • "Fix ML1 train and test envs only using test split" (e227f7b): The for split in ["train", "test"] loop registered lambdas for Meta-World/ML1-train and Meta-World/ML1-test, but the lambda captured split by reference rather than by value. By the time either lambda was called, the loop had finished and split was always "test". This meant ML1-train environments were silently using test tasks instead of training tasks. Fixed by binding the loop variable via a default argument (_split=split).

    Reproduce:

    import gymnasium as gym
    
    import metaworld
    
    train_spec = gym.spec("Meta-World/ML1-train")
    test_spec = gym.spec("Meta-World/ML1-test")
    
    # split is captured by closure reference, not bound as a default argument.
    # The loop variable ends up as "test" for both registrations:
    train_closure = {v.cell_contents for v in train_spec.vector_entry_point.__closure__}
    test_closure = {v.cell_contents for v in test_spec.vector_entry_point.__closure__}
    
    print("ML1-train closure vars:", train_closure)
    # {'test', <function _ml_bench_vector_entry_point ...>}
    print("ML1-test  closure vars:", test_closure)
    # {'test', <function _ml_bench_vector_entry_point ...>}
    # Both contain 'test' — ML1-train should contain 'train'
  • "Make sample_tasks_on_reset True to prevent initial tasks not being set on reset()" (ed5f15b): PseudoRandomTaskSelectWrapper defaulted to sample_tasks_on_reset=False, which meant that on the initial reset() call no task was sampled and the environment had no task set. This caused environments created through the ML entry points to start in an undefined task state. Changing the default to True ensures a task is always sampled when the environment is reset.

    Reproduce:

    import gymnasium as gym
    
    import metaworld
    
    env = gym.make_vec("Meta-World/ML1-train", env_name="pick-place-v3", meta_batch_size=1)
    obs, info = env.reset()
    # Results in
    #     assert self._last_rand_vec is not None
    # AssertionError
  • "Add single env entry point for ML1" (141a8a9): ML1 registrations only provided a vector_entry_point, so calling gym.make("Meta-World/ML1-train", env_name="...") would fail because there was no non-vector entry point. Added a _ml1_entry_point function and registered it as entry_point alongside the existing vector_entry_point, enabling single-environment creation via gym.make().

    Reproduce:

    import gymnasium as gym
    
    import metaworld
    
    env = gym.make("Meta-World/ML1-train", env_name="pick-place-v3")
    obs, info = env.reset()
    # Results in
    #     raise error.Error(f"{env_spec.id} registered but entry_point is not specified")
    # gymnasium.error.Error: Meta-World/ML1-train registered but entry_point is not specified
  • "Refactor meta_batch_size handling in ML environment functions to allow None values" (1fa1b11): meta_batch_size defaulted to 20 across all ML entry points, which was arbitrary and would silently fail for benchmarks where the number of environment classes didn't divide 20. Changed the default to None, which now auto-defaults to the number of environment classes (one sub-environment per class) and can be overwritten by specifying meta_batch_size. I Also added descriptive assertion messages that report the actual values when meta_batch_size is too small or not evenly divisible.

    Reproduce:

    import gymnasium as gym
    
    
    import metaworld
    
    envs = gym.make_vec("Meta-World/ML1-train", env_name="pick-place-v3")
    # Results in
    #     len(tasks_for_subenv) == len(tasks) // tasks_per_env
    # AssertionError: Invalid division of subtasks, expected 2 got 3

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant