[RLlib] Clarify extra model output docstrings (ray-project#63524)

## Why

`set_extra_model_outputs` currently asserts that the provided key
already exists before calling `InfiniteLookbackBuffer.set`. The
docstrings for `SingleAgentEpisode` and `MultiAgentEpisode` also said
the method could insert a new key, which made the behavior ambiguous.

This updates the docstrings to describe the current overwrite-only
behavior.

Closes ray-project#63217

## Tests

Not run. Documentation-only change.

Signed-off-by: GoparapukethaN <goparapukethan01@gmail.com>

Author: GoparapukethaN

rllib/env/multi_agent_episode.py

@@ -2042,12 +2042,13 @@ def set_extra_model_outputs(
     ) -> None:
         """Overwrites all or some of this Episode's extra model outputs with `new_data`.
 
-        This is a helper method to batch `SingleAgentEpisode.set_extra_model_outputs`.
-        For more detail, see `SingleAgentEpisode.set_extra_model_outputs`.
+        This batches `SingleAgentEpisode.set_extra_model_outputs` calls for the
+        agent IDs provided in `new_data`. For more detail, see
+        `SingleAgentEpisode.set_extra_model_outputs`.
 
         Args:
-            key: The `key` within `self.extra_model_outputs` to override data on or
-                to insert as a new key into `self.extra_model_outputs`.
+            key: Existing extra model output key to override on each target agent
+                episode.
             new_data: A dict mapping agent IDs to new extra model outputs data.
                 Each value in the dict is the new extra model outputs data to overwrite existing data with.
                 This may be a list of individual reward(s) in case this episode

rllib/env/single_agent_episode.py

@@ -1360,14 +1360,13 @@ def set_extra_model_outputs(
         current `extra_model_output` values are added to the episode either by calling
         `self.add_env_step` or more directly (and manually) via
         `self.extra_model_outputs[key].append|extend()`. However, for certain
-        postprocessing steps, the entirety (or a slice) of an episode's
-        `extra_model_outputs` might have to be rewritten or a new key (a new type of
-        `extra_model_outputs`) must be inserted, which is when
+        postprocessing steps, an existing `extra_model_outputs` entry might have
+        to be overwritten fully or in slices, which is when
         `self.set_extra_model_outputs()` should be used.
 
         Args:
-            key: The `key` within `self.extra_model_outputs` to override data on or
-                to insert as a new key into `self.extra_model_outputs`.
+            key: Existing key in `self.extra_model_outputs` whose buffer should
+                be overwritten.
             new_data: The new data to overwrite existing data with.
                 This may be a list of individual reward(s) in case this episode
                 is still not numpy'ized yet. In case this episode has already been