[Question] Fluidity Index: Next Generation Super-Intelligence Benchmarks

[ngoiyaeric]

Question

I'd like to implement an benchmark RL-Game with a wrapped environment specifically for a humanoid non stationary robot that can charge its own batteries from earning on their job. Optimizing its mechanical activity to charge less often. Based of our lab research found here:
https://arxiv.org/abs/2510.20636v1

I'm curious RL-GAME and wrapped environment applicability in this scenario relative to the internal battery humanoid observability capabilities in the simulation configurations. Perhaps there are transformer based internal tooling RL policies that can be accessed from this software.

Build Info

Describe the versions that you are currently using:

• Isaac Lab Version: [LATEST]
• Isaac Sim Version: [LATEST]

[ngoiyaeric]

https://doorman-humanoid.github.io/

[ngoiyaeric]

https://viral-humanoid.github.io/

[RandomOakForest]

Thank you for posting this. I will move this post to our Discussions for follow up.

[ngoiyaeric]

We're part of Nvidia Inception too, so there's lots of room to get this done!

[ngoiyaeric]

I've lately been wondering how to program the reward state and transfer protocols on the RL-Game. I suppose energy transfer from tasking onto the robot can be simulated relative to token output quality as observed by the environment. A card is a good example to encapsulate this mechanism that can read and write onto doors or direct charging outlets.

[ngoiyaeric]

https://github.com/robotlearning123/awesome-isaac-gym?tab=readme-ov-file#tutorials Here is some more data on Isaac lab.

[ngoiyaeric]

https://nvlabs.github.io/SONIC/ this robot demos seem contextual to our research. @Shay-7278

[Shay-7278]

This seems really relevant. Excited to be looking into this direction and following along.

[ngoiyaeric]

Overview of Implementing Your RL Benchmark Game in Isaac Lab

Based on the diagram you provided, which appears to depict the architecture of NVIDIA's Isaac Lab Arena (a benchmark evaluation framework built on Isaac Lab), this setup is highly applicable for your scenario. Isaac Lab is a modular, GPU-accelerated simulation framework for robot learning, particularly reinforcement learning (RL), and it supports creating custom benchmarks for humanoid robots. It includes tools for task definitions, environment configurations, evaluation, and integration with partner benchmarks like RoboCasa or GROOT Bench. While the arXiv paper you linked (2510.20636v1) discusses general AI adaptability benchmarks (Fluidity Index), it doesn't directly address robotics—perhaps you intended a related paper like 2511.04831v1 on Isaac Lab itself, which focuses on multi-task robot learning in simulation.

Isaac Lab uses Gymnasium-compatible environments, making it straightforward to create a "RL-Game" (i.e., a custom RL task/environment) with wrapped layers for added functionality. For your humanoid non-stationary robot that earns "jobs" to charge batteries and optimizes mechanical activity for efficiency, you can extend existing humanoid tasks (e.g., locomotion or manipulation) to include battery dynamics. This aligns well with Isaac Lab's support for observability of internal states (like battery levels), reward shaping, and non-stationary elements via randomization.

I'll break this down step-by-step: applicability, environment setup, reward and transfer protocols, observability, and policy options (including transformers).

Applicability of RL-Game and Wrapped Environments

• RL-Game Fit: Isaac Lab treats RL tasks as "games" or environments where agents learn policies through trial-and-error. Your benchmark can be framed as a multi-objective task: the robot performs jobs (e.g., picking/placing objects, navigating) to earn "tokens" (simulated rewards or currency), which enable battery charging. This encourages long-term planning to minimize charging frequency. Isaac Lab's Arena extension (shown in your diagram) is designed for such benchmarks, allowing task definitions via configs (Scene Cfg, Robot Cfg, Task Cfg, etc.) and evaluation metrics (success rates, efficiency).

• Wrapped Environments: Environments in Isaac Lab are Gym-like, so you can wrap them for custom logic (e.g., using gymnasium.wrappers). This is ideal for your non-stationary robot:

  • Base env: Start with a humanoid locomotion/manipulation task.
  • Wrapper: Add layers for battery simulation, job earning, and charging mechanics.
  • Example: Wrap to monitor energy drain per action and reset if battery hits zero without "earning" a charge.

• Simulation Configurations: Isaac Lab runs on NVIDIA Isaac Sim for high-fidelity physics (e.g., PhysX). It supports non-stationary robots through domain randomization (varying mass, friction, joint damping) and adversarial motion priors (AMP) for dynamic behaviors. Your robot can be non-stationary (mobile, with changing energy states), and simulations can scale to thousands of parallel envs for fast training.

Humanoid support is strong—pre-built envs include generic humanoids and specific models like Unitree G1 or Agibot A2D for tasks like walking, picking, or stacking. These can be extended for your energy-optimized benchmark.

Setting Up the Environment

Use Isaac Lab's Manager-Based workflow (recommended for benchmarks, as per your diagram) for high-level orchestration. Here's how to implement:

1. Install and Setup:

   • Download from NVIDIA Developer (requires Omniverse Isaac Sim).
   • Clone the repo: git clone https://github.com/isaac-sim/IsaacLab.git.
   • Run: ./isaaclab.sh --install (handles dependencies like rl_games).

2. Define Custom Task Configs (Based on Diagram's Task Definition Interface):

   • Create a new config file (e.g., source/extensions/isaaclab_tasks/manager_based/custom/humanoid_energy_benchmark_env_cfg.py).
   • Inherit from ManagerBasedRLEnvCfg.
   • Key sections:
     • Robot Cfg: Load a humanoid URDF (e.g., Unitree G1). Add custom actuators/sensors for battery.
     • Scene Cfg: Define job areas (e.g., kitchen for tasks) and charging stations (e.g., outlets as interactive objects).
     • Task Cfg: Specify jobs (e.g., "pick microwave" as a goal). Use affordances (e.g., operable objects) from the diagram.
     • Data Gen Cfg: For randomization (e.g., varying job difficulties).
     • Object Cfg: Model the "card" as an asset (e.g., a keycard object the robot carries/manipulates to access doors/outlets).

   Example config snippet (Python):

   from omni.isaac.lab.envs import ManagerBasedRLEnvCfg
   from omni.isaac.lab.managers import SceneEntityCfg, RewardTermCfg
   
   class HumanoidEnergyBenchmarkCfg(ManagerBasedRLEnvCfg):
       num_envs = 4096  # Parallel sims for efficiency
       # Robot: Humanoid with battery sensor
       robot_cfg = SceneEntityCfg("robot", articulation_props={"battery_capacity": 100.0})
       # Task: Job earning + charging
       rewards = {
           "job_success": RewardTermCfg(func=job_reward_func, weight=1.0),
           "energy_efficiency": RewardTermCfg(func=energy_penalty_func, weight=-0.1)
       }
   

3. Implement Environment Logic:

   • In a companion file (e.g., humanoid_energy_benchmark_env.py), define the env class inheriting from ManagerBasedRLEnv.
   • Handle steps: Simulate energy drain based on actions (e.g., torque * time).
   • Job simulation: Robot completes tasks to earn tokens (e.g., quality based on precision/speed).
   • Charging: When tokens suffice, robot interacts with outlet (via card) to recharge.

4. Register and Run:

   • Add to list_envs.py.
   • Train: ./isaaclab.sh -p source/standalone/workflows/rl/train.py --task=HumanoidEnergyBenchmark --rl-library=rl_games.
   • Benchmark: Use Isaac Lab Arena for evaluation (e.g., metrics like charge frequency, job output quality).

For wrapping:

import gymnasium as gym
from isaaclab.envs import ManagerBasedRLEnv

env = ManagerBasedRLEnv(cfg=HumanoidEnergyBenchmarkCfg())
env = gym.wrappers.TimeLimit(env, max_episode_steps=1000)  # Wrap for episode limits
env = CustomBatteryWrapper(env)  # Your wrapper for battery logic

Programming Reward State and Transfer Protocols

• Reward State Design:

  • Sparse rewards for job completion (e.g., +10 for high-quality output).
  • Dense shaping for efficiency: Penalize high mechanical activity (e.g., -0.001 * sum(joint_torques^2)) to encourage minimal energy use.
  • Battery integration: -1.0 if battery < 10%, +5.0 for completing jobs without charging.
  • Token quality: Observe env (e.g., via metrics like task precision) to scale rewards (e.g., reward = quality_score * base_reward).
  • Overall: reward = job_success + efficiency_bonus - energy_penalty - charge_penalty.

  Example function:

  def compute_rewards(self):
      job_quality = self.env.compute_task_quality()  # e.g., 0-1 based on observation
      energy_drain = torch.norm(self.actions, dim=-1)  # Proxy for mechanical activity
      return job_quality * 10 - 0.01 * energy_drain - (self.charge_count * 2)
  

• Transfer Protocols (Energy/Charging):

  • Simulate energy transfer: Battery drains proportionally to action magnitude (e.g., battery -= k * ||actions|| * dt).
  • Earning to charge: Accumulate tokens from jobs; when threshold met, enable charging action (e.g., robot moves to outlet, uses card to "write" access).
  • Card mechanism: Model as a gripper-attached object with physics interactions (e.g., collision detection for doors/outlets). Use affordances (e.g., "operable" from diagram) to define interactions.
  • Protocol: On contact, transfer energy (battery += amount_based_on_tokens). This can be event-based (from diagram's Events/Term).

This setup optimizes for less frequent charging by rewarding sustained activity.

Internal Battery Humanoid Observability

• Capabilities: Fully supported. Battery can be an internal state added to the observation space.
  • In Manager-Based: Use observation manager to include "battery_level" (e.g., normalized 0-1).
  • In Direct: Add to obs dict: obs["battery"] = self.battery_state / max_capacity.
  • Non-stationary aspect: Battery changes dynamically, making the env partially observable Markov decision process (POMDP)—handle with history in obs (e.g., last 5 battery values for trends).
• Simulation Configs: Use sensors (e.g., custom BatterySensor) for realism. Randomize initial battery for robustness.

Transformer-Based Internal Tooling RL Policies

• Accessibility: Isaac Lab integrates with RL libraries like rl_games (PPO-based), which supports custom policies. Transformers aren't native but can be added as network architectures.
  • rl_games allows defining transformer encoders for policies (e.g., for sequential obs like battery history or job sequences).
  • Example: Extend rl_games.common with a transformer policy:

    import torch.nn as nn
    from rl_games.common import ActorCritic
    
    class TransformerPolicy(ActorCritic):
        def __init__(self, obs_shape, action_shape):
            super().__init__()
            self.transformer = nn.TransformerEncoderLayer(d_model=512, nhead=8)
            self.actor_head = nn.Linear(512, action_shape)
        
        def forward(self, obs):
            # Assume obs includes history
            emb = self.transformer(obs.unsqueeze(0))
            return self.actor_head(emb.mean(dim=1))
    

  • Register: In training config, set network: TransformerPolicy.
  • Applicability: Useful for your scenario—transformers can model long-term dependencies (e.g., predicting when to charge based on job patterns). Libraries like skrl (also supported) can integrate with Hugging Face for pre-trained transformers.
  • Examples in Robotics: Draw from papers like RT-1/RT-2 (transformer RL for robots), adaptable to Isaac Lab via custom scripts.

This framework should get you started—it's flexible for your energy-optimized humanoid. If you share more details (e.g., specific robot model or code snippets), I can refine further. For hands-on, check the Isaac Lab docs at isaac-sim.github.io/IsaacLab/.

[ngoiyaeric]

[ngoiyaeric]

https://www.youtube.com/watch?v=rrUHZKlrxms Here is an example of Boston Dynamics recent Self Swap mechanism on 4 hours of the humanoid's work on two batteries. Atlas seems to already self-replenish before battery swap however the work Atlas does is not correlate to its replenishing/swap/charging costs.

[ngoiyaeric]

I've found https://github.com/isaac-sim/IsaacLabEvalTasks for task specific benchmarks. The project's challenges are looking for pre-existing battery specific policies or configurations and generalizing this across task specific evaluations. These are necessary to coordinate with the whole body control mechanisms instructed by the vision language action.

[ngoiyaeric]

https://www.youtube.com/shorts/yFaQvjIWfPg here is a real life experiment that could be used to benchmark humanoids