[Question] Franka Lift Cube Task with Visual Observation

[Ozzey]

Franka Lift Cube Task with Visual Observation

I was trying to modify the Franka-Lift task to use visual observation. The camera is mounted on the end-effector and from the camera's pov, the cube is visible at all the times (screenshot below).

The only change I made was to modify the observation space:

@configclass
class LiftCameraSceneCfg(ObjectTableSceneCfg):

    # add camera to the scene
    
        gripper_camera = TiledCameraCfg(
            prim_path="{ENV_REGEX_NS}/Robot/panda_hand/camera",
            height=480,
            width=640,
            data_types=["distance_to_image_plane"],
            spawn=sim_utils.PinholeCameraCfg(
                focal_length=24.0,
                focus_distance=400.0,
                horizontal_aperture=53.7,
                clipping_range=(0.01, 1.0e5),
            ),
            offset=TiledCameraCfg.OffsetCfg(
                pos=(0.05, 0, 0.05), rot=(0.70441603, -0.06162842, -0.06162842, 0.70441603), convention="ros"
            ),
        )
    ##


@configclass
class DepthObservationsCfg:
    """Observation specifications for the MDP."""

    @configclass
    class DepthCameraPolicyCfg(ObsGroup):
        """Observations for policy group with depth images."""

        image = ObsTerm(
            func=mdp.image, params={"sensor_cfg": SceneEntityCfg("gripper_camera"), "data_type": "distance_to_image_plane"}
        )

    policy: ObsGroup = DepthCameraPolicyCfg()




@configclass
class LiftCameraEnvCfg(LiftEnvCfg):
    """Configuration for the lift environment with depth camera."""

    scene: LiftCameraSceneCfg = LiftCameraSceneCfg(num_envs=256, env_spacing=20)
    observations: DepthObservationsCfg = DepthObservationsCfg()

    def __post_init__(self):
        super().__post_init__()
        # remove ground as it obstructs the camera
        # self.scene.ground = None
        # viewer settings
        self.viewer.eye = (-5.0, 0.0, 2.5)
        self.viewer.lookat = (0.0, 0.0, 1.5)

This env cfg is then used by

class FrankaLiftCameraEnvCfg(LiftCameraEnvCfg):

which contains details about the manipulator.

The agent was trained for roughly 45,109,248 steps but there is still no success at all, infact the manipulator doesnt even try to pickup the cube and stays fixed in a single position (as shown in the screenshot above). The average reward is 0.95

Algorithm: PPO
Library: RL_Games
GPU: Nvidia RTX 4060ti
VRAM: 16GB
RAM: 64GB
Envs: 16

horizon_length: 64
minibatch_size: 512

[RandomOakForest]

Thank you for posting this. I will move the post to our Discussions for the team to follow up.

[RandomOakForest]

Following up, have you tried using 100 environments? It's definitely looks as an under-training problem.

[Ozzey]

Thank you for your response.

Unfortunately with my current setup (mentioned above) I can only train visual tasks with a maximum of 16 parallel envs. Although I can try training more but overall I was expecting atleast a bit better performance after so many interactions.

Also I have made a similar implementation for Isaac-Open-Drawer-Franka task and after similar number of interactions I still had much higher average return (~30 reward, for non-visual variant the average return was around 90 after similar no. of interactions).