Cross-Episode Experience Memory for Robot Learning

[robotmem]

Type

Feature Request / Improvement

Problem

Robot learning policies (RL/IL) start each episode from scratch — they can't remember what worked in previous episodes. This limits sample efficiency and prevents experience transfer across tasks and environments.

Proposed Solution: robotmem

We built robotmem, an open-source cross-episode experience memory system for robot learning.

Core API:

• learn() — Record episode outcomes with structured context (parameters, spatial info, success/failure)
• recall() — Retrieve relevant past experiences using semantic search + spatial nearest-neighbor
• session() — Automatic session-level experience consolidation

Experiment Results

Tested across 3 Gymnasium Robotics environments with heuristic policies:

Environment	Baseline	With Memory	Delta	Setup
FetchPush-v4	42%	67%	+25%	10 seeds × 300 episodes
FetchPush → FetchSlide	4%	12%	+8%	Cross-environment transfer
Meta-World push-v3	21.5%	32.6%	+11.1%	5 seeds × 20 instances

Mechanism: After exploration episodes, the policy recall()s the most relevant past successful experiences based on spatial similarity (e.g., object position), then adjusts strategy parameters accordingly.

All experiments are reproducible with --seed flags.

Integration Example

from robotmem.sdk import RobotMemory

mem = RobotMemory(collection="lerobot_experiment")

with mem.session(context={"task": "push"}) as sid:
    for episode in range(num_episodes):
        # Before episode — recall similar successful experiences
        memories = mem.recall(
            "successful push strategy",
            n=3,
            context_filter={"task.success": True},
            spatial_sort={
                "field": "spatial.object_position",
                "target": current_obj_pos,
            },
        )
        # ... run episode with memory-guided policy ...

        # After episode — record experience
        mem.learn(
            insight=f"push: reward={reward:.1f}, success={success}",
            context={
                "params": {"push_speed": {"value": speed, "type": "scalar"}},
                "spatial": {"object_position": obj_pos.tolist()},
                "task": {"success": success, "reward": float(reward)},
            },
            session_id=sid,
        )

We also have a RobotMemCallback class that hooks into training loops (on_train_begin, on_episode_end, on_train_end).

ROS 2 Node

Related to #2093 — we have a ROS 2 Node with 7 services + 1 topic that wraps the Python SDK, enabling persistent memory for ROS-based robot deployments.

Install

pip install git+https://github.com/robotmem/robotmem.git

Zero external service dependencies — SQLite + local ONNX embeddings. No GPU required.

Links

• Website: www.robotmem.com
• Repository: robotmem/robotmem
• Experiment scripts: examples/fetch_push/, examples/metaworld/
• LeRobot callback: examples/lerobot_callback.py
• Colab demo: examples/colab_demo.ipynb

[robotmem]

Try it now — Colab Demo

You can try robotmem in 60 seconds without any local setup:

The demo runs a 3-phase experiment (baseline → learn → recall) and shows memory-guided improvement in ~5 seconds after install.

[robotmem]

Update: Plugin module ready

robotmem now ships a dedicated LeRobot integration module:

from robotmem.lerobot import RobotMemCallback

with RobotMemCallback(collection="my_experiment") as cb:
    cb.on_train_begin({"robot": "aloha", "task": "pick_place", "policy": "act"})
    
    for ep in range(num_episodes):
        # recall past successful experiences before episode
        tips = cb.recall_tips("successful strategy", context_filter={"task.success": True})
        
        # ... run episode ...
        
        cb.on_episode_end({"episode": ep, "reward": reward, "success": success})
    
    cb.on_train_end({"success_rate": final_rate})

Install: pip install git+https://github.com/robotmem/robotmem.git

The callback is non-intrusive — all exceptions are caught internally and logged as warnings, so it never interrupts training.

Entry point registered as lerobot.callbacks:robotmem for plugin discovery.

[robotmem]

Update: We've published the LeRobot callback plugin module.

Install & Use

pip install git+https://github.com/robotmem/robotmem.git

from robotmem.lerobot import RobotMemCallback

with RobotMemCallback(collection="my_experiment") as cb:
    cb.on_train_begin({"robot": "aloha", "task": "pick_place"})
    
    for ep in range(num_episodes):
        # ... training ...
        cb.on_episode_end({"episode": ep, "reward": reward, "success": success})
    
    cb.on_train_end({"success_rate": 0.85})

The callback is registered via entry_points in pyproject.toml:

[project.entry-points."lerobot.callbacks"]
robotmem = "robotmem.lerobot:RobotMemCallback"

Source: src/robotmem/lerobot/