Skip to content

Dependency errors with RL policy for joint level control working with Spot with arm #10

Description

@insitelab

Dear RAI support team,
We have been trying to work with BD support for about 6 months to resolve a series of dependency issues with our Spot Explorer.

They have referred us to you because they do not have the expertise on Issac Sim or the RL policy for the Spot with arm.
This is the response we received from them first:


You are absolutely correct. The NVIDIA Isaac Lab tutorial and the associated RL policy in that blog post are designed exclusively for the 12-DOF locomotion of the base Spot robot (walking without the arm).

To achieve "Whole-Body Control" or "Loco-Manipulation" (coordinating both the legs and the 6-DOF arm), you would need to expand the environment in Isaac Lab. This involves replacing the robot asset with the 19-DOF arm version, adjusting the observation/action spaces, and rewriting the reward functions.

Since developing a combined policy from scratch is highly complex, the best approach is to leverage the work of the Boston Dynamics AI Institute (RAI Institute). They have specifically developed frameworks to solve this exact problem.

Here are the most relevant resources from the RAI Institute:

RAI Institute Homepage: rai-inst.com
ReLIC Framework (Reinforcement Learning for Interlimb Coordination): github.com/bdaiinstitute/relic (This is their state-of-the-art repository specifically designed for dynamic coordination between Spot's legs and arm).
Spot ROS 2 Wrapper: github.com/bdaiinstitute/spot_ros2 (For integrating Spot—including arm support—into ROS 2).
Spot RL Deployment Code: github.com/boston-dynamics/spot-rl-example (Co-developed by the AI Institute, required to deploy your custom Isaac Lab policies onto the real robot's Jetson Orin).
Please note that the RAI Institute is a research lab and does not offer traditional customer support. If you have technical questions regarding the code or need help setting up the Sim-to-Real pipeline, I suggest filing an issue in the respective GitHub repository to talk to the developers and researchers directly.


We went through all of the suggestions on the RAI site but then had more dependency problems. We reached out to them again (below)

Your email gave us some really important pointers, however, we are still running into dependency issues with training the new RL policy. While running the RL policy for interlimb coordination as instructed here, we run into the following issues:
When we try to run 'pixi install', it fails with the following error

Error: × Failed to update PyPI packages for environment 'default'
├─▶ Failed to prepare distributions
├─▶ Failed to build flatdict==4.0.1
├─▶ The build backend returned an error
╰─▶ Call to setuptools.build_meta:__legacy__.build_wheel failed (exit
status: 1)

  [stderr]
  Traceback (most recent call last):
    File "<string>", line 14, in <module>
    File "/home/nafis/.cache/rattler/cache/uv-cache/builds-
  v0/.tmp9hQnmX/lib/python3.10/site-packages/setuptools/
  build_meta.py", line 333, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings,
  requirements=[])
    File "/home/nafis/.cache/rattler/cache/uv-cache/builds-
  v0/.tmp9hQnmX/lib/python3.10/site-packages/setuptools/
  build_meta.py", line 301, in _get_build_requires
      self.run_setup()
    File "/home/nafis/.cache/rattler/cache/uv-cache/builds-
  v0/.tmp9hQnmX/lib/python3.10/site-packages/setuptools/
  build_meta.py", line 520, in run_setup
      super().run_setup(setup_script=setup_script)
    File "/home/nafis/.cache/rattler/cache/uv-cache/builds-
  v0/.tmp9hQnmX/lib/python3.10/site-packages/setuptools/
  build_meta.py", line 317, in run_setup
      exec(code, locals())
    File "<string>", line 1, in <module>
  ModuleNotFoundError: No module named 'pkg_resources'
 
  hint: This error likely indicates that `flatdict@4.0.1` depends on
  `pkg_resources`, but doesn't declare it as a build dependency. If
  `flatdict` is a first-party package, consider adding `pkg_resources`
  to its `build-system.requires`. Otherwise, either add it to your
  `pyproject.toml` under:
 
  [tool.uv.extra-build-dependencies]
  flatdict = ["pkg_resources"]
 
  or `uv pip install pkg_resources` into the environment and re-run
  with `--no-build-isolation`.

Please see the attached error file. We tried the instructions under the hint, but it didn't fix the issue.


They referred us again to your Github Issue page this morning:


While I would love to assist you further, providing direct support for these RL policies falls outside our scope. The RAI Institute operates as a separate research organization and manages their own open-source projects, which is why their developers are the best point of contact for these specific codes.

To get in touch with them and ask your technical questions, you can reach out directly to the researchers by opening an "Issue" in their GitHub repository here: https://github.com/bdaiinstitute/relic/issues


We would sincerely appreciate your help. We are a research lab at Colby College a small liberal arts college in Maine. We were one of the first labs adapting Spot as a 'guide dog' in 2021. We have now progressed to developing LLM voice-based interfaces and spatial graphs with VLMs using Spot as a potential navigation guide for blind and low vision navigators. We just presented at HRI 2026 on this work. I am appealing to you for help in keeping this research going as a fellow HRI researcher.

Sincerely,
Dr. Stacy Doore
sadoore@colby.edu

error (1).txt

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions