Skip to content

policy.md

Latest commit

 

History

History
253 lines (186 loc) · 10.4 KB

policy.md

File metadata and controls

253 lines (186 loc) · 10.4 KB
Raw

Integrating a Policy

Like many aspects of computing, AI terms such as model and policy are used in many contexts and can have differing meanings. The following diagram shows how these terms are used in the software blocks of the AI for Industry Challenge:

Block diagram

A policy is the software which consumes sensor data and produces output commands to the robot. Creating a policy is at the heart of the AI for Industry Challenge, since it is the critical block that "closes the loop" between sensors and actuators.

More specifically, the policy can receive the following data at up to 20 Hz:

  • 📷 📷 📷 images from three cameras mounted on the robot wrist
  • 🦾 joint angles of the robot arm and gripper
  • ⚖️ 3d force and 3d torque measurements at the robot wrist
  • 📐 target and actual poses of the gripper-fingers tool center point (TCP)
  • ☄️ velocity of the gripper-fingers tool center point (TCP)

For convenience, the aic_adapter in the Challenge environment combines time-synchronized values of the sensor suite into a single composite Observation data structure, which is delivered to the aic_model block at 20 Hz. The user-defined policy is dynamically loaded at runtime into aic_model and can retrieve the latest Observation at any time.

The policy is responsible for issuing position or velocity targets to the aic_controller, which provides low-level control of the arm to manage contact forces. The targets can be sent to aic_controller at any rate.

Several API styles are possible when writing a policy. Because the Challenge is implemented in ROS, the simplest API uses the data structures generated by the ROS 2 Python client library, rclpy, as shown in the following section.

Policy API

To integrate a policy using ROS data structures, such as geometry_msgs.msg.Pose, sensor_msgs.msg.Image, and so on:

  • define a Python class which derives from aic_model.Policy
  • implement the insert_cable() method, which is called when aic_engine requests a new task.
  • supply this Python class name as a parameter to aic_model at runtime.

The insert_cable() function receives several Callable methods as parameters:

The policy can invoke API functions which issue motion commands to the robot. As an implementation detail, those API functions use the aic_model ROS node to publish data to the aic_controller, which is implemented using the ros2_control framework.

Baseline Policies

We provide several baseline policy implementations in the aic_example_policies package that demonstrate different approaches to the cable insertion task:

  • WaveArm - A minimal example showing the basic Policy API structure
  • CheatCode - A "cheating" policy that uses ground truth data for training and debugging
  • RunACT - An ACT (Action Chunking with Transformers) policy implementation

For detailed descriptions, usage instructions, and source code, see the Example Policies README.

To see expected scoring results for each baseline policy, see the Scoring Test & Evaluation Guide.

Tutorial: Creating a new policy node

A policy node is essentially a ROS 2 node that subscribes to observations and publishes actions to be executed.

For this tutorial, we will be using aic_model to implement a policy node.

Important

Take note of the prompt in the bash examples. If it starts with (aic) $, then it should be run from inside the pixi environment.

Example:

$ pixi shell # This is outside pixi environment.
(aic) $ ros2 pkg list # This is inside the environment.

Create a new ROS 2 package

# Run "pixi shell" to enter the pixi environment 
(aic) $ ros2 pkg create my_policy_node --build-type ament_python

Add AIC dependencies

Add the following to package.xml:

	<depend>aic_control_interfaces</depend>
	<depend>aic_model</depend>
	<depend>aic_model_interfaces</depend>
	<depend>aic_task_interfaces</depend>
	<depend>geometry_msgs</depend>
	<depend>rclpy</depend>
	<depend>sensor_msgs</depend>
	<depend>std_srvs</depend>
	<depend>trajectory_msgs</depend>

Create a pixi package

Create a pixi.toml in the directory of the my_policy_node package with the following contents:

[package.build.backend]
name = "pixi-build-ros"
version = "==0.3.3.20260113.c8b6a54"
channels = [
	"https://prefix.dev/pixi-build-backends",
	"robostack-kilted",
	"conda-forge",
]

[package.host-dependencies]
ros-kilted-aic-control-interfaces = { path = "../aic_interfaces/aic_control_interfaces" }
ros-kilted-aic-model = { path = "../aic_model" }
ros-kilted-aic-model-interfaces = { path = "../aic_interfaces/aic_model_interfaces" }
ros-kilted-aic-task-interfaces = { path = "../aic_interfaces/aic_task_interfaces" }

[package.build-dependencies]
ros-kilted-aic-control-interfaces = { path = "../aic_interfaces/aic_control_interfaces" }
ros-kilted-aic-model = { path = "../aic_model" }
ros-kilted-aic-model-interfaces = { path = "../aic_interfaces/aic_model_interfaces" }
ros-kilted-aic-task-interfaces = { path = "../aic_interfaces/aic_task_interfaces" }

Tip

Normally, pixi will automatically discover the dependencies from package.xml. But because we are building the aic interfaces from source, we need to tell pixi where to find them.

Add the pixi package to the workspace

In the root pixi.toml, add the new package to [dependencies].

It should look like this

[dependencies]
# ...
ros-kilted-my-policy-node = { path = "my_policy_node" }

Implement PolicyRos

For brevity, we will reuse the code from aic_example_policies. See the ROS Policy API section above for implementation details.

(aic) $ cp aic_example_policies/aic_example_policies/ros/WaveArm.py my_policy_node/my_policy_node/WaveArm.py

Test the policy node

Terminal 1:

# Make sure to run 'export DBX_CONTAINER_MANAGER=docker'
$ distrobox enter -r aic_eval -- /entrypoint.sh

Terminal 2:

$ pixi reinstall ros-kilted-my-policy-node
$ pixi run ros2 run aic_model aic_model --ros-args -p use_sim_time:=true -p policy:=my_policy_node.WaveArm

Note

The command above runs the aic_model node, which then dynamically loads and runs your specific policy implementation (my_policy_node.WaveArm).

Dependency Management

This is a quick guide on how to manage dependencies under a pixi workspace. Unlike a typical ROS workspace, you will not be using any system dependencies; all dependencies must come from conda or pypi.

Adding dependencies

ROS dependencies

The pixi workspace is set up with the robostack-kilted channel. You can install most ROS packages with pixi.

$ pixi add ros-kilted-ros-core
pypi dependencies

Unlike a native ROS workspace, a pixi workspace can mix ROS and pypi dependencies.

$ pixi add --pypi torch
Local dependencies

If your package requires a local dependency in the same workspace, those dependencies must be declared on both the package's pixi.toml and the root pixi.toml.

For example, if my_policy_node requires my_local_dep,

my_policy_node/pixi.toml:

[package.host-dependencies]
# ...
ros-kilted-my-local-dep = { path = "../my_local_dep" }

[package.build-dependencies]
# ...
ros-kilted-my-local-dep = { path = "../my_local_dep" }

pixi.toml:

[dependencies]
# ...
ros-kilted-my-policy-node = { path = "my_policy_node" }
ros-kilted-my-local-dep = { path = "my_local_dep" }

Tip

pixi automatically prefixes a ROS package with ros-<distro>- and converts underscores to hyphens.

Build-Run-Debug Cycle (Python)

Important

Changes to packages within a Pixi environment are not tracked automatically. To apply updates, you must run pixi reinstall <package_name>.

$ pixi reinstall <package>

Tip

You may enter the pixi environment with pixi shell and force an "editable" install with pip install -e. But note that this circumvents pixi and may cause unintended side effects.

Preparing for Submission

After you are satisfied with your policy, you will need to prepare a Docker image for submission. See Submission for details.

Conclusion

Congratulations! You have successfully created, tested, and packaged a policy node.

In this tutorial, you have learned how to:

  • Create and set up a new ROS 2 package for a policy node.
  • Manage Python and ROS dependencies within a pixi workspace.
  • Understand the build, run, and debug cycle for developing your policy.
  • Prepare a Docker image for your policy for submission.

You are now equipped with the fundamental skills to develop, test, and submit your own policies for the AI for Industry Challenge. Feel free to explore the provided example policies and other documentation for more advanced concepts and inspiration.