Skip to content

Commit d42f91f

Browse files
LucaKroLucaKro
authored
Merge pull request #119 from HoangGiang93/PleaseMergeMultiSim
Let me merge to main
2 parents d3a1387 + ff0665c commit d42f91f

16 files changed

Lines changed: 2917 additions & 235 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,20 @@ jobs:
4040
git submodule update
4141
pip install -U pip && pip install -r requirements.txt && pip install -e . && pip install pytest
4242
pip install -e ./src/ripple_down_rules_meta
43+
if [[ $(python3 --version) == "Python 3.10"* ]]; then
44+
PACKAGE_URL="https://nc.uni-bremen.de/public.php/dav/files/fxWQigKXPBtnXsy/?accept=zip"
45+
PACKAGE_HASH="5249b21f6a6dbbb86eee9e64659cf19b9b09585ffbc888d13af9bd9c2bcce99c"
46+
elif [[ $(python3 --version) == "Python 3.12"* ]]; then
47+
PACKAGE_URL="https://nc.uni-bremen.de/public.php/dav/files/nxxi4QSBcYmmxba/?accept=zip"
48+
PACKAGE_HASH="a111ac524d6eb9161ed90972cd60670de887066b553a91a397da985f3a60f9aa"
49+
fi
50+
if [[ -n "$PACKAGE_URL" ]]; then
51+
TMP_PKG="/tmp/multiverse_parser-0.0.3.tar.gz"
52+
curl -fsSL "$PACKAGE_URL" -o "$TMP_PKG"
53+
echo "$PACKAGE_HASH $TMP_PKG" | sha256sum -c -
54+
pip install "$TMP_PKG"
55+
rm -f "$TMP_PKG"
56+
fi
4357
# echo "PATH=/opt/ros/overlay_ws/src/Multiverse-Parser/ext/blender:/opt/ros/overlay_ws/src/Multiverse-Parser/USD/linux/lib/python:/opt/ros/overlay_ws/src/Multiverse-Parser/USD/linux/plugin/usd:$PATH" >> $GITHUB_ENV
4458
# echo "PYTHONPATH=/opt/ros/overlay_ws/src/Multiverse-Parser/USD/linux/lib/python:/opt/ros/overlay_ws/src/src" >> $GITHUB_ENV
4559

‎doc/_toc.yml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ parts:
1919
- file: examples/semantic_annotation_factories
2020
- file: examples/persistence_of_annotated_worlds
2121
- file: examples/graph_of_convex_sets
22+
- file: examples/multiverse
2223
- caption: Developer Guide
2324
chapters:
2425
- file: developer_guide

‎doc/user_guide.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ The user guide is divided into multiple chapters teaching you the following topi
3535
- [](persistence-of-annotated-worlds)
3636
- Synchronizing worlds across multiple processes
3737
- Pipelines
38-
- Simulation
38+
- [](multiverse)
3939
- Inverse Kinematics
4040
- Collision Checking
4141
- Casadi

‎examples/multiverse.md‎

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
---
2+
jupyter:
3+
jupytext:
4+
text_representation:
5+
extension: .md
6+
format_name: myst
7+
format_version: 0.13
8+
jupytext_version: 1.16.4
9+
kernelspec:
10+
display_name: Python 3
11+
language: python
12+
name: python3
13+
---
14+
15+
(multiverse)=
16+
# Simulation with Multiverse
17+
18+
Multiverse Framework is a system that connects different robotics simulation and hardware components together.
19+
It serves as a unifying infrastructure that addresses the fragmentation across simulation and controller software.
20+
Instead of being a single simulator extended with controllers and reasoning modules,
21+
it provides an architectural foundation with interoperability mechanisms that allow simulators, controllers,
22+
and reasoning systems to remain independent while still operating together as a coherent ecosystem.
23+
24+
This package uses Multiverse Simulators (MultiSim) to provide a unified abstraction layer over different simulation engines.
25+
This high-level abstraction enables us to switch between multiple backends with minimal effort,
26+
while maintaining a consistent logic for world representation and manipulation.
27+
Communication between the simulator and other components running in separate processes is managed by the Multiverse Framework through its plugins.
28+
29+
Below is an example of how to set up an empty simulation using the Mujoco backend and run it for 5 seconds.
30+
31+
```{code-cell} ipython3
32+
from semantic_digital_twin.world import World
33+
from multiverse_simulator import MultiverseSimulatorState, MultiverseViewer
34+
from semantic_digital_twin.adapters.multi_sim import MujocoSim
35+
import os
36+
import time
37+
38+
world = World()
39+
viewer = MultiverseViewer()
40+
headless = os.environ.get("CI", "false").lower() == "true" # headless in CI environments
41+
multi_sim = MujocoSim(world=world, viewer=viewer, headless=headless, step_size=1E-3)
42+
multi_sim.start_simulation()
43+
44+
start_time = time.time()
45+
time.sleep(5.0)
46+
multi_sim.stop_simulation()
47+
```
48+
49+
A MultiSim simulator takes a `World` object as input and synchronizes its state with the simulation engine,
50+
a `MultiverseViewer` for reading and write data in run-time, and other configuration parameters.
51+
52+
You can spawn objects in the simulation in run-time just by changing the `World` object.
53+
The world changes will be automatically synchronized with the simulation engine.
54+
55+
```{code-cell} ipython3
56+
from semantic_digital_twin.world import World
57+
from multiverse_simulator import MultiverseSimulatorState, MultiverseViewer
58+
from semantic_digital_twin.adapters.multi_sim import MujocoSim
59+
from semantic_digital_twin.datastructures.prefixed_name import PrefixedName
60+
from semantic_digital_twin.spatial_types import TransformationMatrix
61+
from semantic_digital_twin.world_description.connections import Connection6DoF
62+
from semantic_digital_twin.world_description.geometry import Box, Scale, Color
63+
from semantic_digital_twin.world_description.world_entity import Body
64+
from semantic_digital_twin.world_description.shape_collection import ShapeCollection
65+
import os
66+
import time
67+
68+
world = World()
69+
viewer = MultiverseViewer()
70+
headless = os.environ.get("CI", "false").lower() == "true" # headless in CI environments
71+
multi_sim = MujocoSim(world=world, viewer=viewer, headless=headless, step_size=1E-3)
72+
multi_sim.start_simulation()
73+
74+
start_time = time.time()
75+
time.sleep(1.0)
76+
print(f"Time to start creating a new body: {time.time() - start_time}s")
77+
new_body = Body(name=PrefixedName("test_body"))
78+
box_origin = TransformationMatrix.from_xyz_rpy(
79+
x=0.2, y=0.4, z=-0.3, roll=0, pitch=0.5, yaw=0, reference_frame=new_body
80+
)
81+
box = Box(
82+
origin=box_origin,
83+
scale=Scale(1.0, 1.5, 0.5),
84+
color=Color(
85+
1.0,
86+
0.0,
87+
0.0,
88+
1.0,
89+
),
90+
)
91+
new_body.collision = ShapeCollection([box], reference_frame=new_body)
92+
93+
with world.modify_world():
94+
world.add_connection(
95+
Connection6DoF.create_with_dofs(
96+
world=world,
97+
parent=world.root,
98+
child=new_body,
99+
)
100+
)
101+
print(f"Time to add new body: {time.time() - start_time}s")
102+
103+
time.sleep(5.0)
104+
multi_sim.stop_simulation()
105+
```

‎requirements.txt‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,4 +30,6 @@ fbxloader
3030
coacd
3131
pymysql
3232
pyglet == 1.5.31
33-
pycollada
33+
pycollada
34+
mujoco==3.3.5
35+
multiverse-simulators>=0.0.7

‎resources/mjcf/cube_on_the_ground.xml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
<mujoco model="panda scene">
2+
<compiler angle="radian" autolimits="true"/>
3+
4+
<option timestep="0.005" iterations="5" ls_iterations="8" integrator="implicitfast">
5+
<flag eulerdamp="disable"/>
6+
</option>
7+
8+
<custom>
9+
<numeric data="12" name="max_contact_points"/>
10+
</custom>
11+
12+
<visual>
13+
<headlight diffuse="0.6 0.6 0.6" ambient="0.3 0.3 0.3" specular="0 0 0"/>
14+
<rgba haze="0.15 0.25 0.35 1"/>
15+
<global azimuth="120" elevation="-20"/>
16+
<scale contactwidth="0.075" contactheight="0.025" forcewidth="0.05" com="0.05" framewidth="0.01"
17+
framelength="0.2"/>
18+
</visual>
19+
20+
<asset>
21+
<texture type="skybox" builtin="gradient" rgb1="0.3 0.5 0.7" rgb2="0 0 0" width="512" height="3072"/>
22+
<texture type="2d" name="groundplane" builtin="checker" mark="edge" rgb1="0.2 0.3 0.4" rgb2="0.1 0.2 0.3"
23+
markrgb="0.8 0.8 0.8" width="300" height="300"/>
24+
<material name="groundplane" texture="groundplane" texuniform="true" texrepeat="5 5" reflectance="0.2"/>
25+
</asset>
26+
27+
<worldbody>
28+
<light pos="0 0 1.5" dir="0 0 -1" directional="true"/>
29+
<geom name="floor" size="0 0 0.05" type="plane" material="groundplane" contype="1"/>
30+
</worldbody>
31+
32+
<worldbody>
33+
<body name="box" pos="0.5 0 0.03">
34+
<freejoint/>
35+
<geom type="box" name="box" size="0.02 0.02 0.03" condim="3"
36+
friction="1 .03 .003" rgba="0 1 0 1" contype="2" conaffinity="1" solref="0.01 1"/>
37+
</body>
38+
</worldbody>
39+
</mujoco>

‎resources/mjcf/mjx_single_cube_no_mesh.xml‎

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
<mujoco model="panda scene">
2+
<compiler angle="radian" autolimits="true"/>
3+
4+
<default>
5+
<default class="panda">
6+
<material specular="0.5" shininess="0.25"/>
7+
<joint armature="0.1" damping="1" axis="0 0 1" range="-2.8973 2.8973"/>
8+
<general dyntype="none" biastype="affine" ctrlrange="-2.8973 2.8973" forcerange="-87 87"/>
9+
<position forcerange="-100 100"/>
10+
<default class="finger">
11+
<joint axis="0 1 0" type="slide" range="0 0.04"/>
12+
</default>
13+
14+
<default class="visual">
15+
<geom type="mesh" contype="0" conaffinity="0" group="2"/>
16+
</default>
17+
<default class="collision">
18+
<geom group="3" type="mesh" contype="0" conaffinity="0"/>
19+
<default class="primitive_collision">
20+
<geom group="3" type="box" contype="0" conaffinity="0"
21+
condim="3" solimp="0.99 0.995 0.01" solref="0.01 1" friction="1 0.005 0.0001"/>
22+
<default class="fingertip_pad_collision_1">
23+
<geom type="box" size="0.011 0.0075 0.01" pos="0 0.0185 0.011"/>
24+
</default>
25+
<default class="fingertip_pad_collision_2">
26+
<geom type="box" size="0.011 0.0044 0.0019" pos="0 0.0068 0.0022"/>
27+
</default>
28+
<default class="fingertip_pad_collision_3">
29+
<geom type="box" size="0.00875 0.0035 0.01175" pos="0 0.0159 0.02835" quat="1 0.25 0 0"/>
30+
</default>
31+
<default class="fingertip_pad_collision_4">
32+
<geom type="box" size="0.00875 0.0076 0.00825" pos="0 0.00758 0.04525" conaffinity="3"/>
33+
</default>
34+
</default>
35+
</default>
36+
</default>
37+
</default>
38+
39+
<worldbody>
40+
<light name="top" pos="0 0 2" mode="trackcom"/>
41+
<body name="link0" childclass="panda">
42+
<inertial mass="0.629769" pos="-0.041018 -0.00014 0.049974"
43+
fullinertia="0.00315 0.00388 0.004285 8.2904e-7 0.00015 8.2299e-6"/>
44+
<body name="link1" pos="0 0 0.333">
45+
<inertial mass="4.970684" pos="0.003875 0.002081 -0.04762"
46+
fullinertia="0.70337 0.70661 0.0091170 -0.00013900 0.0067720 0.019169"/>
47+
<joint name="joint1" damping="40"/>
48+
<body name="link2" quat="1 -1 0 0">
49+
<inertial mass="0.646926" pos="-0.003141 -0.02872 0.003495"
50+
fullinertia="0.0079620 2.8110e-2 2.5995e-2 -3.925e-3 1.0254e-2 7.04e-4"/>
51+
<joint name="joint2" range="-1.7628 1.7628" damping="40"/>
52+
<body name="link3" pos="0 -0.316 0" quat="1 1 0 0">
53+
<joint name="joint3" damping="40"/>
54+
<inertial mass="3.228604" pos="2.7518e-2 3.9252e-2 -6.6502e-2"
55+
fullinertia="3.7242e-2 3.6155e-2 1.083e-2 -4.761e-3 -1.1396e-2 -1.2805e-2"/>
56+
<body name="link4" pos="0.0825 0 0" quat="1 1 0 0">
57+
<inertial mass="3.587895" pos="-5.317e-2 1.04419e-1 2.7454e-2"
58+
fullinertia="2.5853e-2 1.9552e-2 2.8323e-2 7.796e-3 -1.332e-3 8.641e-3"/>
59+
<joint name="joint4" range="-3.0718 -0.0698" damping="40"/>
60+
<body name="link5" pos="-0.0825 0.384 0" quat="1 -1 0 0">
61+
<inertial mass="1.225946" pos="-1.1953e-2 4.1065e-2 -3.8437e-2"
62+
fullinertia="3.5549e-2 2.9474e-2 8.627e-3 -2.117e-3 -4.037e-3 2.29e-4"/>
63+
<joint name="joint5" damping="2"/>
64+
<body name="link6" quat="1 1 0 0">
65+
<inertial mass="1.666555" pos="6.0149e-2 -1.4117e-2 -1.0517e-2"
66+
fullinertia="1.964e-3 4.354e-3 5.433e-3 1.09e-4 -1.158e-3 3.41e-4"/>
67+
<joint name="joint6" range="-0.0175 3.7525" damping="2"/>
68+
<body name="link7" pos="0.088 0 0" quat="1 1 0 0">
69+
<inertial mass="7.35522e-01" pos="1.0517e-2 -4.252e-3 6.1597e-2"
70+
fullinertia="1.2516e-2 1.0027e-2 4.815e-3 -4.28e-4 -1.196e-3 -7.41e-4"/>
71+
<joint name="joint7" damping="2"/>
72+
<body name="hand" pos="0 0 0.107" quat="0.9238795 0 0 -0.3826834">
73+
<inertial mass="0.73" pos="-0.01 0 0.03" diaginertia="0.001 0.0025 0.0017"/>
74+
<site name="gripper" pos="0 0 0.1"/>
75+
<body name="left_finger" pos="0 0 0.0584">
76+
<inertial mass="0.015" pos="0 0 0"
77+
diaginertia="2.375e-6 2.375e-6 7.5e-7"/>
78+
<joint name="finger_joint1" class="finger" damping="10"/>
79+
</body>
80+
<body name="right_finger" pos="0 0 0.0584" quat="0 0 0 1">
81+
<inertial mass="0.015" pos="0 0 0"
82+
diaginertia="2.375e-6 2.375e-6 7.5e-7"/>
83+
<joint name="finger_joint2" class="finger" damping="10"/>
84+
</body>
85+
</body>
86+
</body>
87+
</body>
88+
</body>
89+
</body>
90+
</body>
91+
</body>
92+
</body>
93+
</body>
94+
</worldbody>
95+
96+
<equality>
97+
<joint joint1="finger_joint1" joint2="finger_joint2" solimp="0.95 0.99 0.001" solref="0.005 1"/>
98+
</equality>
99+
100+
<actuator>
101+
<position class="panda" name="actuator1" joint="joint1" kp="1000" kv="20"
102+
ctrlrange="-2.8973 2.8973"/>
103+
<position class="panda" name="actuator2" joint="joint2" kp="1000" kv="20"
104+
ctrlrange="-1.7628 1.7628"/>
105+
<position class="panda" name="actuator3" joint="joint3" kp="750" kv="4"
106+
ctrlrange="-2.8973 2.8973"/>
107+
<position class="panda" name="actuator4" joint="joint4" kp="750" kv="4"
108+
ctrlrange="-3.0718 -0.0698"/>
109+
<position class="panda" name="actuator5" joint="joint5" kp="300" kv="2"
110+
forcerange="-12 12" ctrlrange="-2.8973 2.8973"/>
111+
<position class="panda" name="actuator6" joint="joint6" kp="300" kv="2" forcerange="-12 12"
112+
ctrlrange="-0.0175 3.7525"/>
113+
<position class="panda" name="actuator7" joint="joint7" kp="300" kv="2" forcerange="-12 12"/>
114+
<general class="panda" name="actuator8" joint="finger_joint1"
115+
ctrlrange="0 0.04" gainprm="350 0 0" biasprm="0 -350 -10" forcerange="-200 200"/>
116+
</actuator>
117+
118+
<option timestep="0.005" iterations="5" ls_iterations="8" integrator="implicitfast">
119+
<flag eulerdamp="disable"/>
120+
</option>
121+
122+
<custom>
123+
<numeric data="12" name="max_contact_points"/>
124+
</custom>
125+
126+
<visual>
127+
<headlight diffuse="0.6 0.6 0.6" ambient="0.3 0.3 0.3" specular="0 0 0"/>
128+
<rgba haze="0.15 0.25 0.35 1"/>
129+
<global azimuth="120" elevation="-20"/>
130+
<scale contactwidth="0.075" contactheight="0.025" forcewidth="0.05" com="0.05" framewidth="0.01"
131+
framelength="0.2"/>
132+
</visual>
133+
134+
<asset>
135+
<texture type="skybox" builtin="gradient" rgb1="0.3 0.5 0.7" rgb2="0 0 0" width="512" height="3072"/>
136+
<texture type="2d" name="groundplane" builtin="checker" mark="edge" rgb1="0.2 0.3 0.4" rgb2="0.1 0.2 0.3"
137+
markrgb="0.8 0.8 0.8" width="300" height="300"/>
138+
<material name="groundplane" texture="groundplane" texuniform="true" texrepeat="5 5" reflectance="0.2"/>
139+
</asset>
140+
141+
<worldbody>
142+
<light pos="0 0 1.5" dir="0 0 -1" directional="true"/>
143+
<geom name="floor" size="0 0 0.05" type="plane" material="groundplane" contype="1"/>
144+
</worldbody>
145+
146+
<worldbody>
147+
<body name="box" pos="0.5 0 0.03">
148+
<freejoint/>
149+
<geom type="box" name="box" size="0.02 0.02 0.03" condim="3"
150+
friction="1 .03 .003" rgba="0 1 0 1" contype="2" conaffinity="1" solref="0.01 1"/>
151+
</body>
152+
</worldbody>
153+
154+
<keyframe>
155+
<key name="home"
156+
qpos="0 0.3 0 -1.57079 0 2.0 -0.7853 0.04 0.04 0.7 0 0.03 1 0 0 0"
157+
ctrl="0 0.3 0 -1.57079 0 2.0 -0.7853 0.04"/>
158+
<key name="pickup"
159+
qpos="0.2897 0.50732 -0.140016 -2.176 -0.0310497 2.51592 -0.49251 0.04 0.0399982 0.511684 0.0645413 0.0298665 0.665781 2.76848e-17 -2.27527e-17 -0.746147"
160+
ctrl="0.2897 0.423 -0.144392 -2.13105 -0.0291743 2.52586 -0.492492 0.04"/>
161+
<key name="pickup1"
162+
qpos='0.2897 0.496673 -0.142836 -2.14746 -0.0295746 2.52378 -0.492496 0.04 0.0399988 0.529553 0.0731702 0.0299388 0.94209 8.84613e-06 -4.97524e-06 -0.335361'
163+
ctrl="0.2897 0.458 -0.144392 -2.13105 -0.0291743 2.52586 -0.492492 0.04"/>
164+
</keyframe>
165+
</mujoco>

0 commit comments

Comments
 (0)