[Meshcat] Set object from JavaScript code #22683
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR implements:
Meshcat::SetObjectFromThreeJsCode(std::string_view path, std::string_view three_js_lambda)
that lets the user invoke Meshcat's
set_object_from_code
command (meshcat-dev/meshcat#188) by sending a string of native JavaScript/THREE.js code. The code string contains a JavaScript lambda function that creates aTHREE.Object3D
instance; this object is then set atpath
. An example command is:Motivation
Currently, the only way to set objects in Meshcat is via
SetObject()
calls that support a specific set of object types (likeShape
,PointCloud
etc.). By letting the user create objects via native JavaScript/THREE.js code, Drake's Meschat API can support all THREE.js object types.Relation to #20941
#20941 proposes something like
void SetObjectRaw(std::string_view path, std::string_view threejs_object_json)
i.e. passing aTHREE.Object3D
's serialized JSON. In my view the following are the pros/cons of sending THREE.js code instead.Pros
Reflector
object [1, 2] to create a reflective floor, as shown in this demo. However,Reflector
cannot be serialized because it uses ShaderMaterial [3], which in turn uses dynamic textures that are computed at runtime and cannot be serialized. When I try to serialize by callingReflector.toJSON()
, I get the following warning:.toJSON()
function to understand what the serialization format looks like.Cons
TypedArrays
(e.g.Float32Array
). For example, setting aPointCloud
usually requires specifying a largepositions
array. Msgpack can efficiently pack float arrays. On the other hand, embedding the floating values into a string might make the string unnecessarily bulky.In conclusion, this PR might not_resolve #20941, but it can handle most uses cases well. For example, the proposed use case of setting text in Meshcat (#20884) is easily handled by this approach, as demonstrated in the test (see below).
Test example
This PR also adds an example to
//geometry:meshcat_manual_test
that creates a floating text mesh usingMeshcat::SetObjectFromThreeJsCode
, as desired by #20884. On runningwe see
Resolves #20884
This change is