How to show 3D object in a window?

[samsamai]

I've created a basic UI in egui and now I need to have a window where I display 3D objects which the user can pan around, I was wondering what the best way to approach this would be.

So far I've been thinking that I use a linear algebra lib to do the necessary projections on the 3D data and then somehow convert that to triangles for epaint to draw. Or maybe there is an easier way of allowing a 3D lib to use the area within an egui window?

This is a web-base app by the way. Any help would be much appreciated.

Thanks.

[tangmi]

Update: As mentioned by @emilk in another reply, in egui 0.18.1, one can now do 3D drawing directly in egui (and avoid roundtripping the render through CPU memory) with Shape::Calback.

(note: I'm not @emilk, but I'm a fan of both Dear ImGui and egui)

I think the simplest way to show 3D content with the libraries as they exist today:

1. Do an off-screen render using raw WebGL.
2. Read back the pixels to the CPU (using more raw WebGL).
3. Use epi::TextureAllocator (via frame.tex_allocator()) to create and upload an egui user texture.
4. Draw the user texture with ui.texture(...).

If you're like to have egui elements interacting with 3D objects, I imagine an approach similar to CedricGuillemet/ImGuizmo would work (i.e. make a widget that is passed enough information about the 3D scene to draw itself correctly as an overlay).

The Triangles draw command doesn't allow for any kind of custom shading (unless unlit flat shading is your jam) and probably doesn't have great performance characteristics at high triangle counts (you'll have to re-upload the triangles every frame).

Some more options that haven't (yet) been implemented in egui:

• Render to the backbuffer before the egui paint jobs are processed (egui_web doesn't expose a hook for this)
• Let integrations expose an API to directly use raw backend textures (this would let one avoid the round-trip of the texture data through the CPU above)

[samsamai]

Thanks for your response @tangmi, i'm getting a better picture of what I need to do now!

[emilk]

Doing 3D rendering of any kind is way out of scope for the egui project, but of course integrating 3D graphics (or graphics of any kind that is rendered by something other than Egui) is a very important use case to support!

How to integrate with 3D graphics depends on the need and the integration. In some cases you would draw your 3D stuff first and then draw Egui on top of it. This is what people are already doing with other integrations. If you want to show 3D graphics inside an Egui window (as opposed to behind the windows), that I agree with @tangmi that rendering to a texture is the cleanest solution. How to render to texture depends on the integration.

Now for your specific case of targeting web:

Unless you want to write a full 3D renderer yourself, then egui_web is not a great fit. It only have enough rendering implemented to get Egui working, and doing rendering with raw WebGL from Rust is pretty tedious.

However, egui_web is not the only way to use Egui on the web! There is an (outdated) egui integration for miniquad at https://github.com/not-fl3/emigui-miniquad. If that could be brought up to date and improved to the level of egui_web, then you would have a pretty nice rendering library to use to render-to-texture, and then you can add Egui around and on-top of that. And it compiles to the web!

In fact, I think it would be pretty awesome if egui_web could move away from raw WebGL usage and instead build upon miniquad …but I say that not really having tested miniquad myself yet (and don't really have the time at the moment).

[samsamai]

Thank you @emilk for all your work on egui and your points above, I'm not familiar with miniquad, I'll start looking into it. From the discussion so far, the way to go seems to be to use textures, as I imagined the 3D window to be floating above the main 2d drawing area.

[angad-k]

Pretty late to the conversation here but I gather that the best solution at that time was rendering to a texture, copying it back over to the CPU and then using it in egui. But wouldn't this hurt the performance a lot? Does a better solution exist currently? Context : I am working on a project that uses raw WebGL from Rust to render 3d and wanted to use egui in it.

[emilk]

You can now embed 3D graphics in egui using Shape::Callback. See for examples:

https://github.com/emilk/egui/tree/master/examples/custom_3d_glow
https://github.com/emilk/egui/tree/master/examples/custom_3d_three-d

[angad-k]

@emilk is this possible in web too? The simple "Hello Triangle" might work but when I try this for my code, it fails with :

error[E0277]: `*mut u8` cannot be sent between threads safely
   --> src\gui.rs:101:17
    |
101 |         callback: std::sync::Arc::new(move |_info, render_ctx| {
    |  _________________^
102 | |         if let Some(painter) = render_ctx.downcast_ref::<egui_glow::Painter>() {
103 | |           spinning_cube
104 | |             .clone()
...   |
109 | |         }
110 | |       }),
    | |________^ `*mut u8` cannot be sent between threads safely
    |
    = help: within `WebGlUniformLocation`, the trait `Send` is not implemented for `*mut u8`
    = note: required because it appears within the type `PhantomData<*mut u8>`
    = note: required because it appears within the type `wasm_bindgen::JsValue`
    = note: required because it appears within the type `js_sys::Object`
    = note: required because it appears within the type `WebGlUniformLocation`
    = note: required because of the requirements on the impl of `Sync` for `std::sync::Mutex<WebGlUniformLocation>`
    = note: required because of the requirements on the impl of `Send` for `Arc<std::sync::Mutex<WebGlUniformLocation>>`
note: required because it appears within the type `SpinningCube`
   --> src\rendering.rs:10:12
    |
10  | pub struct SpinningCube {
    |            ^^^^^^^^^^^^
    = note: required because of the requirements on the impl of `Sync` for `atomic_refcell::AtomicRefCell<SpinningCube>`
    = note: required because it appears within the type `egui::mutex::Mutex<SpinningCube>`
    = note: required because of the requirements on the impl of `Send` for `Arc<egui::mutex::Mutex<SpinningCube>>`
    = note: required because it appears within the type `[closure@src\gui.rs:101:37: 110:8]`
    = note: required for the cast to the object type `dyn for<'r, 's> Fn(&'r PaintCallbackInfo, &'s mut (dyn Any + 'static)) + Send + Sync`

Also, attaching the struct SpinningCube if it's relevant :

pub struct SpinningCube {
  vertex_position_index: Arc<Mutex<u32>>,
  vertex_color_index: Arc<Mutex<u32>>,
  shader_program: Arc<Mutex<Program>>,
  projection_matrix_index: Arc<Mutex<UniformLocation>>,
  model_view_matrix_index: Arc<Mutex<UniformLocation>>,
  position_buffer: Arc<Mutex<Buffer>>,
  color_buffer: Arc<Mutex<Buffer>>,
  index_buffer: Arc<Mutex<Buffer>>,
  square_rotation: f32,
}

I guess the problem is that the callback function needs the send trait implemented - and for the web, all these things are only working on one thread. So, is there any workaround for this?
Any help's appreciated! Also, should I be opening an issue for this?

EDIT : Nevermind, figured it out with Three-D example provided.

[parker-research]

It seems that the three-d example was removed? Any advice on where to go if we want to use three-d now?

Original 3d example: https://github.com/emilk/egui/tree/master/examples/custom_3d_three-d

[enomado]

According to 636a39c there is no simple path for now?