macOS: enable realsense-viewer (ImGui GLSL 120) + relocatable install RPATH#15393
macOS: enable realsense-viewer (ImGui GLSL 120) + relocatable install RPATH#15393keskinonur wants to merge 3 commits into
Conversation
- tools: drop the stale if(NOT APPLE) around realsense-viewer — it builds and runs on Apple Silicon; the actual breakage was ImGui GLSL: windows are created without version hints so macOS provides a legacy 2.1 compatibility context (needed for the viewer's immediate-mode GL), while the backend's Apple default shader version is 150 (3.2 core only). Pass "#version 120" on Apple in ux-window and rosbag-inspector; the vendored backend has the <130 shader path. Verified live: viewer runs from the install prefix without DYLD_LIBRARY_PATH and attaches to a D555 over DDS (control/notification/metadata readers matched). - python: bind rs2_combined_motion.orientation (quaternion) — previously only angular_velocity/linear_acceleration were exposed. - rs_frame.hpp: get_motion_data() now throws on COMBINED_MOTION frames instead of silently reinterpreting the double payload as floats and returning zeros; use get_combined_motion_data().
A viewer run in a session without WindowServer access (headless/remote exec) persists window.width/height as 0x0 to the config. Every later launch then restores a zero-size window: the process runs and renders but nothing is visible (only a Dock/menubar entry). Only restore a saved size when it is plausibly visible (>=320x240); smaller values fall back to the default window size. Verified on macOS: after the guard the D555 viewer window appears and streams depth/color over DDS.
- CMake: stop CMAKE_INSTALL_RPATH_USE_LINK_PATH from baking absolute build-tree paths into installed GUI tools; tools_target_config now sets INSTALL_RPATH=@loader_path;@loader_path/../lib on Apple+DDS. - viewer: clear sticky GL errors before 3D draw; guard gluPerspective aspect; force GLSL off on Apple so the fixed-function pointcloud path runs (GLSL 1.30 / float FBOs need core 3.2, incompatible with the legacy 2.1 context used for ImGui). - pc-shader: fix texcoord-before-vertex order on the non-GLSL path and enable texturing; corrects a silent off-by-one on point UV indices.
sysrsbuild-gh-agentic
left a comment
There was a problem hiding this comment.
Automated Code Review by rs-agentic bot
This PR enables realsense-viewer and GUI tools on Apple Silicon / macOS. The root cause diagnosis (ImGui OpenGL3 backend defaulting to #version 150 on a GL 2.1 legacy context) is correct, and the approach is sound. Several issues noted below require attention before merging.
Summary of findings:
- BUG:
glEnable(GL_TEXTURE_2D)is added inpc-shader.cppbut never matched withglDisable(GL_TEXTURE_2D)after draw calls -- can corrupt subsequent render state. - BUG: The exception in
rs_frame.hppreferencesget_combined_motion_data()but it is not clear this method is publicly exposed; verify it exists and is documented. - Enhancement: GLSL flags are force-disabled on Apple via
set()(not justset_default()), permanently preventing future macOS users from enabling GLSL even on compatible hardware. - Enhancement: Magic numbers
320/240in window size guard should be named constants. - Enhancement:
double-to-floatcast in the Python orientation getter loses precision silently. - Note:
CMAKE_BUILD_RPATHset globally to only@loader_pathinunix_config.cmakemay be too restrictive for non-tool targets during build. - Note:
BUILD_RPATHintools_common.cmakeincludes absolute build-tree paths, which contradicts the comment above it (though technically harmless since it only affects build-tree binaries, not installed ones). - Missing tests: No unit/integration tests for the new Python
combined_motion.orientationbinding.
|
|
||
| // Fixed-function path (macOS legacy 2.1, or when GLSL is off). | ||
| // Texcoords must be set *before* the vertex they apply to. | ||
| glEnable(GL_TEXTURE_2D); |
There was a problem hiding this comment.
[rs-agentic bot] BUG: Missing glDisable(GL_TEXTURE_2D) after draw calls
glEnable(GL_TEXTURE_2D) is added here to enable texturing for the fixed-function path, but there is no corresponding glDisable(GL_TEXTURE_2D) call after the glEnd() blocks (for both the QUADS and POINTS paths). This leaves texture state globally enabled.
On macOS (and any path where GLSL is off), subsequent rendering passes that do not bind a valid texture will sample garbage, causing visual artifacts or black geometry. Add glDisable(GL_TEXTURE_2D) after each glEnd() block that follows this glEnable, or pair the enable/disable around the entire fill/points conditional block.
glEnable(GL_TEXTURE_2D);
// ... QUADS and POINTS draw calls ...
glDisable(GL_TEXTURE_2D); // <-- add this| { | ||
| // On combined-motion frames the payload is rs2_combined_motion (doubles); | ||
| // reading it as three floats silently yields zeros/garbage. | ||
| if( get_profile().format() == RS2_FORMAT_COMBINED_MOTION ) |
There was a problem hiding this comment.
[rs-agentic bot] BUG + Breaking API Change: Verify get_combined_motion_data() exists and document the breaking change
Two concerns here:
-
Missing alternative method: The exception message tells users to call
get_combined_motion_data(), but this method does not appear in the public C++ API ofmotion_framevisible in this diff. If it does not exist, this exception leaves users with no valid code path at all. Please verify the method is present and accessible (or add it in this PR). -
Breaking API change: Previously,
get_motion_data()on aCOMBINED_MOTIONframe silently returned garbage floats. Now it throwsstd::logic_error. Any existing user code that calledget_motion_data()on combined frames will now crash at runtime. This is a breaking change and should be explicitly called out in the PR description and ideally inCHANGELOG.mdor release notes.
| config_file::instance().set_default(configurations::performance::glsl_for_rendering, false); | ||
| // Force-clear any previously saved GLSL-on flags (e.g. config from Linux | ||
| // or an older build). GLSL 1.30 shaders + FBO float textures need a core | ||
| // 3.2 context which we cannot combine with the fixed-function 2D path. |
There was a problem hiding this comment.
[rs-agentic bot] Enhancement: Force-disabling GLSL prevents future user override
Using config_file::instance().set(...) (not just set_default(...)) permanently overwrites any user-configured GLSL preference on Apple. If a future macOS version or a user's system does support a proper OpenGL 3.2 core context (e.g., via explicit GLFW hints), they will have no way to enable GLSL without modifying source code.
Consider instead detecting the actual OpenGL context version at runtime, e.g.:
#ifdef __APPLE__
// Force GLSL off: macOS legacy 2.1 context cannot support GLSL 1.30 shaders.
// Only override saved config, not a runtime capability check.
const int gl_major = ...; // glGetIntegerv(GL_MAJOR_VERSION, ...)
if (gl_major < 3) {
config_file::instance().set(configurations::performance::glsl_for_processing, false);
config_file::instance().set(configurations::performance::glsl_for_rendering, false);
}
#endifOr at minimum add a comment explaining that this is intentionally permanent until the codebase supports core 3.2 contexts on Apple.
| // A run in a headless/session-less context can persist degenerate | ||
| // metrics (e.g. 0x0), leaving every later launch with an invisible | ||
| // window — only restore a plausibly visible size. | ||
| if (w >= 320 && h >= 240) |
There was a problem hiding this comment.
[rs-agentic bot] Enhancement: Magic numbers 320/240 should be named constants
The minimum window size thresholds 320 and 240 are magic numbers. Per the project's code conventions, these should be defined as named constants to improve readability and maintainability:
constexpr int MIN_PLAUSIBLE_WINDOW_WIDTH = 320;
constexpr int MIN_PLAUSIBLE_WINDOW_HEIGHT = 240;
if (w >= MIN_PLAUSIBLE_WINDOW_WIDTH && h >= MIN_PLAUSIBLE_WINDOW_HEIGHT)This also makes it easier to find and adjust the threshold if the minimum supported display resolution changes.
| # never bake absolute build-tree paths into installed binaries (CMake's | ||
| # INSTALL_RPATH_USE_LINK_PATH would add e.g. …/build/Release). | ||
| if(APPLE) | ||
| set(CMAKE_BUILD_RPATH "@loader_path") |
There was a problem hiding this comment.
[rs-agentic bot] Note: Global CMAKE_BUILD_RPATH "@loader_path" may be too narrow
Setting CMAKE_BUILD_RPATH globally to only "@loader_path" means all Apple targets (not just tools) will have only the loader's directory in the build-tree RPATH. Libraries built into subdirectories of ${CMAKE_BINARY_DIR} (e.g. ${CMAKE_BINARY_DIR}/Release/, ${CMAKE_BINARY_DIR}/lib/) may not be found at link time during the build.
tools_common.cmake overrides this per-target with absolute build paths for tools, but other target types (examples, unit tests, Python bindings) that do not go through tools_target_config will only have @loader_path. Please verify that examples (BUILD_EXAMPLES) and Python bindings (BUILD_PYTHON_BINDINGS) still build and run correctly from the build tree on Apple with just @loader_path.
| # Apple: install tree is bin/ + lib/; never bake absolute build-tree paths. | ||
| if(APPLE) | ||
| set_target_properties(${_target} PROPERTIES | ||
| BUILD_RPATH "@loader_path;${CMAKE_BINARY_DIR}/Release;${CMAKE_BINARY_DIR}" |
There was a problem hiding this comment.
[rs-agentic bot] Note: BUILD_RPATH contains absolute build-tree paths (comment is misleading)
The comment above says "never bake absolute build-tree paths", but BUILD_RPATH here explicitly includes ${CMAKE_BINARY_DIR}/Release and ${CMAKE_BINARY_DIR} -- both absolute paths. While BUILD_RPATH only affects binaries in the build tree (not installed binaries), the comment is misleading.
Please either:
- Clarify the comment to distinguish
BUILD_RPATH(build-tree only, absolute paths acceptable) fromINSTALL_RPATH(installed binaries, must be relocatable), or - Remove the absolute paths from
BUILD_RPATHif they are not needed (since@loader_pathplus CMake's own link path resolution should suffice during build).
As written, otool -L on a build-tree binary will show both @loader_path AND the absolute build paths -- not a bug, but it is inconsistent with the stated goal.
| "orientation", | ||
| []( rs2_combined_motion const & self ) | ||
| { | ||
| return rs2_quaternion{ (float)self.orientation.x, |
There was a problem hiding this comment.
[rs-agentic bot] Enhancement: Double-to-float precision loss in orientation getter
rs2_combined_motion::orientation fields are stored as double (full quaternion double precision for IMU fusion accuracy), but this getter casts them to float via rs2_quaternion. This silently loses ~7 decimal digits of precision.
For orientation data used in sensor fusion or dead reckoning, this loss can accumulate over time. Consider one of:
- Expose the orientation as a Python tuple of doubles directly:
return py::make_tuple(self.orientation.x, self.orientation.y, self.orientation.z, self.orientation.w); - Or at minimum document the precision loss in a Python docstring so users are aware.
The setter path (float -> double) has no precision loss and is fine.
|
[rs-agentic bot] Missing Tests No automated tests are added in this PR for any of the changed areas. In particular:
|
|
@keskinonur With sudo (needed for libusb USB access), it crashes at startup — root can't reach the WindowServer: Cocoa: Failed to find service port for display → map::at: key not found. |
Description
Enable realsense-viewer (and related GUI tools) on Apple Silicon / macOS, where the build was previously gated with
if(NOT APPLE).Root cause of the historical breakage was not the viewer itself, but ImGui's OpenGL3 backend defaulting to
#version 150(GL 3.2 core) while GLFW windows are created without version hints, so macOS provides a legacy 2.1 compatibility context. Pass#version 120on Apple so shaders match the context.Also:
0×0after a headless/session-less run) so later launches are not invisible.@loader_path/@loader_path/../liband never bake absolute build-tree paths into installed tools.Motivation
Verified on macOS arm64 with an Ethernet RealSense D555 (DDS): viewer launches from an install prefix without
DYLD_LIBRARY_PATH, enumerates the device, streams depth/color/IR, and exposes Motion Module.rs-depth-qualityandrs-rosbag-inspectorbuild and install alongside.Changes
tools/CMakeLists.txtif(NOT APPLE)aroundrealsense-viewercommon/ux-window.cpp#version 120; force GLSL flags off; ignore <320×240 saved sizetools/rosbag-inspector/…CMake/unix_config.cmake+tools/tools_common.cmakesrc/gl/pc-shader.cppcommon/viewer.cppgluPerspectivecombined_motion.orientation; throw ifget_motion_data()used on combined framesTesting
BUILD_GRAPHICAL_EXAMPLES=ONrealsense-viewerruns from install prefix;otoolRPATH is only@loader_path+@loader_path/../lib(no build-tree absolute path)Notes
SO_REUSEPORTwork (Set SO_REUSEPORT on multicast input sockets for Apple platforms eProsima/Fast-DDS#6471).