HMD Optical Hand tracking and MANUS automatic installation#235
Open
timvw-manusvr wants to merge 7 commits intoNVIDIA:mainfrom
Open
HMD Optical Hand tracking and MANUS automatic installation#235timvw-manusvr wants to merge 7 commits intoNVIDIA:mainfrom
timvw-manusvr wants to merge 7 commits intoNVIDIA:mainfrom
Conversation
added 3 commits
March 5, 2026 10:03
Added installation bash script HMD hand tracking is now used when available to position the hands
📝 WalkthroughWalkthroughAdds XR_MNDX_xdev_space optical hand-tracking with controller fallback, refactors Manus plugin initialization/injection and logging, and introduces automated MANUS SDK and gRPC dependency installers plus README, docs, launcher, and tooling updates. Changes
Sequence Diagram(s)sequenceDiagram
participant OpenXR as OpenXR Session
participant XDev as XR_MNDX_XDev (Optical)
participant Controller as Controller Input
participant Plugin as Manus Plugin
participant Injector as Hand Injector
Plugin->>OpenXR: query extensions & create session
OpenXR-->>Plugin: extension availability
Plugin->>XDev: enumerate/create XDev trackers
alt XDev trackers available
XDev-->>Plugin: joint data & wrist pose (update_xdev_hand)
Plugin->>Injector: inject optical wrist + joint data
else XDev unavailable
Controller-->>Plugin: controller wrist pose (get_controller_wrist_pose)
Plugin->>Injector: inject controller-derived wrist + joint data
end
Injector-->>Plugin: injection complete
Plugin->>XDev: cleanup_xdev_hand_trackers()
XDev-->>Plugin: resources released
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Pull request overview
This PR adds support for using the HMD's built-in optical hand tracking (via the XR_MNDX_xdev_space extension) as the wrist position source when using Manus gloves, alongside two new automated installation scripts and expanded documentation.
Changes:
- Adds
initialize_xdev_hand_trackers()/cleanup_xdev_hand_trackers()/update_xdev_hand()/get_controller_wrist_pose()to the plugin, with automatic fallback from HMD optical tracking to controller-based wrist positioning. - Introduces two new shell scripts (
install_manus.sh,install-dependencies.sh) for end-to-end automated installation including SDK download, dependency setup (with optional gRPC), and plugin build. - Updates
README.mdwith automated installation instructions, CloudXR setup steps, a new section on the two wrist-tracking modes, and an expanded troubleshooting guide.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
src/plugins/manus/core/manus_hand_tracking_plugin.cpp |
Adds XDev/hand tracking initialization, wrist-source auto-selection, and simplifies injector lifecycle |
src/plugins/manus/inc/core/manus_hand_tracking_plugin.hpp |
Adds new private methods, XDev handle/function-pointer members, and HandTracker member |
src/plugins/manus/tools/manus_hand_tracker_printer.cpp |
Adds [Manus] prefix to all console output lines |
src/plugins/manus/install_manus.sh |
New all-in-one installation and build script |
src/plugins/manus/install-dependencies.sh |
New script for building gRPC v1.28.1 and system dependencies |
src/plugins/manus/README.md |
Expanded with automated install, CloudXR setup, tracking modes, and troubleshooting |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 9
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp`:
- Around line 99-100: The printed diagnostic incorrectly says "Z-up" while the
coordinate system is actually configured with up = AxisPolarity_PositiveY;
update the log string printed before calling
CoreSdk_InitializeCoordinateSystemWithVUH(t_VUH, true) (the std::cout line) to
reflect "Y-up, right-handed, meters" so the diagnostic matches the SDK
configuration (referencing AxisPolarity_PositiveY and the call to
CoreSdk_InitializeCoordinateSystemWithVUH).
- Around line 507-514: The code currently treats a wrist pose as "valid" in
update_xdev_hand by checking XR_SPACE_LOCATION_POSITION_VALID_BIT and
XR_SPACE_LOCATION_ORIENTATION_VALID_BIT but does not ensure the pose is actively
tracked; change the logic in update_xdev_hand where wrist is read (using
joint_locations[XR_HAND_JOINT_WRIST_EXT] and wrist.locationFlags) to also check
XR_SPACE_LOCATION_POSITION_TRACKED_BIT and
XR_SPACE_LOCATION_ORIENTATION_TRACKED_BIT before returning success/setting
out_wrist_pose, or alternatively return/propagate both a valid and a tracked
boolean so inject_hand_data (which uses is_root_tracked) can add tracked bits
only when the wrist is actually tracked; update references to is_root_tracked
accordingly so last-known-but-untracked poses are not advertised as fully
tracked.
- Around line 116-129: The code currently unconditionally adds
XR_EXT_HAND_TRACKING_EXTENSION_NAME (via including core::HandTracker in
trackers) and XR_MNDX_XDEV_SPACE_EXTENSION_NAME to extensions before
constructing the OpenXR session, which causes xrCreateInstance() to fail if the
runtime lacks those optional extensions; modify the flow to query the runtime's
supported extensions (or probe via core::OpenXRSession/DeviceIOSession helper)
before pushing optional names: only include XR_EXT_HAND_TRACKING_EXTENSION_NAME
if the runtime reports it supported (or defer adding core::HandTracker to
trackers until after support is confirmed), and only push
XR_MNDX_XDEV_SPACE_EXTENSION_NAME if supported; construct m_session =
std::make_shared<core::OpenXRSession>(app_name, extensions) after gating
optional extensions and then call initialize_xdev_hand_trackers()/create hand
trackers as needed so controller fallback remains possible.
In `@src/plugins/manus/install_manus.sh`:
- Around line 53-64: The apt install block in install_manus.sh currently omits a
downloader and unzip and doesn't validate downloads; update the apt-get install
list (the sudo apt-get install -y ... block) to include a downloader (curl or
wget) and unzip, change the SDK fetch command (the curl/wget invocation in the
SDK download/extract step around lines referenced) to use a failing download
mode (curl -fL or wget --server-response -O) so HTTP errors fail, save to a
deterministic filename, then compute and verify the archive checksum (e.g.
sha256sum) against a known expected value before extracting, and only run
unzip/tar after the checksum matches; if verification fails, abort with a clear
error.
- Around line 181-187: The script currently builds only the manus_hand_plugin
target but README expects the manus_hand_tracker_printer binary; update the
build/install steps to include the printer binary: adjust the cmake --build
invocation (or add a second cmake --build call) to also build the
manus_hand_tracker_printer target and ensure the corresponding binary/component
is installed (e.g., include its name alongside manus or call cmake --install for
that component). Locate the build/install lines referencing targets
manus_hand_plugin and component manus and add the manus_hand_tracker_printer
target/component so the documented ./build/bin/manus_hand_tracker_printer exists
after the script runs.
- Around line 173-179: The script currently skips CMake configuration when
build/CMakeCache.txt exists, which leaves stale discovery results after copying
the ManusSDK; change the logic so CMake is forced to reconfigure after the SDK
copy by removing or invalidating build/CMakeCache.txt (or always running the
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release command) instead of conditionally
skipping it; update the block that checks "build/CMakeCache.txt" and ensure the
cmake invocation is executed unconditionally (or after deleting the cache) so
the project is deterministically reconfigured.
In `@src/plugins/manus/install-dependencies.sh`:
- Around line 14-22: The script currently writes "deb
http://deb.debian.org/debian bookworm main" directly into
/etc/apt/sources.list.d/bookworm-gcc.list which can allow unrelated Bookworm
packages to be installed; instead, add an apt pin/priority file that pins only
the specific packages you need (e.g., gcc-11, g++-11, libgcc-11) with a high
priority for the target release, or use an isolated toolchain source (PPA or apt
repository that provides only GCC packages). Update the Debian branch of the
install step (the block that runs apt-get install -y debian-archive-keyring and
echoes the repo) to also create a /etc/apt/preferences.d/bookworm-gcc.pref entry
that pins the listed package names to release "n=bookworm" with a high
Pin-Priority, and ensure apt-get update is called before installing the pinned
packages so only those specific packages are allowed from Bookworm rather than
the whole main archive.
- Around line 38-44: The existing check that skips cloning when the grpc repo
directory exists can leave a stale or dirty checkout; change the else branch to
normalize the existing checkout: fetch all tags and branches, checkout the
pinned tag v1.28.1, hard-reset the worktree to that tag, remove untracked files,
and ensure submodules are synchronized and updated/initialized recursively so
the repo is deterministic before continuing with build steps that expect
v1.28.1.
In `@src/plugins/manus/README.md`:
- Around line 44-46: Update the MANUS version references in the README: change
both URLs and any text that point to 3.1.0 to 3.1.1 (specifically replace
https://docs.manus-meta.com/3.1.0/Resources/ and
https://docs.manus-meta.com/3.1.0/Plugins/SDK/Linux/ with the 3.1.1 equivalents)
so the manual-install instructions match the automated installer; ensure the
mention of the ManusSDK folder and MANUS_SDK_ROOT remains the same but reflects
the 3.1.1 archive name where applicable.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: c7bcbfd3-5e8e-4006-9d8b-2a78c775fabb
📒 Files selected for processing (6)
src/plugins/manus/README.mdsrc/plugins/manus/core/manus_hand_tracking_plugin.cppsrc/plugins/manus/inc/core/manus_hand_tracking_plugin.hppsrc/plugins/manus/install-dependencies.shsrc/plugins/manus/install_manus.shsrc/plugins/manus/tools/manus_hand_tracker_printer.cpp
jiwenc-nv
reviewed
Mar 11, 2026
| @@ -15,16 +15,35 @@ This folder provides a Linux-only example of using the Manus SDK for hand tracki | |||
|
|
|||
| ## Prerequisites | |||
Collaborator
There was a problem hiding this comment.
@timvw-manusvr we are moving towards consolidating all docs into the /docs folder. Once it got merged, it automatically deploys to: https://nvidia.github.io/IsaacTeleop/
Can you convert this to rst and put it under docs/source/device/manus.rst?
Collaborator
There was a problem hiding this comment.
btw, since you already have the content here, AI can help me convert it to rst very well.
There was a problem hiding this comment.
Actionable comments posted: 5
♻️ Duplicate comments (1)
src/plugins/manus/install_manus.sh (1)
53-64:⚠️ Potential issue | 🟡 MinorInstall
unzipin the minimal dependency path.The minimal dependency installation doesn't include
unzip, but the SDK extraction step (line 113) requires it and will fail with an error if it's not present.🔧 Proposed fix
sudo apt-get install -y \ build-essential \ cmake \ git \ libssl-dev \ zlib1g-dev \ libc-ares-dev \ libzmq3-dev \ libncurses-dev \ libudev-dev \ - libusb-1.0-0-dev + libusb-1.0-0-dev \ + unzip🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/plugins/manus/install_manus.sh` around lines 53 - 64, The minimal dependency apt-get install block in install_manus.sh is missing unzip, which the later SDK extraction step requires; update the apt-get install command (the sudo apt-get install -y ... block) to include unzip alongside the other packages so the SDK extraction step succeeds without error.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp`:
- Around line 471-522: The code currently hardcodes the expected serials "Head
Device (0)" and "Head Device (1)" when matching xdevs (see left_xdev_id,
right_xdev_id and the comparison against properties.serial obtained via
XrGetXDevInfoMNDX); make these strings configurable by reading environment
variables or a config (e.g. MANUS_LEFT_XDEV_SERIAL / MANUS_RIGHT_XDEV_SERIAL)
with fallbacks to the current defaults, use those configured values for the
comparisons instead of the literals, and update the diagnostic message that
prints seen_serials to include the actual expected strings being used so
operators know what was attempted.
In `@src/plugins/manus/install_manus.sh`:
- Line 27: Change the interactive prompt so the shell read invocation that fills
INSTALL_CHOICE uses the -r flag to avoid backslash mangling; in other words,
update the read command (the read that prompts "Enter your choice (1 or 2): "
and sets INSTALL_CHOICE) to include the -r option so input is read raw without
interpreting backslashes.
In `@src/plugins/manus/install-dependencies.sh`:
- Around line 130-139: Replace the multiple echo append lines that write udev
rules into /etc/udev/rules.d/70-manus-hid.rules with a single heredoc to improve
readability and atomicity: in the install-dependencies.sh script, locate the if
block that checks for /etc/udev/rules.d/70-manus-hid.rules and the series of
echo "..." >> /etc/udev/rules.d/70-manus-hid.rules lines, and change them to
create the directory if needed and write the full rules content with a
here-document (e.g., cat <<'EOF' > /etc/udev/rules.d/70-manus-hid.rules ... EOF)
so all rules for SUBSYSTEMS=="usb" and KERNEL=="hidraw*" are written in one
operation.
- Line 54: In the install-dependencies.sh script update the make invocations
that use unquoted command substitution to prevent word splitting: replace
instances of make -j$(nproc) (and the chained commands `make -j$(nproc) && make
-j$(nproc) check && make install && ldconfig`) with versions that quote the
substitution (i.e., use "$(nproc)"); apply the same change to the other
occurrences flagged on lines 85 and 109 so all make -j$(nproc) uses are changed
to make -j"$(nproc)" throughout the script.
In `@src/plugins/manus/README.md`:
- Line 65: Update the README.md markdown formatting: change the ordered list
prefix that begins with "Build from the TeleopCore root:" to use a `1.` prefix
instead of `4.`, add a blank line after all headings (for example after headings
near "Build from the TeleopCore root:" and the later section headings
mentioned), remove the trailing whitespace on the line containing the stray
space (the lone trailing-space line reported), and ensure fenced code blocks
(the blocks around the commands shown later in the README) have a blank line
before and after the ``` fences so they are separated from surrounding text.
---
Duplicate comments:
In `@src/plugins/manus/install_manus.sh`:
- Around line 53-64: The minimal dependency apt-get install block in
install_manus.sh is missing unzip, which the later SDK extraction step requires;
update the apt-get install command (the sudo apt-get install -y ... block) to
include unzip alongside the other packages so the SDK extraction step succeeds
without error.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 7681132b-821c-447c-8bd2-7ca6a732ef5c
📒 Files selected for processing (5)
scripts/run_isaac_lab.shsrc/plugins/manus/README.mdsrc/plugins/manus/core/manus_hand_tracking_plugin.cppsrc/plugins/manus/install-dependencies.shsrc/plugins/manus/install_manus.sh
Comment on lines
+471
to
+522
| // Find native hand tracking devices by matching against their serial strings. | ||
| // | ||
| // NOTE: The serial values "Head Device (0)" (left) and "Head Device (1)" (right) are | ||
| // NOT defined by the XR_MNDX_xdev_space specification. They are an observed runtime- | ||
| // specific naming convention (e.g. Monado). If a runtime changes these display names | ||
| // across firmware or software updates the match below will silently fail. | ||
| // See: https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html (XR_MNDX_xdev_space) | ||
| XrXDevIdMNDX left_xdev_id = 0; | ||
| XrXDevIdMNDX right_xdev_id = 0; | ||
| std::vector<std::string> seen_serials; | ||
|
|
||
| for (const auto& xdev_id : xdev_ids) | ||
| { | ||
| XrGetXDevInfoMNDX get_info{ XR_TYPE_GET_XDEV_INFO_MNDX }; | ||
| get_info.id = xdev_id; | ||
|
|
||
| XrXDevPropertiesMNDX properties{ XR_TYPE_XDEV_PROPERTIES_MNDX }; | ||
| result = m_pfn_get_xdev_properties(m_xdev_list, &get_info, &properties); | ||
| if (XR_FAILED(result)) | ||
| { | ||
| continue; | ||
| } | ||
|
|
||
| std::string serial_str = properties.serial ? properties.serial : ""; | ||
| seen_serials.push_back(serial_str); | ||
|
|
||
| if (serial_str == "Head Device (0)") | ||
| { | ||
| left_xdev_id = xdev_id; | ||
| } | ||
| else if (serial_str == "Head Device (1)") | ||
| { | ||
| right_xdev_id = xdev_id; | ||
| } | ||
| } | ||
|
|
||
| if (left_xdev_id == 0 || right_xdev_id == 0) | ||
| { | ||
| std::string serials_list; | ||
| for (const auto& s : seen_serials) | ||
| { | ||
| if (!serials_list.empty()) serials_list += ", "; | ||
| serials_list += '"'; | ||
| serials_list += s; | ||
| serials_list += '"'; | ||
| } | ||
| std::cerr << "[Manus] Could not match optical hand-tracking XDevs by serial. " | ||
| << "Expected \"Head Device (0)\" (left) and \"Head Device (1)\" (right), " | ||
| << "but found: [" << serials_list << "]. " | ||
| << "These serial strings are runtime-specific and may have changed." | ||
| << std::endl; | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider making XDev serial strings configurable.
The hardcoded serial strings "Head Device (0)" and "Head Device (1)" are documented as runtime-specific. While the diagnostic logging is helpful, consider making these configurable via environment variables or a config file for easier adaptation to different runtimes.
This is a forward-looking suggestion—the current implementation with good diagnostics is acceptable for an initial version.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/core/manus_hand_tracking_plugin.cpp` around lines 471 -
522, The code currently hardcodes the expected serials "Head Device (0)" and
"Head Device (1)" when matching xdevs (see left_xdev_id, right_xdev_id and the
comparison against properties.serial obtained via XrGetXDevInfoMNDX); make these
strings configurable by reading environment variables or a config (e.g.
MANUS_LEFT_XDEV_SERIAL / MANUS_RIGHT_XDEV_SERIAL) with fallbacks to the current
defaults, use those configured values for the comparisons instead of the
literals, and update the diagnostic message that prints seen_serials to include
the actual expected strings being used so operators know what was attempted.
| echo " 1) Install MANUS Core Integrated dependencies only (faster)" | ||
| echo " 2) Install both MANUS Core Integrated and Remote dependencies (includes gRPC, takes longer)" | ||
| echo "" | ||
| read -p "Enter your choice (1 or 2): " INSTALL_CHOICE |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Use read -r to prevent backslash mangling.
Static analysis flagged that read without -r will interpret backslashes. While unlikely to matter for single-digit input, it's best practice.
🔧 Proposed fix
-read -p "Enter your choice (1 or 2): " INSTALL_CHOICE
+read -rp "Enter your choice (1 or 2): " INSTALL_CHOICE📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| read -p "Enter your choice (1 or 2): " INSTALL_CHOICE | |
| read -rp "Enter your choice (1 or 2): " INSTALL_CHOICE |
🧰 Tools
🪛 Shellcheck (0.11.0)
[info] 27-27: read without -r will mangle backslashes.
(SC2162)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/install_manus.sh` at line 27, Change the interactive prompt
so the shell read invocation that fills INSTALL_CHOICE uses the -r flag to avoid
backslash mangling; in other words, update the read command (the read that
prompts "Enter your choice (1 or 2): " and sets INSTALL_CHOICE) to include the
-r option so input is read raw without interpreting backslashes.
| echo "[3/7] Building protobuf for x86_64..." && \ | ||
| cd /var/local/git/grpc/third_party/protobuf && \ | ||
| ./autogen.sh && ./configure --enable-shared && \ | ||
| make -j$(nproc) && make -j$(nproc) check && make install && ldconfig |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Quote command substitution to prevent word splitting.
Static analysis flagged unquoted $(nproc). While unlikely to cause issues here, quoting is best practice.
🔧 Proposed fix
- make -j$(nproc) && make -j$(nproc) check && make install && ldconfig
+ make -j"$(nproc)" && make -j"$(nproc)" check && make install && ldconfigAlso applies to lines 85 and 109.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| make -j$(nproc) && make -j$(nproc) check && make install && ldconfig | |
| make -j"$(nproc)" && make -j"$(nproc)" check && make install && ldconfig |
🧰 Tools
🪛 Shellcheck (0.11.0)
[warning] 54-54: Quote this to prevent word splitting.
(SC2046)
[warning] 54-54: Quote this to prevent word splitting.
(SC2046)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/install-dependencies.sh` at line 54, In the
install-dependencies.sh script update the make invocations that use unquoted
command substitution to prevent word splitting: replace instances of make
-j$(nproc) (and the chained commands `make -j$(nproc) && make -j$(nproc) check
&& make install && ldconfig`) with versions that quote the substitution (i.e.,
use "$(nproc)"); apply the same change to the other occurrences flagged on lines
85 and 109 so all make -j$(nproc) uses are changed to make -j"$(nproc)"
throughout the script.
Comment on lines
+130
to
+139
| if [ ! -f "/etc/udev/rules.d/70-manus-hid.rules" ]; then | ||
| mkdir -p /etc/udev/rules.d/ | ||
| touch /etc/udev/rules.d/70-manus-hid.rules | ||
| echo "[Setup] Adding read/write permissions for manus devices..." | ||
| echo "# HIDAPI/libusb" >> /etc/udev/rules.d/70-manus-hid.rules | ||
| echo "SUBSYSTEMS==\"usb\", ATTRS{idVendor}==\"3325\", MODE:=\"0666\"" >> /etc/udev/rules.d/70-manus-hid.rules | ||
| echo "SUBSYSTEMS==\"usb\", ATTRS{idVendor}==\"1915\", ATTRS{idProduct}==\"83fd\", MODE:=\"0666\"" >> /etc/udev/rules.d/70-manus-hid.rules | ||
| echo "# HIDAPI/hidraw" >> /etc/udev/rules.d/70-manus-hid.rules | ||
| echo "KERNEL==\"hidraw*\", ATTRS{idVendor}==\"3325\", MODE:=\"0666\"" >> /etc/udev/rules.d/70-manus-hid.rules | ||
| fi |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider using a heredoc for udev rules.
Multiple echo statements with individual redirects can be consolidated for readability.
♻️ Proposed refactor
- mkdir -p /etc/udev/rules.d/
- touch /etc/udev/rules.d/70-manus-hid.rules
- echo "[Setup] Adding read/write permissions for manus devices..."
- echo "# HIDAPI/libusb" >> /etc/udev/rules.d/70-manus-hid.rules
- echo "SUBSYSTEMS==\"usb\", ATTRS{idVendor}==\"3325\", MODE:=\"0666\"" >> /etc/udev/rules.d/70-manus-hid.rules
- echo "SUBSYSTEMS==\"usb\", ATTRS{idVendor}==\"1915\", ATTRS{idProduct}==\"83fd\", MODE:=\"0666\"" >> /etc/udev/rules.d/70-manus-hid.rules
- echo "# HIDAPI/hidraw" >> /etc/udev/rules.d/70-manus-hid.rules
- echo "KERNEL==\"hidraw*\", ATTRS{idVendor}==\"3325\", MODE:=\"0666\"" >> /etc/udev/rules.d/70-manus-hid.rules
+ mkdir -p /etc/udev/rules.d/
+ echo "[Setup] Adding read/write permissions for manus devices..."
+ cat > /etc/udev/rules.d/70-manus-hid.rules << 'EOF'
+# HIDAPI/libusb
+SUBSYSTEMS=="usb", ATTRS{idVendor}=="3325", MODE:="0666"
+SUBSYSTEMS=="usb", ATTRS{idVendor}=="1915", ATTRS{idProduct}=="83fd", MODE:="0666"
+# HIDAPI/hidraw
+KERNEL=="hidraw*", ATTRS{idVendor}=="3325", MODE:="0666"
+EOF🧰 Tools
🪛 Shellcheck (0.11.0)
[style] 134-134: Consider using { cmd1; cmd2; } >> file instead of individual redirects.
(SC2129)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/install-dependencies.sh` around lines 130 - 139, Replace
the multiple echo append lines that write udev rules into
/etc/udev/rules.d/70-manus-hid.rules with a single heredoc to improve
readability and atomicity: in the install-dependencies.sh script, locate the if
block that checks for /etc/udev/rules.d/70-manus-hid.rules and the series of
echo "..." >> /etc/udev/rules.d/70-manus-hid.rules lines, and change them to
create the directory if needed and write the full rules content with a
here-document (e.g., cat <<'EOF' > /etc/udev/rules.d/70-manus-hid.rules ... EOF)
so all rules for SUBSYSTEMS=="usb" and KERNEL=="hidraw*" are written in one
operation.
| ``` | ||
|
|
||
| ## Build | ||
| 4. Build from the TeleopCore root: |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Fix markdown formatting issues.
Several markdown lint warnings were detected:
- Line 65: Ordered list should use
1.prefix (currently4.) - Lines 76, 92, 100, 107: Add blank line after headings
- Line 111: Remove trailing space
- Lines 122-127: Add blank lines around fenced code blocks
📝 Example fix for list prefix and heading spacing
-4. Build from the TeleopCore root:
+1. Build from the TeleopCore root:
-### 1. Setup CloudXR Environment
-Before running the plugin, ensure CloudXR environment is configured:
+### 1. Setup CloudXR Environment
+
+Before running the plugin, ensure CloudXR environment is configured:🧰 Tools
🪛 markdownlint-cli2 (0.21.0)
[warning] 65-65: Ordered list item prefix
Expected: 1; Actual: 4; Style: 1/1/1
(MD029, ol-prefix)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/manus/README.md` at line 65, Update the README.md markdown
formatting: change the ordered list prefix that begins with "Build from the
TeleopCore root:" to use a `1.` prefix instead of `4.`, add a blank line after
all headings (for example after headings near "Build from the TeleopCore root:"
and the later section headings mentioned), remove the trailing whitespace on the
line containing the stray space (the lone trailing-space line reported), and
ensure fenced code blocks (the blocks around the commands shown later in the
README) have a blank line before and after the ``` fences so they are separated
from surrounding text.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
src/plugins/manus/install_manus.sh (1)
188-188: 🧹 Nitpick | 🔵 TrivialQuote
$(nproc)to prevent word splitting.Static analysis flagged unquoted command substitution. While
nprocreturns a single number, quoting is best practice for defensive scripting.🔧 Proposed fix
-cmake --build build --target manus_hand_plugin manus_hand_tracker_printer -j$(nproc) +cmake --build build --target manus_hand_plugin manus_hand_tracker_printer -j"$(nproc)"🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/plugins/manus/install_manus.sh` at line 188, The cmake build invocation uses unquoted command substitution for nproc in the -j flag which can cause word splitting; update the cmake command in install_manus.sh (the line calling cmake --build build --target manus_hand_plugin manus_hand_tracker_printer -j$(nproc)) to quote the substitution so the -j argument uses "$(nproc)" to prevent potential word splitting.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@docs/source/device/manus.rst`:
- Around line 98-104: Update the "Running the Plugin" section to consistently
document CloudXR environment setup by mentioning both options: either source the
provided script (scripts/setup_cloudxr_env.sh) or set the variables manually;
specifically reference the environment variables NV_CXR_RUNTIME_DIR and
XR_RUNTIME_JSON and show the export commands as an alternative, and make the
troubleshooting note (which currently references scripts/setup_cloudxr_env.sh)
match this wording so readers know they can either run the script or export
NV_CXR_RUNTIME_DIR and XR_RUNTIME_JSON themselves.
---
Duplicate comments:
In `@src/plugins/manus/install_manus.sh`:
- Line 188: The cmake build invocation uses unquoted command substitution for
nproc in the -j flag which can cause word splitting; update the cmake command in
install_manus.sh (the line calling cmake --build build --target
manus_hand_plugin manus_hand_tracker_printer -j$(nproc)) to quote the
substitution so the -j argument uses "$(nproc)" to prevent potential word
splitting.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: e1fb9eab-1ae6-4238-8db8-2d4f16a11bdd
📒 Files selected for processing (7)
docs/source/device/index.rstdocs/source/device/manus.rstsrc/plugins/manus/README.mdsrc/plugins/manus/core/manus_hand_tracking_plugin.cppsrc/plugins/manus/inc/core/manus_hand_tracking_plugin.hppsrc/plugins/manus/install-dependencies.shsrc/plugins/manus/install_manus.sh
Comment on lines
+98
to
+104
| Source the CloudXR environment and start the runtime before running the plugin: | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| export NV_CXR_RUNTIME_DIR=~/.cloudxr/run | ||
| export XR_RUNTIME_JSON=~/.cloudxr/openxr_cloudxr.json | ||
|
|
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Minor inconsistency in CloudXR environment setup references.
The "Running the Plugin" section (lines 98-104) shows manual export commands for CloudXR environment variables, but the troubleshooting section (line 161) references scripts/setup_cloudxr_env.sh. Consider clarifying whether users should source the script or set the variables manually, or document both approaches consistently.
Also applies to: 160-161
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@docs/source/device/manus.rst` around lines 98 - 104, Update the "Running the
Plugin" section to consistently document CloudXR environment setup by mentioning
both options: either source the provided script (scripts/setup_cloudxr_env.sh)
or set the variables manually; specifically reference the environment variables
NV_CXR_RUNTIME_DIR and XR_RUNTIME_JSON and show the export commands as an
alternative, and make the troubleshooting note (which currently references
scripts/setup_cloudxr_env.sh) match this wording so readers know they can either
run the script or export NV_CXR_RUNTIME_DIR and XR_RUNTIME_JSON themselves.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HMD Optical Hand Tracking
Summary
This PR adds support for using the HMD's built-in optical hand tracking (via the
XR_MNDX_xdev_spaceextension) as the wrist position source when using Manus gloves, along with automated installation scripts and improved documentation.Changes
HMD optical hand tracking wrist source (
XR_MNDX_xdev_space)XR_MNDX_xdev_spaceandXR_EXT_hand_trackingextension function pointers. If both are available, it creates two nativeXrHandTrackerEXThandles backed by the HMD's optical tracking ("Head Device (0)" = left, "Head Device (1)" = right).inject_hand_data()now auto-selects the wrist pose source: HMD optical tracking when available (m_xdev_available == true), falling back silently to controller aim-pose + wrist offset when not.[Manus] Initialized with wrist source: HandTrackingorControllers.initialize_xdev_hand_trackers(),cleanup_xdev_hand_trackers(),update_xdev_hand(),get_controller_wrist_pose().m_handlesis now stored as a member so it can be reused across the new helper methods without fetching it repeatedly.HandTrackeris now registered alongsideControllerTrackerin theDeviceIOSession.XR_MNDX_xdev_spaceis requested as a required extension at session creation time.std::coutlog messages prefixed with[Manus]for consistent filtering.manus_hand_tracker_printer.cpp
[Manus]to match the plugin log format.install_manus.sh (new file)
All-in-one installation script:
install-dependencies.sh (new file)
Full dependency installer for the Remote mode:
abseil-cpppatch required for GCC 11+ compatibility.ldconfig.README.md
install_manus.sh.setup_cloudxr_env.sh,run_cloudxr.sh) as a prerequisite to running the plugin.Summary by CodeRabbit
New Features
New Utilities
Documentation
Improvements