test: add mock Haply SDK server for integration testing#268
test: add mock Haply SDK server for integration testing#268vickybot911 wants to merge 4 commits intoNVIDIA:mainfrom
Conversation
Add a new plugin for Haply Robotics Inverse3 and VerseGrip devices, bridging the Haply SDK WebSocket interface to the Isaac Teleop framework via OpenXR hand tracking injection. Components: - haply_plugin_core: shared library (libIsaacTeleopPluginsHaply.so) implementing WebSocket client, JSON parsing, OpenXR HandInjector integration, and DeviceIO controller tracking - haply_hand_plugin: plugin executable running at 90Hz - haply_hand_tracker_printer: standalone CLI tool for verifying Haply device data without OpenXR dependencies Architecture: - Background I/O thread communicates with Haply SDK via WebSocket (ws://localhost:10001, configurable via HAPLY_WEBSOCKET_HOST/PORT) - Maps Inverse3 cursor position and VerseGrip orientation to OpenXR XrHandJointLocationEXT format (26 joints) - WRIST and PALM joints receive real tracked data; finger joints are placed at wrist position with VALID but not TRACKED flags - Controller aim pose provides root positioning (same as Manus plugin) - Exponential backoff reconnection on WebSocket errors No build-time SDK dependency — Haply SDK is a runtime service. Vendored: nlohmann/json (MIT, single header) for JSON parsing. Signed-off-by: Vicky <vickybot911@gmail.com>
Add a Python WebSocket server that emulates the Haply Inverse Service, enabling end-to-end testing without physical hardware. Features: - Streams synthetic Inverse3 + VerseGrip data at configurable frequency - Lissajous curve positions, rotating quaternion orientation - Cycling button patterns (3-second intervals) - Accepts and logs force commands - Configurable via CLI: --port, --hz, --handedness, --duration, --verbose Usage: python3 src/plugins/haply/tests/mock_haply_server.py --duration 10 -v Depends on: Haply plugin implementation commit. Signed-off-by: Vicky <vickybot911@gmail.com>
📝 WalkthroughWalkthroughAdds a new Haply hand‑tracking plugin: build configuration, core library with WebSocket + OpenXR integration, an executable app, a debugging tool, Python mock server, docs, and install manifests. Plugin receives Haply SDK messages and injects synthesized hand joint data at ~90 Hz. Changes
Sequence Diagram(s)sequenceDiagram
participant App as haply_hand_plugin
participant Tracker as HaplyTracker
participant IO as I/O Thread
participant WS as HaplyWebSocket
participant SDK as Haply SDK (WebSocket)
participant OpenXR as OpenXR Injector
App->>Tracker: instance()/initialize()
activate Tracker
Tracker->>Tracker: Init OpenXR session & injectors
Tracker->>IO: spawn I/O thread
activate IO
IO->>WS: connect(host,port,path)
WS->>SDK: TCP + WebSocket handshake
SDK-->>WS: upgrade response
WS-->>IO: connected
loop Runtime (~90 Hz)
IO->>SDK: recv_text()
SDK-->>IO: JSON (inverse3, VerseGrip)
IO->>Tracker: update internal state
IO->>SDK: send_text(zero-force)
Tracker->>OpenXR: inject_hand_data()
end
App->>Tracker: shutdown()
Tracker->>IO: stop & join
Tracker->>OpenXR: cleanup
deactivate Tracker
sequenceDiagram
participant Mock as mock_haply_server.py
participant Tool as haply_hand_tracker_printer
Tool->>Tool: read HAPLY_WEBSOCKET_HOST/PORT
Tool->>Mock: connect (WS handshake)
loop streaming
Mock-->>Tool: JSON messages (config + state)
Tool->>Tool: parse & print (periodic)
Tool->>Mock: send_cursor_force (0,0,0)
end
Tool->>Tool: SIGINT/SIGTERM -> close
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
📝 Coding Plan
Comment |
There was a problem hiding this comment.
Actionable comments posted: 8
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/plugins/haply/core/haply_hand_tracking_plugin.cpp`:
- Around line 88-202: The HaplyWebSocket::connect implementation blocks
indefinitely on ::connect and subsequent recv_raw() calls during the handshake;
set socket timeouts after creating fd_ to avoid hangs by applying SO_RCVTIMEO
(and optionally SO_SNDTIMEO) on fd_ before performing ::connect and before the
recv loop, and ensure recv_raw/send_raw handle EWOULDBLOCK/EAGAIN/ETIMEDOUT to
return false on timeout; update HaplyWebSocket::connect to set the timeval
(e.g., 5s) via setsockopt(fd_, SOL_SOCKET, SO_RCVTIMEO, ...) and similarly for
SO_SNDTIMEO or handle non-blocking connect if you prefer that approach,
referencing fd_, send_raw, and recv_raw in your changes.
- Around line 320-326: The code unconditionally allocates std::vector<uint8_t>
data(payload_len) where payload_len is a 64-bit value from the wire; add a
sanity cap check before that allocation (e.g., define a MAX_PAYLOAD_SIZE
constant) and validate payload_len <= MAX_PAYLOAD_SIZE and payload_len fits into
size_t, returning false on violation; then safely cast payload_len to size_t and
proceed to call recv_raw(data.data(), payload_len) in the same block (update the
function or surrounding code that handles payload_len and recv_raw accordingly).
- Around line 409-412: The port parsing needs robust validation: replace the
std::atoi usage for HAPLY_WEBSOCKET_PORT (variables port_env and ws_port) with a
checked parse (e.g., std::from_chars or std::strtol) that detects non-numeric
input and out-of-range values, ensure the parsed value is within 1–65535, and on
parse failure log a warning mentioning the invalid env value and that you'll
fall back to the default 10001 before setting ws_port to the default.
In `@src/plugins/haply/tests/mock_haply_server.py`:
- Line 192: Remove the unnecessary f-string prefixes on the debug print
statements in the mock Haply server tests (e.g., change print(f"[mock-haply]
Starting mock Haply SDK server") and the similar prints at the other occurrences
to plain string prints). Locate the print calls in
src/plugins/haply/tests/mock_haply_server.py that use f"..." with no
interpolation (the messages like "[mock-haply] Starting mock Haply SDK server"
and the other two mentioned) and replace them with print("...") to eliminate the
F541 Ruff warning.
- Around line 87-94: The mock currently builds a buttons dict with numeric
string keys ("0"…"3") using variable button_phase and dict name buttons; change
the emitted keys to match the C++ client schema (use "button_0", "button_1",
"button_2", "button_3") so the printer that reads button_0…button_3 will see the
correct booleans; keep the same boolean expressions (e.g., button_0 =
button_phase == 0 or button_phase == 3) and update whatever
serialization/emission uses buttons to send the renamed keys.
- Around line 145-180: handle_client currently can run faster than hz when
clients reply immediately and crashes for 0<hz<1; validate hz>0 at function
start and replace the ad-hoc sleeps with a monotonic next-frame deadline:
compute period = 1.0/hz, set next_frame = time.monotonic() + period before the
loop, on each iteration send the state then compute recv_timeout = max(0,
next_frame - time.monotonic()) and use that for
asyncio.wait_for(websocket.recv(), timeout=recv_timeout), process the command if
received, then await asyncio.sleep(max(0, next_frame - time.monotonic())) and
advance next_frame += period (bumping forward if behind); also avoid frame_count
% int(hz) failing for hz<1 by computing log_interval = max(1, int(hz)) and use
device.frame_count % log_interval for verbose logging.
In `@src/plugins/haply/tools/CMakeLists.txt`:
- Around line 4-22: The tools executable haply_hand_tracker_printer
(haply_hand_tracker_printer.cpp) uses POSIX-only headers and should be excluded
from Windows builds; locate the add_subdirectory(tools) invocation in
src/plugins/haply/CMakeLists.txt and wrap it with a platform guard (e.g.,
if(UNIX) or if(NOT WIN32)) so the tools folder (and the
haply_hand_tracker_printer target) is only added on non-Windows platforms,
leaving existing target names (haply_hand_tracker_printer) and properties
unchanged.
In `@src/plugins/haply/tools/haply_hand_tracker_printer.cpp`:
- Around line 13-27: The file haply_hand_tracker_printer.cpp uses std::min
(called around the block that computes the clamped/limited value at the
hand-tracker logic) but does not include <algorithm>; add an explicit `#include`
<algorithm> to the top include list so std::min is declared (do not rely on
transitive includes).
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: cd9ff9c4-75f0-4663-992c-b1ef75b65cb3
📒 Files selected for processing (16)
CMakeLists.txtsrc/plugins/haply/.gitignoresrc/plugins/haply/CMakeLists.txtsrc/plugins/haply/CMakePresets.jsonsrc/plugins/haply/CMakePresets.json.licensesrc/plugins/haply/README.mdsrc/plugins/haply/app/CMakeLists.txtsrc/plugins/haply/app/main.cppsrc/plugins/haply/core/CMakeLists.txtsrc/plugins/haply/core/haply_hand_tracking_plugin.cppsrc/plugins/haply/inc/core/haply_hand_tracking_plugin.hppsrc/plugins/haply/plugin.yamlsrc/plugins/haply/tests/mock_haply_server.pysrc/plugins/haply/third_party/nlohmann/json.hppsrc/plugins/haply/tools/CMakeLists.txtsrc/plugins/haply/tools/haply_hand_tracker_printer.cpp
| async def handle_client(websocket, device: MockHaplyDevice, hz: float, verbose: bool): | ||
| """Handle a single WebSocket client connection.""" | ||
| period = 1.0 / hz | ||
| first_message = True | ||
| client_addr = websocket.remote_address | ||
|
|
||
| print(f"[mock-haply] Client connected: {client_addr}") | ||
|
|
||
| try: | ||
| while True: | ||
| # Send device state | ||
| state = device.get_state(first_message=first_message) | ||
| first_message = False | ||
| await websocket.send(json.dumps(state)) | ||
|
|
||
| if verbose and device.frame_count % int(hz) == 0: | ||
| pos = state["inverse3"][0]["state"]["cursor_position"] | ||
| print( | ||
| f"[mock-haply] frame={device.frame_count} " | ||
| f"pos=({pos['x']:.3f}, {pos['y']:.3f}, {pos['z']:.3f}) " | ||
| f"force=({device.last_force['x']:.3f}, {device.last_force['y']:.3f}, {device.last_force['z']:.3f})" | ||
| ) | ||
|
|
||
| # Try to receive a force command (non-blocking with short timeout) | ||
| try: | ||
| raw = await asyncio.wait_for(websocket.recv(), timeout=period * 0.8) | ||
| try: | ||
| cmd = json.loads(raw) | ||
| device.process_command(cmd) | ||
| except json.JSONDecodeError: | ||
| pass | ||
| except asyncio.TimeoutError: | ||
| pass | ||
|
|
||
| # Maintain target frequency | ||
| await asyncio.sleep(period * 0.2) |
There was a problem hiding this comment.
The send loop doesn't actually honor --hz.
If the client replies immediately, Line 170 returns almost at once and the loop only sleeps 0.2 * period at Line 180, so the server can run about 5x faster than requested. The same block also does frame_count % int(hz) at Line 160, which raises for 0 < hz < 1, and hz <= 0 already fails at Line 147. Track an absolute next-frame deadline and validate hz once.
One possible fix
async def handle_client(websocket, device: MockHaplyDevice, hz: float, verbose: bool):
"""Handle a single WebSocket client connection."""
+ if hz <= 0:
+ raise ValueError("hz must be > 0")
period = 1.0 / hz
+ log_every = max(1, int(round(hz)))
+ loop = asyncio.get_running_loop()
+ next_frame_at = loop.time()
first_message = True
client_addr = websocket.remote_address
print(f"[mock-haply] Client connected: {client_addr}")
try:
while True:
+ next_frame_at += period
# Send device state
state = device.get_state(first_message=first_message)
first_message = False
await websocket.send(json.dumps(state))
- if verbose and device.frame_count % int(hz) == 0:
+ if verbose and device.frame_count % log_every == 0:
pos = state["inverse3"][0]["state"]["cursor_position"]
print(
f"[mock-haply] frame={device.frame_count} "
f"pos=({pos['x']:.3f}, {pos['y']:.3f}, {pos['z']:.3f}) "
f"force=({device.last_force['x']:.3f}, {device.last_force['y']:.3f}, {device.last_force['z']:.3f})"
)
# Try to receive a force command (non-blocking with short timeout)
+ remaining = max(0.0, next_frame_at - loop.time())
try:
- raw = await asyncio.wait_for(websocket.recv(), timeout=period * 0.8)
+ raw = await asyncio.wait_for(websocket.recv(), timeout=remaining)
try:
cmd = json.loads(raw)
device.process_command(cmd)
except json.JSONDecodeError:
pass
except asyncio.TimeoutError:
pass
# Maintain target frequency
- await asyncio.sleep(period * 0.2)
+ await asyncio.sleep(max(0.0, next_frame_at - loop.time()))🧰 Tools
🪛 Ruff (0.15.5)
[warning] 145-145: Boolean-typed positional argument in function definition
(FBT001)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/haply/tests/mock_haply_server.py` around lines 145 - 180,
handle_client currently can run faster than hz when clients reply immediately
and crashes for 0<hz<1; validate hz>0 at function start and replace the ad-hoc
sleeps with a monotonic next-frame deadline: compute period = 1.0/hz, set
next_frame = time.monotonic() + period before the loop, on each iteration send
the state then compute recv_timeout = max(0, next_frame - time.monotonic()) and
use that for asyncio.wait_for(websocket.recv(), timeout=recv_timeout), process
the command if received, then await asyncio.sleep(max(0, next_frame -
time.monotonic())) and advance next_frame += period (bumping forward if behind);
also avoid frame_count % int(hz) failing for hz<1 by computing log_interval =
max(1, int(hz)) and use device.frame_count % log_interval for verbose logging.
| add_executable(haply_hand_tracker_printer | ||
| haply_hand_tracker_printer.cpp | ||
| ) | ||
|
|
||
| target_include_directories(haply_hand_tracker_printer | ||
| PRIVATE | ||
| ${CMAKE_CURRENT_SOURCE_DIR}/../third_party | ||
| ) | ||
|
|
||
| target_link_libraries(haply_hand_tracker_printer | ||
| PRIVATE | ||
| Threads::Threads | ||
| ) | ||
|
|
||
| set_target_properties(haply_hand_tracker_printer PROPERTIES | ||
| CXX_VISIBILITY_PRESET hidden | ||
| VISIBILITY_INLINES_HIDDEN YES | ||
| RUNTIME_OUTPUT_DIRECTORY "${CMAKE_BINARY_DIR}/bin" | ||
| ) |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "=== Haply CMake files ==="
fd '^CMakeLists\.txt$' src/plugins/haply -x sh -c 'echo ">>> {}"; sed -n "1,220p" "{}"; echo'
echo "=== Platform guards / target wiring ==="
rg -n -C3 'WIN32|UNIX|APPLE|CMAKE_SYSTEM_NAME|add_subdirectory\(tools\)|haply_hand_tracker_printer' CMakeLists.txt src/plugins/haplyRepository: NVIDIA/IsaacTeleop
Length of output: 9221
🏁 Script executed:
# Check root CMakeLists.txt around line 146 for how haply is included
sed -n '130,160p' CMakeLists.txt
# Also search for haply references in the root file
rg -n 'haply' CMakeLists.txtRepository: NVIDIA/IsaacTeleop
Length of output: 872
🏁 Script executed:
# Check haply_hand_tracker_printer.cpp for POSIX/socket headers
cat -n src/plugins/haply/tools/haply_hand_tracker_printer.cpp | head -50Repository: NVIDIA/IsaacTeleop
Length of output: 1776
Add platform guard to exclude haply_hand_tracker_printer from Windows builds.
haply_hand_tracker_printer.cpp uses POSIX-only APIs (socket headers: arpa/inet.h, netinet/tcp.h, sys/socket.h, netdb.h, signal.h, unistd.h) and will not compile on Windows. The tools subdirectory is added unconditionally in src/plugins/haply/CMakeLists.txt and will cause build failures when BUILD_PLUGINS is enabled on Windows. Wrap the add_subdirectory(tools) call with if(UNIX) or if(NOT WIN32).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/haply/tools/CMakeLists.txt` around lines 4 - 22, The tools
executable haply_hand_tracker_printer (haply_hand_tracker_printer.cpp) uses
POSIX-only headers and should be excluded from Windows builds; locate the
add_subdirectory(tools) invocation in src/plugins/haply/CMakeLists.txt and wrap
it with a platform guard (e.g., if(UNIX) or if(NOT WIN32)) so the tools folder
(and the haply_hand_tracker_printer target) is only added on non-Windows
platforms, leaving existing target names (haply_hand_tracker_printer) and
properties unchanged.
| #include <atomic> | ||
| #include <chrono> | ||
| #include <cstdint> | ||
| #include <cstdlib> | ||
| #include <cstring> | ||
| #include <iomanip> | ||
| #include <iostream> | ||
| #include <mutex> | ||
| #include <netdb.h> | ||
| #include <random> | ||
| #include <signal.h> | ||
| #include <string> | ||
| #include <thread> | ||
| #include <unistd.h> | ||
| #include <vector> |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, locate and check if the file exists
find . -name "haply_hand_tracker_printer.cpp" -type fRepository: NVIDIA/IsaacTeleop
Length of output: 118
🏁 Script executed:
# Read the include block (lines 13-27) to verify it matches the snippet
if [ -f "src/plugins/haply/tools/haply_hand_tracker_printer.cpp" ]; then
echo "=== Lines 13-27 (include block) ==="
sed -n '13,27p' "src/plugins/haply/tools/haply_hand_tracker_printer.cpp"
else
echo "File not found at expected location"
fiRepository: NVIDIA/IsaacTeleop
Length of output: 377
🏁 Script executed:
# Search for all occurrences of std::min in the file
if [ -f "src/plugins/haply/tools/haply_hand_tracker_printer.cpp" ]; then
echo "=== Searching for std::min usage ==="
rg "std::min" "src/plugins/haply/tools/haply_hand_tracker_printer.cpp" -n
fiRepository: NVIDIA/IsaacTeleop
Length of output: 193
🏁 Script executed:
# Check if <algorithm> is included anywhere in the file
if [ -f "src/plugins/haply/tools/haply_hand_tracker_printer.cpp" ]; then
echo "=== Checking for `#include` <algorithm> ==="
rg "#include.*algorithm" "src/plugins/haply/tools/haply_hand_tracker_printer.cpp"
echo "Exit code: $?"
fiRepository: NVIDIA/IsaacTeleop
Length of output: 116
🏁 Script executed:
# Get the full include block at the beginning of the file to confirm context
if [ -f "src/plugins/haply/tools/haply_hand_tracker_printer.cpp" ]; then
echo "=== Full header section (first 30 lines) ==="
head -30 "src/plugins/haply/tools/haply_hand_tracker_printer.cpp"
fiRepository: NVIDIA/IsaacTeleop
Length of output: 900
Include <algorithm> for std::min.
Line 326 calls std::min, but this file does not include <algorithm>. The build currently depends on transitive includes, which is fragile and not guaranteed across toolchains.
Minimal fix
+#include <algorithm>
`#include` <atomic>
`#include` <chrono>📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| #include <atomic> | |
| #include <chrono> | |
| #include <cstdint> | |
| #include <cstdlib> | |
| #include <cstring> | |
| #include <iomanip> | |
| #include <iostream> | |
| #include <mutex> | |
| #include <netdb.h> | |
| #include <random> | |
| #include <signal.h> | |
| #include <string> | |
| #include <thread> | |
| #include <unistd.h> | |
| #include <vector> | |
| `#include` <algorithm> | |
| `#include` <atomic> | |
| `#include` <chrono> | |
| `#include` <cstdint> | |
| `#include` <cstdlib> | |
| `#include` <cstring> | |
| `#include` <iomanip> | |
| `#include` <iostream> | |
| `#include` <mutex> | |
| `#include` <netdb.h> | |
| `#include` <random> | |
| `#include` <signal.h> | |
| `#include` <string> | |
| `#include` <thread> | |
| `#include` <unistd.h> | |
| `#include` <vector> |
🧰 Tools
🪛 Cppcheck (2.20.0)
[information] 13-13: Include file
(missingIncludeSystem)
[information] 14-14: Include file
(missingIncludeSystem)
[information] 15-15: Include file
(missingIncludeSystem)
[information] 16-16: Include file
(missingIncludeSystem)
[information] 17-17: Include file
(missingIncludeSystem)
[information] 18-18: Include file
(missingIncludeSystem)
[information] 19-19: Include file
(missingIncludeSystem)
[information] 20-20: Include file
(missingIncludeSystem)
[information] 21-21: Include file
(missingIncludeSystem)
[information] 22-22: Include file
(missingIncludeSystem)
[information] 23-23: Include file
(missingIncludeSystem)
[information] 24-24: Include file
(missingIncludeSystem)
[information] 25-25: Include file
(missingIncludeSystem)
[information] 26-26: Include file
(missingIncludeSystem)
[information] 27-27: Include file
(missingIncludeSystem)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/haply/tools/haply_hand_tracker_printer.cpp` around lines 13 - 27,
The file haply_hand_tracker_printer.cpp uses std::min (called around the block
that computes the clamped/limited value at the hand-tracker logic) but does not
include <algorithm>; add an explicit `#include` <algorithm> to the top include
list so std::min is declared (do not rely on transitive includes).
- Add WebSocket payload size limit (16 MB) to prevent OOM on malformed frames - Add SO_RCVTIMEO (5s) connect timeout to prevent indefinite blocking - Replace std::atoi with std::stoul + range validation for port parsing - Add signal handler for graceful shutdown in plugin executable - Add TODO comment for unused m_grip_interpolant (future finger synthesis) - Fix British spelling: Synthesised → Synthesized - Apply same payload limit and port validation fixes to printer tool Signed-off-by: Vicky <vickybot911@gmail.com>
- Fix button keys from '0','1','2','3' to 'button_0','button_1','button_2','button_3' to match C++ parser expectations - Fix Hz timing with proper frame pacing using frame_start timestamp - Remove stray f-string prefixes (Ruff F541) Signed-off-by: Vicky <vickybot911@gmail.com>
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (4)
src/plugins/haply/core/haply_hand_tracking_plugin.cpp (2)
419-440:⚠️ Potential issue | 🟠 MajorNormalize the WebSocket env vars and fully validate the port.
src/plugins/haply/README.mdandsrc/plugins/haply/tools/haply_hand_tracker_printer.cppuseHAPLY_WEBSOCKET_HOST/HAPLY_WEBSOCKET_PORT, but the plugin readsHAPLY_WS_HOST/HAPLY_WS_PORThere. Following the docs configures the printer but leaves the plugin on defaults. Also,std::stoulat Line 428 accepts trailing junk, so10001foois silently treated as10001. Please support the documented names (or both aliases) and require full-string consumption when parsing the port.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/plugins/haply/core/haply_hand_tracking_plugin.cpp` around lines 419 - 440, Update the env parsing to accept the documented names and validate the port fully: check for HAPLY_WEBSOCKET_HOST first then fallback to HAPLY_WS_HOST to set ws_host (variables: host_env, ws_host), and check HAPLY_WEBSOCKET_PORT then HAPLY_WS_PORT to get port_env; when parsing the port use std::stoul with the size_t idx parameter and require idx == strlen(port_env) and parsed in 1..65535 before assigning ws_port (variable: ws_port), otherwise log the same error and keep default.
111-125:⚠️ Potential issue | 🟠 Major
SO_RCVTIMEOhere doesn't create a connect timeout.Line 125 only affects later
recv()calls. The blocking::connectat Line 111 can still sit in kernel TCP retry for a long time, which means startup andshutdown()/join()can hang when the target host is unroutable.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/plugins/haply/core/haply_hand_tracking_plugin.cpp` around lines 111 - 125, The current code uses ::connect(...) which can block despite the later SO_RCVTIMEO; change the connect flow to a non-blocking connect with a timeout: set the socket fd_ to non-blocking (use fcntl), call ::connect and if it returns EINPROGRESS use select/poll with the desired timeout (5s), then call getsockopt(fd_, SOL_SOCKET, SO_ERROR, ...) to verify success, restore the socket to blocking mode, and only then proceed to set SO_RCVTIMEO; ensure you still freeaddrinfo(res) and close(fd_) and set fd_ = -1 on any failure. Reference symbols: fd_, ::connect, setsockopt, SO_RCVTIMEO, and freeaddrinfo.src/plugins/haply/tools/haply_hand_tracker_printer.cpp (1)
13-27:⚠️ Potential issue | 🟡 MinorAdd
<algorithm>forstd::min.
std::minis used at Line 353, but this translation unit doesn't include<algorithm>. The build currently depends on transitive includes, which is fragile across toolchains.Minimal fix
+#include <algorithm> `#include` <atomic> `#include` <chrono>🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/plugins/haply/tools/haply_hand_tracker_printer.cpp` around lines 13 - 27, This translation unit uses std::min (around where hand-tracker printing logic computes sizes at the use site in haply_hand_tracker_printer.cpp) but does not include <algorithm>, relying on transitive includes; add `#include` <algorithm> to the top include block (with the other standard headers) so std::min is defined regardless of toolchain; update the include list in haply_hand_tracker_printer.cpp accordingly.src/plugins/haply/tests/mock_haply_server.py (1)
145-163:⚠️ Potential issue | 🟠 Major
--hzbelow 1 still breaks this handler.Line 147 raises on
--hz 0, negative values produce a nonsensical negative period, and Line 162 does% int(hz), which becomes modulo-by-zero for any0 < hz < 1. Validate the input once and compute a nonzerolog_every.Possible fix
async def handle_client(websocket, device: MockHaplyDevice, hz: float, verbose: bool): """Handle a single WebSocket client connection.""" + if hz <= 0: + raise ValueError("hz must be > 0") period = 1.0 / hz + log_every = max(1, int(round(hz))) first_message = True client_addr = websocket.remote_address @@ - if verbose and device.frame_count % int(hz) == 0: + if verbose and device.frame_count % log_every == 0: pos = state["inverse3"][0]["state"]["cursor_position"] print( f"[mock-haply] frame={device.frame_count} " f"pos=({pos['x']:.3f}, {pos['y']:.3f}, {pos['z']:.3f}) " f"force=({device.last_force['x']:.3f}, {device.last_force['y']:.3f}, {device.last_force['z']:.3f})" )Also applies to: 260-261
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/plugins/haply/tests/mock_haply_server.py` around lines 145 - 163, Validate and sanitize the incoming hz in handle_client before computing period or using it in modulo: ensure hz is a positive number (raise or default to 1 for hz <= 0), compute period = 1.0 / hz only after validation, and compute a non-zero log_every = max(1, int(hz)) (or int(round(hz)) wrapped with max(1, ...)) so the verbose check uses device.frame_count % log_every and never divides/modulos by zero; apply the same validation/logic to the other occurrence around the code referenced (lines ~260-261).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@src/plugins/haply/core/haply_hand_tracking_plugin.cpp`:
- Around line 522-539: When the WebSocket drops we must clear the published hand
snapshot so inject_hand_data() doesn't keep emitting the last frame; on both
failure paths (after ws.connect(...) returns false and when ws.recv_text(...)
fails/before breaking out of the while loop) set m_state.has_data = false (and
update any related timestamp/fields if applicable) so the rest of the system
knows there is no current hand data; locate the connection loop that uses
ws.connect, reconnect_delay_ms, ws.recv_text and the m_state struct referenced
by inject_hand_data and add the m_state.has_data=false assignment (ensuring
thread-safety consistent with existing m_state access) before continuing or
breaking.
- Around line 546-625: The code holds m_state_mutex while calling
ws.send_text(cmd.dump()), which can block and stall inject_hand_data(); to fix,
minimize lock scope: while holding m_state_mutex (in the block parsing
inverse3/wireless_verse_grip) copy needed state out (e.g. copy
m_state.inverse3_device_id and any other fields required to build the command)
into local variables, release the lock, then construct the json command (or
construct it up to non-network parts while locked and finish it unlocked) and
call ws.send_text() outside the mutex; update references in this function
(m_state_mutex, m_state.inverse3_device_id, and ws.send_text) accordingly so no
blocking network I/O occurs while m_state_mutex is held.
In `@src/plugins/haply/tools/haply_hand_tracker_printer.cpp`:
- Around line 65-82: The socket connect and subsequent recv_raw calls can block
indefinitely; change the socket setup in the constructor/connector where fd_ is
created and ::connect(fd_, ...) is called to use a non-blocking connect with
select()/poll to wait for writability (or use a timed connect pattern), and
ensure you set SO_RCVTIMEO and SO_SNDTIMEO via setsockopt on fd_ (in addition to
the existing TCP_NODELAY) so read/write syscalls time out; also update the
recv_raw implementation (and the upgrade/frame parsing loops that call it) to
honor a timeout or return on EINTR/EAGAIN so the main loop can handle shutdowns
and retries instead of hanging indefinitely.
---
Duplicate comments:
In `@src/plugins/haply/core/haply_hand_tracking_plugin.cpp`:
- Around line 419-440: Update the env parsing to accept the documented names and
validate the port fully: check for HAPLY_WEBSOCKET_HOST first then fallback to
HAPLY_WS_HOST to set ws_host (variables: host_env, ws_host), and check
HAPLY_WEBSOCKET_PORT then HAPLY_WS_PORT to get port_env; when parsing the port
use std::stoul with the size_t idx parameter and require idx == strlen(port_env)
and parsed in 1..65535 before assigning ws_port (variable: ws_port), otherwise
log the same error and keep default.
- Around line 111-125: The current code uses ::connect(...) which can block
despite the later SO_RCVTIMEO; change the connect flow to a non-blocking connect
with a timeout: set the socket fd_ to non-blocking (use fcntl), call ::connect
and if it returns EINPROGRESS use select/poll with the desired timeout (5s),
then call getsockopt(fd_, SOL_SOCKET, SO_ERROR, ...) to verify success, restore
the socket to blocking mode, and only then proceed to set SO_RCVTIMEO; ensure
you still freeaddrinfo(res) and close(fd_) and set fd_ = -1 on any failure.
Reference symbols: fd_, ::connect, setsockopt, SO_RCVTIMEO, and freeaddrinfo.
In `@src/plugins/haply/tests/mock_haply_server.py`:
- Around line 145-163: Validate and sanitize the incoming hz in handle_client
before computing period or using it in modulo: ensure hz is a positive number
(raise or default to 1 for hz <= 0), compute period = 1.0 / hz only after
validation, and compute a non-zero log_every = max(1, int(hz)) (or
int(round(hz)) wrapped with max(1, ...)) so the verbose check uses
device.frame_count % log_every and never divides/modulos by zero; apply the same
validation/logic to the other occurrence around the code referenced (lines
~260-261).
In `@src/plugins/haply/tools/haply_hand_tracker_printer.cpp`:
- Around line 13-27: This translation unit uses std::min (around where
hand-tracker printing logic computes sizes at the use site in
haply_hand_tracker_printer.cpp) but does not include <algorithm>, relying on
transitive includes; add `#include` <algorithm> to the top include block (with the
other standard headers) so std::min is defined regardless of toolchain; update
the include list in haply_hand_tracker_printer.cpp accordingly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 855ebcfa-112e-4782-8be8-71dc4b33a841
📒 Files selected for processing (5)
src/plugins/haply/README.mdsrc/plugins/haply/app/main.cppsrc/plugins/haply/core/haply_hand_tracking_plugin.cppsrc/plugins/haply/tests/mock_haply_server.pysrc/plugins/haply/tools/haply_hand_tracker_printer.cpp
| if (!ws.connect(host, port, "/")) | ||
| { | ||
| std::cerr << "[HaplyTracker] Connection failed, retrying in " << reconnect_delay_ms << " ms" << std::endl; | ||
| std::this_thread::sleep_for(std::chrono::milliseconds(reconnect_delay_ms)); | ||
| reconnect_delay_ms = std::min(reconnect_delay_ms * 2, max_reconnect_delay_ms); | ||
| continue; | ||
| } | ||
|
|
||
| std::cout << "[HaplyTracker] Connected to Haply SDK" << std::endl; | ||
| reconnect_delay_ms = 500; | ||
|
|
||
| while (m_running.load() && ws.is_connected()) | ||
| { | ||
| std::string msg; | ||
| if (!ws.recv_text(msg)) | ||
| { | ||
| std::cerr << "[HaplyTracker] Connection lost" << std::endl; | ||
| break; |
There was a problem hiding this comment.
Drop published hand data as soon as the socket drops.
On both failure paths here, m_state.has_data keeps its last value. Because inject_hand_data() just snapshots m_state at Lines 642-646, the runtime keeps publishing a frozen hand during reconnect instead of deactivating it.
Possible fix
if (!ws.connect(host, port, "/"))
{
+ {
+ std::lock_guard<std::mutex> lock(m_state_mutex);
+ m_state.has_data = false;
+ }
std::cerr << "[HaplyTracker] Connection failed, retrying in " << reconnect_delay_ms << " ms" << std::endl;
std::this_thread::sleep_for(std::chrono::milliseconds(reconnect_delay_ms));
reconnect_delay_ms = std::min(reconnect_delay_ms * 2, max_reconnect_delay_ms);
continue;
}
@@
if (!ws.recv_text(msg))
{
+ {
+ std::lock_guard<std::mutex> lock(m_state_mutex);
+ m_state.has_data = false;
+ }
std::cerr << "[HaplyTracker] Connection lost" << std::endl;
break;
}Also applies to: 642-646
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/haply/core/haply_hand_tracking_plugin.cpp` around lines 522 -
539, When the WebSocket drops we must clear the published hand snapshot so
inject_hand_data() doesn't keep emitting the last frame; on both failure paths
(after ws.connect(...) returns false and when ws.recv_text(...) fails/before
breaking out of the while loop) set m_state.has_data = false (and update any
related timestamp/fields if applicable) so the rest of the system knows there is
no current hand data; locate the connection loop that uses ws.connect,
reconnect_delay_ms, ws.recv_text and the m_state struct referenced by
inject_hand_data and add the m_state.has_data=false assignment (ensuring
thread-safety consistent with existing m_state access) before continuing or
breaking.
| std::lock_guard<std::mutex> lock(m_state_mutex); | ||
|
|
||
| // Parse inverse3 array | ||
| if (j.contains("inverse3") && j["inverse3"].is_array() && !j["inverse3"].empty()) | ||
| { | ||
| const auto& inv3 = j["inverse3"][0]; | ||
| if (inv3.contains("device_id")) | ||
| { | ||
| m_state.inverse3_device_id = inv3["device_id"].get<std::string>(); | ||
| } | ||
| // Handedness from config (only in first message) | ||
| if (inv3.contains("config") && inv3["config"].contains("handedness")) | ||
| { | ||
| m_state.handedness = inv3["config"]["handedness"].get<std::string>(); | ||
| } | ||
| if (inv3.contains("state")) | ||
| { | ||
| const auto& st = inv3["state"]; | ||
| if (st.contains("cursor_position")) | ||
| { | ||
| const auto& pos = st["cursor_position"]; | ||
| m_state.cursor_position.x = pos.value("x", 0.0f); | ||
| m_state.cursor_position.y = pos.value("y", 0.0f); | ||
| m_state.cursor_position.z = pos.value("z", 0.0f); | ||
| } | ||
| if (st.contains("cursor_velocity")) | ||
| { | ||
| const auto& vel = st["cursor_velocity"]; | ||
| m_state.cursor_velocity.x = vel.value("x", 0.0f); | ||
| m_state.cursor_velocity.y = vel.value("y", 0.0f); | ||
| m_state.cursor_velocity.z = vel.value("z", 0.0f); | ||
| } | ||
| } | ||
| m_state.has_data = true; | ||
| } | ||
|
|
||
| // Parse wireless_verse_grip array | ||
| if (j.contains("wireless_verse_grip") && j["wireless_verse_grip"].is_array() && | ||
| !j["wireless_verse_grip"].empty()) | ||
| { | ||
| const auto& vg = j["wireless_verse_grip"][0]; | ||
| if (vg.contains("device_id")) | ||
| { | ||
| m_state.versegrip_device_id = vg["device_id"].get<std::string>(); | ||
| } | ||
| if (vg.contains("state")) | ||
| { | ||
| const auto& st = vg["state"]; | ||
| if (st.contains("orientation")) | ||
| { | ||
| const auto& ori = st["orientation"]; | ||
| m_state.orientation.w = ori.value("w", 1.0f); | ||
| m_state.orientation.x = ori.value("x", 0.0f); | ||
| m_state.orientation.y = ori.value("y", 0.0f); | ||
| m_state.orientation.z = ori.value("z", 0.0f); | ||
| } | ||
| if (st.contains("buttons")) | ||
| { | ||
| const auto& btn = st["buttons"]; | ||
| m_state.buttons.button_0 = btn.value("button_0", false); | ||
| m_state.buttons.button_1 = btn.value("button_1", false); | ||
| m_state.buttons.button_2 = btn.value("button_2", false); | ||
| m_state.buttons.button_3 = btn.value("button_3", false); | ||
| } | ||
| } | ||
| m_state.has_data = true; | ||
| } | ||
|
|
||
| // Send a command to keep receiving updates (set zero force) | ||
| if (!m_state.inverse3_device_id.empty()) | ||
| { | ||
| json cmd; | ||
| cmd["inverse3"] = json::array(); | ||
| json dev; | ||
| dev["device_id"] = m_state.inverse3_device_id; | ||
| dev["commands"]["set_cursor_force"]["values"]["x"] = 0; | ||
| dev["commands"]["set_cursor_force"]["values"]["y"] = 0; | ||
| dev["commands"]["set_cursor_force"]["values"]["z"] = 0; | ||
| cmd["inverse3"].push_back(dev); | ||
| ws.send_text(cmd.dump()); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
# First, let's locate the file and understand its context
find . -name "haply_hand_tracking_plugin.cpp" -type fRepository: NVIDIA/IsaacTeleop
Length of output: 117
🏁 Script executed:
# Read the file to understand the lock scope
wc -l ./src/plugins/haply/core/haply_hand_tracking_plugin.cppRepository: NVIDIA/IsaacTeleop
Length of output: 121
🏁 Script executed:
# Read the critical section to verify mutex scope
sed -n '540,650p' ./src/plugins/haply/core/haply_hand_tracking_plugin.cpp | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 5705
🏁 Script executed:
# Check lines 644-645 mentioned in the review
sed -n '630,650p' ./src/plugins/haply/core/haply_hand_tracking_plugin.cpp | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 779
🏁 Script executed:
# Read more of inject_hand_data() to understand the full context
sed -n '640,700p' ./src/plugins/haply/core/haply_hand_tracking_plugin.cpp | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 2743
🏁 Script executed:
# Search for all ws.send_text() calls and lock_guard uses with m_state_mutex
echo "=== All ws.send_text() calls ===" && rg 'ws\.send_text' ./src/plugins/haply/core/haply_hand_tracking_plugin.cpp -n
echo ""
echo "=== All lock_guard with m_state_mutex ===" && rg 'lock_guard.*m_state_mutex' ./src/plugins/haply/core/haply_hand_tracking_plugin.cpp -nRepository: NVIDIA/IsaacTeleop
Length of output: 374
🏁 Script executed:
# Check the context around line 392
sed -n '385,400p' ./src/plugins/haply/core/haply_hand_tracking_plugin.cpp | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 441
🏁 Script executed:
# Find where 'ws' is declared and what type it is
rg 'websocket|SimpleWeb|ws\s*=' ./src/plugins/haply/core/haply_hand_tracking_plugin.cpp -n -B 2 -A 2Repository: NVIDIA/IsaacTeleop
Length of output: 349
🏁 Script executed:
# Look for the function signature containing the ws.send_text() call
sed -n '500,550p' ./src/plugins/haply/core/haply_hand_tracking_plugin.cpp | cat -nRepository: NVIDIA/IsaacTeleop
Length of output: 1939
🏁 Script executed:
# Find HaplyWebSocket definition and send_text() implementation
find . -name "*.h" -o -name "*.hpp" | xargs grep -l "class HaplyWebSocket" 2>/dev/nullRepository: NVIDIA/IsaacTeleop
Length of output: 121
🏁 Script executed:
# Find send_text implementation in HaplyWebSocket
rg 'send_text|class HaplyWebSocket' ./src/plugins/haply/inc/core/haply_hand_tracking_plugin.hpp -A 10 -B 2Repository: NVIDIA/IsaacTeleop
Length of output: 977
🏁 Script executed:
# Find the send_text implementation
find . -name "haply_websocket.cpp" -o -name "haply_websocket.cc" 2>/dev/nullRepository: NVIDIA/IsaacTeleop
Length of output: 44
🏁 Script executed:
# Search for send_text implementation in cpp files
rg 'bool.*send_text' . --type cpp -A 20Repository: NVIDIA/IsaacTeleop
Length of output: 5359
🏁 Script executed:
# Find send_raw and send_frame implementations
rg 'bool.*send_raw|bool.*send_frame' ./src/plugins/haply/core/haply_hand_tracking_plugin.cpp -A 15Repository: NVIDIA/IsaacTeleop
Length of output: 846
Don't hold m_state_mutex across ws.send_text().
The lock taken at line 546 is held until after the network send at line 625. Since send_raw() uses blocking ::send() with no timeout, a slow peer can pin the mutex, stalling the 90 Hz update path in inject_hand_data() which also needs m_state_mutex.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/haply/core/haply_hand_tracking_plugin.cpp` around lines 546 -
625, The code holds m_state_mutex while calling ws.send_text(cmd.dump()), which
can block and stall inject_hand_data(); to fix, minimize lock scope: while
holding m_state_mutex (in the block parsing inverse3/wireless_verse_grip) copy
needed state out (e.g. copy m_state.inverse3_device_id and any other fields
required to build the command) into local variables, release the lock, then
construct the json command (or construct it up to non-network parts while locked
and finish it unlocked) and call ws.send_text() outside the mutex; update
references in this function (m_state_mutex, m_state.inverse3_device_id, and
ws.send_text) accordingly so no blocking network I/O occurs while m_state_mutex
is held.
| fd_ = ::socket(res->ai_family, res->ai_socktype, res->ai_protocol); | ||
| if (fd_ < 0) | ||
| { | ||
| freeaddrinfo(res); | ||
| return false; | ||
| } | ||
|
|
||
| if (::connect(fd_, res->ai_addr, res->ai_addrlen) < 0) | ||
| { | ||
| freeaddrinfo(res); | ||
| ::close(fd_); | ||
| fd_ = -1; | ||
| return false; | ||
| } | ||
| freeaddrinfo(res); | ||
|
|
||
| int flag = 1; | ||
| setsockopt(fd_, IPPROTO_TCP, TCP_NODELAY, &flag, sizeof(flag)); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
wc -l src/plugins/haply/tools/haply_hand_tracker_printer.cppRepository: NVIDIA/IsaacTeleop
Length of output: 120
🏁 Script executed:
cat -n src/plugins/haply/tools/haply_hand_tracker_printer.cpp | sed -n '60,130p'Repository: NVIDIA/IsaacTeleop
Length of output: 2865
🏁 Script executed:
rg -n "setsockopt|fcntl|SO_RCVTIMEO|SO_SNDTIMEO|setblocking|select|poll|timeout" src/plugins/haply/tools/haply_hand_tracker_printer.cppRepository: NVIDIA/IsaacTeleop
Length of output: 136
🏁 Script executed:
rg -n "recv_raw|send_raw" src/plugins/haply/tools/haply_hand_tracker_printer.cpp -A 10Repository: NVIDIA/IsaacTeleop
Length of output: 3325
🏁 Script executed:
rg -n "class MiniWebSocket|struct MiniWebSocket" src/plugins/haply/tools/haply_hand_tracker_printer.cpp -A 30Repository: NVIDIA/IsaacTeleop
Length of output: 948
🏁 Script executed:
rg -n "void close\(\)|bool close\(\)" src/plugins/haply/tools/haply_hand_tracker_printer.cpp -A 10Repository: NVIDIA/IsaacTeleop
Length of output: 297
Socket operations lack timeout mechanisms and can hang the tool indefinitely.
The blocking ::connect() at line 72 will hang if an unroutable host accepts the connection without responding. Additionally, all recv_raw() calls throughout the code (upgrade loop at line 124, frame parsing at lines 159, 168, 175, 182, 193) block indefinitely with no timeout. If a peer accepts the connection but never sends data, the tool will hang until the OS/network timeout expires, ignoring shutdown signals.
Consider using non-blocking sockets with select() or poll(), or set socket timeouts via setsockopt() with SO_RCVTIMEO and SO_SNDTIMEO.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@src/plugins/haply/tools/haply_hand_tracker_printer.cpp` around lines 65 - 82,
The socket connect and subsequent recv_raw calls can block indefinitely; change
the socket setup in the constructor/connector where fd_ is created and
::connect(fd_, ...) is called to use a non-blocking connect with select()/poll
to wait for writability (or use a timed connect pattern), and ensure you set
SO_RCVTIMEO and SO_SNDTIMEO via setsockopt on fd_ (in addition to the existing
TCP_NODELAY) so read/write syscalls time out; also update the recv_raw
implementation (and the upgrade/frame parsing loops that call it) to honor a
timeout or return on EINTR/EAGAIN so the main loop can handle shutdowns and
retries instead of hanging indefinitely.
Summary
Add a mock Haply SDK WebSocket server for integration testing the Haply plugin without physical hardware.
Mock Server
src/plugins/haply/tests/mock_haply_server.pyemulates the Haply Inverse Service:--port,--hz,--handedness,--duration,--verboseUsage
Dependencies
websocketspackageDepends on #267 (Haply plugin implementation).
Summary by CodeRabbit
New Features
Documentation
Tests