Skip to content

Fix multiple bugs in Ops.Devices.WebXr.Vr.VrHand#8102

Closed
andrejhronco wants to merge 2 commits intocables-gl:developfrom
andrejhronco:webxr-vr-vrhand
Closed

Fix multiple bugs in Ops.Devices.WebXr.Vr.VrHand#8102
andrejhronco wants to merge 2 commits intocables-gl:developfrom
andrejhronco:webxr-vr-vrhand

Conversation

@andrejhronco
Copy link
Copy Markdown

Summary

  • Fix crashes when specific joints are not yet tracked — null guard added before accessing joint positions for rays and pinch detection
  • Guard against null gripSpace on hand-tracking inputs (grip space is a controller concept and may not be present)
  • Clear stale outputs (outBones, outIndexRay, outWristRay, outGp) when hand is lost, preventing stale data in the patch
  • Reset wasClicked state when hand is lost to prevent the next pinch being silently swallowed
  • Add xrFrame null guard alongside the existing xrSession check
  • Pre-allocate bone buffer as Float32Array to avoid per-frame GC pressure (was allocating a new array + pushing every frame)
  • Replace dist3d (with Math.sqrt) with a squared distance comparison — threshold updated from 0.01 to 0.0001 (= 0.01²)

Test plan

  • Hand tracking works normally when all joints are detected
  • No crash when hand is partially tracked (some joints missing)
  • Outputs clear correctly when hand leaves tracking volume
  • Pinch (click) detection fires correctly and resets after hand is lost and reappears
  • Bone lines render correctly via the Bones output array

- fix crashes when specific joints not tracked (null guard before ray/pinch access)
- guard against null gripSpace on hand-tracking inputs
- clear stale outputs (bones, rays, hand object) when hand is lost
- reset wasClicked state when hand is lost to prevent swallowed pinches
- add xrFrame null guard alongside xrSession check
- pre-allocate bone buffer as Float32Array to avoid per-frame GC pressure
- replace dist3d with squared comparison to avoid Math.sqrt
andrejhronco

This comment was marked as off-topic.

Copy link
Copy Markdown
Author

@andrejhronco andrejhronco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review of changes

Line 78 — xrFrame null guard
Added && cgl.tempData.xrFrame alongside the existing xrSession check. getJointPose is called inside this block; if xrFrame is null (e.g. during session teardown) it would throw.

Lines 102–122 — Joint position null guards before ray outputs
getJointPose can return null for any individual joint — partial tracking is common at the edges of the sensor field of view. The original code accessed jointPositions["index-finger-tip"][0] etc. unconditionally, causing a runtime crash whenever a joint was missing. Each ray output is now only written when both required joints are present.

Lines 124–138 — Null guard before pinch distance check
Same issue: dist3dSq would receive undefined if thumb-tip or index-finger-tip were not tracked that frame.

Lines 65–72 — dist3dSq replaces dist3d
The pinch check only compares the result to a constant threshold. Math.sqrt is unnecessary — comparing squared distance to 0.0001 (= 0.01²) is equivalent and avoids the square root entirely.

Lines 152–168 — gripSpace null guard
Hand-tracking inputs (XRHand) do not always have a gripSpace — that is a controller concept. Calling getPose(null, ...) throws. The position/transform outputs are only set when grip space is available.

Lines 74, 142–160 — Pre-allocated Float32Array bone buffer
The original code allocated a new JS array and pushed 150 values into it every frame. At 90–120 Hz in VR, this creates per-frame GC pressure. boneBuffer is allocated once at module level (bones.length * 6 floats = 150 entries); each frame only the valid slice is passed via subarray(0, boneCount), so partially-tracked hands still produce correct output.

Lines 184–191 — Clear stale outputs and reset wasClicked when hand is lost
Previously, outBones, outIndexRay, and outWristRay were never cleared when tracking was lost — downstream ops would keep reading data from the last known frame. wasClicked also stayed true if the hand disappeared mid-pinch, causing the next pinch to be silently swallowed.

}

function dist3d(a, b)
function dist3dSq(a, b)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to dist3dSq — the pinch check only needs to compare against a constant threshold, so Math.sqrt is unnecessary. Threshold updated from 0.01 to 0.0001 (= 0.01²).

return xd * xd + yd * yd + zd * zd;
}

const boneBuffer = new Float32Array(bones.length * 6);
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-allocated once at module level instead of allocating a new array every frame. At 90–120 Hz in VR this was creating ~150 allocations/second of GC pressure. Each frame writes into this buffer and passes only the valid slice via subarray(0, boneCount).

let found = false;

if (op.patch.cgl.tempData.xrSession)
if (op.patch.cgl.tempData.xrSession && cgl.tempData.xrFrame)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added && cgl.tempData.xrFrame guard. The Vr op sets xrFrame inside the animation frame callback — if this op fires during session teardown xrFrame can be null, causing getJointPose to throw.

@@ -99,77 +101,94 @@ inUpdate.onTriggered = () =>

/// ///////////////////
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getJointPose returns null for joints outside the sensor's field of view. The original code accessed jointPositions["index-finger-tip"][0] unconditionally, crashing whenever a joint was not tracked. Only written when both required joints are present.

jointPositions["index-finger-tip"][2]
]);

if (jointPositions.wrist && jointPositions["index-finger-phalanx-proximal"])
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same null-guard pattern for the wrist ray — wrist and index-finger-phalanx-proximal may both be missing during partial tracking.

jointPositions["index-finger-phalanx-proximal"][2]
]);

if (jointPositions["thumb-tip"] && jointPositions["index-finger-tip"])
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Null guard before passing to dist3dSq. If either thumb-tip or index-finger-tip was not tracked, the original dist3d call would receive undefined and return NaN.


let controlPose = cgl.tempData.xrFrame.getPose(inputSources[i].gripSpace, cgl.tempData.xrReferenceSpace);
if (controlPose && controlPose.transform)
if (inputSources[i].gripSpace)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

XRHand inputs do not always have a gripSpace — that is a controller concept. Calling getPose(null, referenceSpace) throws a TypeError. Position/transform outputs are only set when grip space is present.

}
}
if (!found) outGp.setRef(null);
if (!found)
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When tracking is lost, stale data from the last tracked frame was left in all outputs. Downstream ops reading outBones or the ray arrays would continue processing invalid positions. wasClicked is also reset here — if the hand disappeared mid-pinch it would stay true, silently swallowing the next pinch gesture.

@pandrr pandrr closed this Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants