problem about hand estimation

[wishinger1219-lang]

Thank you very much for your excellent work. When running the code, I encountered a phenomenon: when the input image only shows one hand, or the other hand is not clearly visible, the algorithm still estimates it as if there are two hands, causing inaccurate hand estimation. Is this phenomenon known? Or are there any methods to avoid or mitigate this phenomenon?

[yangtiming]

Yes, this is expected — SAM 3D Body always predicts both hands by design, including the pose of the hand that is not visible.

If you encounter inaccurate hand predictions, you can try tuning the parameters in run_demo.sh:

Primary adjustments:

• Layer Tuning: While fewer layers increase speed, our testing shows that export HAND_INTERM_PRED_LAYERS=0,1 (0-5) provides the best balance between performance and accuracy.
• Hand Box Scaling: Adjust the hand_box_scale factor, which controls the size of the hand bounding box cropped for the model. Please refer to the visualizations in the paper appendix (Fig. 10, 11 Hand bbox) for guidance.

Secondary adjustments:

• export MHR_NO_CORRECTIVES=1
• export SKIP_KEYPOINT_PROMPT=1

If the hand prediction is still inaccurate after tuning, it may also be worth checking whether SAM 3D Body itself struggles with the overall pose estimation for that particular image.

[wishinger1219-lang]

OK, I will try this.thanks.