Improve spacemouse by squarefk · Pull Request #561 · ARISE-Initiative/robosuite

squarefk · 2024-11-15T16:24:13Z

This PR adds support for bimanual teleoperation using the existing SpaceMouse class.
When two space mice are connected, the system will automatically switch to bimanual mode.

To test the functionality without setting up an environment, users can connect either one or two space mice and run python -m robosuite.devices.spacemouse.

snasiriany

Thanks Yu! Left some comments!

snasiriany · 2024-11-15T23:23:48Z

        ac_dict[f"{active_arm}_delta"] = arm_action["delta"]
-        ac_dict[f"{active_arm}_gripper"] = np.array([grasp] * gripper_dof)
+        # ac_dict[f"{active_arm}_gripper"] = np.array([grasp] * gripper_dof)
+        ac_dict[f"{active_arm}_gripper"] = [1, 0.45 * grasp, grasp, grasp, grasp, grasp]


what does this part change?

0.45 * grasp

this seems specific to the Inspire hand. We should make it a case by case basis
Also not sure if it works for non-humanoids

Agreed. This would impact other robots. Moved the changes into class InspireLeftHand and InspireRightHand

snasiriany · 2024-11-15T23:24:54Z

+            "robot0_torso_waist_yaw": 30.0,
+            "robot0_torso_waist_pitch": 100.0,
+            "robot0_torso_waist_roll": 100.0,


these changes are making the robot less agile. for example reaching up inside cabinets or farther back in the counter, etc. can we try to make use of the waist only when really needed? I think this part just needs more tuning, perhaps along with the overall gains of the controller.

Parameters further tuned.

snasiriany · 2024-11-15T23:33:24Z



-class SpaceMouse(Device):
+class SpaceMouseSdkWrapper:


maybe call this SpaceMouseListener instead

snasiriany · 2024-11-15T23:40:19Z

            <inertial diaginertia="0 0 0" mass="0" pos="0 0 0"/>
            <body name="eef" pos="0 0 0" quat="0.707 0. -0.707 0.">
-                <site name="grip_site" pos="0 0 0" size="0.01 0.01 0.01" rgba="1 1 0 1" type="sphere" group="2"/>
+                <site name="grip_site" pos="0 0 0" size="0.01 0.01 0.01" rgba="1 1 0 0" type="sphere" group="2"/>


reason for change?

It has been reverted.

snasiriany · 2024-11-15T23:40:24Z

                <site name="ee_z" pos="0 0 0.1" size="0.005 .1" quat="1 0 0 0" rgba="0 0 1 0" type="cylinder" group="1"/>
                <!-- This site was added for visualization. -->
-                <site name="grip_site_cylinder" pos="0 0 0" quat="-0.5 -0.5 -0.5 0.5" size="0.005 0.5" rgba="0 1 0 0.3" type="cylinder" group="1"/>
+                <site name="grip_site_cylinder" pos="0 0 0" quat="-0.5 -0.5 -0.5 0.5" size="0.005 0.5" rgba="0 1 0 0" type="cylinder" group="1"/>


reason for change?

Make the long ray invisible.

snasiriany · 2024-11-15T23:41:17Z

can we add some red sites for when each of the other 3 fingers are closed? in a lighter shade of red than the current red color marker of the hand

snasiriany · 2024-11-16T00:19:32Z

@squarefk also I wanted to press S and start moving the other arm (at the start). but if I do it at the start the other arm does not move. can we fix this?

squarefk · 2024-11-22T02:33:58Z

also I wanted to press S and start moving the other arm (at the start). but if I do it at the start the other arm does not move. can we fix this?

Repost the solution here:

In the is_empty_input_spacemouse function from robocasa/robocasa/scripts/collect_demos.py , we only checked right_delta. Need to update the condition into:

    if (
        ("left_delta" not in action_dict or np.all(action_dict["left_delta"] == 0))
        and np.all(action_dict["right_delta"] == 0)
        and action_dict["base_mode"] == -1
        and np.all(action_dict["base"] == 0)
    ):

snasiriany reviewed Nov 15, 2024

View reviewed changes

squarefk force-pushed the improve-spacemouse branch from 297db1d to d10de3e Compare November 22, 2024 02:26

Yu Fang added 8 commits November 26, 2024 10:32

Reduce waist movement, modify finger movement speed and remake sites

3e9e92f

Add pinch spheres and tune mink

64d4e74

SpaceMouse supports bimanual teleop

fdea8d5

Fix out-of-bound with single SpaceMouse

8a04241

Add missing comments

a2b74ed

Reformat files

0f0a66a

Adjust based on reviews

7e7df3e

Revert unnecessary changes

fc86b58

squarefk force-pushed the improve-spacemouse branch from d10de3e to fc86b58 Compare November 26, 2024 18:33

Revert changes in the inspire hands

e44486a

Conversation

squarefk commented Nov 15, 2024

Uh oh!

snasiriany left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

snasiriany commented Nov 16, 2024

Uh oh!

squarefk commented Nov 22, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants