Add Fii mobile manipulator for teleoperation and locomanipulation SDG

[jaybdub]

Description

This PR adds support for the Fii mobile manipulator with teleoperation and locomanipulation SDG.

This includes

1. Adds fiibot.py to isaaclab_assets to support the Fii mobile manipulator.
2. Adds locomanipulation_fii_env_cfg.py to isaaclab_tasks

• This is a simple task for picking up a cube and placing it.
• This includes the environment with upper and lower body IK control

3. Adds fii_locomanipulation_sdg_env.py to the isaaclab_mimic locomanipulation SDG environments

• This allows the user to use the locomanipulation SDG pipeline to generate data from static manipulation recordings with the Fii mobile manipulator.

4. Adds FiiRetargeter to isaaclab openxr devices. This enables the user to record teleoperation samples
5. Adds extra arguments to the locomanipulation SDG generate_data.py script to support more diverse scenarios. Updates the existing G1 example documentation to use these arguments.

This was tested by

1. Recording a teleoperation sample using the AVP
2. Replaying the teleoperation sample with replay_demos.py script
3. Generating locomanipulation data using the locomanipulation SDG pipeline

Please note, this PR requires an asset be uploaded to the IsaacLab nucleus server, the USD is in the attached zip file:

Type of change

• New feature (non-breaking change which adds functionality)

Screenshots

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Overview

Greptile Summary

This PR introduces comprehensive support for the Fii mobile manipulator robot in Isaac Lab, adding both simulation environments and teleoperation capabilities. The implementation includes a new locomanipulation task environment that handles dual-arm manipulation with swerve drive mobility, integrating custom inverse kinematics for the 3-wheel swerve drive system and Pink IK for upper body control. The PR also adds OpenXR hand tracking retargeting functionality that translates human hand movements into robot commands for end-effector poses, gripper control, and base velocity. The changes follow Isaac Lab's established architecture patterns, extending the existing locomanipulation framework with Fii-specific configurations while maintaining compatibility with the broader ecosystem.

Important Files Changed

Filename	Score	Overview
source/isaaclab/isaaclab/devices/openxr/retargeters/init.py	5/5	Adds import registration for Fii retargeter classes following established patterns
source/isaaclab_tasks/isaaclab_tasks/manager_based/locomanipulation/pick_place/.gitignore	5/5	Excludes Fii robot URDF directory from version control
source/isaaclab_tasks/isaaclab_tasks/manager_based/locomanipulation/pick_place/init.py	4/5	Registers new Fii mobile manipulator environment with Gym
source/isaaclab_tasks/isaaclab_tasks/manager_based/locomanipulation/pick_place/swerve_ik.py	5/5	Implements mathematically correct inverse kinematics for 3-wheel swerve drive system
source/isaaclab_tasks/isaaclab_tasks/manager_based/locomanipulation/pick_place/locomanipulation_fii_env_cfg.py	4/5	Comprehensive environment configuration integrating swerve drive and dual-arm control
source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/fii/fii_retargeter.py	3/5	Hand tracking retargeter with redundant operations and missing error handling

Confidence score: 3/5

• This PR requires careful review due to code quality issues in the retargeter implementation and incomplete integration
• Score reflects redundant tensor operations, hardcoded values, missing error handling in the retargeter, and no-op assignments that suggest incomplete implementation
• Pay close attention to the FiiRetargeter class and consider testing the environment registration thoroughly

Sequence Diagram

sequenceDiagram
    participant User
    participant OpenXRDevice
    participant FiiRetargeter
    participant ManagerBasedEnv
    participant FiibotLowerBodyAction
    participant swerve_ik
    participant PinkIKController
    participant Articulation

    User->>OpenXRDevice: "Hand tracking input"
    OpenXRDevice->>FiiRetargeter: "retarget(hand_data)"
    
    FiiRetargeter->>FiiRetargeter: "_hand_data_to_gripper_values(left_hand)"
    FiiRetargeter->>FiiRetargeter: "_hand_data_to_gripper_values(right_hand)"
    FiiRetargeter->>FiiRetargeter: "Extract wrist poses from hand data"
    FiiRetargeter-->>OpenXRDevice: "Combined action tensor [left_eef, right_eef, grippers, base_vel, base_height]"
    
    OpenXRDevice-->>ManagerBasedEnv: "Retargeted actions"
    
    ManagerBasedEnv->>FiibotLowerBodyAction: "process_actions([vx, vy, wz, jack_height])"
    FiibotLowerBodyAction->>swerve_ik: "swerve_isosceles_ik(vx, vy, wz, L1, d, w, R)"
    swerve_ik-->>FiibotLowerBodyAction: "wheel angles and velocities"
    FiibotLowerBodyAction->>FiibotLowerBodyAction: "Calculate joint targets"
    
    ManagerBasedEnv->>PinkIKController: "Upper body IK control"
    PinkIKController->>PinkIKController: "Process EEF poses for left/right arms"
    
    FiibotLowerBodyAction->>Articulation: "set_joint_position_target(lower_body_joints)"
    PinkIKController->>Articulation: "set_joint_position_target(upper_body_joints)"
    
    Articulation-->>ManagerBasedEnv: "Robot state update"
    ManagerBasedEnv-->>User: "Visual feedback in simulation"

Loading

[greptile-apps]

Additional Comments (8)

1. source/isaaclab_tasks/isaaclab_tasks/manager_based/locomanipulation/pick_place/__init__.py, line 34 (link)

   style: Consider adding version suffix '-v0' to maintain consistency with other environment IDs in the codebase

   Is there a specific reason for omitting the version suffix that other environments use?

   _{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

2. source/isaaclab_tasks/isaaclab_tasks/manager_based/locomanipulation/pick_place/swerve_ik.py, line 50-52 (link)

   style: references contain placeholder text :contentReference[oaicite:x]{index=x} that should be replaced with actual references

   Are these placeholders meant to be replaced with actual citations?

3. source/isaaclab_tasks/isaaclab_tasks/manager_based/locomanipulation/pick_place/locomanipulation_fii_env_cfg.py, line 273-276 (link)

   logic: link name mismatch: observations use 'left_7_Link' but action config uses 'Fiibot_W_2_V2_left_7_Link'

   Should the observation link names match the full names used in the action configuration?

4. source/isaaclab_tasks/isaaclab_tasks/manager_based/locomanipulation/pick_place/locomanipulation_fii_env_cfg.py, line 42 (link)

   syntax: typo: 'exit' should be 'exist' in comment

5. source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/fii/fii_retargeter.py, line 40 (link)

   logic: This line performs a no-op assignment. The height coordinate is assigned to itself without any transformation. Was there supposed to be a height adjustment or transformation applied to the z-coordinate?

6. source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/fii/fii_retargeter.py, line 43 (link)

   logic: Same no-op assignment issue as the left wrist. The height coordinate is not being modified.

7. source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/fii/fii_retargeter.py, line 39 (link)

   style: Redundant tensor creation - torch.from_numpy() already creates a tensor, then torch.tensor() creates another tensor from it.

   _{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

8. source/isaaclab/isaaclab/devices/openxr/retargeters/humanoid/fii/fii_retargeter.py, line 42 (link)

   style: Same redundant tensor creation issue as line 39.

   _{Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!}

_{6 files reviewed, 8 comments}

_{Edit Code Review Agent Settings | Greptile}

[zoctipus]

is this file commit by mistake?

[zoctipus]

please using string style for registration entry point rather than direct import https://github.com/isaac-sim/IsaacLab/pull/3803/changes

[zoctipus]

wrong comment?

[zoctipus]

nice work, some nit fixes but overall seems good. I would be a bit more thoughtful about handling the urdf, is it possible to upload it to server instead rather than do local convertion?

[rwiltz]

Missing super().__init__(cfg) to properly handle the sim_device