Nix equivalent of mc-rtc-superbuild (upcoming mc_rtc 2.14)

[arntanguy]

• Convert to flakes
• Re-use as much of Gepetto's as possible
  • Use gepetto flake and same nixpkgs/ros inputs
• Support mc-rtc-superbuild intent: building mc_rtc and its dependencies, along with a set of runtime dependencies (and their dependencies) such as controllers, plugins, observers, robots, etc.
• Allow to setup a developpement environment:
  • Full devshell with all runtime dependencies available
  • Allow local development with fast recompilation
    • Local overrides of runtime dependencies controllers/plugins/observers/robots are possible provided that the right mc_rtc.yaml and LD_LIBRARY_PATH is provided
    • Development of upstream packages to mc_rtc is possible, but requires a full nix rebuild which takes forever as it needs to rebuild everything from scratch (depencies of mc_rtc, mc_rtc itself, and its runtime dependencies). Maybe ccacheStdenv could be an option there.
• Sync local dev environment between machines (cachix)
• ROS2 support
• Python support
  • mc_rtc python utilities (mc_log_ui, mc_rtc_new_fsm_controller, etc)
  • python bindings: either finalize the new nanobind bindings, or figure out how to build the cython ones
• Release mc_rtc and all updated dependencies

[arntanguy]

In its current state, this builds a nix-shell that builds mc-rtc and its dependencies, and runs a controller.
ROS2 does not yet work, and python bindings are not yet handled.

[arntanguy]

cbb9f48 is the first truly exploitable prototype. Dependencies are still in branches, but their ref is up-to-date and everything has been pushed to mc-rtc-cachix (no build required).

It provides an mc-rtc-superbuild derivation that can be configured with a list of robots/controllers/plugins/apps to build (or fetch from mc-rtc-nix cachix cache).
This derivation creates an mc_rtc.yaml file with all the runtime paths to the runtime dependencies (robots/plugin/controller/...) pre-filled to their nix store path. Each dependency is installed into its own isolated store path. We can define an additional mc_rtc.yaml configuration to load on top of that, e.g which controller to run by default.
To support loading these additional mc_rtc.yaml I introduced MC_RTC_CONTROLLER_CONFIG=<path to mc_rtc.yaml>:<other path to mc_rtc.yaml>:<nix generated mc_rtc.yaml>, and most tools requiring runtime paths have been updated to properly take it into account (mc_rtc-magnum, mc_robot_visualization, etc).

It looks like this for now:

  mc-rtc-superbuild = prev.callPackage ./pkgs/mc-rtc/mc-rtc-superbuild.nix { 
    robots = [
      mc-hrp2
      mc-panda
      mc-panda-lirmm
      # note that panda-prosthesis is not strictly-speaking a robot, but it builds a robot module so we need it here as well to populate the robots runtime paths
      # it (obviously) won't be built multiple times ;)
      panda-prosthesis
    ];
    # MainRobot = "HRP2DRC";
    # Enabled = "CoM";
    # controllers = [lipm-walking-controller];
    controllers = [ panda-prosthesis ];
    configs = [ "${panda-prosthesis}/lib/mc_controller/etc/mc_rtc.yaml" ]; # extra mc_rtc.yaml
    observers = [];
    plugins = [];
    apps = [ mc-rtc-magnum mc-franka ];
  };

and can be tested with

git clone --recursive https://github.com/mc-rtc/nixpkgs -b upgrade
git checkout cbb9f483f1453eea52deb99e8d46f45c79ca1d24
nix develop
mc-rtc-magnum &
mc_rtc_ticker

Note on non NixOS system, one might need to use system-manager to get opengl apps to work, but I haven't tested it yet. It ough to be setup in the flake though.

For now, derivations can be overridden with a local git repository, but this is done manually, and triggers a full rebuild on change.

This has been successfully deployed on Seanan's system with Windows 11 + WSL2 + NixOS today!

[arntanguy]

Still unfinished, but will merge as-is as:

• main branch is not working anyways
• the mc-rtc-superbuild devShell is stable, and works along with PR Updates for nixpkg jrl-umi3218/mc_rtc#495, and all inputs are pushed + corresponding cachix cache
• ongoing changes are more experimental (magnum packaging / mc-rtc-magnum / mc-mujoco standalone versions)
• or controller-specific (hugo's polytopeController or rolkneematics demo)