feat: Autocompletion on pixi

[EliaCereda]

Pixi has implemented support for shell autocompletion scripts a few weeks ago in (prefix-dev/pixi#3219 and prefix-dev/pixi#2366)

To take advantage of this feature, a package needs to install autocompletion files in one of these directories depending on the user's shell:

• share/bash-completion/completions
• share/zsh/site-functions
• share/fish/vendor_completions.d

I tested this with both Bash and Zsh on macOS, using the attached scripts (unzip in the root of your pixi workspace) and it's working great. What's missing is just RoboStack installing these files automatically as part of the distribution.

Is this the correct repo to discuss how to possibly get this merged? We encountered this as part of a university course on ROS we are teaching, so we'd really benefit from a fix. I can send a pull request after we define how you'd like to approach this.

Thank you!

ros-autocompletion.zip

[Tobias-Fischer]

A question for @wolfv I think how to best achieve this

[Tobias-Fischer]

PS: which uni / course do you teach?

[traversaro]

I guess also @ruben-arts may be interested.

[EliaCereda]

It's Robotics in the Master in AI at USI (Lugano, Switzerland), https://search.usi.ch/en/courses/35270726/robotics

Thanks to Pixi we've been able to migrate from a cumbersome VM that we used to prepare and distribute to students, to a single set of instructions that they can follow themselves to setup a native environment in macOS / Ubuntu / Windows. It's a great improvement!

[ruben-arts]

That's great to hear @EliaCereda, exactly what we are designing pixi for!

The quickest way would be to add these scripts to the colcon and the package the ros2 cli comes from. It's also pretty normal to create a separate package like colcon-completions which you would version similarly to colcon. This way, a user can decide to install the completions.

The extra package has my preference but might not be worth the work as these environments are huge anyway.

[traversaro]

I am bit confused on how this works anyhow. I have a bit of experience with feedstock providing autocomplete scripts (see https://github.com/conda-forge/yarp-feedstock, even if it is not clear how that works from the feedstock as the installation is handled by CMake), but I am not sure how the proposed one works.

In particular if the autocomplete script is:

eval "$(register-python-argcomplete colcon)"

who installs the register-python-argcomplete executable? Python? In that case, for colcon we can add it in https://github.com/conda-forge/colcon-core-feedstock/blob/main/recipe/meta.yaml, that already installs the colcon entry point, but to follow @ruben-arts suggestions we would need to turn that in a multi-output recipe. @ruben-arts do you have any example feedstock implementing that?

Just to understand, what is the reason not to just add the autocomplete script in the main package? The increase time spent in the activation scripts, or people not wanting autocomplete for some reason?

[traversaro]

On the colcon side I found https://github.com/colcon/colcon-argcomplete, that is packaged in https://github.com/conda-forge/colcon-argcomplete-feedstock, have you tried if just installing colcon-argcomplete works fine? (in that case, a possible action is just to document that this needs to be installed for autocompletion).

[traversaro]

On the colcon side I found https://github.com/colcon/colcon-argcomplete, that is packaged in https://github.com/conda-forge/colcon-argcomplete-feedstock, have you tried if just installing colcon-argcomplete works fine? (in that case, a possible action is just to document that this needs to be installed for autocompletion).

By looking at the installed files ( https://conda-metadata-app.streamlit.app/?q=conda-forge%2Fnoarch%2Fcolcon-argcomplete-0.3.3-pyhd8ed1ab_1.conda ) I guess it does not work out of the box, we probably need to install some scripts in share/bash-completion/completions and share/zsh/site-functions that call the existing scripts, and verify if they do not conflict with then the same script is called as part of the colcon workspace activation.

[traversaro]

See also colcon/colcon-argcomplete#41 and colcon/colcon-argcomplete#43 .

[EliaCereda]

I haven't looked deep into how it works, the register-python-argcomplete commands are from this answer: https://robotics.stackexchange.com/a/100987

There should be a cleaner way of doing this. Installing ROS from Ubuntu APT does indeed install autocompletion by default, but I could not find how that is setup exactly

[traversaro]

There should be a cleaner way of doing this. Installing ROS from Ubuntu APT does indeed install autocompletion by default, but I could not find how that is setup exactly

Is ROS autocompletion on by default, or just after sourcing the setup.sh ? I would be surprised if it works even without sourcing the setup.sh of the ROS installation. What about colcon instead?

[EliaCereda]

No, it's enabled by the setup.sh. This doesn't work in Pixi because I think it executes the setup.sh in a sub-shell which collects what environment variables are set and then only sets those in the actual user shell. This is true for both the setup.sh of the ROS distribution and for any setup.sh that are added as activation scripts in the pixi.toml.

[traversaro]

Ah, hence the need for actually installing the scripts in share/bash-completion/completions.

[EliaCereda]

I just spent a bit more time looking at what ROS actually does to enable autocompletion.

The sequence for a normal ROS installation is:

• The user sources the setup.sh for a given workspace
• setup.sh runs source local_setup.sh for the current workspace and all underlying workspaces
• Each local_setup.sh runs _local_setup_util.py to collect the activation commands of all installed packages
• Activation commands are loaded from share/<package name/package.dsv if it exists, otherwise default to share/<package name/local_setup.{sh,bash,zsh}
• local_setup.sh evals the output of _local_setup_util.py to run the commands inside the user's shell

Here's the output I get from _local_setup_util.py on my system:

AMENT_CURRENT_PREFIX="/<path>/.pixi/envs/default" _ament_prefix_sh_source_script "/<path>/.pixi/envs/default/share/ros2cli/environment/ros2-argcomplete.zsh"
AMENT_CURRENT_PREFIX="/<path>/.pixi/envs/default" _ament_prefix_sh_source_script "/<path>/.pixi/envs/default/share/ros_environment/environment/0.ros_distro_check.sh"
export ROS_DISTRO="humble"
# export ROS_LOCALHOST_ONLY="0"
export ROS_PYTHON_VERSION="3"
export ROS_VERSION="2"
AMENT_CURRENT_PREFIX="/<path>/.pixi/envs/default" _ament_prefix_sh_source_script "/<path>/.pixi/envs/default/share/rosidl_cli/environment/rosidl-argcomplete.zsh"

_ament_prefix_sh_source_script is defined in local_setup.sh, it just sources the file if it exists, and ignores it otherwise.

Out of those commands, ros2-argcomplete.sh doesn't do anything except calling register-python-argcomplete:

if type register-python-argcomplete3 > /dev/null 2>&1; then
  eval "$(register-python-argcomplete3 ros2)"
elif type register-python-argcomplete > /dev/null 2>&1; then
  eval "$(register-python-argcomplete ros2)"
fi

So it seems to me we have two problems here:

• When launching a pixi shell, Pixi sources the setup.sh with its custom system that preserves environment variables but not other shell customization (such as autocompletion). This breaks the assumption of ROS packages that they'll have their activation scripts sourced directly from the user shell.
• Even if I source the setup.sh inside a pixi shell manually, it doesn't seem to enable the autocompletion correctly. Running eval "$(register-python-argcomplete ros2)" directly works instead.

The latter might be the same problem from the StackOverflow answer I linked above (I haven't investigated further)

I haven't looked deep into how it works, the register-python-argcomplete commands are from this answer:
https://robotics.stackexchange.com/a/100987

The former however appears to me as a fundamental problem in the way Pixi handles activation scripts. Do you have ideas for how ROS should fit in your design?

[traversaro]

Thanks for the in-detail analysis. From my point of view, the simple solution for this is:

• Modify/patch the colcon-argcomplete conda-forge recipe to install its argcomplete script in the folders supported by pixi. In this case, also just copying the files in the build.sh script is doable.
• Modify/patch the ros-humble-ros2cli robostack package to install its argcomplete script in the folders supported by pixi. In this case, modifying the build.sh is more tricky as that is generated by vinca, an easier option may be to do a simple patch to the CMakeLists.txt of ros2cli to install them in the correct place.

My only doubt is for @ruben-arts , i.e. quoting my earlier comment:

Just to understand, what is the reason not to just add the autocomplete script in the main package? The increase time spent in the activation scripts, or people not wanting autocomplete for some reason?

Unless there is a compelling reason do to that, I would avoid the complexity of adding an output for the recipe just for the autocomplete script, that especially for robostack packages is really far from trivial.