Skip to content

Cannot run sub-commands on RHL because optimum mishandles the lib64 symlink to lib #2417

Description

@gh-andre

System Info

optimum 2.1.0
optimum-onnx 0.1.0
Python 3.12.
CentOS Stream 10

Who can help?

No response

Information

  • The official example scripts
  • My own modified scripts

Tasks

  • An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
  • My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)

When running a command that takes a sub-command, such as export onnx on a CentOS Stream 10:

.venv/bin/optimum-cli export onnx --help

, the command fails with this error:

Traceback (most recent call last):
  File "/home/user/.venv/bin/optimum-cli", line 6, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/user/.venv/lib64/python3.12/site-packages/optimum/commands/optimum_cli.py", line 209, in main
    register_optimum_cli_subcommand(command_or_command_info, parent_command=parent_command_instance)
  File "/home/user/.venv/lib64/python3.12/site-packages/optimum/commands/optimum_cli.py", line 185, in register_optimum_cli_subcommand
    parent_command.register_subcommand(command_info)
  File "/home/user/.venv/lib64/python3.12/site-packages/optimum/commands/base.py", line 135, in register_subcommand
    self.registered_subcommands.append(command_info.subcommand_class(self.subparsers, command=command_info))
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.venv/lib64/python3.12/site-packages/optimum/commands/base.py", line 86, in __init__
    self.parser = subparsers.add_parser(self.COMMAND.name, help=self.COMMAND.help)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/argparse.py", line 1223, in add_parser
    raise ArgumentError(self, _('conflicting subparser: %s') % name)
argparse.ArgumentError: argument {onnx}: conflicting subparser: onnx

I can run sub-commands successfully on Windows or Ubuntu.

It took me a bunch of time and a bunch Claude prompts to find out what the problem was, which is triggered by how optimum registers sub-commands.

It all comes down to RedHat distributions add both lib and lib64 to sys.path and Ubuntu does not. This confuses optimum-cli because it uses a plain set call to remove duplicates, which does not work for sym-linked paths and registers the onnx sub-parser twice, causing this error.

That is, running this command on CentOS Stream 10 and Ubuntu:

.venv/bin/python3 -c "import sys; [print(p) for p in sys.path if 'site-packages' in p]"

, yields these results:

RHL:

/home/user/.venv/lib64/python3.12/site-packages
/home/user/.venv/lib/python3.12/site-packages

Ubuntu:

/home/user/.venv/lib/python3.12/site-packages

These directories get picked up by the code in optimum/commands/optimum_cli.py, which does this:

for register_path in set(commands_register_spec.submodule_search_locations):

So, the set in this case will fail because it will evaluate different paths without realizing that it's the same location, triggering the duplicate sub-parser error.

This is Claude-provided fix that worked for me. I'm not suggesting this fix, however - it's a quick hack that makes the problem go away. I'm sure a better fix that resolves locations before deduplicating them can be implemented.

sed -i 's/for register_path in set(commands_register_spec.submodule_search_locations):/for register_path in {__import__("pathlib").Path(p).resolve() for p in commands_register_spec.submodule_search_locations}:/' \
    ~/.venv/lib/python3.12/site-packages/optimum/commands/optimum_cli.py

Expected behavior

I can run optimum-cli sub-commands on a RHL distribution.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions