System Info
optimum 2.1.0
optimum-onnx 0.1.0
Python 3.12.
CentOS Stream 10
Who can help?
No response
Information
Tasks
Reproduction (minimal, reproducible, runnable)
When running a command that takes a sub-command, such as export onnx on a CentOS Stream 10:
.venv/bin/optimum-cli export onnx --help
, the command fails with this error:
Traceback (most recent call last):
File "/home/user/.venv/bin/optimum-cli", line 6, in <module>
sys.exit(main())
^^^^^^
File "/home/user/.venv/lib64/python3.12/site-packages/optimum/commands/optimum_cli.py", line 209, in main
register_optimum_cli_subcommand(command_or_command_info, parent_command=parent_command_instance)
File "/home/user/.venv/lib64/python3.12/site-packages/optimum/commands/optimum_cli.py", line 185, in register_optimum_cli_subcommand
parent_command.register_subcommand(command_info)
File "/home/user/.venv/lib64/python3.12/site-packages/optimum/commands/base.py", line 135, in register_subcommand
self.registered_subcommands.append(command_info.subcommand_class(self.subparsers, command=command_info))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/user/.venv/lib64/python3.12/site-packages/optimum/commands/base.py", line 86, in __init__
self.parser = subparsers.add_parser(self.COMMAND.name, help=self.COMMAND.help)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/argparse.py", line 1223, in add_parser
raise ArgumentError(self, _('conflicting subparser: %s') % name)
argparse.ArgumentError: argument {onnx}: conflicting subparser: onnx
I can run sub-commands successfully on Windows or Ubuntu.
It took me a bunch of time and a bunch Claude prompts to find out what the problem was, which is triggered by how optimum registers sub-commands.
It all comes down to RedHat distributions add both lib and lib64 to sys.path and Ubuntu does not. This confuses optimum-cli because it uses a plain set call to remove duplicates, which does not work for sym-linked paths and registers the onnx sub-parser twice, causing this error.
That is, running this command on CentOS Stream 10 and Ubuntu:
.venv/bin/python3 -c "import sys; [print(p) for p in sys.path if 'site-packages' in p]"
, yields these results:
RHL:
/home/user/.venv/lib64/python3.12/site-packages
/home/user/.venv/lib/python3.12/site-packages
Ubuntu:
/home/user/.venv/lib/python3.12/site-packages
These directories get picked up by the code in optimum/commands/optimum_cli.py, which does this:
for register_path in set(commands_register_spec.submodule_search_locations):
So, the set in this case will fail because it will evaluate different paths without realizing that it's the same location, triggering the duplicate sub-parser error.
This is Claude-provided fix that worked for me. I'm not suggesting this fix, however - it's a quick hack that makes the problem go away. I'm sure a better fix that resolves locations before deduplicating them can be implemented.
sed -i 's/for register_path in set(commands_register_spec.submodule_search_locations):/for register_path in {__import__("pathlib").Path(p).resolve() for p in commands_register_spec.submodule_search_locations}:/' \
~/.venv/lib/python3.12/site-packages/optimum/commands/optimum_cli.py
Expected behavior
I can run optimum-cli sub-commands on a RHL distribution.
System Info
Who can help?
No response
Information
Tasks
examplesfolder (such as GLUE/SQuAD, ...)Reproduction (minimal, reproducible, runnable)
When running a command that takes a sub-command, such as
export onnxon a CentOS Stream 10:, the command fails with this error:
I can run sub-commands successfully on Windows or Ubuntu.
It took me a bunch of time and a bunch Claude prompts to find out what the problem was, which is triggered by how
optimumregisters sub-commands.It all comes down to RedHat distributions add both
libandlib64tosys.pathand Ubuntu does not. This confusesoptimum-clibecause it uses a plainsetcall to remove duplicates, which does not work for sym-linked paths and registers theonnxsub-parser twice, causing this error.That is, running this command on CentOS Stream 10 and Ubuntu:
, yields these results:
RHL:
Ubuntu:
These directories get picked up by the code in
optimum/commands/optimum_cli.py, which does this:So, the
setin this case will fail because it will evaluate different paths without realizing that it's the same location, triggering the duplicate sub-parser error.This is Claude-provided fix that worked for me. I'm not suggesting this fix, however - it's a quick hack that makes the problem go away. I'm sure a better fix that resolves locations before deduplicating them can be implemented.
Expected behavior
I can run
optimum-clisub-commands on a RHL distribution.