Summary
register_onnx_provider() lets a plugin add an execution provider and pass
provider-level options (provider_options, e.g. device_id). Sometimes,
though, the fix a provider needs isn't a provider option at all - it's a
session-level setting (onnxruntime.SessionOptions.add_session_config_entry),
which today has no seam a plugin can reach. Would it be reasonable to extend
the registration API to cover that case too?
Background: the two option surfaces onnxruntime has
onnxruntime sessions take options at two different levels:
- Provider options - per execution provider, e.g.
{"device_id": 0}.
Already covered by register_onnx_provider(name, options, ...).
- Session config entries -
sess_options.add_session_config_entry(key, value),
applied once for the whole session regardless of which provider ends up
running which node. Things like disabling a specific graph optimizer
(optimization.disable_specified_optimizers) live here, and onnxruntime
exposes no environment variable for most of these - the only way to set
one is through SessionOptions at session-construction time.
Core builds SessionOptions itself, once, before it knows anything about a
plugin's provider - there's currently no way for a plugin to say "when you
build a session that uses my provider, also set this session config entry."
Concrete use-case I ran into
On older AMD Polaris (RX 460 to RX 590) cards, CLAP sometimes intermittently crashes. This because of a feature that doesn't work for ROCm with this model. To avoid that faulty feature, I can turn it off for the session. I am testing these cards and want to support them, because the 8GB mining edition (no video output) ones are so dirt cheap, everyone can use them in their homelab.
Currently, I use a monkeypatch to do this:
sess_options.add_session_config_entry("optimization.disable_specified_optimizers", "ConvActivationFusion")However, I only discovered it today, so in my initial PR I didn't think to add session options. So I have a solution and it's not super urgent, but it could probably be cleaner.
The feature request
Would it make sense for register_onnx_provider() (or a small companion
call) to also accept session config entries, scoped the same way provider
registration already is (only_models/exclude_models)? Something in the
shape of:
ctx.register_onnx_provider(
"ROCMExecutionProvider",
{"device_id": 0},
only_models=["clap"],
session_config_entries={"optimization.disable_specified_optimizers": "ConvActivationFusion"},
)...with core applying those entries to sess_options whenever it builds a
session that ends up using that provider registration - same scoping logic
that already decides which provider chain a given label gets.
Wanted to ask if this was wanted/doable first before just submitting a PR. Also might just be faster if you implement it rather than me writing bad python code.
Summary
register_onnx_provider()lets a plugin add an execution provider and passprovider-level options (
provider_options, e.g.device_id). Sometimes,though, the fix a provider needs isn't a provider option at all - it's a
session-level setting (
onnxruntime.SessionOptions.add_session_config_entry),which today has no seam a plugin can reach. Would it be reasonable to extend
the registration API to cover that case too?
Background: the two option surfaces onnxruntime has
onnxruntime sessions take options at two different levels:
{"device_id": 0}.Already covered by
register_onnx_provider(name, options, ...).sess_options.add_session_config_entry(key, value),applied once for the whole session regardless of which provider ends up
running which node. Things like disabling a specific graph optimizer
(
optimization.disable_specified_optimizers) live here, and onnxruntimeexposes no environment variable for most of these - the only way to set
one is through
SessionOptionsat session-construction time.Core builds
SessionOptionsitself, once, before it knows anything about aplugin's provider - there's currently no way for a plugin to say "when you
build a session that uses my provider, also set this session config entry."
Concrete use-case I ran into
On older AMD Polaris (RX 460 to RX 590) cards, CLAP sometimes intermittently crashes. This because of a feature that doesn't work for ROCm with this model. To avoid that faulty feature, I can turn it off for the session. I am testing these cards and want to support them, because the 8GB mining edition (no video output) ones are so dirt cheap, everyone can use them in their homelab.
Currently, I use a monkeypatch to do this:
However, I only discovered it today, so in my initial PR I didn't think to add session options. So I have a solution and it's not super urgent, but it could probably be cleaner.
The feature request
Would it make sense for
register_onnx_provider()(or a small companioncall) to also accept session config entries, scoped the same way provider
registration already is (
only_models/exclude_models)? Something in theshape of:
...with core applying those entries to
sess_optionswhenever it builds asession that ends up using that provider registration - same scoping logic
that already decides which provider chain a given label gets.
Wanted to ask if this was wanted/doable first before just submitting a PR. Also might just be faster if you implement it rather than me writing bad python code.