Cannot import NeuronWhisperForConditionalGeneration from optimum-neuron==0.4.0

[wolanlu]

System Info

I installed optimum-neuron==0.4.0 on top of pytorch-training-neuronx:2.8.0-neuronx-py311-sdk2.26.0-ubuntu22.04 image

optimum-cli also does not work: `/usr/local/lib/python3.11/site-packages/optimum/commands/env.py:18: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  from pkg_resources import get_distribution
Traceback (most recent call last):
  File "/usr/local/bin/optimum-cli", line 3, in <module>
    from optimum.commands.optimum_cli import main
  File "/usr/local/lib/python3.11/site-packages/optimum/commands/__init__.py", line 17, in <module>
    from .export import ExportCommand, ONNXExportCommand, TFLiteExportCommand
  File "/usr/local/lib/python3.11/site-packages/optimum/commands/export/__init__.py", line 16, in <module>
    from .base import ExportCommand
  File "/usr/local/lib/python3.11/site-packages/optimum/commands/export/base.py", line 18, in <module>
    from .onnx import ONNXExportCommand
  File "/usr/local/lib/python3.11/site-packages/optimum/commands/export/onnx.py", line 23, in <module>
    from ...exporters import TasksManager
  File "/usr/local/lib/python3.11/site-packages/optimum/exporters/__init__.py", line 16, in <module>
    from .tasks import TasksManager  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/optimum/exporters/tasks.py", line 173, in <module>
    class TasksManager:
  File "/usr/local/lib/python3.11/site-packages/optimum/exporters/tasks.py", line 348, in TasksManager
    "t5-encoder": supported_tasks_mapping(
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/optimum/exporters/tasks.py", line 117, in supported_tasks_mapping
    importlib.import_module(f"optimum.exporters.{backend}.model_configs"), config_cls_name
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/optimum/exporters/onnx/model_configs.py", line 72, in <module>
    from .base import ConfigBehavior, OnnxConfig, OnnxConfigWithPast, OnnxSeq2SeqConfigWithPast
  File "/usr/local/lib/python3.11/site-packages/optimum/exporters/onnx/base.py", line 55, in <module>
    from .model_patcher import ModelPatcher, Seq2SeqModelPatcher
  File "/usr/local/lib/python3.11/site-packages/optimum/exporters/onnx/model_patcher.py", line 36, in <module>
    from transformers.models.clip.modeling_clip import CLIPAttention, CLIPSdpaAttention
ImportError: cannot import name 'CLIPSdpaAttention' from 'transformers.models.clip.modeling_clip' (/usr/local/lib/python3.11/site-packages/transformers/models/clip/modeling_clip.py)`

Who can help?

@dacorvo

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction (minimal, reproducible, runnable)

Trying to run inference for Whisper model according to official guide

when importing 'from optimum.neuron import NeuronWhisperForConditionalGeneration' I get exception ImportError: cannot import name 'CLIPSdpaAttention' from 'transformers.models.clip.modeling_clip'.

seems CLIPSdpaAttention was removed from transformer in version v4.52.1 but optimum-neuron uses 4.55.4 as of commit

Expected behavior

can import NeuronWhisperForConditionalGeneration and run inference