KeyError: 'sdpa'

[pblack476]

Hi! When I try to run the simple server I get this error:

config.json: 100%|████████████████████████████████████████████████████████████████████| 811/811 [00:00<?, ?B/s]
model.safetensors: 100%|████████████████████████████████████████████████████| 154M/154M [00:07<00:00, 20.4MB/s]
Loaded detection model vikp/surya_det3 on device cpu with dtype torch.float32
preprocessor_config.json: 100%|███████████████████████████████████████████████████████| 675/675 [00:00<?, ?B/s]
config.json: 100%|████████████████████████████████████████████████████████████████| 1.32k/1.32k [00:00<?, ?B/s]
model.safetensors: 100%|████████████████████████████████████████████████████| 154M/154M [00:07<00:00, 21.1MB/s]
Loaded detection model vikp/surya_layout3 on device cpu with dtype torch.float32
preprocessor_config.json: 100%|███████████████████████████████████████████████████████| 373/373 [00:00<?, ?B/s]
config.json: 100%|████████████████████████████████████████████████████████████████| 5.04k/5.04k [00:00<?, ?B/s]
model.safetensors: 100%|████████████████████████████████████████████████████| 550M/550M [00:29<00:00, 18.8MB/s]
ERROR: Traceback (most recent call last):
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\starlette\routing.py", line 693, in lifespan
async with self.lifespan_context(app) as maybe_state:
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\gradio\routes.py", line 1650, in new_lifespan
async with old_lifespan(
^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\contextlib.py", line 210, in aenter
return await anext(self.gen)
^^^^^^^^^^^^^^^^^^^^^
File "D:\ppgbe\Documents\GitHub\marker-api\server.py", line 39, in lifespan
model_list = load_all_models()
^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\marker\models.py", line 74, in load_all_models
order = setup_order_model(device, dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\marker\models.py", line 61, in setup_order_model
model = load_order_model()
^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\surya\model\ordering\model.py", line 27, in load_model
model = OrderVisionEncoderDecoderModel.from_pretrained(checkpoint, config=config, torch_dtype=dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\transformers\models\vision_encoder_decoder\modeling_vision_encoder_decoder.py", line 376, in from_pretrained
return super().from_pretrained(pretrained_model_name_or_path, *model_args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\transformers\modeling_utils.py", line 4097, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\transformers\models\vision_encoder_decoder\modeling_vision_encoder_decoder.py", line 199, in init
decoder = AutoModelForCausalLM.from_config(config.decoder)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\transformers\models\auto\auto_factory.py", line 440, in from_config
return model_class._from_config(config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\transformers\modeling_utils.py", line 1544, in _from_config
model = cls(config, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\surya\model\ordering\decoder.py", line 495, in init
self.model = MBartOrderDecoderWrapper(config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\surya\model\ordering\decoder.py", line 480, in init
self.decoder = MBartOrderDecoder(config)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\surya\model\ordering\decoder.py", line 294, in init
self.layers = nn.ModuleList([MBartOrderDecoderLayer(config) for _ in range(config.decoder_layers)])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\ppgbe\AppData\Local\Programs\Python\Python312\Lib\site-packages\surya\model\ordering\decoder.py", line 209, in init
self.self_attn = MBART_ATTENTION_CLASSES[config._attn_implementation](
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'sdpa'

ERROR: Application startup failed. Exiting.

[sagaruprety]

I have the same issue, both on Mac M3 and on Linux

[sagaruprety]

I got it to go away by changing the version of transformers to "4.45.2" as mentioned here and here

[liwuxuan]

It's useful to me,thx a lot