PaddleOCR-vl sft后无法vllm部署 #1349
Description
(APIServer pid=76722) The argument trust_remote_code is to be used with Auto classes. It has no effect here and is ignored.
(APIServer pid=76722) INFO 11-09 08:49:56 [model.py:547] Resolved architecture: PaddleOCRVLForConditionalGeneration
(APIServer pid=76722) INFO 11-09 08:49:56 [model.py:1510] Using max model len 16384
(APIServer pid=76722) INFO 11-09 08:49:56 [scheduler.py:205] Chunked prefill is enabled with max_num_batched_tokens=131072.
(APIServer pid=76722) Traceback (most recent call last):
(APIServer pid=76722) File "/usr/local/bin/paddlex_genai_server", line 8, in
(APIServer pid=76722) sys.exit(main())
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/genai/server.py", line 113, in main
(APIServer pid=76722) run_genai_server(args)
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/genai/server.py", line 100, in run_genai_server
(APIServer pid=76722) run_server_func(
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/paddlex/inference/genai/backends/vllm.py", line 68, in run_vllm_server
(APIServer pid=76722) uvloop.run(run_server(args))
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/uvloop/init.py", line 69, in run
(APIServer pid=76722) return loop.run_until_complete(wrapper())
(APIServer pid=76722) File "uvloop/loop.pyx", line 1518, in uvloop.loop.Loop.run_until_complete
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/uvloop/init.py", line 48, in wrapper
(APIServer pid=76722) return await main
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1884, in run_server
(APIServer pid=76722) await run_server_worker(listen_address, sock, args, **uvicorn_kwargs)
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 1902, in run_server_worker
(APIServer pid=76722) async with build_async_engine_client(
(APIServer pid=76722) File "/usr/lib/python3.10/contextlib.py", line 199, in aenter
(APIServer pid=76722) return await anext(self.gen)
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 180, in build_async_engine_client
(APIServer pid=76722) async with build_async_engine_client_from_engine_args(
(APIServer pid=76722) File "/usr/lib/python3.10/contextlib.py", line 199, in aenter
(APIServer pid=76722) return await anext(self.gen)
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/openai/api_server.py", line 225, in build_async_engine_client_from_engine_args
(APIServer pid=76722) async_llm = AsyncLLM.from_vllm_config(
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/vllm/utils/init.py", line 1572, in inner
(APIServer pid=76722) return fn(*args, **kwargs)
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/async_llm.py", line 207, in from_vllm_config
(APIServer pid=76722) return cls(
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/vllm/v1/engine/async_llm.py", line 114, in init
(APIServer pid=76722) self.tokenizer = init_tokenizer_from_configs(
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/vllm/transformers_utils/tokenizer.py", line 286, in init_tokenizer_from_configs
(APIServer pid=76722) return get_tokenizer(
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/vllm/transformers_utils/tokenizer.py", line 238, in get_tokenizer
(APIServer pid=76722) raise e
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/vllm/transformers_utils/tokenizer.py", line 217, in get_tokenizer
(APIServer pid=76722) tokenizer = AutoTokenizer.from_pretrained(
(APIServer pid=76722) File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/tokenization_auto.py", line 1113, in from_pretrained
(APIServer pid=76722) raise ValueError(
(APIServer pid=76722) ValueError: Tokenizer class Ernie4_5_Tokenizer does not exist or is not currently imported.
使用文心组件微调后，产出的tokenizer_config.py中tokenizer_class:"Ernie4_5_Tokenizer"
然后通过vllm部署启动，就报上面的错误，Ernie4_5_Tokenizer这个class是在ERNIE里的，检查了ERNIE是正常流程安装的。