Replace with QWen2.5-3B-Instruct but failed with log. Any update to support QWen2.5?
13 Namespace(model_path='./checkpoints', llm_path='/DATA/Qwen/Qwen2.5-3B-Instruct', top_k=20, top_p=0.8, temperature=0.8, ip='172.16.75.123', port='8081', max_users=3, llm_exec_nums=1, timeout=180)
14 [2025-02-13 17:06:53.299] the number of speech encoder params: 341.3681640625M
15 ^MLoading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]^MLoading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 15.74it/s]^MLoading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 15.72it/s]
16 /home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/torch/nn/utils/weight_norm.py:143: FutureWarning: `torch.nn.utils.weight_norm` is deprecated in favor of `torch.nn.utils.parametrizations.wei ght_norm`.
17 WeightNorm.apply(module, name, dim)
18 [2025-02-13 17:06:55.275] Checkpoint: loading from checkpoint ./checkpoints/audiollm/final.pt for GPU
19 [2025-02-13 17:06:58.125] reading a config file from ./checkpoints/decoder/model.json
20 [2025-02-13 17:06:58.126] reading a config file from ./checkpoints/decoder/model.json
21 [2025-02-13 17:06:58.126] reading a config file from ./checkpoints/decoder/model.json
22 [2025-02-13 17:07:01.790] Removing weight norm...
23 [2025-02-13 17:07:01.797] Removing weight norm...
24 [2025-02-13 17:07:01.800] Removing weight norm...
25 [2025-02-13 17:07:01.803] Removing weight norm...
26 [2025-02-13 17:07:01.804] Removing weight norm...
27 [2025-02-13 17:07:01.809] Removing weight norm...
28 [2025-02-13 17:07:01.821] Start Freeze-Omni sever
29 * Serving Flask app 'server'
30 * Debug mode: off
31 ^[[31m^[[1mWARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.^[[0m
32 * Running on https://172.16.75.123:8081
33 ^[[33mPress CTRL+C to quit^[[0m
34 10.176.245.8 - - [13/Feb/2025 17:07:03] "GET /socket.io/?EIO=4&transport=polling&t=mxce1ske HTTP/1.1" 200 -
35 /home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/transformers/models/qwen2/modeling_qwen2.py:216: UserWarning: Attempting to use hipBLASLt on an unsupported architecture! Overriding blas bac kend to hipblas (Triggered internally at /pytorch/aten/src/ATen/Context.cpp:310.)
36 freqs = (inv_freq_expanded.float() @ position_ids_expanded.float()).transpose(1, 2)
37 10.176.245.8 - - [13/Feb/2025 17:07:04] "GET /socket.io/?EIO=4&transport=polling&t=mxcn6kxp&sid=ytVjS-kcmfultMxuAAAA HTTP/1.1" 200 -
38 message handler error
39 [2025-02-13 17:07:04.209] Traceback (most recent call last):
40 [2025-02-13 17:07:04.210] File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/flask_socketio/__init__.py", line 824, in _handle_event
41 ret = handler(auth)
42 ^^^^^^^^^^^^^
43 [2025-02-13 17:07:04.210] TypeError: handle_connect() takes 0 positional arguments but 1 was given
44 [2025-02-13 17:07:04.210]
45 During handling of the above exception, another exception occurred:
46
47 [2025-02-13 17:07:04.210] Traceback (most recent call last):
48 [2025-02-13 17:07:04.210] File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/engineio/server.py", line 450, in run_handler
49 return self.handlers[event](*args)
50 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
51 [2025-02-13 17:07:04.210] File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/socketio/server.py", line 652, in _handle_eio_message
52 self._handle_connect(eio_sid, pkt.namespace, pkt.data)
53 [2025-02-13 17:07:04.210] File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/socketio/server.py", line 538, in _handle_connect
54 success = self._trigger_event(
55 ^^^^^^^^^^^^^^^^^^^^
56 [2025-02-13 17:07:04.210] File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/socketio/server.py", line 617, in _trigger_event
57 return handler(*args)
58 ^^^^^^^^^^^^^^
59 [2025-02-13 17:07:04.210] File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/flask_socketio/__init__.py", line 282, in _handler
60 return self._handle_event(handler, message, namespace, sid,
61 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
62 [2025-02-13 17:07:04.210] File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/flask_socketio/__init__.py", line 826, in _handle_event
63 ret = handler()
64 ^^^^^^^^^
65 [2025-02-13 17:07:04.210] File "/DATA/Demo/Freeze-Omni/Freeze-Omni/bin/server.py", line 326, in handle_connect
66 connected_users[sid].append(GlobalParams(tts_pool, pipeline_pool))
67 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
68 [2025-02-13 17:07:04.210] File "/DATA/Demo/Freeze-Omni/Freeze-Omni/web/parms.py", line 44, in __init__
69 init_outputs = self.pipeline_obj.pipeline_proc.speech_dialogue(None, stat='pre',
70 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
71 [2025-02-13 17:07:04.210] File "/DATA/Demo/Freeze-Omni/Freeze-Omni/models/pipeline.py", line 62, in speech_dialogue
72 past_key_values = self.model.set_system_role(extra_inputs)
73 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
74 [2025-02-13 17:07:04.210] File "/DATA/Demo/Freeze-Omni/Freeze-Omni/models/audioLLM.py", line 241, in set_system_role
75 _, past_key_values, stat, _ = self._generate_one_step(
76 ^^^^^^^^^^^^^^^^^^^^^^^^
77 [2025-02-13 17:07:04.210] File "/DATA/Demo/Freeze-Omni/Freeze-Omni/models/audioLLM.py", line 413, in _generate_one_step
78 state_logits = self.predictor_head(
79 ^^^^^^^^^^^^^^^^^^^^
80 [2025-02-13 17:07:04.210] File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
81 return self._call_impl(*args, **kwargs)
82 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
83 [2025-02-13 17:07:04.210] File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
84 return forward_call(*args, **kwargs)
85 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
86 [2025-02-13 17:07:04.210] File "/home/amd/anaconda3/envs/rocm624/lib/python3.11/site-packages/torch/nn/modules/linear.py", line 125, in forward
87 return F.linear(input, self.weight, self.bias)
88 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
89 [2025-02-13 17:07:04.210] RuntimeError: mat1 and mat2 shapes cannot be multiplied (36x2048 and 3584x4)
90 10.176.245.8 - - [13/Feb/2025 17:07:04] "POST /socket.io/?EIO=4&transport=polling&t=mxcn5ij9&sid=ytVjS-kcmfultMxuAAAA HTTP/1.1" 200 -
Replace with QWen2.5-3B-Instruct but failed with log. Any update to support QWen2.5?