vllm适配BUG: vllm 0.8.3不支持Modelscope魔搭社区下载的Telechat2-7b-32k模型推理部署

[WjMessi1]

部署代码：
CUDA_VISIBLE_DEVICES=0,1 vllm serve /home/gpu/modelscope/TeleChat2-7B-32K --host 0.0.0.0 --port 30004 --trust-remote-code --dtype bfloat16 --disable-custom-all-reduce

报错输出：

ERROR 04-14 14:27:42 [core.py:390] File "/home/gpu/.local/lib/python3.9/site-packages/vllm/v1/worker/gpu_worker.py", line 136, in load_model
ERROR 04-14 14:27:42 [core.py:390] self.model_runner.load_model()
ERROR 04-14 14:27:42 [core.py:390] File "/home/gpu/.local/lib/python3.9/site-packages/vllm/v1/worker/gpu_model_runner.py", line 1261, in load_model
ERROR 04-14 14:27:42 [core.py:390] self.model = get_model(vllm_config=self.vllm_config)
ERROR 04-14 14:27:42 [core.py:390] File "/home/gpu/.local/lib/python3.9/site-packages/vllm/model_executor/model_loader/init.py", line 14, in get_model
ERROR 04-14 14:27:42 [core.py:390] return loader.load_model(vllm_config=vllm_config)
ERROR 04-14 14:27:42 [core.py:390] File "/home/gpu/.local/lib/python3.9/site-packages/vllm/model_executor/model_loader/loader.py", line 441, in load_model
ERROR 04-14 14:27:42 [core.py:390] model = _initialize_model(vllm_config=vllm_config)
ERROR 04-14 14:27:42 [core.py:390] File "/home/gpu/.local/lib/python3.9/site-packages/vllm/model_executor/model_loader/loader.py", line 127, in _initialize_model
ERROR 04-14 14:27:42 [core.py:390] return model_class(vllm_config=vllm_config, prefix=prefix)
ERROR 04-14 14:27:42 [core.py:390] File "/home/gpu/.local/lib/python3.9/site-packages/vllm/model_executor/models/llama.py", line 486, in init
ERROR 04-14 14:27:42 [core.py:390] self.model = self._init_model(vllm_config=vllm_config,
ERROR 04-14 14:27:42 [core.py:390] TypeError: _init_model() got an unexpected keyword argument 'layer_type'
ERROR 04-14 14:27:42 [core.py:390]
CRITICAL 04-14 14:27:42 [core_client.py:361] Got fatal signal from worker processes, shutting down. See stack trace above for root cause issue.
Killed