[Bug]: qwen2.5-omni model failed to start

[hackerHiJu]

Your current environment

vllm：0.8.2
transformers：4.51.0.dev0

export VLLM_USE_V1=0 && vllm serve Qwen2.5-Omni-7B --dtype half --cpu-offload-gb 1 --gpu-memory-utilization 0.9 --host 0.0.0.0 --port 9000

🐛 Describe the bug

qwen2.5-omni model failed to start

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

You need to install the transformers PR branch from source, since it's not merged into main branch of transformers yet.

[hackerHiJu]

You need to install the transformers PR branch from source, since it's not merged into main branch of transformers yet.

I downloaded it through the official branch

pip install git+https://github.com/huggingface/transformers@f742a644ca32e65758c3adb36225aef1731bd2a8

[DarkLight1337]

Can you check out the discussion in #15754 (comment)? See if it can solve your issue

[hackerHiJu]

Can you check out the discussion in #15754 (comment)? See if it can solve your issue

Still no good, the startup still failed.

pip install git+https://github.com/BakerBunker/transformers.git@qwen25omni

export CUDA_VISIBLE_DEVICES=1,2 && export VLLM_USE_V1=0 && vllm serve Qwen2.5-Omni-7B
--dtype half
--cpu-offload-gb 1
--gpu-memory-utilization 0.9
--host 0.0.0.0
--port 9000

[DarkLight1337]

Please note that you also need to install PR branch for vLLM: #15130

[hackerHiJu]

Please note that you also need to install PR branch for vLLM: #15130

Still no good, the startup still failed.

[fyabc]

Please note that you also need to install PR branch for vLLM: #15130

Still no good, the startup still failed.

Hi @hackerHiJu , you need to pull the latest model & config from HuggingFace repo. we have updated tokenizer_config.json and other configs.

You can also check this issue.