running Qwen2-7B-Instruct-GPTQ-Int4 reported an error

[sunj0104]

hello, when I run "python tools/run_pipeline.py -o /root/QwenQwen2-7B-Instruct-GPTQ-Int4 -m gptq-auto -q int_n" report error like this:

......
INFO:hf-to-gguf:blk.27.attn_v.bias, torch.float16 --> F32, shape = {512}
INFO:hf-to-gguf:blk.27.attn_v.weight, torch.float16 --> I4, shape = {3584, 512}
INFO:hf-to-gguf:output_norm.weight, torch.float16 --> F32, shape = {3584}
Traceback (most recent call last):
File "convert_hf_to_gguf.py", line 4543, in
main()
File "convert_hf_to_gguf.py", line 4537, in main
model_instance.write()
File "convert_hf_to_gguf.py", line 527, in write
self.prepare_tensors()
File "convert_hf_to_gguf.py", line 352, in prepare_tensors
for name, data_torch in chain(self.generate_extra_tensors(), self.get_tensors()):
File "convert_hf_to_gguf.py", line 190, in get_tensors
raise ValueError("Mismatch between weight map and model parts for tensor names:\n"
ValueError: Mismatch between weight map and model parts for tensor names:
Missing tensors: []
Extra tensors: ['model.layers.0.mlp.down_proj.bias', 'model.layers.0.mlp.gate_proj.bias',......

Thanks for your help~

[QingtaoLi1]

@sunj0104 Do you pull the latest T-MAC and llama.cpp submodule code? We have solved this problem, and I did a test just now, which is all right.

[sunj0104]

@sunj0104 Do you pull the latest T-MAC and llama.cpp submodule code? We have solved this problem, and I did a test just now, which is all right.

Thanks for you reply, I can run it now.
The reason is the model files. before, I download the model files one by one manually from hf web page; now, I download the mode files by the command 'huggingface-cli download....'

[sunj0104]

@sunj0104 Do you pull the latest T-MAC and llama.cpp submodule code? We have solved this problem, and I did a test just now, which is all right.

Thanks for you reply, I can run it now. The reason is the model files. before, I download the model files one by one manually from hf web page; now, I download the mode files by the command 'huggingface-cli download....'

Actually, after I download the model files maually one by one , I checked the files' size, it seems normal, I don't know why the method doesn't work.

[QingtaoLi1]

@sunj0104 Do you pull the latest T-MAC and llama.cpp submodule code? We have solved this problem, and I did a test just now, which is all right.

Thanks for you reply, I can run it now. The reason is the model files. before, I download the model files one by one manually from hf web page; now, I download the mode files by the command 'huggingface-cli download....'

Actually, after I download the model files maually one by one , I checked the files' size, it seems normal, I don't know why the method doesn't work.

@sunj0104 We read some meta information from config.json. The T-MAC configuration may be wrong without it.

[sunj0104]

@sunj0104 Do you pull the latest T-MAC and llama.cpp submodule code? We have solved this problem, and I did a test just now, which is all right.

Thanks for you reply, I can run it now. The reason is the model files. before, I download the model files one by one manually from hf web page; now, I download the mode files by the command 'huggingface-cli download....'

Actually, after I download the model files maually one by one , I checked the files' size, it seems normal, I don't know why the method doesn't work.

@sunj0104 We read some meta information from config.json. The T-MAC configuration may be wrong without it.

I also download ‘config.json’ manually, and I compare the two files named ‘config.json’ downloaded by two method, they are same.