You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
while I use quantize_model.py(in script folder) to quantize ChatGLM3 model, it can be saved successfully, but when I try to load the model by torch.load(), it was return None.
To Reproduce
Steps to reproduce the behavior:
1.run 'python quantize_model.py -m .\chatglm3-6b -b 4 -o .\chatglm3-6b' to quantize the model, it will export below files
try to load model
return none
Except:
It should return a model object
The text was updated successfully, but these errors were encountered:
By another way, just I can quantize the model use 'model = intel_npu_acceleration_library.compile(model, dtype=int4) ', however, it will need much time to quantize and compile.
Describe the bug
while I use quantize_model.py(in script folder) to quantize ChatGLM3 model, it can be saved successfully, but when I try to load the model by torch.load(), it was return None.
To Reproduce

Steps to reproduce the behavior:
1.run 'python quantize_model.py -m .\chatglm3-6b -b 4 -o .\chatglm3-6b' to quantize the model, it will export below files
Except:
It should return a model object
The text was updated successfully, but these errors were encountered: