how to convert bitnet model? looks like the script is broken

[nigelzzzzz]

Hi
I using the steps in readme.md. In steps 3, i can't find convert-hf-to-gguf-t-mac.py in 3party/llama.cpp/....
but i can find the file in master branch. can i know which step i must follow. thanks

Running STEP.3: Convert HF to GGUF
  Running command in /Users/user/jianyu/T-MAC/3rdparty/llama.cpp:
    python convert-hf-to-gguf-t-mac.py /Users/user/Downloads/test_models/hf-bitnet-3B --outtype i2 --outfile /Users/user/Downloads/test_models/hf-bitnet-3B/ggml-model.i2.gguf --kcfg /Users/user/jianyu/T-MAC/install/lib/kcfg.ini

[nigelzzzzz]

https://github.com/kaleid-liner/llama.cpp/tree/fde57d01c01d1cb5f97b4df9780c756e672976d9
above repo doesn't have tmac script, but master branch has it.

[nigelzzzzz]

Because i want to test bitnet 3b model, so firstly i need to convert model to i2.

def convert_models():
    model_dir = str(FLAGS.model_dir).rstrip('\\').rstrip('/')
    if not os.path.exists(model_dir):
        raise FileNotFoundError(model_dir)
    
    out_type = FLAGS.quant_type
    if FLAGS.quant_type == "q4_0":
        out_type = "f16"
    
    model_name = f"{os.path.split(model_dir)[-1]}.{str(out_type).upper()}.gguf"
    out_path = os.path.join(model_dir, model_name)
    kcfg_path = os.path.join(ROOT_DIR, "install", "lib", "kcfg.ini")
    llamacpp_dir = os.path.join(ROOT_DIR, "3rdparty", "llama.cpp")
    command = [
        'python',
        'convert_hf_to_gguf.py',
        f'{model_dir}',
        '--outtype', f'{out_type}',
        '--outfile', f'{out_path}',
        '--kcfg', f'{kcfg_path}',
        '--enable-t-mac',
        '--verbose',
    ]
    run_command(command, llamacpp_dir)

    if FLAGS.quant_type == "q4_0":
        quantized_model_name = f"{os.path.split(model_dir)[-1]}.Q4_0.gguf"
        quantized_out_path = os.path.join(model_dir, quantized_model_name)
        command = [
            './build/bin/llama-quantize',
            '--token-embedding-type', 'f16',
            '--output-tensor-type', 'f16',
            f'{out_path}',
            f'{quantized_out_path}',
            'q4_0',
        ]
        run_command(command, llamacpp_dir)

[QingtaoLi1]

@nigelzzzzz You can try running run_pipeline.py directly. The file convert-hf-to-gguf-t-mac.py does not exist now, we will update the README later. Thank you for pointing it out!

[nigelzzzzz]

Hi @QingtaoLi1 ,
thanks for your reply, I think the project is very amazing,
i can help to fix the type, or do you have any task can assign to me, i can do it.
I am an master student in taiwan (NYCU), i am interesting in this project.

[QingtaoLi1]

@nigelzzzzz Thank you, but I don't quite understand what do you mean by "fix the type". The current pipeline should work if you use the correct command.