Eval bug: Issue when trying to run exaone model after hf to gguf conversion

[Zhaeong]

Name and Version

llama-cli.exe --version
version: 3813 (116efee)
built with MSVC 19.39.33523.0 for x64

When running convert_hf_to_gguf.py on this model https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct

After conversion, during inference this error message is shown:

llama_model_load: error loading model: check_tensor_dims: tensor 'output.weight' not found
llama_load_model_from_file: failed to load model

The fix seems to be in convert_hf_to_gguf.py for class ExaoneModel(Model), add this function:

def modify_tensors(self, data_torch: Tensor, name: str, bid: int | None) -> Iterable[tuple[str, Tensor]]:
        new_name = self.map_tensor_name(name)


        tensors: list[tuple[str, Tensor]] = [(new_name, data_torch)]

        if new_name == self.format_tensor_name(gguf.MODEL_TENSOR.TOKEN_EMBD):
            assert self.tensor_names is not None

            if all(s not in self.tensor_names for s in ("output.weight")):
                # copy tok_embd.weight to output.weight
                tensors.append((self.format_tensor_name(gguf.MODEL_TENSOR.OUTPUT), data_torch))

        return tensors

Operating systems

Windows

GGML backends

Vulkan

Hardware

Ryzen 7900X + AMD 7900XT

Models

https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct

Problem description & steps to reproduce

run convert_hf_to_gguf.py on this model https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct

First Bad Commit

No response

Relevant log output

llama_model_load: error loading model: check_tensor_dims: tensor 'output.weight' not found
llama_load_model_from_file: failed to load model