Skip to content

Conversation

@HaroldBenoit
Copy link

Hello, thank you very much for this toolkit. This is very useful for the community, and this PR aims to make a few improvements.

  • It adds an explicit requirements.txt & installation instructions in the README.md.

    • The current dependencies specified in the README.md don't work out of box with pip or uv.
  • It adds explicitly llama.cpp as a submodule.

    • This allows a clear way to obtain llama binaries such as llama-quantize necessary for scripts such as quant/gguf/run_quant.sh .
    • This also fixes import errors (e.g. MistralTokenizerType is not importable) when running quant/gptq/pack_gptq_into_gguf.py, as the pypi version of gguf is not up-to-date, and llama.cpp implictly requires to point to the local upstream version stored in the gguf-py folder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant