Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Gemma-2 models #17

Open
nerdimite opened this issue Aug 19, 2024 · 2 comments
Open

Support for Gemma-2 models #17

nerdimite opened this issue Aug 19, 2024 · 2 comments

Comments

@nerdimite
Copy link

Hi, I wanted to ask if the support for Gemma2 models can be added in this binding? Specifically I was trying to load the gemma2-2b-it model but wasn't able to because the ModelType enum only supports 2b and 7b and passing gemma 2 weights in the path doesnt work.

Also, if it's possible, I think it would be better to make these bindings compatible directly with gemma.cpp's CLI args instead of custom enums mapping to an integer, so that this package in itself remains compatible with any new model types that come in the original gemma.cpp. Like directly passing the cpp args ./gemma --tokenizer tokenizer.spm --model gemma2-2b-it --compressed_weights 2.0-2b-it-sfp.sbs in the Gemma() class object

@namtranase
Copy link
Owner

Hi @nerdimite, thank you for using the package, we will try to update gemma-cpp-python to the latest version compatible with the gemma.cpp within this week!

@JustinSo1
Copy link

Has any progress been made on this? I would love to see it implemented

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants