Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop Gemma 3 chatbot with Llama.cpp & Gradio #156

Open
sitamgithub-MSIT opened this issue Mar 20, 2025 · 0 comments
Open

Develop Gemma 3 chatbot with Llama.cpp & Gradio #156

sitamgithub-MSIT opened this issue Mar 20, 2025 · 0 comments
Labels
wishlist A wish list of cookbooks

Comments

@sitamgithub-MSIT
Copy link
Contributor

sitamgithub-MSIT commented Mar 20, 2025

Description of the feature request:

The feature involves putting the Gemma 3 1b text-only English model into action. The goal is to create a chatbot using Llama.cpp, with the UI being developed by Gradio.

What problem are you trying to solve with this feature?

The current Gemma repository lacks any notebook on this idea. We have notebooks related to the concept of running it with Llama.cpp, but this idea involves an additional step to create an end-to-end Gradio chat app, allowing for easy testing via a web UI and deployment as well.

Any other information you'd like to share?

Since we are currently utilizing the lightest model in the Gemma family, there is no need for any GPU involvement, which also holds for deployment. Here is a live demo of the application hosted on Hugging Face Spaces. Feel free to give it a try: https://huggingface.co/spaces/sitammeur/Gemma-llamacpp

cc: @bebechien @windmaple

@bebechien bebechien added the wishlist A wish list of cookbooks label Apr 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wishlist A wish list of cookbooks
Projects
None yet
Development

No branches or pull requests

2 participants