This project provides a collective list of Generative AI (GenAI) and Retrieval-Augmented Generation (RAG) examples such as chatbot with question and answering (ChatQnA), code generation (CodeGen), document summary (DocSum), etc.
All the examples are well-validated on Intel platforms. In addition, these examples are:
-
Easy to use. Use ecosystem-compliant APIs to build the end-to-end GenAI examples
-
Easy to customize. Customize the example using different framework, LLM, embedding, serving etc.
-
Easy to deploy. Deploy the GenAI examples with performance on Intel platforms
Note: The below support matrix gives the validated configurations. Feel free to customize per your needs.
ChatQnA is an example of chatbot for question and answering through retrieval argumented generation (RAG).
| Framework | LLM | Embedding | Vector Database | Serving | HW | Description |
| LangChain | NeuralChat-7B | BGE-Base | Redis | TGI TEI | Xeon/Gaudi2 | Chatbot |
| LangChain | NeuralChat-7B | BGE-Base | Chroma | TGI TEI | Xeon/Gaudi2 | Chatbot |
| LangChain | Mistral-7B | BGE-Base | Redis | TGI TEI | Xeon/Gaudi2 | Chatbot |
| LangChain | Mistral-7B | BGE-Base | Qdrant | TGI TEI | Xeon/Gaudi2 | Chatbot |
CodeGen is an example of copilot designed for code generation in Visual Studio Code.
| Framework | LLM | Serving | HW | Description |
| LangChain | Deepseek-Coder-33B | TGI | Xeon/Gaudi2 | Copilot |
CodeTrans is an example of chatbot for converting code written in one programming language to another programming language while maintaining the same functionality.
| Framework | LLM | Serving | HW | Description |
| LangChain | HuggingFaceH4/mistral-7b-grok | TGI | Xeon/Gaudi2 | Code Translation |
DocSum is an example of chatbot for summarizing the content of documents or reports.
| Framework | LLM | Serving | HW | Description |
| LangChain | NeuralChat-7B | TGI | Xeon/Gaudi2 | Chatbot |
| LangChain | Mistral-7B | TGI | Xeon/Gaudi2 | Chatbot |
Language Translation is an example of chatbot for converting a source-language text to an equivalent target-language text.
| Framework | LLM | Serving | HW | Description |
| LangChain | haoranxu/ALMA-13B | TGI | Xeon/Gaudi2 | Language Translation |
SearchQnA is an example of chatbot for using search engine to enhance QA quality.
| Framework | LLM | Serving | HW | Description |
| LangChain | NeuralChat-7B | TGI | Xeon/Gaudi2 | Chatbot |
| LangChain | Mistral-7B | TGI | Xeon/Gaudi2 | Chatbot |
VisualQnA is an example of chatbot for question and answering based on the images.
| Framework | LLM | Serving | HW | Description |
| LangChain | LLaVA-1.5-7B | TGI | Gaudi2 | Chatbot |