Loading Local Large Language Model in DictTango

Steps to Use Local Large Language Model Files

1. Download the local large language model file and save it to your device.
2. In DictTango, add the AI service and select "Local LLM" as the service provider.
3. Select the local LLM model file
4. Add an AI prompt or directly click "AI Chat" on the home page.

Large Language Model Download Methods

• You can use models from Ollama directly: https://ollama.com/library
• Llama 3.2 1B : https://huggingface.co/bartowski/Llama-3.2-1B-Instruct-GGUF/blob/main/Llama-3.2-1B-Instruct-Q4_0_4_4.gguf
• Llama 3.2 3B : https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-GGUF/blob/main/Llama-3.2-3B-Instruct-Q4_0_4_4.gguf
• TinyLlama 1.1B (f16, 2.2 GiB): Download here
• Phi-2 7B (Q4_0, 1.6 GiB): Download here
• Phi 2 DPO (Q3_K_M, 1.48 GiB): Download here