Skip to content

Latest commit

 

History

History
139 lines (114 loc) · 11 KB

README.md

File metadata and controls

139 lines (114 loc) · 11 KB

LLaMA3_Cookbook 🦙✨

d

LLaMA3 (Large Language Model by META AI) is a leading-edge large language model that excels in AI technology. 🌟 This repository📁 is intended to provide information necessary to kick-start various projects🚀 using LLaMA3.

📜Introducing Meta Llama 3: The most capable openly available LLM to date review

Official Website and Information 🌐

⚡️ Cloud API

API Calls Available 🔌

Name Description Link
Grok High-performance AI Chip enabling LLaMA3 inference and API calls Grok 🌐
AWS Bedrock support for LLaMA at AWS, currently only Llama2 available AWS 🌐
Azure Support for 8B/70B models on Microsoft Azure, searchable via Azure Marketplace Azure 🌐
GCP Google Cloud Vertax AI support for LLaMA3 GCP 🌐
together.ai Support for Llama2, CodeLlama, Llama3 8b/70b instances together.ai 🌐
replicate Llama3 API support (Node.js, Python, HTTP) replicate 🌐
llama AI Support for Llama3 8B/70B, supports other OpenLLMs llama AI 🌐
aimlapi Supports various openLLMs as APIs AI/ML API
Nvidia API Multiple OpenLLM models available Nvidia devloper llama AI 🌐
Meta AI(github) Connect to Meta AI api MetaAI 🌐

🤖 Inference 🧠

Inference Platforms 🖥️

Platform Name Description Link
HuggingFace Llama 8B model Link 🌐
HuggingFace Llama 70B model Link 🌐
HuggingFace Llama 8B Instruct model Link 🌐
HuggingFace Llama 70B Instruct model Link 🌐
HuggingFace Llama Guard-2-8B(policy model) Link 🌐
HuggingFace Llama 3 70B - FP8(friendliAI) Link 🌐
HuggingFace Llama 3 70B Instruct - FP8(friendliAI) Link 🌐
HuggingFace Llama 3 8B - FP8(friendliAI) Link 🌐
HuggingFace Llama 3 8B Instruct - FP8(friendliAI) Link 🌐
HuggingFace Llama 8B KO (made beomi) Link 🌐
Ollama Support for various lightweight Llama3 models Link 🌐

HuggingFace Models 🐥

Name Description Link
gradientai/Llama-3-8B-Instruct-Gradient-1048k 1M Long Context Link 🌐
Trelis/Meta-Llama-3-70B-Instruct-function-calling function calling Link 🌐
Trelis/Meta-Llama-3-8B-Instruct-function-calling function calling Link 🌐
cognitivecomputations/dolphin-2.9-llama3-8b Uncensored fine-tuning Link 🌐
McGill-NLP/Llama-3-8B-Web Zero-shot internet link selection capability Link 🌐
teddylee777/Llama-3-Open-Ko-8B-Instruct-preview-gguf Korean quantizatied GGUF model for ollama use Link 🌐
beomi/Llama-3-Open-Ko-8B-Instruct-preview Korean model trained with the Chat vector method Link 🌐

💬 Chat Interface (Related Information) 💻

Name Link
HuggingChat Link 🌐
Groq Link 🌐
together.ai Link 🌐
replicate Llama chat(local) Link 🌐
perplexity.ai(lightweight model) Link 🌐
openrouter.ai Link 🌐
MetaAI (Not available in Korea) Link 🌐
Morphic(multimodal offerings) Link 🌐
Nvidia AI [Link 🌐

LLaMA Framework 📘

Name Type Link
Langchain RAG Link 🌐
llamaindex RAG Link 🌐
llama.cpp convert Link 🌐

🛠️ Fine-tuning 🔧

Name Link
Meta Link 🌐
torchrune Link 🌐
LLaMAFactory Link 🌐
axolotl Link 🌐

LLAMA3_Cookbook 👩‍🍳

Information Link
Prompt Engineering Guide Link 🌐
Using llama3 with WEB UI Link 🌐
API with Ollama, LangChain and ChromaDB with Flask API and PDF upload Link 🌐
Guide for tuning and inference with Llama on MacBook Link 🌐
Fine-tune Llama 3 with ORPO Link 🌐
Qlora_aplaca_llama3 finetune Link 🌐
fully local RAG agents with LLama3 Link 🌐
RAG Chatbot LLama3(HF) Link 🌐
llama index RAG llama3 Link 🌐
ollama RAG + UI(Gradio) Link 🌐
LangGraph + Llama3 Link 🌐
RAG(Re-Ranking) Link 🌐

LLM Dataset 🗂️

Information Link
HuggingFaceFW/fineweb Link 🌐
mlabonne/orpo-dpo-mix-40k Link 🌐

LLM skills 📌

Information Link
FSDP+QLORA finetunning Link 🌐

Mac vs 4090 Comparison 🖥️🆚🖥️

Category M3 Max M1 Pro RTX 4090
CPU Cores 16 cores 10 cores 16 cores AMD
Memory 128GB 16GB /32GB 32GB
GPU Memory 16 core CPU & 40 core GPU, 400GB/s memory bandwidth 10 core CPU(8 performance cores & 2 efficiency cores) 16 core GPU 200GB/s memory bandwidth 24GB
Model 7B - Performs well on all computers - Performs well on all computers - Performs well on all computers, similar performance to M3 Max
Model 13B - Good performance - Third best performance - Best performance
Model 70B - Runs quickly, utilizing 128GB memory - Lacks memory at 16GB, prone to crashes and reboots - Cannot run on GPU, very slow on CPU
Lightening Not necessary with sufficient memory Should be considered Quantization compromises necessary
Power Consumption 65W 250-300W
Value for Money Excellent ($4600) Relatively low ($6000 for A6000 GPU)

🙌 Contributing 💖

Would you like to contribute to this repository? Feel free to leave your comments on Issues or send a Pull Request. All types of contributions are welcome!

📩 Contact Us 💌

Need more information or wish to collaborate? Click here to send me a message. Let's share knowledge together!