simple-sequential-chain-with-llama-2-and-langchain.ipynb

{"cells":[{"source":"<a href=\"https://www.kaggle.com/code/gpreda/simple-sequential-chain-with-llama-2-and-langchain?scriptVersionId=172016669\" target=\"_blank\"><img align=\"left\" alt=\"Kaggle\" title=\"Open in Kaggle\" src=\"https://kaggle.com/static/images/open-in-kaggle.svg\"></a>","metadata":{},"cell_type":"markdown"},{"cell_type":"markdown","id":"7228dd9e","metadata":{"papermill":{"duration":0.005956,"end_time":"2024-04-14T19:03:29.654328","exception":false,"start_time":"2024-04-14T19:03:29.648372","status":"completed"},"tags":[]},"source":["<center><h1>Simple sequential chain with Llama 2 and Langchain</h1></center>\n","\n","<center><img src=\"https://eu-images.contentstack.com/v3/assets/blt6b0f74e5591baa03/blt98d8a946b63c9b5f/64b7170ab314c94aa481d8c3/Untitled_design_(1).jpg\" width=400></center>\n","\n","\n","# Introduction\n","\n","\n","## Objective  \n","\n","Use Llama 2 and Langchain to create a multi-step task chain. \n","\n","## Models details  \n","\n","* **Model #1**: Llama 2  \n","* **Variation**: 7b-chat-hf    \n","* **Version**: V1  \n","* **Framework**: PyTorch  \n","\n","\n","LlaMA 2 model is pretrained and fine-tuned with 2 Trillion tokens and 7 to 70 Billion parameters which makes it one of the powerful open source models. It is a highly improvement over LlaMA 1 model. "]},{"cell_type":"markdown","id":"7135d4ad","metadata":{"papermill":{"duration":0.005103,"end_time":"2024-04-14T19:03:29.66522","exception":false,"start_time":"2024-04-14T19:03:29.660117","status":"completed"},"tags":[]},"source":["# InstalIing, imports, utils"]},{"cell_type":"markdown","id":"65c571af","metadata":{"papermill":{"duration":0.005277,"end_time":"2024-04-14T19:03:29.676394","exception":false,"start_time":"2024-04-14T19:03:29.671117","status":"completed"},"tags":[]},"source":["Install packages."]},{"cell_type":"code","execution_count":1,"id":"9ccb2fa6","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.execute_input":"2024-04-14T19:03:29.689346Z","iopub.status.busy":"2024-04-14T19:03:29.688485Z","iopub.status.idle":"2024-04-14T19:07:01.628184Z","shell.execute_reply":"2024-04-14T19:07:01.626995Z"},"papermill":{"duration":211.948904,"end_time":"2024-04-14T19:07:01.630803","exception":false,"start_time":"2024-04-14T19:03:29.681899","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Requirement already satisfied: transformers==4.33.0 in /opt/conda/lib/python3.10/site-packages (4.33.0)\r\n","Requirement already satisfied: accelerate==0.22.0 in /opt/conda/lib/python3.10/site-packages (0.22.0)\r\n","Collecting einops==0.6.1\r\n","  Downloading einops-0.6.1-py3-none-any.whl (42 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m42.2/42.2 kB\u001b[0m \u001b[31m842.2 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting langchain==0.0.300\r\n","  Downloading langchain-0.0.300-py3-none-any.whl (1.7 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.7/1.7 MB\u001b[0m \u001b[31m10.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting xformers==0.0.21\r\n","  Downloading xformers-0.0.21-cp310-cp310-manylinux2014_x86_64.whl (167.0 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m167.0/167.0 MB\u001b[0m \u001b[31m7.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting bitsandbytes==0.41.1\r\n","  Downloading bitsandbytes-0.41.1-py3-none-any.whl (92.6 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m92.6/92.6 MB\u001b[0m \u001b[31m13.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting sentence_transformers==2.2.2\r\n","  Downloading sentence-transformers-2.2.2.tar.gz (85 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.0/86.0 kB\u001b[0m \u001b[31m8.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25h  Preparing metadata (setup.py) ... \u001b[?25l-\b \bdone\r\n","\u001b[?25hCollecting chromadb==0.4.12\r\n","  Downloading chromadb-0.4.12-py3-none-any.whl (426 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m426.5/426.5 kB\u001b[0m \u001b[31m29.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hRequirement already satisfied: filelock in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (3.12.2)\r\n","Requirement already satisfied: huggingface-hub<1.0,>=0.15.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (0.16.4)\r\n","Requirement already satisfied: numpy>=1.17 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (1.23.5)\r\n","Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (21.3)\r\n","Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (6.0)\r\n","Requirement already satisfied: regex!=2019.12.17 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (2023.6.3)\r\n","Requirement already satisfied: requests in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (2.31.0)\r\n","Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (0.13.3)\r\n","Requirement already satisfied: safetensors>=0.3.1 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (0.3.3)\r\n","Requirement already satisfied: tqdm>=4.27 in /opt/conda/lib/python3.10/site-packages (from transformers==4.33.0) (4.66.1)\r\n","Requirement already satisfied: psutil in /opt/conda/lib/python3.10/site-packages (from accelerate==0.22.0) (5.9.3)\r\n","Requirement already satisfied: torch>=1.10.0 in /opt/conda/lib/python3.10/site-packages (from accelerate==0.22.0) (2.0.0)\r\n","Requirement already satisfied: SQLAlchemy<3,>=1.4 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (2.0.17)\r\n","Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (3.8.4)\r\n","Requirement already satisfied: anyio<4.0 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (3.7.0)\r\n","Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (4.0.2)\r\n","Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (0.6.0)\r\n","Collecting jsonpatch<2.0,>=1.33 (from langchain==0.0.300)\r\n","  Downloading jsonpatch-1.33-py2.py3-none-any.whl (12 kB)\r\n","Collecting langsmith<0.1.0,>=0.0.38 (from langchain==0.0.300)\r\n","  Downloading langsmith-0.0.92-py3-none-any.whl (56 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m56.5/56.5 kB\u001b[0m \u001b[31m4.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hRequirement already satisfied: numexpr<3.0.0,>=2.8.4 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (2.8.5)\r\n","Requirement already satisfied: pydantic<3,>=1 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (1.10.9)\r\n","Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /opt/conda/lib/python3.10/site-packages (from langchain==0.0.300) (8.2.2)\r\n","Collecting torch>=1.10.0 (from accelerate==0.22.0)\r\n","  Downloading torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m619.9/619.9 MB\u001b[0m \u001b[31m2.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hRequirement already satisfied: torchvision in /opt/conda/lib/python3.10/site-packages (from sentence_transformers==2.2.2) (0.15.1)\r\n","Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.10/site-packages (from sentence_transformers==2.2.2) (1.2.2)\r\n","Requirement already satisfied: scipy in /opt/conda/lib/python3.10/site-packages (from sentence_transformers==2.2.2) (1.11.2)\r\n","Requirement already satisfied: nltk in /opt/conda/lib/python3.10/site-packages (from sentence_transformers==2.2.2) (3.2.4)\r\n","Requirement already satisfied: sentencepiece in /opt/conda/lib/python3.10/site-packages (from sentence_transformers==2.2.2) (0.1.99)\r\n","Collecting chroma-hnswlib==0.7.3 (from chromadb==0.4.12)\r\n","  Downloading chroma_hnswlib-0.7.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.4 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.4/2.4 MB\u001b[0m \u001b[31m58.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hRequirement already satisfied: fastapi<0.100.0,>=0.95.2 in /opt/conda/lib/python3.10/site-packages (from chromadb==0.4.12) (0.98.0)\r\n","Requirement already satisfied: uvicorn[standard]>=0.18.3 in /opt/conda/lib/python3.10/site-packages (from chromadb==0.4.12) (0.22.0)\r\n","Collecting posthog>=2.4.0 (from chromadb==0.4.12)\r\n","  Downloading posthog-3.5.0-py2.py3-none-any.whl (41 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m41.3/41.3 kB\u001b[0m \u001b[31m3.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hRequirement already satisfied: typing-extensions>=4.5.0 in /opt/conda/lib/python3.10/site-packages (from chromadb==0.4.12) (4.6.3)\r\n","Collecting pulsar-client>=3.1.0 (from chromadb==0.4.12)\r\n","  Downloading pulsar_client-3.5.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.4/5.4 MB\u001b[0m \u001b[31m74.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting onnxruntime>=1.14.1 (from chromadb==0.4.12)\r\n","  Downloading onnxruntime-1.17.3-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.8 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.8/6.8 MB\u001b[0m \u001b[31m62.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting pypika>=0.48.9 (from chromadb==0.4.12)\r\n","  Downloading PyPika-0.48.9.tar.gz (67 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m67.3/67.3 kB\u001b[0m \u001b[31m5.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25h  Installing build dependencies ... \u001b[?25l-\b \b\\\b \b|\b \b/\b \b-\b \b\\\b \bdone\r\n","\u001b[?25h  Getting requirements to build wheel ... \u001b[?25l-\b \bdone\r\n","\u001b[?25h  Preparing metadata (pyproject.toml) ... \u001b[?25l-\b \bdone\r\n","\u001b[?25hCollecting overrides>=7.3.1 (from chromadb==0.4.12)\r\n","  Downloading overrides-7.7.0-py3-none-any.whl (17 kB)\r\n","Requirement already satisfied: importlib-resources in /opt/conda/lib/python3.10/site-packages (from chromadb==0.4.12) (5.12.0)\r\n","Collecting bcrypt>=4.0.1 (from chromadb==0.4.12)\r\n","  Downloading bcrypt-4.1.2-cp39-abi3-manylinux_2_28_x86_64.whl (698 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m698.9/698.9 kB\u001b[0m \u001b[31m31.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hRequirement already satisfied: typer>=0.9.0 in /opt/conda/lib/python3.10/site-packages (from chromadb==0.4.12) (0.9.0)\r\n","Requirement already satisfied: sympy in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.22.0) (1.12)\r\n","Requirement already satisfied: networkx in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.22.0) (3.1)\r\n","Requirement already satisfied: jinja2 in /opt/conda/lib/python3.10/site-packages (from torch>=1.10.0->accelerate==0.22.0) (3.1.2)\r\n","Collecting nvidia-cuda-nvrtc-cu11==11.7.99 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_cuda_nvrtc_cu11-11.7.99-2-py3-none-manylinux1_x86_64.whl (21.0 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m21.0/21.0 MB\u001b[0m \u001b[31m48.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting nvidia-cuda-runtime-cu11==11.7.99 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_cuda_runtime_cu11-11.7.99-py3-none-manylinux1_x86_64.whl (849 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m849.3/849.3 kB\u001b[0m \u001b[31m47.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting nvidia-cuda-cupti-cu11==11.7.101 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_cuda_cupti_cu11-11.7.101-py3-none-manylinux1_x86_64.whl (11.8 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m11.8/11.8 MB\u001b[0m \u001b[31m61.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting nvidia-cudnn-cu11==8.5.0.96 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_cudnn_cu11-8.5.0.96-2-py3-none-manylinux1_x86_64.whl (557.1 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m557.1/557.1 MB\u001b[0m \u001b[31m1.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting nvidia-cublas-cu11==11.10.3.66 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_cublas_cu11-11.10.3.66-py3-none-manylinux1_x86_64.whl (317.1 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m317.1/317.1 MB\u001b[0m \u001b[31m3.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting nvidia-cufft-cu11==10.9.0.58 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_cufft_cu11-10.9.0.58-py3-none-manylinux1_x86_64.whl (168.4 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m168.4/168.4 MB\u001b[0m \u001b[31m7.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting nvidia-curand-cu11==10.2.10.91 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_curand_cu11-10.2.10.91-py3-none-manylinux1_x86_64.whl (54.6 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m54.6/54.6 MB\u001b[0m \u001b[31m21.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting nvidia-cusolver-cu11==11.4.0.1 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_cusolver_cu11-11.4.0.1-2-py3-none-manylinux1_x86_64.whl (102.6 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m102.6/102.6 MB\u001b[0m \u001b[31m11.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting nvidia-cusparse-cu11==11.7.4.91 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_cusparse_cu11-11.7.4.91-py3-none-manylinux1_x86_64.whl (173.2 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m173.2/173.2 MB\u001b[0m \u001b[31m7.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting nvidia-nccl-cu11==2.14.3 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_nccl_cu11-2.14.3-py3-none-manylinux1_x86_64.whl (177.1 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m177.1/177.1 MB\u001b[0m \u001b[31m6.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting nvidia-nvtx-cu11==11.7.91 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading nvidia_nvtx_cu11-11.7.91-py3-none-manylinux1_x86_64.whl (98 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m98.6/98.6 kB\u001b[0m \u001b[31m5.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting triton==2.0.0 (from torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading triton-2.0.0-1-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (63.3 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m63.3/63.3 MB\u001b[0m \u001b[31m19.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hRequirement already satisfied: setuptools in /opt/conda/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.10.0->accelerate==0.22.0) (68.0.0)\r\n","Requirement already satisfied: wheel in /opt/conda/lib/python3.10/site-packages (from nvidia-cublas-cu11==11.10.3.66->torch>=1.10.0->accelerate==0.22.0) (0.40.0)\r\n","Collecting cmake (from triton==2.0.0->torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading cmake-3.29.2-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (26.7 MB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m26.7/26.7 MB\u001b[0m \u001b[31m36.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hCollecting lit (from triton==2.0.0->torch>=1.10.0->accelerate==0.22.0)\r\n","  Downloading lit-18.1.3-py3-none-any.whl (96 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m96.4/96.4 kB\u001b[0m \u001b[31m519.0 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hRequirement already satisfied: attrs>=17.3.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (23.1.0)\r\n","Requirement already satisfied: charset-normalizer<4.0,>=2.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (3.1.0)\r\n","Requirement already satisfied: multidict<7.0,>=4.5 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (6.0.4)\r\n","Requirement already satisfied: yarl<2.0,>=1.0 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (1.9.2)\r\n","Requirement already satisfied: frozenlist>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (1.3.3)\r\n","Requirement already satisfied: aiosignal>=1.1.2 in /opt/conda/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.0.300) (1.3.1)\r\n","Requirement already satisfied: idna>=2.8 in /opt/conda/lib/python3.10/site-packages (from anyio<4.0->langchain==0.0.300) (3.4)\r\n","Requirement already satisfied: sniffio>=1.1 in /opt/conda/lib/python3.10/site-packages (from anyio<4.0->langchain==0.0.300) (1.3.0)\r\n","Requirement already satisfied: exceptiongroup in /opt/conda/lib/python3.10/site-packages (from anyio<4.0->langchain==0.0.300) (1.1.1)\r\n","Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /opt/conda/lib/python3.10/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain==0.0.300) (3.20.1)\r\n","Requirement already satisfied: typing-inspect<1,>=0.4.0 in /opt/conda/lib/python3.10/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain==0.0.300) (0.9.0)\r\n","Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /opt/conda/lib/python3.10/site-packages (from fastapi<0.100.0,>=0.95.2->chromadb==0.4.12) (0.27.0)\r\n","Requirement already satisfied: fsspec in /opt/conda/lib/python3.10/site-packages (from huggingface-hub<1.0,>=0.15.1->transformers==4.33.0) (2023.9.0)\r\n","Requirement already satisfied: jsonpointer>=1.9 in /opt/conda/lib/python3.10/site-packages (from jsonpatch<2.0,>=1.33->langchain==0.0.300) (2.0)\r\n","Collecting coloredlogs (from onnxruntime>=1.14.1->chromadb==0.4.12)\r\n","  Downloading coloredlogs-15.0.1-py2.py3-none-any.whl (46 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m46.0/46.0 kB\u001b[0m \u001b[31m175.0 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hRequirement already satisfied: flatbuffers in /opt/conda/lib/python3.10/site-packages (from onnxruntime>=1.14.1->chromadb==0.4.12) (23.5.26)\r\n","Requirement already satisfied: protobuf in /opt/conda/lib/python3.10/site-packages (from onnxruntime>=1.14.1->chromadb==0.4.12) (3.20.3)\r\n","Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/conda/lib/python3.10/site-packages (from packaging>=20.0->transformers==4.33.0) (3.0.9)\r\n","Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.10/site-packages (from posthog>=2.4.0->chromadb==0.4.12) (1.16.0)\r\n","Collecting monotonic>=1.5 (from posthog>=2.4.0->chromadb==0.4.12)\r\n","  Downloading monotonic-1.6-py2.py3-none-any.whl (8.2 kB)\r\n","Requirement already satisfied: backoff>=1.10.0 in /opt/conda/lib/python3.10/site-packages (from posthog>=2.4.0->chromadb==0.4.12) (2.2.1)\r\n","Requirement already satisfied: python-dateutil>2.1 in /opt/conda/lib/python3.10/site-packages (from posthog>=2.4.0->chromadb==0.4.12) (2.8.2)\r\n","Requirement already satisfied: certifi in /opt/conda/lib/python3.10/site-packages (from pulsar-client>=3.1.0->chromadb==0.4.12) (2023.7.22)\r\n","Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.10/site-packages (from requests->transformers==4.33.0) (1.26.15)\r\n","Requirement already satisfied: greenlet!=0.4.17 in /opt/conda/lib/python3.10/site-packages (from SQLAlchemy<3,>=1.4->langchain==0.0.300) (2.0.2)\r\n","Requirement already satisfied: click<9.0.0,>=7.1.1 in /opt/conda/lib/python3.10/site-packages (from typer>=0.9.0->chromadb==0.4.12) (8.1.7)\r\n","Requirement already satisfied: h11>=0.8 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (0.14.0)\r\n","Requirement already satisfied: httptools>=0.5.0 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (0.6.0)\r\n","Requirement already satisfied: python-dotenv>=0.13 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (1.0.0)\r\n","Requirement already satisfied: uvloop!=0.15.0,!=0.15.1,>=0.14.0 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (0.17.0)\r\n","Requirement already satisfied: watchfiles>=0.13 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (0.20.0)\r\n","Requirement already satisfied: websockets>=10.4 in /opt/conda/lib/python3.10/site-packages (from uvicorn[standard]>=0.18.3->chromadb==0.4.12) (11.0.3)\r\n","Requirement already satisfied: joblib>=1.1.1 in /opt/conda/lib/python3.10/site-packages (from scikit-learn->sentence_transformers==2.2.2) (1.3.2)\r\n","Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.10/site-packages (from scikit-learn->sentence_transformers==2.2.2) (3.1.0)\r\n","Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /opt/conda/lib/python3.10/site-packages (from torchvision->sentence_transformers==2.2.2) (9.5.0)\r\n","Requirement already satisfied: mypy-extensions>=0.3.0 in /opt/conda/lib/python3.10/site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain==0.0.300) (1.0.0)\r\n","Collecting humanfriendly>=9.1 (from coloredlogs->onnxruntime>=1.14.1->chromadb==0.4.12)\r\n","  Downloading humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\r\n","\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m86.8/86.8 kB\u001b[0m \u001b[31m9.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\r\n","\u001b[?25hRequirement already satisfied: MarkupSafe>=2.0 in /opt/conda/lib/python3.10/site-packages (from jinja2->torch>=1.10.0->accelerate==0.22.0) (2.1.3)\r\n","Requirement already satisfied: mpmath>=0.19 in /opt/conda/lib/python3.10/site-packages (from sympy->torch>=1.10.0->accelerate==0.22.0) (1.3.0)\r\n","Building wheels for collected packages: sentence_transformers, pypika\r\n","  Building wheel for sentence_transformers (setup.py) ... \u001b[?25l-\b \b\\\b \bdone\r\n","\u001b[?25h  Created wheel for sentence_transformers: filename=sentence_transformers-2.2.2-py3-none-any.whl size=125926 sha256=bd11d5b1de378ec8eb02b9aeeeb50f6e93a8c2e8a68e0f8e584c96b699781492\r\n","  Stored in directory: /root/.cache/pip/wheels/62/f2/10/1e606fd5f02395388f74e7462910fe851042f97238cbbd902f\r\n","  Building wheel for pypika (pyproject.toml) ... \u001b[?25l-\b \bdone\r\n","\u001b[?25h  Created wheel for pypika: filename=PyPika-0.48.9-py2.py3-none-any.whl size=53724 sha256=210f93daa0245c053103a1a6e1e0810c44b2901d3094846bfff30a911541af4b\r\n","  Stored in directory: /root/.cache/pip/wheels/e1/26/51/d0bffb3d2fd82256676d7ad3003faea3bd6dddc9577af665f4\r\n","Successfully built sentence_transformers pypika\r\n","Installing collected packages: pypika, monotonic, lit, bitsandbytes, pulsar-client, overrides, nvidia-nvtx-cu11, nvidia-nccl-cu11, nvidia-cusparse-cu11, nvidia-curand-cu11, nvidia-cufft-cu11, nvidia-cuda-runtime-cu11, nvidia-cuda-nvrtc-cu11, nvidia-cuda-cupti-cu11, nvidia-cublas-cu11, jsonpatch, humanfriendly, einops, cmake, chroma-hnswlib, bcrypt, posthog, nvidia-cusolver-cu11, nvidia-cudnn-cu11, langsmith, coloredlogs, onnxruntime, langchain, chromadb, triton, torch, xformers, sentence_transformers\r\n","  Attempting uninstall: overrides\r\n","    Found existing installation: overrides 6.5.0\r\n","    Uninstalling overrides-6.5.0:\r\n","      Successfully uninstalled overrides-6.5.0\r\n","  Attempting uninstall: jsonpatch\r\n","    Found existing installation: jsonpatch 1.32\r\n","    Uninstalling jsonpatch-1.32:\r\n","      Successfully uninstalled jsonpatch-1.32\r\n","  Attempting uninstall: torch\r\n","    Found existing installation: torch 2.0.0\r\n","    Uninstalling torch-2.0.0:\r\n","      Successfully uninstalled torch-2.0.0\r\n","\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\r\n","google-cloud-pubsublite 1.8.2 requires overrides<7.0.0,>=6.0.1, but you have overrides 7.7.0 which is incompatible.\r\n","jupyterlab-lsp 4.2.0 requires jupyter-lsp>=2.0.0, but you have jupyter-lsp 1.5.1 which is incompatible.\r\n","torchdata 0.6.0 requires torch==2.0.0, but you have torch 2.0.1 which is incompatible.\u001b[0m\u001b[31m\r\n","\u001b[0mSuccessfully installed bcrypt-4.1.2 bitsandbytes-0.41.1 chroma-hnswlib-0.7.3 chromadb-0.4.12 cmake-3.29.2 coloredlogs-15.0.1 einops-0.6.1 humanfriendly-10.0 jsonpatch-1.33 langchain-0.0.300 langsmith-0.0.92 lit-18.1.3 monotonic-1.6 nvidia-cublas-cu11-11.10.3.66 nvidia-cuda-cupti-cu11-11.7.101 nvidia-cuda-nvrtc-cu11-11.7.99 nvidia-cuda-runtime-cu11-11.7.99 nvidia-cudnn-cu11-8.5.0.96 nvidia-cufft-cu11-10.9.0.58 nvidia-curand-cu11-10.2.10.91 nvidia-cusolver-cu11-11.4.0.1 nvidia-cusparse-cu11-11.7.4.91 nvidia-nccl-cu11-2.14.3 nvidia-nvtx-cu11-11.7.91 onnxruntime-1.17.3 overrides-7.3.1 posthog-3.5.0 pulsar-client-3.5.0 pypika-0.48.9 sentence_transformers-2.2.2 torch-2.0.1 triton-2.0.0 xformers-0.0.21\r\n"]}],"source":["!pip install transformers==4.33.0 accelerate==0.22.0 einops==0.6.1 langchain==0.0.300 xformers==0.0.21 \\\n","bitsandbytes==0.41.1 sentence_transformers==2.2.2 chromadb==0.4.12"]},{"cell_type":"markdown","id":"0a74702e","metadata":{"papermill":{"duration":0.079386,"end_time":"2024-04-14T19:07:01.790242","exception":false,"start_time":"2024-04-14T19:07:01.710856","status":"completed"},"tags":[]},"source":["Import packages."]},{"cell_type":"code","execution_count":2,"id":"f864c333","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.execute_input":"2024-04-14T19:07:01.949064Z","iopub.status.busy":"2024-04-14T19:07:01.948265Z","iopub.status.idle":"2024-04-14T19:07:11.383284Z","shell.execute_reply":"2024-04-14T19:07:11.382502Z"},"papermill":{"duration":9.516963,"end_time":"2024-04-14T19:07:11.385583","exception":false,"start_time":"2024-04-14T19:07:01.86862","status":"completed"},"tags":[]},"outputs":[],"source":["from torch import cuda, bfloat16\n","import torch\n","import transformers\n","from transformers import AutoTokenizer\n","from time import time\n","\n","from langchain.llms import HuggingFacePipeline\n","from langchain.chains import LLMChain, SimpleSequentialChain\n","from langchain import PromptTemplate\n"]},{"cell_type":"markdown","id":"7f813903","metadata":{"papermill":{"duration":0.079019,"end_time":"2024-04-14T19:07:11.544231","exception":false,"start_time":"2024-04-14T19:07:11.465212","status":"completed"},"tags":[]},"source":["## Initialize model, tokenizer, query pipeline  \n","\n","Define the model, the device, and the bitsandbytes configuration."]},{"cell_type":"code","execution_count":3,"id":"1bdc8a70","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2024-04-14T19:07:11.707114Z","iopub.status.busy":"2024-04-14T19:07:11.706019Z","iopub.status.idle":"2024-04-14T19:07:11.832602Z","shell.execute_reply":"2024-04-14T19:07:11.831676Z"},"papermill":{"duration":0.211974,"end_time":"2024-04-14T19:07:11.834757","exception":false,"start_time":"2024-04-14T19:07:11.622783","status":"completed"},"tags":[]},"outputs":[],"source":["model_id = '/kaggle/input/llama-2/pytorch/7b-chat-hf/1'\n","\n","device = f'cuda:{cuda.current_device()}' if cuda.is_available() else 'cpu'\n","\n","# set quantization configuration to load large model with less GPU memory\n","# this requires the `bitsandbytes` library\n","bnb_config = transformers.BitsAndBytesConfig(\n","    load_in_4bit=True,\n","    bnb_4bit_quant_type='nf4',\n","    bnb_4bit_use_double_quant=True,\n","    bnb_4bit_compute_dtype=bfloat16\n",")"]},{"cell_type":"markdown","id":"81a2beb7","metadata":{"papermill":{"duration":0.084803,"end_time":"2024-04-14T19:07:11.999809","exception":false,"start_time":"2024-04-14T19:07:11.915006","status":"completed"},"tags":[]},"source":["Prepare the model and the tokenizer.  \n","\n","We perform this operation for the model 7b."]},{"cell_type":"code","execution_count":4,"id":"0609b3ea","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2024-04-14T19:07:12.175598Z","iopub.status.busy":"2024-04-14T19:07:12.174659Z","iopub.status.idle":"2024-04-14T19:09:49.87703Z","shell.execute_reply":"2024-04-14T19:09:49.875837Z"},"papermill":{"duration":157.795855,"end_time":"2024-04-14T19:09:49.879555","exception":false,"start_time":"2024-04-14T19:07:12.0837","status":"completed"},"tags":[]},"outputs":[{"name":"stderr","output_type":"stream","text":["/opt/conda/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.5\n","  warnings.warn(f\"A NumPy version >={np_minversion} and <{np_maxversion}\"\n"]},{"data":{"application/vnd.jupyter.widget-view+json":{"model_id":"c8890e270e22423899342d9fbebb8661","version_major":2,"version_minor":0},"text/plain":["Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"]},"metadata":{},"output_type":"display_data"},{"name":"stderr","output_type":"stream","text":["/opt/conda/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:362: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.9` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.\n","  warnings.warn(\n","/opt/conda/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:367: UserWarning: `do_sample` is set to `False`. However, `top_p` is set to `0.6` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `top_p`. This was detected when initializing the generation config instance, which means the corresponding file may hold incorrect parameterization and should be fixed.\n","  warnings.warn(\n"]},{"name":"stdout","output_type":"stream","text":["Prepare model #1, tokenizer: 157.695 sec.\n"]}],"source":["time_1 = time()\n","model_config = transformers.AutoConfig.from_pretrained(\n","    model_id,\n",")\n","model = transformers.AutoModelForCausalLM.from_pretrained(\n","    model_id,\n","    trust_remote_code=True,\n","    config=model_config,\n","    quantization_config=None,\n","    device_map='auto',\n",")\n","tokenizer = AutoTokenizer.from_pretrained(model_id)\n","time_2 = time()\n","print(f\"Prepare model #1, tokenizer: {round(time_2-time_1, 3)} sec.\")"]},{"cell_type":"markdown","id":"de4994a9","metadata":{"papermill":{"duration":0.080341,"end_time":"2024-04-14T19:09:50.044322","exception":false,"start_time":"2024-04-14T19:09:49.963981","status":"completed"},"tags":[]},"source":["Define a pipeline."]},{"cell_type":"code","execution_count":5,"id":"908bba62","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2024-04-14T19:09:50.207668Z","iopub.status.busy":"2024-04-14T19:09:50.207279Z","iopub.status.idle":"2024-04-14T19:09:52.642546Z","shell.execute_reply":"2024-04-14T19:09:52.641562Z"},"papermill":{"duration":2.519073,"end_time":"2024-04-14T19:09:52.644782","exception":false,"start_time":"2024-04-14T19:09:50.125709","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Prepare pipeline #1: 2.429 sec.\n"]}],"source":["time_1 = time()\n","query_pipeline = transformers.pipeline(\n","        \"text-generation\",\n","        model=model,\n","        tokenizer=tokenizer,\n","        torch_dtype=torch.float16,\n","        device_map=\"auto\",)\n","time_2 = time()\n","print(f\"Prepare pipeline #1: {round(time_2-time_1, 3)} sec.\")\n","\n","llm = HuggingFacePipeline(pipeline=query_pipeline)\n"]},{"cell_type":"markdown","id":"1ad3b76b","metadata":{"papermill":{"duration":0.084082,"end_time":"2024-04-14T19:09:52.810816","exception":false,"start_time":"2024-04-14T19:09:52.726734","status":"completed"},"tags":[]},"source":["\n","We test it by running a simple query."]},{"cell_type":"code","execution_count":6,"id":"62b0662e","metadata":{"execution":{"iopub.execute_input":"2024-04-14T19:09:52.973775Z","iopub.status.busy":"2024-04-14T19:09:52.973216Z","iopub.status.idle":"2024-04-14T19:09:52.978798Z","shell.execute_reply":"2024-04-14T19:09:52.977964Z"},"papermill":{"duration":0.08782,"end_time":"2024-04-14T19:09:52.980522","exception":false,"start_time":"2024-04-14T19:09:52.892702","status":"completed"},"tags":[]},"outputs":[],"source":["from IPython.display import display, Markdown\n","\n","def colorize_text(text):\n","    for word, color in zip([\"Reasoning\", \"Question\", \"Answer\", \"Total time\"], [\"blue\", \"red\", \"green\", \"magenta\"]):\n","        text = text.replace(f\"{word}:\", f\"\\n\\n**<font color='{color}'>{word}:</font>**\")\n","    return text"]},{"cell_type":"code","execution_count":7,"id":"5dc40576","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2024-04-14T19:09:53.141706Z","iopub.status.busy":"2024-04-14T19:09:53.140699Z","iopub.status.idle":"2024-04-14T19:09:58.802845Z","shell.execute_reply":"2024-04-14T19:09:58.801877Z"},"papermill":{"duration":5.744585,"end_time":"2024-04-14T19:09:58.805139","exception":false,"start_time":"2024-04-14T19:09:53.060554","status":"completed"},"tags":[]},"outputs":[{"name":"stderr","output_type":"stream","text":["/opt/conda/lib/python3.10/site-packages/transformers/generation/utils.py:1417: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation )\n","  warnings.warn(\n"]},{"data":{"text/markdown":["\n","\n","\n","\n","**<font color='green'>Answer:</font>** Escargots (Snails)\n","\n","\n","\n","**<font color='magenta'>Total time:</font>** 5.65 sec."],"text/plain":["<IPython.core.display.Markdown object>"]},"metadata":{},"output_type":"display_data"}],"source":["# checking model\n","t = time()\n","response = llm(prompt=\"What is the most popular food in France for tourists? Just return the name of the food.\")\n","display(Markdown(colorize_text(f\"{response}\\n\\nTotal time: {round(time()-t, 2)} sec.\")))"]},{"cell_type":"markdown","id":"a7f6cc51","metadata":{"papermill":{"duration":0.082033,"end_time":"2024-04-14T19:09:58.973431","exception":false,"start_time":"2024-04-14T19:09:58.891398","status":"completed"},"tags":[]},"source":["# Define and execute the sequential chain\n","\n","\n","We define a chain with two tasks in sequence.  \n","The input for the second step is the output of the first step.  "]},{"cell_type":"code","execution_count":8,"id":"3e1a1eee","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2024-04-14T19:09:59.138912Z","iopub.status.busy":"2024-04-14T19:09:59.138543Z","iopub.status.idle":"2024-04-14T19:09:59.145555Z","shell.execute_reply":"2024-04-14T19:09:59.144687Z"},"papermill":{"duration":0.091837,"end_time":"2024-04-14T19:09:59.147426","exception":false,"start_time":"2024-04-14T19:09:59.055589","status":"completed"},"tags":[]},"outputs":[],"source":["def sequential_chain(country, llm):\n","    \"\"\"\n","    Args:\n","        country: country selected\n","    Returns:\n","        None\n","    \"\"\"\n","    time_1 = time()\n","    template = \"What is the most popular food in {country} for tourists? Just return the name of the food.\"\n","\n","    #  first task in chain\n","    first_prompt = PromptTemplate(\n","\n","    input_variables=[\"country\"],\n","\n","    template=template)\n","\n","    chain_one = LLMChain(llm = llm, prompt = first_prompt)\n","\n","    # second step in chain\n","    second_prompt = PromptTemplate(\n","\n","    input_variables=[\"food\"],\n","\n","    template=\"What are the top three ingredients in {food}. Just return the answer as three bullet points.\",)\n","\n","    chain_two = LLMChain(llm=llm, prompt=second_prompt)\n","\n","    # combine the two steps and run the chain sequence\n","    overall_chain = SimpleSequentialChain(chains=[chain_one, chain_two], verbose=True)\n","    overall_chain.run(country)\n","    time_2 = time()\n","    print(f\"Run sequential chain: {round(time_2-time_1, 3)} sec.\")"]},{"cell_type":"markdown","id":"b46ce39f","metadata":{"papermill":{"duration":0.080322,"end_time":"2024-04-14T19:09:59.308794","exception":false,"start_time":"2024-04-14T19:09:59.228472","status":"completed"},"tags":[]},"source":["Test the sequence with Llama v2 **7b** chat HF model."]},{"cell_type":"code","execution_count":9,"id":"bfa106a4","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2024-04-14T19:09:59.476265Z","iopub.status.busy":"2024-04-14T19:09:59.47591Z","iopub.status.idle":"2024-04-14T19:10:03.046876Z","shell.execute_reply":"2024-04-14T19:10:03.0456Z"},"papermill":{"duration":3.657172,"end_time":"2024-04-14T19:10:03.049343","exception":false,"start_time":"2024-04-14T19:09:59.392171","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["\n","\n","\u001b[1m> Entering new SimpleSequentialChain chain...\u001b[0m\n","\u001b[36;1m\u001b[1;3m\n","\n","Answer: Escargots (Snails)\u001b[0m\n","\u001b[33;1m\u001b[1;3m\n","\n","* Escargots (Snails)\n","* Garlic\n","* Butter\u001b[0m\n","\n","\u001b[1m> Finished chain.\u001b[0m\n","Run sequential chain: 3.566 sec.\n"]}],"source":["final_answer = sequential_chain(\"France\", llm)"]},{"cell_type":"markdown","id":"8bb0ed66","metadata":{"papermill":{"duration":0.07978,"end_time":"2024-04-14T19:10:03.210885","exception":false,"start_time":"2024-04-14T19:10:03.131105","status":"completed"},"tags":[]},"source":["Test the sequence with Llama v2 **7b** chat HF model."]},{"cell_type":"code","execution_count":10,"id":"8b965fc3","metadata":{"_kg_hide-input":true,"execution":{"iopub.execute_input":"2024-04-14T19:10:03.371411Z","iopub.status.busy":"2024-04-14T19:10:03.371019Z","iopub.status.idle":"2024-04-14T19:10:07.25637Z","shell.execute_reply":"2024-04-14T19:10:07.255429Z"},"papermill":{"duration":3.968509,"end_time":"2024-04-14T19:10:07.25853","exception":false,"start_time":"2024-04-14T19:10:03.290021","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["\n","\n","\u001b[1m> Entering new SimpleSequentialChain chain...\u001b[0m\n","\u001b[36;1m\u001b[1;3m\n","\n","Answer: Pizza.\u001b[0m\n","\u001b[33;1m\u001b[1;3m\n","\n","Top three ingredients in pizza:\n","\n","• Cheese\n","• Tomato sauce\n","• Pepperoni\u001b[0m\n","\n","\u001b[1m> Finished chain.\u001b[0m\n","Run sequential chain: 3.881 sec.\n"]}],"source":["final_answer = sequential_chain(\"Italy\", llm)"]},{"cell_type":"markdown","id":"06ad8ffe","metadata":{"papermill":{"duration":0.085131,"end_time":"2024-04-14T19:10:07.425728","exception":false,"start_time":"2024-04-14T19:10:07.340597","status":"completed"},"tags":[]},"source":["# Conclusions\n","\n","The model answers are correct and the time for answering is in acceptable limits (less than 5 seconds for two steps-task)."]}],"metadata":{"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"modelInstanceId":3093,"sourceId":4298,"sourceType":"modelInstanceVersion"}],"dockerImageVersionId":30559,"isGpuEnabled":true,"isInternetEnabled":true,"language":"python","sourceType":"notebook"},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.12"},"papermill":{"default_parameters":{},"duration":405.577643,"end_time":"2024-04-14T19:10:10.548765","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-04-14T19:03:24.971122","version":"2.4.0"},"widgets":{"application/vnd.jupyter.widget-state+json":{"state":{"0bbd90eb42d7410eb5369cb3f301e3cf":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"516d6fc18f2f44aab18032daf4d5794b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_e4e5432631d24c81b45f4ffc81c51288","placeholder":"​","style":"IPY_MODEL_ca0b543716ed4559b0bf53019f95d90b","value":"Loading checkpoint shards: 100%"}},"5aeda9151c164b80ac745356ad48adca":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"FloatProgressModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"FloatProgressModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"ProgressView","bar_style":"success","description":"","description_tooltip":null,"layout":"IPY_MODEL_0bbd90eb42d7410eb5369cb3f301e3cf","max":2.0,"min":0.0,"orientation":"horizontal","style":"IPY_MODEL_c03969415485488a93844963d8f6014a","value":2.0}},"698643c6bd014d6ebf9e17ab64b1974a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"6ed89abe1ff24064b61143dd403d7c44":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c03969415485488a93844963d8f6014a":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"ProgressStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"ProgressStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","bar_color":null,"description_width":""}},"c3103eda23e347a9b203609ab48ca235":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}},"c8890e270e22423899342d9fbebb8661":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HBoxModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HBoxModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HBoxView","box_style":"","children":["IPY_MODEL_516d6fc18f2f44aab18032daf4d5794b","IPY_MODEL_5aeda9151c164b80ac745356ad48adca","IPY_MODEL_ce529187f867476bbcef4b8266d4156b"],"layout":"IPY_MODEL_6ed89abe1ff24064b61143dd403d7c44"}},"ca0b543716ed4559b0bf53019f95d90b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"DescriptionStyleModel","state":{"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"DescriptionStyleModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"StyleView","description_width":""}},"ce529187f867476bbcef4b8266d4156b":{"model_module":"@jupyter-widgets/controls","model_module_version":"1.5.0","model_name":"HTMLModel","state":{"_dom_classes":[],"_model_module":"@jupyter-widgets/controls","_model_module_version":"1.5.0","_model_name":"HTMLModel","_view_count":null,"_view_module":"@jupyter-widgets/controls","_view_module_version":"1.5.0","_view_name":"HTMLView","description":"","description_tooltip":null,"layout":"IPY_MODEL_c3103eda23e347a9b203609ab48ca235","placeholder":"​","style":"IPY_MODEL_698643c6bd014d6ebf9e17ab64b1974a","value":" 2/2 [02:17&lt;00:00, 63.25s/it]"}},"e4e5432631d24c81b45f4ffc81c51288":{"model_module":"@jupyter-widgets/base","model_module_version":"1.2.0","model_name":"LayoutModel","state":{"_model_module":"@jupyter-widgets/base","_model_module_version":"1.2.0","_model_name":"LayoutModel","_view_count":null,"_view_module":"@jupyter-widgets/base","_view_module_version":"1.2.0","_view_name":"LayoutView","align_content":null,"align_items":null,"align_self":null,"border":null,"bottom":null,"display":null,"flex":null,"flex_flow":null,"grid_area":null,"grid_auto_columns":null,"grid_auto_flow":null,"grid_auto_rows":null,"grid_column":null,"grid_gap":null,"grid_row":null,"grid_template_areas":null,"grid_template_columns":null,"grid_template_rows":null,"height":null,"justify_content":null,"justify_items":null,"left":null,"margin":null,"max_height":null,"max_width":null,"min_height":null,"min_width":null,"object_fit":null,"object_position":null,"order":null,"overflow":null,"overflow_x":null,"overflow_y":null,"padding":null,"right":null,"top":null,"visibility":null,"width":null}}},"version_major":2,"version_minor":0}}},"nbformat":4,"nbformat_minor":5}