sentence_transformers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f506b3a9-ea3d-485f-b6ce-a397fa1733e0",
   "metadata": {
    "id": "f506b3a9-ea3d-485f-b6ce-a397fa1733e0"
   },
   "source": [
    "# Sentence Transformers 학습과 활용"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b90a5cb-440e-42b4-850d-11b651870aa5",
   "metadata": {
    "id": "0b90a5cb-440e-42b4-850d-11b651870aa5"
   },
   "source": [
    "본 노트북에서는 `klue/roberta-base` 모델을 **KLUE** 내 **STS** 데이터셋을 활용하여 모델을 훈련하는 예제를 다루게 됩니다.\n",
    "\n",
    "학습을 통해 얻어질 `sentence-klue-roberta-base` 모델은 입력된 문장의 임베딩을 계산해 유사도를 예측하는데 사용할 수 있게 됩니다.\n",
    "\n",
    "학습 과정 이후에는 간단한 예제 코드를 통해 모델이 어떻게 활용되는지도 함께 알아보도록 할 것입니다.\n",
    "\n",
    "모든 소스 코드는 [`sentence-transformers`](https://github.com/UKPLab/sentence-transformers) 원 라이브러리를 참고하였습니다.\n",
    "\n",
    "먼저, 노트북을 실행하는데 필요한 라이브러리를 설치합니다. 모델 훈련을 위해서는 `sentence-transformers`가, 학습 데이터셋 로드를 위해서는 `datasets` 라이브러리의 설치가 필요합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cad90326-2557-4b86-a42a-ca5033a09cfd",
   "metadata": {
    "id": "cad90326-2557-4b86-a42a-ca5033a09cfd"
   },
   "outputs": [],
   "source": [
    "!pip install sentence-transformers datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b50e01af-199f-4fdf-b934-4211d8bf4592",
   "metadata": {
    "id": "b50e01af-199f-4fdf-b934-4211d8bf4592"
   },
   "source": [
    "## Sentence Transformers 학습"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e431cd47-5f9b-44a0-a1a9-8aa500eb89ff",
   "metadata": {
    "id": "e431cd47-5f9b-44a0-a1a9-8aa500eb89ff"
   },
   "source": [
    "노트북을 실행하는데 필요한 라이브러리들을 모두 임포트합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d21b5400",
   "metadata": {
    "id": "d21b5400"
   },
   "outputs": [],
   "source": [
    "import math\n",
    "import logging\n",
    "from datetime import datetime\n",
    "\n",
    "import torch\n",
    "from torch.utils.data import DataLoader\n",
    "from datasets import load_dataset\n",
    "from sentence_transformers import SentenceTransformer,  LoggingHandler, losses, models, util\n",
    "from sentence_transformers.evaluation import EmbeddingSimilarityEvaluator\n",
    "from sentence_transformers.readers import InputExample"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bb62c66-83b4-43c3-a709-18080a816105",
   "metadata": {
    "id": "9bb62c66-83b4-43c3-a709-18080a816105"
   },
   "source": [
    "학습 경과를 지켜보는데 사용될 *logger* 를 초기화합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "238f04b0",
   "metadata": {
    "id": "238f04b0"
   },
   "outputs": [],
   "source": [
    "logging.basicConfig(\n",
    "    format=\"%(asctime)s - %(message)s\",\n",
    "    datefmt=\"%Y-%m-%d %H:%M:%S\",\n",
    "    level=logging.INFO,\n",
    "    handlers=[LoggingHandler()],\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28fb0c2f",
   "metadata": {
    "id": "28fb0c2f"
   },
   "source": [
    "학습에 필요한 정보를 변수로 기록합니다.\n",
    "\n",
    "본 노트북에서는 `klue-roberta-base` 모델을 활용하지만, https://huggingface.co/klue 페이지에서 더 다양한 사전학습 언어 모델을 확인하실 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "536bc892",
   "metadata": {
    "id": "536bc892"
   },
   "outputs": [],
   "source": [
    "model_name = \"klue/roberta-base\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef35e70f-1322-4dc2-b903-25248555e779",
   "metadata": {
    "id": "ef35e70f-1322-4dc2-b903-25248555e779"
   },
   "source": [
    "모델 정보 외에도 학습에 필요한 하이퍼 파라미터를 정의합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c7484dcc",
   "metadata": {
    "id": "c7484dcc"
   },
   "outputs": [],
   "source": [
    "train_batch_size = 32\n",
    "num_epochs = 4\n",
    "model_save_path = \"output/training_klue_sts_\" + model_name.replace(\"/\", \"-\") + \"-\" + datetime.now().strftime(\"%Y-%m-%d_%H-%M-%S\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8e0d438e-5f39-419a-a25b-4c78a0a4d2a7",
   "metadata": {
    "id": "8e0d438e-5f39-419a-a25b-4c78a0a4d2a7"
   },
   "source": [
    "앞서 정의한 사전학습 언어 모델을 로드합니다.\n",
    "\n",
    "`sentence-transformers`는 HuggingFace의 `transformers`와 호환이 잘 이루어지고 있기 때문에, [모델 허브](https://huggingface.co/models)에 올라와있는 대부분의 언어 모델을 임베딩을 추출할 *Embedder* 로 사용할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5dd44fce",
   "metadata": {
    "id": "5dd44fce"
   },
   "outputs": [],
   "source": [
    "embedding_model = models.Transformer(model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97e09dc6-5115-499e-a409-8b9b3876ce30",
   "metadata": {
    "id": "97e09dc6-5115-499e-a409-8b9b3876ce30"
   },
   "source": [
    "*Embedder* 에서 추출된 토큰 단위 임베딩들을 가지고 문장 임베딩을 어떻게 계산할 것인지를 결정하는 *Pooler* 를 정의합니다.\n",
    "\n",
    "여러 Pooling 기법이 있겠지만, 예제 노트북에서는 **Mean Pooling**을 사용하기로 합니다.\n",
    "\n",
    "**Mean Pooling**이란 모델이 반환한 모든 토큰 임베딩을 더해준 후, 더해진 토큰 개수만큼 나누어 문장을 대표하는 임베딩으로 사용하는 기법을 의미합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "361b75b0",
   "metadata": {
    "id": "361b75b0"
   },
   "outputs": [],
   "source": [
    "pooler = models.Pooling(\n",
    "    embedding_model.get_word_embedding_dimension(),\n",
    "    pooling_mode_mean_tokens=True,\n",
    "    pooling_mode_cls_token=False,\n",
    "    pooling_mode_max_tokens=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34e61463-27d9-4538-ad54-74aa08383857",
   "metadata": {
    "id": "34e61463-27d9-4538-ad54-74aa08383857"
   },
   "source": [
    "*Embedder* 와 *Pooler* 를 정의했으므로, 이 두 모듈로 구성된 하나의 모델을 정의합니다.\n",
    "\n",
    "`modules`에 입력으로 들어가는 모듈이 순차적으로 임베딩 과정에 사용이 된다고 생각하시면 됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50a967d3-6118-4b02-a5b6-1815c8125510",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "50a967d3-6118-4b02-a5b6-1815c8125510",
    "outputId": "521faef8-873c-445b-e3c7-5b5b1a5b6be5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-06-18 16:26:30 - Use pytorch device: cuda\n"
     ]
    }
   ],
   "source": [
    "model = SentenceTransformer(modules=[embedding_model, pooler])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06323b50-d9ac-4c37-8d16-d81ea1104a11",
   "metadata": {
    "id": "06323b50-d9ac-4c37-8d16-d81ea1104a11"
   },
   "source": [
    "이제 학습에 사용될 KLUE STS 데이터셋을 다운로드 및 로드합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "635169fc-d2aa-4113-99f8-a64ab5bde282",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 218,
     "referenced_widgets": [
      "fc7374f1bb9e4c5185bf9dc2ba45966c",
      "f366b4862135486c925f1a6d55cf0b3e",
      "25d5b898e09a485792aab85b8e1adcd9",
      "89206a925fce40c282ad389ddac46b92",
      "72ae9ecb01e744d0a5746c3e1d06e16e",
      "4e4ab7edd35b4cbf98bf936d9c9083e9",
      "96a3579f62064b0f83e652dd51b2bd14",
      "b444af2d690c44598824edacfcbe2c7f",
      "356cf5005b7d4685b03bb45e8aaa1955",
      "0cc015495ef1473c9185df6ef619acc8",
      "ca29160383634760a651f8168fd4619d",
      "80c9223d0d334dcd86ef2da07592d841",
      "386ca11e61b240f8bb96e5ced006c2b5",
      "f4ea9066b3b0497f8440537580e478d9",
      "7051593e8d1c46f8ae18992ef7d565b6",
      "1b38b2bb686641cfb62702391e752c01",
      "0a51622362424fc0a86fdc9dcc470a97",
      "f6ab48b6dff043f985ffd9fc5140506a",
      "fc31aa88c29342a4b0e94eee9013e2fc",
      "8d4f688dd2ce4a54b5811590f65c5ac6",
      "d04c49de6dfe4ebd994f20e6e9c1fba3",
      "08bd4b003c174df2a25bd8788dd516f8",
      "91fff2991acc45d4a466d5e79e01dfe8",
      "032acc3e44044975ae678ec3fdcced83",
      "3e9c9dcd669044a283e864e894e6576b",
      "41a8570a0c1e48a0bb0e5cb3319139c7",
      "105822d482834e55b75da6280a149144",
      "db287f5471e846eb8f188f7d34d48a0f",
      "4f4a5d94054047c9910b9861ead6325d",
      "8e7be76c92394ab292eda03240ac54a7",
      "48c3976b084a4ada9fc4225904eceab8",
      "c5a3f0ee7f56484e85ab570a53fedd79",
      "f373dc73be2b4ee88c1bc3f6a6a2f935",
      "dcb49ed472364f54958664bb39bdaec2",
      "e8c4cd2098ff4152879ae0deaf230e2f",
      "00548206f84149dfa4f9fbc82c307006",
      "8d0142893b3f404d83971d3886c2d9b9",
      "4d0afb67f54a42af9976d167fe9606c5",
      "a9bd945bb0f741c3964b8a0894df72bd",
      "622e299187ff4e97ac193396b3065f91"
     ]
    },
    "id": "635169fc-d2aa-4113-99f8-a64ab5bde282",
    "outputId": "bb16dbdf-f9ff-4e9b-8596-76252f3ecd91"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fc7374f1bb9e4c5185bf9dc2ba45966c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=5191.0, style=ProgressStyle(description…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "356cf5005b7d4685b03bb45e8aaa1955",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=2932.0, style=ProgressStyle(description…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Downloading and preparing dataset klue/sts (download: 1.29 MiB, generated: 2.82 MiB, post-processed: Unknown size, total: 4.11 MiB) to /root/.cache/huggingface/datasets/klue/sts/1.0.0/55ff8f92b7a4b9842be6514ce0b4b5295b46d5e493f8bb5760da4be717018f90...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0a51622362424fc0a86fdc9dcc470a97",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=1349875.0, style=ProgressStyle(descript…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3e9c9dcd669044a283e864e894e6576b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f373dc73be2b4ee88c1bc3f6a6a2f935",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "Dataset klue downloaded and prepared to /root/.cache/huggingface/datasets/klue/sts/1.0.0/55ff8f92b7a4b9842be6514ce0b4b5295b46d5e493f8bb5760da4be717018f90. Subsequent calls will reuse this data.\n"
     ]
    }
   ],
   "source": [
    "datasets = load_dataset(\"klue\", \"sts\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c899baa-e1c5-43f1-9324-9222b7ea495f",
   "metadata": {
    "id": "7c899baa-e1c5-43f1-9324-9222b7ea495f"
   },
   "source": [
    "다운로드 혹은 로드 후 얻어진 `datasets` 객체를 살펴보면, 훈련 데이터와 검증 데이터가 포함되어 있는 것을 확인할 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11ec808e-acb8-4770-ad3e-7e8aa5de2b22",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "11ec808e-acb8-4770-ad3e-7e8aa5de2b22",
    "outputId": "e0ebcde7-b99c-49b9-a4e0-3d49588e1a35"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['guid', 'labels', 'sentence1', 'sentence2', 'source'],\n",
       "        num_rows: 11668\n",
       "    })\n",
       "    validation: Dataset({\n",
       "        features: ['guid', 'labels', 'sentence1', 'sentence2', 'source'],\n",
       "        num_rows: 519\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 10,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datasets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbdb3c8a-e64f-490e-a864-534018c05fea",
   "metadata": {
    "id": "bbdb3c8a-e64f-490e-a864-534018c05fea"
   },
   "source": [
    "각 예시 데이터는 아래와 같이 두 개의 문장과 두 문장의 유사도를 라벨로 지니고 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c81ed4e3-8ccc-4362-96ea-8ac5e8d42523",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "c81ed4e3-8ccc-4362-96ea-8ac5e8d42523",
    "outputId": "a21b8695-f9f9-45a0-8cfc-544c4a3426a7"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'guid': 'klue-sts-v1_train_00000',\n",
       " 'labels': {'binary-label': 1, 'label': 3.7, 'real-label': 3.714285714285714},\n",
       " 'sentence1': '숙소 위치는 찾기 쉽고 일반적인 한국의 반지하 숙소입니다.',\n",
       " 'sentence2': '숙박시설의 위치는 쉽게 찾을 수 있고 한국의 대표적인 반지하 숙박시설입니다.',\n",
       " 'source': 'airbnb-rtt'}"
      ]
     },
     "execution_count": 11,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "datasets[\"train\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c6780db-0594-4bfe-a9fd-46e03e171092",
   "metadata": {
    "id": "0c6780db-0594-4bfe-a9fd-46e03e171092"
   },
   "source": [
    "이제 테스트에 활용할 데이터를 얻어야 할 차례입니다.\n",
    "\n",
    "위에서 살펴본 바와 같이 KLUE 내 STS 데이터셋은 테스트 데이터셋을 포함하고 있지 않습니다.\n",
    "\n",
    "따라서 실습의 원활한 진행을 위해 다른 벤치마크 STS 데이터셋인 KorSTS 데이터셋을 다운로드 및 로드하여 사용하도록 하겠습니다.\n",
    "\n",
    "(\\* 두 데이터셋은 제작 과정이 엄밀히 다르므로, KLUE STS 데이터에 대해 학습된 모델이 KorSTS 테스트셋에 대해 기록하는 점수은 사실상 큰 의미가 없을 수 있습니다. 전체적인 훈련 프로세스의 이해를 돕기 위해 사용한다고 생각해주시는게 좋습니다.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c09d37e9-9424-4bfb-abef-67d40cda624a",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 316,
     "referenced_widgets": [
      "b159bcf2d5d74211bc391b8fc08d9d8a",
      "dff1f823541f4c5896e2c8c34ea07953",
      "7e578efc091c458d8bda6899cee9b913",
      "dd315967ced947bb98bc2cc95a62c23f",
      "d18c42f6f51242728f111f5a1ca9cfcb",
      "91899de43539471ea34e27161c126799",
      "ea1abe864b4f4fdb92cf00373dde56ac",
      "d79df4c82bdb405f91623f1157d652c6",
      "b40b278c5d3548d3a35e8cb39a2233aa",
      "94cd2773d9e44c908e0a411c6fc6a90d",
      "e529a66bfd2f4eec9e2a98bc28fdf94a",
      "525993c10abe4052b21da590ff3f9c88",
      "0b6ecc67d864453593098647fc18c78e",
      "07e1587a70e749fd85a4d77813e67bcd",
      "52c2e05505974578adacb96202e4ac03",
      "c672854515244b07a840750c81d93486",
      "7491d0c59e314c17a340880286cace2c",
      "692225963d2d434881ebbb7ad5ecab42",
      "c1d67854eea545699c72a282bf169725",
      "6b9d09dd3ddb430a82b6342b29908b6c",
      "55c4117d3cae41a39db19c10f09653bb",
      "e4a0935bcfc74f34a0fab107506696a0",
      "7139cfd2150743af994aad6e26d3a73c",
      "f172860e4aea40f796f0671df12393eb",
      "d951edfda0a84ba196ab9c1ed2e00856",
      "942e110b651d4a9ab8c19fe1b16b6b66",
      "8942aa54279f4c048d1132fd31857c7a",
      "331b193159854002a67c02e7252b61c4",
      "9a67fd098c9f4ceb9e73d3181e1b90eb",
      "0497dd703317433a8369d67b805ce5d8",
      "fee8569d1c9e47baab3439c2661549ce",
      "a48999c9c915448eb18c0a059353ee9e",
      "97b1a3c1e5a84ba9805d2a888d8fbd04",
      "b7ca38c550e04d83a7d872dcc4e76b5d",
      "f35ba4753cf94e12adaac95b15e832e2",
      "1576fda5d9a84b20824081442e71c579",
      "28bb99538bf64d78b8430add8ca8bede",
      "babef37d61e24cafba6a4fee968a5d50",
      "ddb97d3ec0be4d43a0c500b3b976d3ff",
      "0e33d6a4ecdf4e9690b59d7252d5628d",
      "ae6ee389236a4af09075d2e46ffed8fc",
      "a1ecc0d99a12498abc32c6e68beeda61",
      "67cf4d8223b34fb69d8ceb4cb0b6baf8",
      "fe3a2818127a4406ae53682ab075563a",
      "334b671a7f5240679fafae3dc367dacc",
      "814859a2218f4ae69e24ad9067e36d1d",
      "51576b520f4f4b8aa53e20d7aef543b3",
      "46e0a3dd08474d32ae86816095225f12",
      "be68bd601d554ca693a7c4d5e1ce7b20",
      "47fc6e8fc71d4664ac375beb323577ad",
      "ec14d701ee314261920530616c8dbfad",
      "e6ca2ed570eb40bdbb67f15fee0d8ed1",
      "c924ed150174410f8f6fbf0da83aee40",
      "9a48e63c2a3b4c5bb0b206f382635301",
      "9fc608766a5146df913f35e5b9c0810a",
      "7e5fbb8dbce945e399add4d7d675159a",
      "ed529787a715476583c1c2ca5353e39b",
      "92644e0d7b4146eb9fc0fa6e53788935",
      "b2755bab103540fc815bfd5c25ec1aab",
      "aa519372a89c4fc39fcc0c5d1735a03c",
      "d537eecb55e04535958882b45b91d217",
      "3b43878106ca40b0ae0780456848ba5b",
      "55c9425d605b4105bcdc014e1a946e84",
      "ee9d8918d3ce4ae284ebceda8563f005"
     ]
    },
    "id": "c09d37e9-9424-4bfb-abef-67d40cda624a",
    "outputId": "650ae2e2-bc75-4579-bef7-4071150544c3"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b159bcf2d5d74211bc391b8fc08d9d8a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=1781.0, style=ProgressStyle(description…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b40b278c5d3548d3a35e8cb39a2233aa",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=1371.0, style=ProgressStyle(description…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Downloading and preparing dataset kor_nlu/sts (download: 1.53 MiB, generated: 1.54 MiB, post-processed: Unknown size, total: 3.07 MiB) to /root/.cache/huggingface/datasets/kor_nlu/sts/1.0.0/4facbba77df60b0658056ced2052633e681a50187b9428bd5752ebd59d332ba8...\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7491d0c59e314c17a340880286cace2c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=282044.0, style=ProgressStyle(descripti…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d951edfda0a84ba196ab9c1ed2e00856",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=89895.0, style=ProgressStyle(descriptio…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "97b1a3c1e5a84ba9805d2a888d8fbd04",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Downloading', max=66129.0, style=ProgressStyle(descriptio…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ae6ee389236a4af09075d2e46ffed8fc",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "be68bd601d554ca693a7c4d5e1ce7b20",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ed529787a715476583c1c2ca5353e39b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=1.0, bar_style='info', max=1.0), HTML(value='')))"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "Dataset kor_nlu downloaded and prepared to /root/.cache/huggingface/datasets/kor_nlu/sts/1.0.0/4facbba77df60b0658056ced2052633e681a50187b9428bd5752ebd59d332ba8. Subsequent calls will reuse this data.\n"
     ]
    }
   ],
   "source": [
    "testsets = load_dataset(\"kor_nlu\", \"sts\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0155ae3e-ef28-46d2-a4c0-0624c05b2ea2",
   "metadata": {
    "id": "0155ae3e-ef28-46d2-a4c0-0624c05b2ea2"
   },
   "source": [
    "KorSTS 데이터셋은 훈련, 검증 그리고 테스트셋을 지니고 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "92474875-b1bb-46fb-9865-7a4b27de753b",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "92474875-b1bb-46fb-9865-7a4b27de753b",
    "outputId": "58359e99-1ffe-428c-f26a-137ffc4ee375"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatasetDict({\n",
       "    train: Dataset({\n",
       "        features: ['filename', 'genre', 'id', 'score', 'sentence1', 'sentence2', 'year'],\n",
       "        num_rows: 5703\n",
       "    })\n",
       "    validation: Dataset({\n",
       "        features: ['filename', 'genre', 'id', 'score', 'sentence1', 'sentence2', 'year'],\n",
       "        num_rows: 1471\n",
       "    })\n",
       "    test: Dataset({\n",
       "        features: ['filename', 'genre', 'id', 'score', 'sentence1', 'sentence2', 'year'],\n",
       "        num_rows: 1379\n",
       "    })\n",
       "})"
      ]
     },
     "execution_count": 13,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "testsets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72659878-2651-4286-b049-3e1d1cfafe34",
   "metadata": {
    "id": "72659878-2651-4286-b049-3e1d1cfafe34"
   },
   "source": [
    "KorSTS의 예시 데이터도 마찬가지로 두 문장과 두 문장 간 유사도를 지니고 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9db255fd-627d-46a8-a2e1-cf8b7a4cad93",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9db255fd-627d-46a8-a2e1-cf8b7a4cad93",
    "outputId": "c57fa8e2-529f-4b30-dc7a-2432d6b3b7df"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'filename': 2,\n",
       " 'genre': 1,\n",
       " 'id': 24,\n",
       " 'score': 2.5,\n",
       " 'sentence1': '한 소녀가 머리를 스타일링하고 있다.',\n",
       " 'sentence2': '한 소녀가 머리를 빗고 있다.',\n",
       " 'year': 6}"
      ]
     },
     "execution_count": 14,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "testsets[\"test\"][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eadcf99a-dcdc-4496-8380-978cfe5f3322",
   "metadata": {
    "id": "eadcf99a-dcdc-4496-8380-978cfe5f3322"
   },
   "source": [
    "이제 앞서 얻어진 데이터셋을 `sentence-transformers` 훈련 양식에 맞게 변환해주는 작업을 거쳐야 합니다.\n",
    "\n",
    "두 데이터 모두 0점에서 5점 사이의 값으로 유사도가 기록되었기 때문에, 0.0 ~ 1.0 스케일로 정규화를 시켜주는 작업을 거치게 됩니다.\n",
    "\n",
    "(\\* KorSTS 내 테스트셋의 경우 `None`으로 기록된 문장이 몇 개 존재하여, `None`을 걸러주는 조건이 추가되었습니다.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "647c049e",
   "metadata": {
    "id": "647c049e"
   },
   "outputs": [],
   "source": [
    "train_samples = []\n",
    "dev_samples = []\n",
    "test_samples = []\n",
    "\n",
    "# KLUE STS 내 훈련, 검증 데이터 예제 변환\n",
    "for phase in [\"train\", \"validation\"]:\n",
    "    examples = datasets[phase]\n",
    "\n",
    "    for example in examples:\n",
    "        score = float(example[\"labels\"][\"label\"]) / 5.0  # 0.0 ~ 1.0 스케일로 유사도 정규화\n",
    "\n",
    "        inp_example = InputExample(\n",
    "            texts=[example[\"sentence1\"], example[\"sentence2\"]], \n",
    "            label=score,\n",
    "        )\n",
    "\n",
    "        if phase == \"validation\":\n",
    "            dev_samples.append(inp_example)\n",
    "        else:\n",
    "            train_samples.append(inp_example)\n",
    "\n",
    "# KorSTS 내 테스트 데이터 예제 변환\n",
    "for example in testsets[\"test\"]:\n",
    "    score = float(example[\"score\"]) / 5.0\n",
    "\n",
    "    if example[\"sentence1\"] and example[\"sentence2\"]:\n",
    "        inp_example = InputExample(\n",
    "            texts=[example[\"sentence1\"], example[\"sentence2\"]],\n",
    "            label=score,\n",
    "        )\n",
    "\n",
    "    test_samples.append(inp_example)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0905167a-b301-495d-97fa-0c33e773939c",
   "metadata": {
    "id": "0905167a-b301-495d-97fa-0c33e773939c"
   },
   "source": [
    "앞선 로직을 통해 각 데이터 예제는 다음과 같이 `InputExample` 객체로 변환되게 됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5de3d5b8-6572-4d6e-bb5b-88efc9daf1be",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5de3d5b8-6572-4d6e-bb5b-88efc9daf1be",
    "outputId": "1d97316c-b8b8-4afc-e034-8246f658ea52"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(['숙소 위치는 찾기 쉽고 일반적인 한국의 반지하 숙소입니다.',\n",
       "  '숙박시설의 위치는 쉽게 찾을 수 있고 한국의 대표적인 반지하 숙박시설입니다.'],\n",
       " 0.74)"
      ]
     },
     "execution_count": 16,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_samples[0].texts, train_samples[0].label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "afb8c68a-fa6e-4730-af55-824868661135",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "afb8c68a-fa6e-4730-af55-824868661135",
    "outputId": "31eb4faa-9548-4c75-858b-7087b2891a09"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(['한 소녀가 머리를 스타일링하고 있다.', '한 소녀가 머리를 빗고 있다.'], 0.5)"
      ]
     },
     "execution_count": 17,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_samples[0].texts, test_samples[0].label"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52bd7bb4-a621-4814-b770-e5ecc8004fab",
   "metadata": {
    "id": "52bd7bb4-a621-4814-b770-e5ecc8004fab"
   },
   "source": [
    "이제 학습에 사용될 `DataLoader`와 **Loss**를 설정해주도록 합니다.\n",
    "\n",
    "`CosineSimilarityLoss`는 입력된 두 문장의 임베딩 간 코사인 유사도와 골드 라벨 간 차이를 통해 계산되게 됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c30838cd",
   "metadata": {
    "id": "c30838cd"
   },
   "outputs": [],
   "source": [
    "train_dataloader = DataLoader(\n",
    "    train_samples,\n",
    "    shuffle=True,\n",
    "    batch_size=train_batch_size,\n",
    ")\n",
    "train_loss = losses.CosineSimilarityLoss(model=model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dc603d4-77ad-4cb4-9276-161e42f11f4d",
   "metadata": {
    "id": "4dc603d4-77ad-4cb4-9276-161e42f11f4d"
   },
   "source": [
    "모델 검증에 활용할 **Evaluator** 를 정의해줍니다.\n",
    "\n",
    "앞서 얻어진 검증 데이터를 활용하여, 모델의 문장 임베딩 간 코사인 유사도가 얼마나 골드 라벨에 가까운지 계산하는 역할을 수행합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8dee8198",
   "metadata": {
    "id": "8dee8198"
   },
   "outputs": [],
   "source": [
    "evaluator = EmbeddingSimilarityEvaluator.from_input_examples(\n",
    "    dev_samples,\n",
    "    name=\"sts-dev\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "947d09d6-f086-4ccb-a213-46f95a9b1d37",
   "metadata": {
    "id": "947d09d6-f086-4ccb-a213-46f95a9b1d37"
   },
   "source": [
    "모델 학습에 사용될 **Warm up Steps**를 설정합니다.\n",
    "\n",
    "다양한 방법으로 스텝 수를 결정할 수 있겠지만, 예제 노트북에서는 원 예제 코드를 따라 훈련 배치 수의 10% 만큼으로 값을 설정합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8fdbcaa7",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "8fdbcaa7",
    "outputId": "a34bc6e9-e824-43a7-8e08-64d1de959fe5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-06-18 16:28:19 - Warmup-steps: 146\n"
     ]
    }
   ],
   "source": [
    "warmup_steps = math.ceil(len(train_dataloader) * num_epochs  * 0.1)  # 10% of train data for warm-up\n",
    "logging.info(f\"Warmup-steps: {warmup_steps}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "505e262b",
   "metadata": {
    "id": "505e262b"
   },
   "source": [
    "이제 앞서 얻어진 객체, 값들을 가지고 모델의 훈련을 진행합니다.\n",
    "\n",
    "`sentence-transformers`에서는 다음과 같이 `fit` 함수를 통해 간단히 모델의 훈련과 검증이 가능합니다.\n",
    "\n",
    "훈련 과정을 통해 매 에폭 마다 얻어지는 체크포인트에 대해 *Evaluator* 가 학습된 모델의 코사인 유사도와 골드 라벨 간 피어슨, 스피어만 상관 계수를 계산해 기록을 남기게 됩니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "360274d4",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 670,
     "referenced_widgets": [
      "7a2f0db32cc24798a728e9a674b1ac96",
      "f3ca9da02dd64784a0287359e95a9dd7",
      "7c5bd980c44044a897fcc5d9121e39b0",
      "65e86b2504194c8bb8ff28715e729143",
      "5cfa167c683a40ae8cc4d0bf6c007588",
      "98f8203831204f3491002c875292829a",
      "f6c37b9336b24293a0d94ce7bf31273d",
      "127a23b75b164c8f8a8a59deedc17a1f",
      "8398ef9da1464b008ebc053f65a52b40",
      "25059b0da3d344dfb872c5091c52ca0b",
      "88c3864ad3bf4c159123d54b8316d1ca",
      "600b916033c34c22beb328bd72d5b881",
      "76ff42ef55ab467cab41a570ac238113",
      "2bdac7dbb3744d06bdb135c15309dfd3",
      "c739a8e4d6ac4df48c6ca224b32c6a4c",
      "9b167f9f16ea489d8a9dd358f0a63f1c",
      "840487532d124f5f8750d23ca26f6ec3",
      "548a46f134704494a685ebe87327609d",
      "5488a2eaa030452a9f63563296a53e55",
      "d9bdb1a3edfc46d59a6e36593b2b201f",
      "235558cc682b4b6da15a83d9595c5b96",
      "b484aaddb8684cd9b03c5acb45a6fb4a",
      "321f8883d1044d75a9aa5b9bcd365262",
      "8b897879b24c4fe9938b4d7dd9596b12",
      "d4bfe78967354804835d382cb47be8fd",
      "2bb48f10c57049b487d49dd89453aa3a",
      "2b0a0cc6d8814f68968d447d27081d5a",
      "78502be99cd04a0698042f42a028f5a1",
      "d26767b5d78046b5a0acc0cfa4a2f6e6",
      "b99b10f6c8f94ef7ad06f8c23f6cfaa8",
      "67373d86bc164f09a3d983f2c64864fa",
      "f70d3ae8707c4cccaca58ef8f5e50627",
      "123cd4f17f9e435da9ba2d2b8fa0da35",
      "767236879fd44eef92b8589f96779821",
      "526bf3d71aec4c34ad20bbdfcfcf532a",
      "f61668e72b954552bc03d5427f6376d0",
      "cbbe6b1733504a9c86fba24a8a4e0200",
      "7fa4d5bc2e294984924b977bba075a46",
      "cdfdfa26dc774f79becef859fdbabb51",
      "2c5fb679777f4863a5b24068dab16459"
     ]
    },
    "id": "360274d4",
    "outputId": "49491951-7094-4b0a-bbf9-9b07047f3e48"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "7a2f0db32cc24798a728e9a674b1ac96",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Epoch', max=4.0, style=ProgressStyle(description_width='i…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8398ef9da1464b008ebc053f65a52b40",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Iteration', max=365.0, style=ProgressStyle(description_wi…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "2021-06-18 16:31:47 - EmbeddingSimilarityEvaluator: Evaluating the model on sts-dev dataset after epoch 0:\n",
      "2021-06-18 16:31:49 - Cosine-Similarity :\tPearson: 0.8754\tSpearman: 0.8715\n",
      "2021-06-18 16:31:49 - Manhattan-Distance:\tPearson: 0.8727\tSpearman: 0.8678\n",
      "2021-06-18 16:31:49 - Euclidean-Distance:\tPearson: 0.8729\tSpearman: 0.8685\n",
      "2021-06-18 16:31:49 - Dot-Product-Similarity:\tPearson: 0.8691\tSpearman: 0.8643\n",
      "2021-06-18 16:31:49 - Save model to output/training_klue_sts_klue-roberta-base-2021-06-18_16-26-03\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "840487532d124f5f8750d23ca26f6ec3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Iteration', max=365.0, style=ProgressStyle(description_wi…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "2021-06-18 16:35:01 - EmbeddingSimilarityEvaluator: Evaluating the model on sts-dev dataset after epoch 1:\n",
      "2021-06-18 16:35:03 - Cosine-Similarity :\tPearson: 0.8836\tSpearman: 0.8839\n",
      "2021-06-18 16:35:03 - Manhattan-Distance:\tPearson: 0.8841\tSpearman: 0.8789\n",
      "2021-06-18 16:35:03 - Euclidean-Distance:\tPearson: 0.8843\tSpearman: 0.8798\n",
      "2021-06-18 16:35:03 - Dot-Product-Similarity:\tPearson: 0.8743\tSpearman: 0.8708\n",
      "2021-06-18 16:35:03 - Save model to output/training_klue_sts_klue-roberta-base-2021-06-18_16-26-03\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d4bfe78967354804835d382cb47be8fd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Iteration', max=365.0, style=ProgressStyle(description_wi…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "2021-06-18 16:38:16 - EmbeddingSimilarityEvaluator: Evaluating the model on sts-dev dataset after epoch 2:\n",
      "2021-06-18 16:38:17 - Cosine-Similarity :\tPearson: 0.8867\tSpearman: 0.8877\n",
      "2021-06-18 16:38:17 - Manhattan-Distance:\tPearson: 0.8870\tSpearman: 0.8824\n",
      "2021-06-18 16:38:17 - Euclidean-Distance:\tPearson: 0.8874\tSpearman: 0.8828\n",
      "2021-06-18 16:38:17 - Dot-Product-Similarity:\tPearson: 0.8790\tSpearman: 0.8759\n",
      "2021-06-18 16:38:17 - Save model to output/training_klue_sts_klue-roberta-base-2021-06-18_16-26-03\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "123cd4f17f9e435da9ba2d2b8fa0da35",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Iteration', max=365.0, style=ProgressStyle(description_wi…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "2021-06-18 16:41:29 - EmbeddingSimilarityEvaluator: Evaluating the model on sts-dev dataset after epoch 3:\n",
      "2021-06-18 16:41:31 - Cosine-Similarity :\tPearson: 0.8875\tSpearman: 0.8895\n",
      "2021-06-18 16:41:31 - Manhattan-Distance:\tPearson: 0.8871\tSpearman: 0.8834\n",
      "2021-06-18 16:41:31 - Euclidean-Distance:\tPearson: 0.8876\tSpearman: 0.8839\n",
      "2021-06-18 16:41:31 - Dot-Product-Similarity:\tPearson: 0.8784\tSpearman: 0.8766\n",
      "2021-06-18 16:41:31 - Save model to output/training_klue_sts_klue-roberta-base-2021-06-18_16-26-03\n",
      "\n"
     ]
    }
   ],
   "source": [
    "model.fit(\n",
    "    train_objectives=[(train_dataloader, train_loss)],\n",
    "    evaluator=evaluator,\n",
    "    epochs=num_epochs,\n",
    "    evaluation_steps=1000,\n",
    "    warmup_steps=warmup_steps,\n",
    "    output_path=model_save_path,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16458123",
   "metadata": {
    "id": "16458123"
   },
   "source": [
    "학습이 완료되었다면 이제 학습된 모델을 테스트 할 시간입니다.\n",
    "\n",
    "앞서 KorSTS 데이터를 활용해 구축한 테스트 데이터셋을 앞서와 마찬가지로 *Evaluator* 로 초기화해주도록 합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bc63a151",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bc63a151",
    "outputId": "3bbd1ba5-e242-473f-b42a-efd322d4fbee"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-06-18 16:43:02 - Load pretrained SentenceTransformer: output/training_klue_sts_klue-roberta-base-2021-06-18_16-26-03\n",
      "2021-06-18 16:43:02 - Load SentenceTransformer from folder: output/training_klue_sts_klue-roberta-base-2021-06-18_16-26-03\n",
      "2021-06-18 16:43:03 - Use pytorch device: cuda\n"
     ]
    }
   ],
   "source": [
    "model = SentenceTransformer(model_save_path)\n",
    "test_evaluator = EmbeddingSimilarityEvaluator.from_input_examples(test_samples, name='sts-test')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99746cbe-3980-49ed-8880-58f71d04d8d0",
   "metadata": {
    "id": "99746cbe-3980-49ed-8880-58f71d04d8d0"
   },
   "source": [
    "이제 테스트 *Evaluator* 를 활용하여 테스트셋에 대해 각 상관 계수를 계산하도록 합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57a412da-8106-4a9b-835f-9b9ff954a999",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "57a412da-8106-4a9b-835f-9b9ff954a999",
    "outputId": "c6f615ce-96ad-4688-9572-df2014a6fc21"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2021-06-18 16:43:07 - EmbeddingSimilarityEvaluator: Evaluating the model on sts-test dataset:\n",
      "2021-06-18 16:43:11 - Cosine-Similarity :\tPearson: 0.7758\tSpearman: 0.7639\n",
      "2021-06-18 16:43:11 - Manhattan-Distance:\tPearson: 0.7695\tSpearman: 0.7660\n",
      "2021-06-18 16:43:11 - Euclidean-Distance:\tPearson: 0.7694\tSpearman: 0.7660\n",
      "2021-06-18 16:43:11 - Dot-Product-Similarity:\tPearson: 0.7429\tSpearman: 0.7300\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0.766013168080944"
      ]
     },
     "execution_count": 24,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_evaluator(model, output_path=model_save_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9598bd1-bf9a-4abc-bf0a-d79f0b0f45a9",
   "metadata": {
    "id": "d9598bd1-bf9a-4abc-bf0a-d79f0b0f45a9"
   },
   "source": [
    "역시 검증 데이터에 비해 좋지 않은 점수를 기록하였습니다.\n",
    "\n",
    "KLUE 내 검증 데이터셋 중 일부를 샘플링하여 테스트셋으로 활용하는 방안도 있겠지만,\n",
    "\n",
    "본 노트북은 전체 훈련 프로세스를 파악하는데 초점을 맞추었으므로 실험을 마치도록 합니다."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59d23d68-36ca-4709-b74a-b548887ad93a",
   "metadata": {
    "id": "59d23d68-36ca-4709-b74a-b548887ad93a"
   },
   "source": [
    "## Sentence Transformers 활용"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b08565e0-549a-4344-b901-8a67cd60fd86",
   "metadata": {
    "id": "b08565e0-549a-4344-b901-8a67cd60fd86"
   },
   "source": [
    "### 시맨틱 서치"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13a8d2bd-29c8-4575-84d3-2c948ade5cd4",
   "metadata": {
    "id": "13a8d2bd-29c8-4575-84d3-2c948ade5cd4"
   },
   "source": [
    "입력된 문장 간 유사도를 쉽고 빠르게 구할 수 있도록 설계된 `sentence-transformers`를 이용한다면 임베딩을 활용해 다양한 어플리케이션을 고안할 수 있습니다.\n",
    "\n",
    "먼저 여러 문장 후보군이 주어졌을 때, 입력된 문장과 가장 유사한 문장을 계산하는 예제를 살펴보도록 합시다.\n",
    "\n",
    "이를 위해 검색의 대상이 되는 문장 후보군을 다음과 같이 정의할 필요가 있습니다. 이후, 정의된 문장 후보군을 미리 임베딩합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c7639791-83ba-4a9b-bfd2-936d358c0c41",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 66,
     "referenced_widgets": [
      "fd518e8d58a048fc933020524bb298a5",
      "91050df0419b49f284952ac697af5941",
      "2c60ed2462a34fc6bbddcd938443faf8",
      "b6c03c0eedd845eaada0e17cd184b46f",
      "716f62e6b2ba4e9da8180d0c529f679a",
      "140930a6d8f4422283546c5c99db167c",
      "257ac4a8485146b9afcd9d7cab4bf004",
      "d9ec0cf4628f4ee7b799bdb8e63672bc"
     ]
    },
    "id": "c7639791-83ba-4a9b-bfd2-936d358c0c41",
    "outputId": "d3595d31-c390-494b-b057-e603d9e6507a"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "fd518e8d58a048fc933020524bb298a5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Batches', max=1.0, style=ProgressStyle(description_width=…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "docs = [\n",
    "    \"1992년 7월 8일 손흥민은 강원도 춘천시 후평동에서 아버지 손웅정과 어머니 길은자의 차남으로 태어나 그곳에서 자랐다.\",\n",
    "    \"형은 손흥윤이다.\",\n",
    "    \"춘천 부안초등학교를 졸업했고, 춘천 후평중학교에 입학한 후 2학년때 원주 육민관중학교 축구부에 들어가기 위해 전학하여 졸업하였으며, 2008년 당시 FC 서울의 U-18팀이었던 동북고등학교 축구부에서 선수 활동 중 대한축구협회 우수선수 해외유학 프로젝트에 선발되어 2008년 8월 독일 분데스리가의 함부르크 유소년팀에 입단하였다.\",\n",
    "    \"함부르크 유스팀 주전 공격수로 2008년 6월 네덜란드에서 열린 4개국 경기에서 4게임에 출전, 3골을 터뜨렸다.\",\n",
    "    \"1년간의 유학 후 2009년 8월 한국으로 돌아온 후 10월에 개막한 FIFA U-17 월드컵에 출전하여 3골을 터트리며 한국을 8강으로 이끌었다.\",\n",
    "    \"그해 11월 함부르크의 정식 유소년팀 선수 계약을 체결하였으며 독일 U-19 리그 4경기 2골을 넣고 2군 리그에 출전을 시작했다.\",\n",
    "    \"독일 U-19 리그에서 손흥민은 11경기 6골, 2부 리그에서는 6경기 1골을 넣으며 재능을 인정받아 2010년 6월 17세의 나이로 함부르크의 1군 팀 훈련에 참가, 프리시즌 활약으로 함부르크와 정식 계약을 한 후 10월 18세에 함부르크 1군 소속으로 독일 분데스리가에 데뷔하였다.\",\n",
    "]\n",
    "document_embeddings = model.encode(docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f40059cf-6c9a-423c-a1e6-da0105512fd8",
   "metadata": {
    "id": "f40059cf-6c9a-423c-a1e6-da0105512fd8"
   },
   "source": [
    "이제 입력 문장을 임베딩 할 차례입니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "366338dd-1894-4b0f-9392-6afaeb11d1da",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 66,
     "referenced_widgets": [
      "a729b4890b954fc68273ad4db545d9c8",
      "1348afbc25204780ac866f0ec08149e1",
      "56dca2ceb71d4ff39b537877065fb42d",
      "dd796e4d932547509d02a45c7f7d596e",
      "2cfc03bda6a149bba24dd36a0e377136",
      "bb37968ba0594c34afed35ee7d613d67",
      "50d8892d85b3494186e30ad6253e626a",
      "285316b400ec4de0aa4ebb1911d998a8"
     ]
    },
    "id": "366338dd-1894-4b0f-9392-6afaeb11d1da",
    "outputId": "95498dfb-0b90-4a43-d4da-5a1fb1a98eec"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a729b4890b954fc68273ad4db545d9c8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Batches', max=1.0, style=ProgressStyle(description_width=…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    }
   ],
   "source": [
    "query = \"손흥민은 어린 나이에 유럽에 진출하였다.\"\n",
    "query_embedding = model.encode(query)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "466fccb3-ea9b-4eec-b342-939c3f38a154",
   "metadata": {
    "id": "466fccb3-ea9b-4eec-b342-939c3f38a154"
   },
   "source": [
    "아래는 입력된 문장의 임베딩과 미리 임베딩 된 후보군 문장 임베딩 간 유사도를 계산해 유사도가 높은 순서대로 `top_k` 개 문장을 뽑아주는 예제 코드입니다.\n",
    "\n",
    "`top_k`는 전체 문장 후보군의 개수를 넘지 않아야 하므로, `min()` 함수를 통해 예외 처리를 해줍니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f0a5354-91ba-4471-8241-43d8269298cf",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9f0a5354-91ba-4471-8241-43d8269298cf",
    "outputId": "e26b19cc-37e8-47a2-db60-9549c7705a46"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "입력 문장: 손흥민은 어린 나이에 유럽에 진출하였다.\n",
      "\n",
      "<입력 문장과 유사한 5 개의 문장>\n",
      "\n",
      "1: 독일 U-19 리그에서 손흥민은 11경기 6골, 2부 리그에서는 6경기 1골을 넣으며 재능을 인정받아 2010년 6월 17세의 나이로 함부르크의 1군 팀 훈련에 참가, 프리시즌 활약으로 함부르크와 정식 계약을 한 후 10월 18세에 함부르크 1군 소속으로 독일 분데스리가에 데뷔하였다. (유사도: 0.5897)\n",
      "\n",
      "2: 그해 11월 함부르크의 정식 유소년팀 선수 계약을 체결하였으며 독일 U-19 리그 4경기 2골을 넣고 2군 리그에 출전을 시작했다. (유사도: 0.4857)\n",
      "\n",
      "3: 1992년 7월 8일 손흥민은 강원도 춘천시 후평동에서 아버지 손웅정과 어머니 길은자의 차남으로 태어나 그곳에서 자랐다. (유사도: 0.4047)\n",
      "\n",
      "4: 함부르크 유스팀 주전 공격수로 2008년 6월 네덜란드에서 열린 4개국 경기에서 4게임에 출전, 3골을 터뜨렸다. (유사도: 0.3953)\n",
      "\n",
      "5: 춘천 부안초등학교를 졸업했고, 춘천 후평중학교에 입학한 후 2학년때 원주 육민관중학교 축구부에 들어가기 위해 전학하여 졸업하였으며, 2008년 당시 FC 서울의 U-18팀이었던 동북고등학교 축구부에서 선수 활동 중 대한축구협회 우수선수 해외유학 프로젝트에 선발되어 2008년 8월 독일 분데스리가의 함부르크 유소년팀에 입단하였다. (유사도: 0.3183)\n",
      "\n"
     ]
    }
   ],
   "source": [
    "top_k = min(5, len(docs))\n",
    "\n",
    "# 입력 문장 - 문장 후보군 간 코사인 유사도 계산 후,\n",
    "cos_scores = util.pytorch_cos_sim(query_embedding, document_embeddings)[0]\n",
    "\n",
    "# 코사인 유사도 순으로 `top_k` 개 문장 추출\n",
    "top_results = torch.topk(cos_scores, k=top_k)\n",
    "\n",
    "print(f\"입력 문장: {query}\")\n",
    "print(f\"\\n<입력 문장과 유사한 {top_k} 개의 문장>\\n\")\n",
    "\n",
    "for i, (score, idx) in enumerate(zip(top_results[0], top_results[1])):\n",
    "    print(f\"{i+1}: {docs[idx]} {'(유사도: {:.4f})'.format(score)}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2edc7686-4318-498c-9261-e4b92a5d0c4b",
   "metadata": {
    "id": "2edc7686-4318-498c-9261-e4b92a5d0c4b"
   },
   "source": [
    "반환된 문장 중 `유럽`, `어린 나이` 등의 키워드가 없음애도 높은 유사도로 두 문장이 반환된 것을 확인할 수 있습니다."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8fc9bc75-1413-410b-bd44-9f68b0f78032",
   "metadata": {
    "id": "8fc9bc75-1413-410b-bd44-9f68b0f78032"
   },
   "source": [
    "### 클러스터링"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c0fdc0c-7cb6-4538-9cd8-eb983dc086bf",
   "metadata": {
    "id": "7c0fdc0c-7cb6-4538-9cd8-eb983dc086bf"
   },
   "source": [
    "`sentence-transformers`를 통해 얻어진 임베딩을 활용해 클러스터링을 수행할 수도 있습니다.\n",
    "\n",
    "다양한 클러스터링 기법의 적용이 가능하겠지만, 본 노트북에서는 **k-Means** 클러스터링을 수행한 결과를 살펴보도록 합니다.\n",
    "\n",
    "예제 수행을 위해 `scikit-learn`의 설치가 추가로 필요합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "be615a4d-f482-4d24-b77c-431eeefa5b6c",
   "metadata": {
    "id": "be615a4d-f482-4d24-b77c-431eeefa5b6c"
   },
   "outputs": [],
   "source": [
    "# !pip install scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a55f9168-ba61-4f18-b458-b10b1e3c90ac",
   "metadata": {
    "id": "a55f9168-ba61-4f18-b458-b10b1e3c90ac"
   },
   "outputs": [],
   "source": [
    "from sklearn.cluster import KMeans"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65b2096a-d683-4713-889d-46a368e9888c",
   "metadata": {
    "id": "65b2096a-d683-4713-889d-46a368e9888c"
   },
   "source": [
    "마찬가지로 앞서 구축한 문장 후보군들에 대해 임베딩을 수행합니다.\n",
    "\n",
    "이후, `num_clusters` 변수를 통해 클러스터의 개수를 설정한 후 임베딩을 활용한 **k-Means** 클러스터링을 수행하도록 합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9771e341-3902-427e-a7c2-72d71804d63f",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 117,
     "referenced_widgets": [
      "b0d98be053714494acf93d879637686c",
      "288049376d58424b8a61234d8275ea73",
      "28b77981ed8c47608b589bc22fe96385",
      "9e7359c250184c63a69614e0ebbea177",
      "c3bbb86623e4415da6417b1cc6610742",
      "0829b66884274a509808088e57eef941",
      "1cb5512595454961bbc2c943068fbbed",
      "34576bd9bc784d73941789765ad06a30"
     ]
    },
    "id": "9771e341-3902-427e-a7c2-72d71804d63f",
    "outputId": "53540386-993b-4fc4-c25d-2dcfe6037164",
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b0d98be053714494acf93d879637686c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "HBox(children=(FloatProgress(value=0.0, description='Batches', max=1.0, style=ProgressStyle(description_width=…"
      ]
     },
     "metadata": {
      "tags": []
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "KMeans(algorithm='auto', copy_x=True, init='k-means++', max_iter=300,\n",
       "       n_clusters=3, n_init=10, n_jobs=None, precompute_distances='auto',\n",
       "       random_state=None, tol=0.0001, verbose=0)"
      ]
     },
     "execution_count": 31,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "document_embeddings = model.encode(docs)\n",
    "\n",
    "num_clusters = 3\n",
    "\n",
    "k_means = KMeans(n_clusters=num_clusters)\n",
    "k_means.fit(document_embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d82676eb-3046-40b7-af1a-fac8521b4002",
   "metadata": {
    "id": "d82676eb-3046-40b7-af1a-fac8521b4002"
   },
   "source": [
    "이제 클러스터링을 통해 각 문장이 어떤 클러스터에 포함되었는지 확인해봅시다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a84c048-594a-4c09-ae1b-732c96a06c73",
   "metadata": {
    "id": "9a84c048-594a-4c09-ae1b-732c96a06c73"
   },
   "outputs": [],
   "source": [
    "cluster_assignment = k_means.labels_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19df02ff-2b3a-4934-92e6-84d14b653974",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "19df02ff-2b3a-4934-92e6-84d14b653974",
    "outputId": "623d0ae4-ebc9-4642-a9f3-d977459358ab"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 2, 0, 1, 0, 1, 1], dtype=int32)"
      ]
     },
     "execution_count": 33,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cluster_assignment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8539ad61-5d5c-4d6a-9f38-0002aeb3c7f1",
   "metadata": {
    "id": "8539ad61-5d5c-4d6a-9f38-0002aeb3c7f1"
   },
   "source": [
    "클러스터링 결과를 토대로 각 문장을 클러스터로 분리한 후, 결과를 출력합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e25e6d9-2b2c-4564-a414-b470f8473b80",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "9e25e6d9-2b2c-4564-a414-b470f8473b80",
    "outputId": "54e6ad14-30e3-4a1e-ca43-c27976259fd1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "< 클러스터 1 >\n",
      "1992년 7월 8일 손흥민은 강원도 춘천시 후평동에서 아버지 손웅정과 어머니 길은자의 차남으로 태어나 그곳에서 자랐다.\n",
      "춘천 부안초등학교를 졸업했고, 춘천 후평중학교에 입학한 후 2학년때 원주 육민관중학교 축구부에 들어가기 위해 전학하여 졸업하였으며, 2008년 당시 FC 서울의 U-18팀이었던 동북고등학교 축구부에서 선수 활동 중 대한축구협회 우수선수 해외유학 프로젝트에 선발되어 2008년 8월 독일 분데스리가의 함부르크 유소년팀에 입단하였다.\n",
      "1년간의 유학 후 2009년 8월 한국으로 돌아온 후 10월에 개막한 FIFA U-17 월드컵에 출전하여 3골을 터트리며 한국을 8강으로 이끌었다.\n",
      "\n",
      "< 클러스터 2 >\n",
      "함부르크 유스팀 주전 공격수로 2008년 6월 네덜란드에서 열린 4개국 경기에서 4게임에 출전, 3골을 터뜨렸다.\n",
      "그해 11월 함부르크의 정식 유소년팀 선수 계약을 체결하였으며 독일 U-19 리그 4경기 2골을 넣고 2군 리그에 출전을 시작했다.\n",
      "독일 U-19 리그에서 손흥민은 11경기 6골, 2부 리그에서는 6경기 1골을 넣으며 재능을 인정받아 2010년 6월 17세의 나이로 함부르크의 1군 팀 훈련에 참가, 프리시즌 활약으로 함부르크와 정식 계약을 한 후 10월 18세에 함부르크 1군 소속으로 독일 분데스리가에 데뷔하였다.\n",
      "\n",
      "< 클러스터 3 >\n",
      "형은 손흥윤이다.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 클러스터 개수 만큼 문장을 담을 리스트 초기화\n",
    "clustered_sentences = [[] for _ in range(num_clusters)]\n",
    "\n",
    "# 클러스터링 결과를 돌며 각 클러스터에 맞게 문장 삽입\n",
    "for sentence_id, cluster_id in enumerate(cluster_assignment):\n",
    "    clustered_sentences[cluster_id].append(docs[sentence_id])\n",
    "\n",
    "for i, cluster in enumerate(clustered_sentences):\n",
    "    result = \"\\n\".join(cluster)\n",
    "    print(f\"< 클러스터 {i+1} >\\n{result}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e837f7d-19b3-47d7-b08a-2e35d41bc937",
   "metadata": {
    "id": "9e837f7d-19b3-47d7-b08a-2e35d41bc937"
   },
   "source": [
    "국내 관련 문장과 해외 관련 문장, 가족 관계와 같은 느낌으로 클러스터가 형성된 것을 확인할 수 있습니다.\n",
    "\n",
    "지금까지 `sentence-transformers`를 학습하는 과정을 KLUE STS 데이터셋을 통해 알아보았습니다.\n",
    "\n",
    "`sentence-transformers`는 다양한 문장 임베딩 기법과 이를 활용한 응용 사례에 대해서 크게 고민하는 **UKPLab**에서 관리되는 라이브러리이니 만큼 앞으로 더 발전하고 관리될 가능성이 높은 도구입니다.\n",
    "\n",
    "본 노트북을 통해 습득한 지식이 여러분의 업무와 학습에 도움이 되었으면 좋겠습니다.\n",
    "\n",
    "```\n",
    "허 훈 (huffonism@gmail.com)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "QAdhBQpFBmCw",
   "metadata": {
    "id": "QAdhBQpFBmCw"
   },
   "source": [
    "APPENDIX: 앞서 학습된 모델을 HuggingFace 모델 허브에 업로드하였으니, 아래 예제와 같이 사용이 가능합니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "lZI-DoJfRsgZ",
   "metadata": {
    "id": "lZI-DoJfRsgZ"
   },
   "outputs": [],
   "source": [
    "import torch\n",
    "from sentence_transformers import SentenceTransformer, util\n",
    "\n",
    "model = SentenceTransformer(\"Huffon/sentence-klue-roberta-base\")\n",
    "\n",
    "docs = [\n",
    "    \"1992년 7월 8일 손흥민은 강원도 춘천시 후평동에서 아버지 손웅정과 어머니 길은자의 차남으로 태어나 그곳에서 자랐다.\",\n",
    "    \"형은 손흥윤이다.\",\n",
    "    \"춘천 부안초등학교를 졸업했고, 춘천 후평중학교에 입학한 후 2학년때 원주 육민관중학교 축구부에 들어가기 위해 전학하여 졸업하였으며, 2008년 당시 FC 서울의 U-18팀이었던 동북고등학교 축구부에서 선수 활동 중 대한축구협회 우수선수 해외유학 프로젝트에 선발되어 2008년 8월 독일 분데스리가의 함부르크 유소년팀에 입단하였다.\",\n",
    "    \"함부르크 유스팀 주전 공격수로 2008년 6월 네덜란드에서 열린 4개국 경기에서 4게임에 출전, 3골을 터뜨렸다.\",\n",
    "    \"1년간의 유학 후 2009년 8월 한국으로 돌아온 후 10월에 개막한 FIFA U-17 월드컵에 출전하여 3골을 터트리며 한국을 8강으로 이끌었다.\",\n",
    "    \"그해 11월 함부르크의 정식 유소년팀 선수 계약을 체결하였으며 독일 U-19 리그 4경기 2골을 넣고 2군 리그에 출전을 시작했다.\",\n",
    "    \"독일 U-19 리그에서 손흥민은 11경기 6골, 2부 리그에서는 6경기 1골을 넣으며 재능을 인정받아 2010년 6월 17세의 나이로 함부르크의 1군 팀 훈련에 참가, 프리시즌 활약으로 함부르크와 정식 계약을 한 후 10월 18세에 함부르크 1군 소속으로 독일 분데스리가에 데뷔하였다.\",\n",
    "]\n",
    "document_embeddings = model.encode(docs)\n",
    "\n",
    "query = \"손흥민은 어린 나이에 유럽에 진출하였다.\"\n",
    "query_embedding = model.encode(query)\n",
    "\n",
    "top_k = min(5, len(docs))\n",
    "\n",
    "# 입력 문장 - 문장 후보군 간 코사인 유사도 계산 후,\n",
    "cos_scores = util.pytorch_cos_sim(query_embedding, document_embeddings)[0]\n",
    "\n",
    "# 코사인 유사도 순으로 `top_k` 개 문장 추출\n",
    "top_results = torch.topk(cos_scores, k=top_k)\n",
    "\n",
    "print(f\"입력 문장: {query}\")\n",
    "print(f\"\\n<입력 문장과 유사한 {top_k} 개의 문장>\\n\")\n",
    "\n",
    "for i, (score, idx) in enumerate(zip(top_results[0], top_results[1])):\n",
    "    print(f\"{i+1}: {docs[idx]} {'(유사도: {:.4f})'.format(score)}\\n\")"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "sentence-transformers.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "00548206f84149dfa4f9fbc82c307006": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_622e299187ff4e97ac193396b3065f91",
      "placeholder": "​",
      "style": "IPY_MODEL_a9bd945bb0f741c3964b8a0894df72bd",
      "value": " 519/0 [00:00&lt;00:00, 6193.59 examples/s]"
     }
    },
    "032acc3e44044975ae678ec3fdcced83": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0497dd703317433a8369d67b805ce5d8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "07e1587a70e749fd85a4d77813e67bcd": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0829b66884274a509808088e57eef941": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "08bd4b003c174df2a25bd8788dd516f8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0a51622362424fc0a86fdc9dcc470a97": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_fc31aa88c29342a4b0e94eee9013e2fc",
       "IPY_MODEL_8d4f688dd2ce4a54b5811590f65c5ac6"
      ],
      "layout": "IPY_MODEL_f6ab48b6dff043f985ffd9fc5140506a"
     }
    },
    "0b6ecc67d864453593098647fc18c78e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "0cc015495ef1473c9185df6ef619acc8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0e33d6a4ecdf4e9690b59d7252d5628d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "105822d482834e55b75da6280a149144": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8e7be76c92394ab292eda03240ac54a7",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_4f4a5d94054047c9910b9861ead6325d",
      "value": 1
     }
    },
    "123cd4f17f9e435da9ba2d2b8fa0da35": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_526bf3d71aec4c34ad20bbdfcfcf532a",
       "IPY_MODEL_f61668e72b954552bc03d5427f6376d0"
      ],
      "layout": "IPY_MODEL_767236879fd44eef92b8589f96779821"
     }
    },
    "127a23b75b164c8f8a8a59deedc17a1f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1348afbc25204780ac866f0ec08149e1": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "140930a6d8f4422283546c5c99db167c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1576fda5d9a84b20824081442e71c579": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0e33d6a4ecdf4e9690b59d7252d5628d",
      "placeholder": "​",
      "style": "IPY_MODEL_ddb97d3ec0be4d43a0c500b3b976d3ff",
      "value": " 250k/? [55:40&lt;00:00, 74.7B/s]"
     }
    },
    "1b38b2bb686641cfb62702391e752c01": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1cb5512595454961bbc2c943068fbbed": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "235558cc682b4b6da15a83d9595c5b96": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "25059b0da3d344dfb872c5091c52ca0b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "257ac4a8485146b9afcd9d7cab4bf004": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "25d5b898e09a485792aab85b8e1adcd9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: ",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4e4ab7edd35b4cbf98bf936d9c9083e9",
      "max": 5191,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_72ae9ecb01e744d0a5746c3e1d06e16e",
      "value": 5191
     }
    },
    "285316b400ec4de0aa4ebb1911d998a8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "288049376d58424b8a61234d8275ea73": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "28b77981ed8c47608b589bc22fe96385": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Batches: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0829b66884274a509808088e57eef941",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c3bbb86623e4415da6417b1cc6610742",
      "value": 1
     }
    },
    "28bb99538bf64d78b8430add8ca8bede": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "2b0a0cc6d8814f68968d447d27081d5a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Iteration: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b99b10f6c8f94ef7ad06f8c23f6cfaa8",
      "max": 365,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d26767b5d78046b5a0acc0cfa4a2f6e6",
      "value": 365
     }
    },
    "2bb48f10c57049b487d49dd89453aa3a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2bdac7dbb3744d06bdb135c15309dfd3": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2c5fb679777f4863a5b24068dab16459": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2c60ed2462a34fc6bbddcd938443faf8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Batches: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_140930a6d8f4422283546c5c99db167c",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_716f62e6b2ba4e9da8180d0c529f679a",
      "value": 1
     }
    },
    "2cfc03bda6a149bba24dd36a0e377136": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "321f8883d1044d75a9aa5b9bcd365262": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "331b193159854002a67c02e7252b61c4": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a48999c9c915448eb18c0a059353ee9e",
      "placeholder": "​",
      "style": "IPY_MODEL_fee8569d1c9e47baab3439c2661549ce",
      "value": " 307k/? [00:00&lt;00:00, 1.32MB/s]"
     }
    },
    "334b671a7f5240679fafae3dc367dacc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "34576bd9bc784d73941789765ad06a30": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "356cf5005b7d4685b03bb45e8aaa1955": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ca29160383634760a651f8168fd4619d",
       "IPY_MODEL_80c9223d0d334dcd86ef2da07592d841"
      ],
      "layout": "IPY_MODEL_0cc015495ef1473c9185df6ef619acc8"
     }
    },
    "386ca11e61b240f8bb96e5ced006c2b5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "3b43878106ca40b0ae0780456848ba5b": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3e9c9dcd669044a283e864e894e6576b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_105822d482834e55b75da6280a149144",
       "IPY_MODEL_db287f5471e846eb8f188f7d34d48a0f"
      ],
      "layout": "IPY_MODEL_41a8570a0c1e48a0bb0e5cb3319139c7"
     }
    },
    "41a8570a0c1e48a0bb0e5cb3319139c7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "46e0a3dd08474d32ae86816095225f12": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "47fc6e8fc71d4664ac375beb323577ad": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "48c3976b084a4ada9fc4225904eceab8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "4d0afb67f54a42af9976d167fe9606c5": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4e4ab7edd35b4cbf98bf936d9c9083e9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4f4a5d94054047c9910b9861ead6325d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "50d8892d85b3494186e30ad6253e626a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "51576b520f4f4b8aa53e20d7aef543b3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "525993c10abe4052b21da590ff3f9c88": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c672854515244b07a840750c81d93486",
      "placeholder": "​",
      "style": "IPY_MODEL_52c2e05505974578adacb96202e4ac03",
      "value": " 4.49k/? [00:00&lt;00:00, 21.0kB/s]"
     }
    },
    "526bf3d71aec4c34ad20bbdfcfcf532a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Iteration: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7fa4d5bc2e294984924b977bba075a46",
      "max": 365,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_cbbe6b1733504a9c86fba24a8a4e0200",
      "value": 365
     }
    },
    "52c2e05505974578adacb96202e4ac03": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5488a2eaa030452a9f63563296a53e55": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Iteration: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b484aaddb8684cd9b03c5acb45a6fb4a",
      "max": 365,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_235558cc682b4b6da15a83d9595c5b96",
      "value": 365
     }
    },
    "548a46f134704494a685ebe87327609d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "55c4117d3cae41a39db19c10f09653bb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "55c9425d605b4105bcdc014e1a946e84": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "56dca2ceb71d4ff39b537877065fb42d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Batches: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_bb37968ba0594c34afed35ee7d613d67",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2cfc03bda6a149bba24dd36a0e377136",
      "value": 1
     }
    },
    "5cfa167c683a40ae8cc4d0bf6c007588": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "600b916033c34c22beb328bd72d5b881": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9b167f9f16ea489d8a9dd358f0a63f1c",
      "placeholder": "​",
      "style": "IPY_MODEL_c739a8e4d6ac4df48c6ca224b32c6a4c",
      "value": " 365/365 [53:56&lt;00:00,  8.87s/it]"
     }
    },
    "622e299187ff4e97ac193396b3065f91": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "65e86b2504194c8bb8ff28715e729143": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_127a23b75b164c8f8a8a59deedc17a1f",
      "placeholder": "​",
      "style": "IPY_MODEL_f6c37b9336b24293a0d94ce7bf31273d",
      "value": " 4/4 [53:56&lt;00:00, 809.11s/it]"
     }
    },
    "67373d86bc164f09a3d983f2c64864fa": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "67cf4d8223b34fb69d8ceb4cb0b6baf8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_814859a2218f4ae69e24ad9067e36d1d",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_334b671a7f5240679fafae3dc367dacc",
      "value": 1
     }
    },
    "692225963d2d434881ebbb7ad5ecab42": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "6b9d09dd3ddb430a82b6342b29908b6c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f172860e4aea40f796f0671df12393eb",
      "placeholder": "​",
      "style": "IPY_MODEL_7139cfd2150743af994aad6e26d3a73c",
      "value": " 1.05M/? [00:00&lt;00:00, 2.24MB/s]"
     }
    },
    "7051593e8d1c46f8ae18992ef7d565b6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7139cfd2150743af994aad6e26d3a73c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "716f62e6b2ba4e9da8180d0c529f679a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "72ae9ecb01e744d0a5746c3e1d06e16e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "7491d0c59e314c17a340880286cace2c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_c1d67854eea545699c72a282bf169725",
       "IPY_MODEL_6b9d09dd3ddb430a82b6342b29908b6c"
      ],
      "layout": "IPY_MODEL_692225963d2d434881ebbb7ad5ecab42"
     }
    },
    "767236879fd44eef92b8589f96779821": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "76ff42ef55ab467cab41a570ac238113": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "78502be99cd04a0698042f42a028f5a1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f70d3ae8707c4cccaca58ef8f5e50627",
      "placeholder": "​",
      "style": "IPY_MODEL_67373d86bc164f09a3d983f2c64864fa",
      "value": " 365/365 [47:32&lt;00:00,  7.81s/it]"
     }
    },
    "7a2f0db32cc24798a728e9a674b1ac96": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_7c5bd980c44044a897fcc5d9121e39b0",
       "IPY_MODEL_65e86b2504194c8bb8ff28715e729143"
      ],
      "layout": "IPY_MODEL_f3ca9da02dd64784a0287359e95a9dd7"
     }
    },
    "7c5bd980c44044a897fcc5d9121e39b0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Epoch: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_98f8203831204f3491002c875292829a",
      "max": 4,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_5cfa167c683a40ae8cc4d0bf6c007588",
      "value": 4
     }
    },
    "7e578efc091c458d8bda6899cee9b913": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: ",
      "description_tooltip": null,
      "layout": "IPY_MODEL_91899de43539471ea34e27161c126799",
      "max": 1781,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d18c42f6f51242728f111f5a1ca9cfcb",
      "value": 1781
     }
    },
    "7e5fbb8dbce945e399add4d7d675159a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7fa4d5bc2e294984924b977bba075a46": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "80c9223d0d334dcd86ef2da07592d841": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1b38b2bb686641cfb62702391e752c01",
      "placeholder": "​",
      "style": "IPY_MODEL_7051593e8d1c46f8ae18992ef7d565b6",
      "value": " 22.7k/? [00:00&lt;00:00, 253kB/s]"
     }
    },
    "814859a2218f4ae69e24ad9067e36d1d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8398ef9da1464b008ebc053f65a52b40": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_88c3864ad3bf4c159123d54b8316d1ca",
       "IPY_MODEL_600b916033c34c22beb328bd72d5b881"
      ],
      "layout": "IPY_MODEL_25059b0da3d344dfb872c5091c52ca0b"
     }
    },
    "840487532d124f5f8750d23ca26f6ec3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5488a2eaa030452a9f63563296a53e55",
       "IPY_MODEL_d9bdb1a3edfc46d59a6e36593b2b201f"
      ],
      "layout": "IPY_MODEL_548a46f134704494a685ebe87327609d"
     }
    },
    "88c3864ad3bf4c159123d54b8316d1ca": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Iteration: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2bdac7dbb3744d06bdb135c15309dfd3",
      "max": 365,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_76ff42ef55ab467cab41a570ac238113",
      "value": 365
     }
    },
    "89206a925fce40c282ad389ddac46b92": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b444af2d690c44598824edacfcbe2c7f",
      "placeholder": "​",
      "style": "IPY_MODEL_96a3579f62064b0f83e652dd51b2bd14",
      "value": " 22.5k/? [00:02&lt;00:00, 9.36kB/s]"
     }
    },
    "8942aa54279f4c048d1132fd31857c7a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: ",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0497dd703317433a8369d67b805ce5d8",
      "max": 89895,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9a67fd098c9f4ceb9e73d3181e1b90eb",
      "value": 89895
     }
    },
    "8b897879b24c4fe9938b4d7dd9596b12": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8d0142893b3f404d83971d3886c2d9b9": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "8d4f688dd2ce4a54b5811590f65c5ac6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_032acc3e44044975ae678ec3fdcced83",
      "placeholder": "​",
      "style": "IPY_MODEL_91fff2991acc45d4a466d5e79e01dfe8",
      "value": " 1.35M/1.35M [00:01&lt;00:00, 1.13MB/s]"
     }
    },
    "8e7be76c92394ab292eda03240ac54a7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "91050df0419b49f284952ac697af5941": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "91899de43539471ea34e27161c126799": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "91fff2991acc45d4a466d5e79e01dfe8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "92644e0d7b4146eb9fc0fa6e53788935": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "942e110b651d4a9ab8c19fe1b16b6b66": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "94cd2773d9e44c908e0a411c6fc6a90d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "96a3579f62064b0f83e652dd51b2bd14": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "97b1a3c1e5a84ba9805d2a888d8fbd04": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_f35ba4753cf94e12adaac95b15e832e2",
       "IPY_MODEL_1576fda5d9a84b20824081442e71c579"
      ],
      "layout": "IPY_MODEL_b7ca38c550e04d83a7d872dcc4e76b5d"
     }
    },
    "98f8203831204f3491002c875292829a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9a48e63c2a3b4c5bb0b206f382635301": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9a67fd098c9f4ceb9e73d3181e1b90eb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "9b167f9f16ea489d8a9dd358f0a63f1c": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9e7359c250184c63a69614e0ebbea177": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_34576bd9bc784d73941789765ad06a30",
      "placeholder": "​",
      "style": "IPY_MODEL_1cb5512595454961bbc2c943068fbbed",
      "value": " 1/1 [34:10&lt;00:00, 2050.06s/it]"
     }
    },
    "9fc608766a5146df913f35e5b9c0810a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a1ecc0d99a12498abc32c6e68beeda61": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a48999c9c915448eb18c0a059353ee9e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a729b4890b954fc68273ad4db545d9c8": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_56dca2ceb71d4ff39b537877065fb42d",
       "IPY_MODEL_dd796e4d932547509d02a45c7f7d596e"
      ],
      "layout": "IPY_MODEL_1348afbc25204780ac866f0ec08149e1"
     }
    },
    "a9bd945bb0f741c3964b8a0894df72bd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "aa519372a89c4fc39fcc0c5d1735a03c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ee9d8918d3ce4ae284ebceda8563f005",
      "placeholder": "​",
      "style": "IPY_MODEL_55c9425d605b4105bcdc014e1a946e84",
      "value": " 1379/0 [00:00&lt;00:00, 10927.95 examples/s]"
     }
    },
    "ae6ee389236a4af09075d2e46ffed8fc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_67cf4d8223b34fb69d8ceb4cb0b6baf8",
       "IPY_MODEL_fe3a2818127a4406ae53682ab075563a"
      ],
      "layout": "IPY_MODEL_a1ecc0d99a12498abc32c6e68beeda61"
     }
    },
    "b0d98be053714494acf93d879637686c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_28b77981ed8c47608b589bc22fe96385",
       "IPY_MODEL_9e7359c250184c63a69614e0ebbea177"
      ],
      "layout": "IPY_MODEL_288049376d58424b8a61234d8275ea73"
     }
    },
    "b159bcf2d5d74211bc391b8fc08d9d8a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_7e578efc091c458d8bda6899cee9b913",
       "IPY_MODEL_dd315967ced947bb98bc2cc95a62c23f"
      ],
      "layout": "IPY_MODEL_dff1f823541f4c5896e2c8c34ea07953"
     }
    },
    "b2755bab103540fc815bfd5c25ec1aab": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_3b43878106ca40b0ae0780456848ba5b",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d537eecb55e04535958882b45b91d217",
      "value": 1
     }
    },
    "b40b278c5d3548d3a35e8cb39a2233aa": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e529a66bfd2f4eec9e2a98bc28fdf94a",
       "IPY_MODEL_525993c10abe4052b21da590ff3f9c88"
      ],
      "layout": "IPY_MODEL_94cd2773d9e44c908e0a411c6fc6a90d"
     }
    },
    "b444af2d690c44598824edacfcbe2c7f": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b484aaddb8684cd9b03c5acb45a6fb4a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b6c03c0eedd845eaada0e17cd184b46f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d9ec0cf4628f4ee7b799bdb8e63672bc",
      "placeholder": "​",
      "style": "IPY_MODEL_257ac4a8485146b9afcd9d7cab4bf004",
      "value": " 1/1 [00:04&lt;00:00,  4.67s/it]"
     }
    },
    "b7ca38c550e04d83a7d872dcc4e76b5d": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "b99b10f6c8f94ef7ad06f8c23f6cfaa8": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "babef37d61e24cafba6a4fee968a5d50": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "bb37968ba0594c34afed35ee7d613d67": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "be68bd601d554ca693a7c4d5e1ce7b20": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_ec14d701ee314261920530616c8dbfad",
       "IPY_MODEL_e6ca2ed570eb40bdbb67f15fee0d8ed1"
      ],
      "layout": "IPY_MODEL_47fc6e8fc71d4664ac375beb323577ad"
     }
    },
    "c1d67854eea545699c72a282bf169725": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: ",
      "description_tooltip": null,
      "layout": "IPY_MODEL_e4a0935bcfc74f34a0fab107506696a0",
      "max": 282044,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_55c4117d3cae41a39db19c10f09653bb",
      "value": 282044
     }
    },
    "c3bbb86623e4415da6417b1cc6610742": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "c5a3f0ee7f56484e85ab570a53fedd79": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c672854515244b07a840750c81d93486": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c739a8e4d6ac4df48c6ca224b32c6a4c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "c924ed150174410f8f6fbf0da83aee40": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "ca29160383634760a651f8168fd4619d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: ",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f4ea9066b3b0497f8440537580e478d9",
      "max": 2932,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_386ca11e61b240f8bb96e5ced006c2b5",
      "value": 2932
     }
    },
    "cbbe6b1733504a9c86fba24a8a4e0200": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "cdfdfa26dc774f79becef859fdbabb51": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "d04c49de6dfe4ebd994f20e6e9c1fba3": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "d18c42f6f51242728f111f5a1ca9cfcb": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "d26767b5d78046b5a0acc0cfa4a2f6e6": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "d4bfe78967354804835d382cb47be8fd": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2b0a0cc6d8814f68968d447d27081d5a",
       "IPY_MODEL_78502be99cd04a0698042f42a028f5a1"
      ],
      "layout": "IPY_MODEL_2bb48f10c57049b487d49dd89453aa3a"
     }
    },
    "d537eecb55e04535958882b45b91d217": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": "initial"
     }
    },
    "d79df4c82bdb405f91623f1157d652c6": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d951edfda0a84ba196ab9c1ed2e00856": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_8942aa54279f4c048d1132fd31857c7a",
       "IPY_MODEL_331b193159854002a67c02e7252b61c4"
      ],
      "layout": "IPY_MODEL_942e110b651d4a9ab8c19fe1b16b6b66"
     }
    },
    "d9bdb1a3edfc46d59a6e36593b2b201f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8b897879b24c4fe9938b4d7dd9596b12",
      "placeholder": "​",
      "style": "IPY_MODEL_321f8883d1044d75a9aa5b9bcd365262",
      "value": " 365/365 [50:46&lt;00:00,  8.35s/it]"
     }
    },
    "d9ec0cf4628f4ee7b799bdb8e63672bc": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "db287f5471e846eb8f188f7d34d48a0f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c5a3f0ee7f56484e85ab570a53fedd79",
      "placeholder": "​",
      "style": "IPY_MODEL_48c3976b084a4ada9fc4225904eceab8",
      "value": " 11668/0 [00:00&lt;00:00, 19.42 examples/s]"
     }
    },
    "dcb49ed472364f54958664bb39bdaec2": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "dd315967ced947bb98bc2cc95a62c23f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_d79df4c82bdb405f91623f1157d652c6",
      "placeholder": "​",
      "style": "IPY_MODEL_ea1abe864b4f4fdb92cf00373dde56ac",
      "value": " 6.49k/? [00:00&lt;00:00, 17.3kB/s]"
     }
    },
    "dd796e4d932547509d02a45c7f7d596e": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_285316b400ec4de0aa4ebb1911d998a8",
      "placeholder": "​",
      "style": "IPY_MODEL_50d8892d85b3494186e30ad6253e626a",
      "value": " 1/1 [00:02&lt;00:00,  2.50s/it]"
     }
    },
    "ddb97d3ec0be4d43a0c500b3b976d3ff": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "dff1f823541f4c5896e2c8c34ea07953": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e4a0935bcfc74f34a0fab107506696a0": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "e529a66bfd2f4eec9e2a98bc28fdf94a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: ",
      "description_tooltip": null,
      "layout": "IPY_MODEL_07e1587a70e749fd85a4d77813e67bcd",
      "max": 1371,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_0b6ecc67d864453593098647fc18c78e",
      "value": 1371
     }
    },
    "e6ca2ed570eb40bdbb67f15fee0d8ed1": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_7e5fbb8dbce945e399add4d7d675159a",
      "placeholder": "​",
      "style": "IPY_MODEL_9fc608766a5146df913f35e5b9c0810a",
      "value": " 1471/0 [00:00&lt;00:00, 12448.17 examples/s]"
     }
    },
    "e8c4cd2098ff4152879ae0deaf230e2f": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_4d0afb67f54a42af9976d167fe9606c5",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_8d0142893b3f404d83971d3886c2d9b9",
      "value": 1
     }
    },
    "ea1abe864b4f4fdb92cf00373dde56ac": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ec14d701ee314261920530616c8dbfad": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_9a48e63c2a3b4c5bb0b206f382635301",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_c924ed150174410f8f6fbf0da83aee40",
      "value": 1
     }
    },
    "ed529787a715476583c1c2ca5353e39b": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_b2755bab103540fc815bfd5c25ec1aab",
       "IPY_MODEL_aa519372a89c4fc39fcc0c5d1735a03c"
      ],
      "layout": "IPY_MODEL_92644e0d7b4146eb9fc0fa6e53788935"
     }
    },
    "ee9d8918d3ce4ae284ebceda8563f005": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f172860e4aea40f796f0671df12393eb": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f35ba4753cf94e12adaac95b15e832e2": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: ",
      "description_tooltip": null,
      "layout": "IPY_MODEL_babef37d61e24cafba6a4fee968a5d50",
      "max": 66129,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_28bb99538bf64d78b8430add8ca8bede",
      "value": 66129
     }
    },
    "f366b4862135486c925f1a6d55cf0b3e": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f373dc73be2b4ee88c1bc3f6a6a2f935": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_e8c4cd2098ff4152879ae0deaf230e2f",
       "IPY_MODEL_00548206f84149dfa4f9fbc82c307006"
      ],
      "layout": "IPY_MODEL_dcb49ed472364f54958664bb39bdaec2"
     }
    },
    "f3ca9da02dd64784a0287359e95a9dd7": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f4ea9066b3b0497f8440537580e478d9": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f61668e72b954552bc03d5427f6376d0": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_2c5fb679777f4863a5b24068dab16459",
      "placeholder": "​",
      "style": "IPY_MODEL_cdfdfa26dc774f79becef859fdbabb51",
      "value": " 365/365 [04:43&lt;00:00,  1.29it/s]"
     }
    },
    "f6ab48b6dff043f985ffd9fc5140506a": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f6c37b9336b24293a0d94ce7bf31273d": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f70d3ae8707c4cccaca58ef8f5e50627": {
     "model_module": "@jupyter-widgets/base",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "fc31aa88c29342a4b0e94eee9013e2fc": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "Downloading: 100%",
      "description_tooltip": null,
      "layout": "IPY_MODEL_08bd4b003c174df2a25bd8788dd516f8",
      "max": 1349875,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_d04c49de6dfe4ebd994f20e6e9c1fba3",
      "value": 1349875
     }
    },
    "fc7374f1bb9e4c5185bf9dc2ba45966c": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_25d5b898e09a485792aab85b8e1adcd9",
       "IPY_MODEL_89206a925fce40c282ad389ddac46b92"
      ],
      "layout": "IPY_MODEL_f366b4862135486c925f1a6d55cf0b3e"
     }
    },
    "fd518e8d58a048fc933020524bb298a5": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_2c60ed2462a34fc6bbddcd938443faf8",
       "IPY_MODEL_b6c03c0eedd845eaada0e17cd184b46f"
      ],
      "layout": "IPY_MODEL_91050df0419b49f284952ac697af5941"
     }
    },
    "fe3a2818127a4406ae53682ab075563a": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_46e0a3dd08474d32ae86816095225f12",
      "placeholder": "​",
      "style": "IPY_MODEL_51576b520f4f4b8aa53e20d7aef543b3",
      "value": " 5703/0 [00:00&lt;00:00, 8021.54 examples/s]"
     }
    },
    "fee8569d1c9e47baab3439c2661549ce": {
     "model_module": "@jupyter-widgets/controls",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}