Kaggle_Python3_Bronze_Medal_Optimized_LightGBM_with_Optuna.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.007305,
     "end_time": "2020-12-24T22:00:33.076194",
     "exception": false,
     "start_time": "2020-12-24T22:00:33.068889",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## \"Optimized LightGBM with Optuna on [Riiid! Answer Correctness Prediction](https://www.kaggle.com/c/riiid-test-answer-prediction)\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.007632,
     "end_time": "2020-12-24T22:00:33.090501",
     "exception": false,
     "start_time": "2020-12-24T22:00:33.082869",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Change points from [Riiid model LGBM](https://www.kaggle.com/ragnar123/riiid-model-lgbm):\n",
    "1. I added hyperparameters by Optuna.\n",
    "1. The hyperparameters by Optuna were obtained from [this notebook](https://github.com/satorushibata0627/Publishment/blob/main/Kaggle_Python3_LightGBM_on_GPU_with_Feature_Engineering_Optuna_and_Visualization.ipynb).\n",
    "    - The submission version: [this one](https://www.kaggle.com/satorushibata/lightgbm-with-the-inference-empirical-analysis).\n",
    "1. I added the parameter: `num_iteration = lgb.best_iteration` to the predection function. For example, `model.predict (test_df [FEATURES], num_iteration = model.best_iteration)`, it is."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.006003,
     "end_time": "2020-12-24T22:00:33.102939",
     "exception": false,
     "start_time": "2020-12-24T22:00:33.096936",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Comments\n",
    "Thanks to tito for this great script https://www.kaggle.com/its7171/lgbm-with-loop-feature-engineering\n",
    "\n",
    "* Creating predictive feature is very important, here I just used 14 features and 15M data points to train the model.\n",
    "* The dataset is big to preprocess using python with a for loop, their are other tools and frameworks like (SQL, Spark, Apache Beam, Dask) where you could make feature engineering much faster but if we are smart and make predictive feature it's ok to just use for loops.\n",
    "* Foward feature engineering seems a good technique to try in this problem (create 1 new feature that you think it could be predective based on the problem, run the pipeline and check if val score increase, if it increase that feature is predictive and you should add it. Care when you just get some minor improvement, sometime is better to discard that feature because your experimentation process is going to get slower)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2020-12-24T22:00:33.119721Z",
     "iopub.status.busy": "2020-12-24T22:00:33.119013Z",
     "iopub.status.idle": "2020-12-24T22:00:35.102729Z",
     "shell.execute_reply": "2020-12-24T22:00:35.103305Z"
    },
    "papermill": {
     "duration": 1.994264,
     "end_time": "2020-12-24T22:00:35.103524",
     "exception": false,
     "start_time": "2020-12-24T22:00:33.109260",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style type='text/css'>\n",
       ".datatable table.frame { margin-bottom: 0; }\n",
       ".datatable table.frame thead { border-bottom: none; }\n",
       ".datatable table.frame tr.coltypes td {  color: #FFFFFF;  line-height: 6px;  padding: 0 0.5em;}\n",
       ".datatable .bool    { background: #DDDD99; }\n",
       ".datatable .object  { background: #565656; }\n",
       ".datatable .int     { background: #5D9E5D; }\n",
       ".datatable .float   { background: #4040CC; }\n",
       ".datatable .str     { background: #CC4040; }\n",
       ".datatable .row_index {  background: var(--jp-border-color3);  border-right: 1px solid var(--jp-border-color0);  color: var(--jp-ui-font-color3);  font-size: 9px;}\n",
       ".datatable .frame tr.coltypes .row_index {  background: var(--jp-border-color0);}\n",
       ".datatable th:nth-child(2) { padding-left: 12px; }\n",
       ".datatable .hellipsis {  color: var(--jp-cell-editor-border-color);}\n",
       ".datatable .vellipsis {  background: var(--jp-layout-color0);  color: var(--jp-cell-editor-border-color);}\n",
       ".datatable .na {  color: var(--jp-cell-editor-border-color);  font-size: 80%;}\n",
       ".datatable .footer { font-size: 9px; }\n",
       ".datatable .frame_dimensions {  background: var(--jp-border-color3);  border-top: 1px solid var(--jp-border-color0);  color: var(--jp-ui-font-color3);  display: inline-block;  opacity: 0.6;  padding: 1px 10px 1px 5px;}\n",
       ".datatable .frame thead tr.colnames {  background-image: url('data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABwAAAA4CAYAAADuMJi0AAAGR0lEQVR42rVZ21IbRxBtCbQrkIR2dQVjsLmDLBsET3nTQ8ouYRkQVf6e/E9+Im958qMfkgoXAaKSSj6C9Jnd2R2NeiRSRaZqitVOT5+Z6dNnWoKGlN94JFp8Ipofkb/7SOXjGyp8wF+z35K3f0uUp/GW4XfLQ8v2gefj3ZCCzojoNfue+43o1Q3l3xB/yA3JO7jnF2pCLnI+pNyx/qw7L+SQ7T2N9p2f8c60QcfcK6KGXsAd+ZvA4LlZYuSSAoOhMs5vwJkEGDlbPMaJoA+FcQ0IH38QLWkbAFLkOOhoMF5tU6/eBRhNjro0ZgKiPRAt3FLhCO/vqdgmNTm32LkmKpvBmQY4q5uAaAgbwDBG2BVv3bfI8KKAMWj2kfw9+pkZREIbEql4ST1x7hgHIANkbJ//MF8mAH/ilTCJ2tIi4ASr1IC3VNqXHKOxjy34mgoImnOQtx1g81fkqTiMOBVGcTogNhiT5iBHET8R8C+iApJUmgim3SQAXhsLQz7ee2G8gOAQNtJckBEplADiAxtX+G9NmhDl0qJKnTvyWlAMPYZnvIviGXRg6/Dh824DBXhP/tbfREXJEIvQ+aaPGjG7pvw6r3xdx+9hqb4dgZaP2XmdHO2K/B0c1+oUph6k8kShBryl/Ft0DYgjTlOieOACHFFpVyUl72T9V3cM1jUoYvxIC2vpCSys/ck70mDYuYvdvKjlMdKAUThneWVU1aAsyjv6PURDiwNsHGBZzY+JtAAgE2TFxdRHJdyIp/f+zqu09M5cDP2F08Ukkpj4YNSdX950HY2pNCCUK/Hhx5ZMBfjNSEzdsIihVzzAMdn9dz4eDYhnyQb9SSCiAryiJcQk82LiTbJ4x2FZJaUenpKnzP95WyDf4Y+QN9EFHHSeDLGdBjjKNQ5vKHf4XMA7KrY0y0GEObBOO/8e1ywuQExOHXktuQyJALEBpcEqhwtHqgiDuCK5b6i0p2MQpcckIIoh+6hYgTZtO8xlMi6O4tKCF/kOGHEg/W0UUpHW0ZoGNZ1ExZWcn7EErgwt4uj50E/sFBjXXIayWvh7WryjasxarZKssXon0zxvvkc32Q0bqbBCuZiKt9dWFysfQefeL29JYFaeztX6tePaZdz5mYx8+6Zq3Mk0wXECQxlhdzgS2wjBHju3j1RIgKyOMdNUE8X0+RAdbSapS11MRCv1SzUXmO6wGZe2SQYrv2MvCSWEv2VODE6DN7bz8ufypgQKW7uQskFTQHULLKyaEyrnlZbgOGLrV5qrn9U79jjm2HJmgkaVN98AfBub91lGPLZBqdroN5LYgjSu4zYZDDHXZOIPC691HqrWI1900I8qLzgKP4ft8DxEWigprPfrO+KcXno9gZz4jjGewWdUcpGCj0qVFuGPYbl2VturndZ2qRvlL8acDO6lF/DY/VjsFesiUK+ypJ+r/ep+cJkSQxEK4PG4WozgA75TYrDDqStE69K8/mzGEM+JXTeqvmedEElMmwCMm2SLd6bNNF9su02zEtoW6nAQtpMj5Gd7fKa//wqonF7UdtHFsVn+6hf1o7AfriPH7M6EeIUEF5zKVxXbYo7kS/OEtOqDYZKPoBsETIixn0uYrasThmzDkhdKPkz2EnaX0HdQbIgr59vAdGYDqjHrxkjS7WOxkTD8sqEqhiwcJETgBYigrBqF08KyDaje9SZ/I1A7MzaTzMGDEulPtZUkuKcyIRAjxEJPVrnVlb/9wkfij31D/pQt1IN+iL8bGJcstBIO7Y5VI/cwDqURbXhMuJxBqD0KLoK3esWFs0Jz5i5ZvJUAfFJMFb9XmGIOnzGpijpcWYCaMqXSQWp8EnCABepQ0Elyi4wfKfsw78ikIqif1pe1AGPlLmojl1SKxHHXp1L+Ut7AmDQHvhI5xHGi4EooO2BR7k78PEkJOdL7cAxQUZ/Tyclu9gnfwGgOmm2lNHGNmZXsq4Pqgc1EG1ATrvKl8s4R9ywwnqulGUnaRLVhxy8v3ieUwy2hbooT68uscW++DCDH0WSzuoyN2D4LUJ/tLECbcSKznwMIFs0ChF4mRTCnQbIIfk4SHJo6A9BMuTnXTs3Ku/KxsgZWqzuSe+Os8cEUfnMBY6UF5gi3SUbd5K7vDjq5WW0UENJlRsWn4sy21Er/E/AvPQSFHy1p4fgAAAAASUVORK5CYII=');  background-repeat: repeat-x;  background-size: 14px;  height: 28px;}\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import gc\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from collections import defaultdict\n",
    "from tqdm.notebook import tqdm\n",
    "import lightgbm as lgb\n",
    "import riiideducation\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import random\n",
    "import os\n",
    "import pickle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.00714,
     "end_time": "2020-12-24T22:00:35.118967",
     "exception": false,
     "start_time": "2020-12-24T22:00:35.111827",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Show the parameters by Optuna from [this Notebook](https://github.com/satorushibata0627/Publishment/blob/main/Kaggle_Python3_LightGBM_on_GPU_with_Feature_Engineering_Optuna_and_Visualization.ipynb). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-24T22:00:35.141894Z",
     "iopub.status.busy": "2020-12-24T22:00:35.140996Z",
     "iopub.status.idle": "2020-12-24T22:00:35.725799Z",
     "shell.execute_reply": "2020-12-24T22:00:35.724612Z"
    },
    "papermill": {
     "duration": 0.599079,
     "end_time": "2020-12-24T22:00:35.725943",
     "exception": false,
     "start_time": "2020-12-24T22:00:35.126864",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The result of Optuna:\n",
      "{'objective': 'binary', 'seed': 123, 'metric': 'AUC', 'boosting_type': 'gbdt', 'lambda_l1': 0.23094442978103413, 'lambda_l2': 0.10970746111051044, 'num_leaves': 206, 'feature_fraction': 0.7548698723252225, 'bagging_fraction': 0.7186994268081899, 'bagging_freq': 7, 'min_child_samples': 45}\n"
     ]
    }
   ],
   "source": [
    "Parameters = pd.read_pickle('../input/the-best-of-all-parameters-on-lightgbm/The_best_of_all_parameters_on_LightGBM.pickle')\n",
    "print('The result of Optuna:')\n",
    "print(Parameters)\n",
    "del Parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.011265,
     "end_time": "2020-12-24T22:00:35.749203",
     "exception": false,
     "start_time": "2020-12-24T22:00:35.737938",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Main functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "_cell_guid": "79c7e3d0-c299-4dcb-8224-4455121ee9b0",
    "_uuid": "d629ff2d2480ee46fbb7e2d37f6b5fab8052498a",
    "execution": {
     "iopub.execute_input": "2020-12-24T22:00:35.793830Z",
     "iopub.status.busy": "2020-12-24T22:00:35.788108Z",
     "iopub.status.idle": "2020-12-24T23:27:45.807509Z",
     "shell.execute_reply": "2020-12-24T23:27:45.808260Z"
    },
    "papermill": {
     "duration": 5230.048114,
     "end_time": "2020-12-24T23:27:45.808639",
     "exception": false,
     "start_time": "2020-12-24T22:00:35.760525",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "User feature calculation started...\n",
      "\n",
      "\n",
      "User feature calculation completed...\n",
      "\n",
      "\n",
      "Traning with 15000000 rows and 14 features\n",
      "Training until validation scores don't improve for 10 rounds\n",
      "[50]\ttraining's auc: 0.770203\tvalid_1's auc: 0.774857\n",
      "[100]\ttraining's auc: 0.773587\tvalid_1's auc: 0.777699\n",
      "[150]\ttraining's auc: 0.774888\tvalid_1's auc: 0.778489\n",
      "[200]\ttraining's auc: 0.775739\tvalid_1's auc: 0.778818\n",
      "[250]\ttraining's auc: 0.776491\tvalid_1's auc: 0.779042\n",
      "[300]\ttraining's auc: 0.777163\tvalid_1's auc: 0.779156\n",
      "[350]\ttraining's auc: 0.777754\tvalid_1's auc: 0.779268\n",
      "[400]\ttraining's auc: 0.778335\tvalid_1's auc: 0.779372\n",
      "[450]\ttraining's auc: 0.778908\tvalid_1's auc: 0.77941\n",
      "[500]\ttraining's auc: 0.779418\tvalid_1's auc: 0.779463\n",
      "[550]\ttraining's auc: 0.779955\tvalid_1's auc: 0.779493\n",
      "Early stopping, best iteration is:\n",
      "[580]\ttraining's auc: 0.780273\tvalid_1's auc: 0.779516\n",
      "Our ROC AUC score for the validation data is: 0.7795164620326916\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAxYAAAKdCAYAAAC+vv8IAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdebydVX3v8c+XSYZAIpMYRhVUhKpVcEZoHRgUUK84MRQtAtpqbetYrYCKoleLUgcEraCAIKJUBC7ScglQRQuKA0OvTDGQgJAQSJiH3/3jWafZnJ6Tc5Inycnweb9e+5W9n7XWs37r2ef1yvPba629U1VIkiRJUh+rTXQAkiRJklZ8JhaSJEmSejOxkCRJktSbiYUkSZKk3kwsJEmSJPVmYiFJkiSpNxMLSZIkSb2ZWEiSFkmSGuNx8DKO5eJl1d+SlOSkZX29JkqSi5P4w1nSSm6NiQ5AkrTCOmqU41ct0ygkScsFEwtJ0mKpqiMnOgZJ0vLDpVCSpKUmyYZJPpPk2iT3J7k7yb8nefUIdScn+UCSi5LckuShJHck+VGSFw2re/DA0ppdhy3FOrLV2W3w9Qj93Zzk5pHO2/7doy3huXtwGU+SNZK8O8nlSe5Jcl+SXyX56yS9/18dWjaUZM0kH09yQ5IHklyX5J0D9Q5P8tt2XW9JctTw/pNs0851UpJnJjk7yZwk9ya5bKT3obV7QpIPJ/lNG989SS5N8qYR6g728fQkZyT5Y5LHBt6nXVvdwffp4oFz/FmSE5Jc0/q6P8nvkhyRZO0R+jyynWO3JG9M8osW55wkpyfZfJRxbZjk6Hbu+9p7++skxyRZb4S64/rbldRxxkKStFQk2Rq4GNgGuBT4P8B6wGuB/5PksKo6caDJ9sDRwCXAucBdwFbAPsCeSfauqv/T6l5FtxTrCGA6cNLAeS5eAuG/EdgDOB84vo2BJGsC5wC7A/8FnAY8APwZ8M/AC4EDl0D/AKe3850HPNxiOiHJw8Czgb8Afgz8O901+jhwH/DZEc71FOBnwO+ArwNPBt4MnJ/kbVV1xlDFJGsBF9AlA9cBXwHWbf2fkeS5VfUPI/TxNODnwP8DTgXWAX5D9z4dDGzN45fP3Tzw/EPAM4Gf0r33awMvBY4Edkvyyqp6dIQ+393G/iNgWrtebwae0+J8cGBcTwH+b4vjSuBrdB+wPh34W7r3+d5Wd1H/diUBVJUPHz58+PAx7gdQ7XHkCI+DB+pdDDwGvGVY+yl0icH9wJMGjk8GNh6hvy2AmcC1o8Ry8Shx7jYU5yjlNwM3Dzt2cGvzGLDHCG2ObOX/DKw+cHx14JutbN9xXseTWv2Dhx2/uB3/T2DKwPGnAg/RJVw3AZsPu6Z3AncAawwc32bg/frfw/rZiS5huQvYYOD4R1r984ada9N2zQp4ySh9fHqUsV7c3XKMei2eCmSE459s533zKO/DPcCfDCs7rZW9adjx/2jHPzJCPxsDay/u364PHz66h0uhJEmL64gRHgcDJHkO3SfeZ1XV6YONqmpuq7s28L8Gjt9dVXcO76SqbgG+DzwzyVZLZST/07/WgtkRANoyo78GbgP+tgY+QW/P/57uxnX/JRTDh9u1GurjRuAyupvbT1bVrQNlc+lmUjYGRloGdDfwicEDVXUF3czCFOD1A0XvoBvH31XVIwP1/0h3ow9wyAh93M7oG/oXqqpurKqRvjXqi+3f3UdpelxV/XbYsaGZhBcMHUjyfOAldEnB/5jRqao7q+qBVneR/3YldVwKJUlaLFWVhRS/uP07eZQ9Dpu0f7cfPJjkpcDftPabAmsNa7c58IdFDnbR/WKEY08HNgJ+D3wsGXH49zNsTD1cMcKxme3fK0coG0o0tqBbHjbol1U1b4Q2F9MtqfpT4OQk6wPbArdW1XUj1L+o/funI5T9ugaWHi2Ktr/hb+gSnKcD6wODF3jEPROMfI1mtH+fOHBsaI/OBVX12BjhLNbfriQTC0nS0rFR+/dV7TGaSUNPkryebmbiAeBC4Aa6Ne+P0S1r2hV4wlKIdSS3jXBsaEzb0X1qPZpJCykbt6q6e4TDQzMICytbc4Sy20fpZmick4f9O2uU+kPHpyzkXIuk7Vu5iG6G4XfAGXRLuh5uVY5g9Pd97gjHhq7D6gPHhuK9lbEt8t+upI6JhSRpaRi68f2bqjpunG0+SbeHYKequnawIMnXad8stAiGPpke7f+6yYx8gw7dUqDhhur+sKresIixTLQnjXJ8s/bv3cP+3WyEutBt+h6sN2hxfwBvX7qk4uSqOniwIMmTWXgSN15DCchoMx+DFudvVxJ+3awkaem4vP27yyK02Ra4ZoSkYjXgZaO0eYzHfzI96K7275bDC5Jsy8ifui/MdXQ3qC9qn7KvSJ7XljkNt1v791cAbbnUDcDmSbYbof6ftX9/uYj9PwqQZKT3atv271kjlC1qMjmaob/H3cfxlcCL87crCRMLSdJS0DYGXwq8Ick7RqqT5E+SbDpw6GZguyRTB+qE7hPrZ43S1WxGSBya6+i+NWjfwX6SrAMs8ifRbSPzP9N9an9cO8/jJHlyktFinUiT6b6O9r8l2Yluo/ndwA8Hiv6Fbn/D/x5MBJJsDPzjQJ1FMbv9O9Lm+5vbv7sNi++pjPzVuYusqq6k+yrb59J9te3jJNlo6PcyFvNvVxIuhZIkLT1vo1s7/80k76X7jYO5dJuLnw3sSLdR9o+t/rF0vyXwqyRn0a2xfyldUnEOsPcIffw78JYk59BtaH4EuKSqLqmqh5N8ie5m+FdJfkj3/96r6DZBzxzhfGP5JPAc4HBg7yQX0a3b35Ru78VLgY8C1yzGuZemS4BDkryQ7mtXh37HYjXgsKq6Z6Du54E96ZYo/TrJeXS/Y7Ef3Tg/V1WXLWL//97a/6Cd735gelV9h+69vR74uyR/Qjd7shXdb0acy8jJyOI4gG6z+qeT/K/2PHTv26vpfkfj5lZ3Uf92JWFiIUlaSqrqlvY1n++h+2rO/emWLd1Gd+P9z8BvB+p/PcmDwPvovqnofrpPjt/e2o+UWPwN3dr+VwB70d0oH0V3Iw3dbMd9wDuBQ1vfp9P9DsIi3/y3ZOV1dDepB9Pd/E6i22x8E10Sc+qinncZuIkuGTqm/fsEuuVMn6iqCwYrVtVDSV4F/B3dDfZ76BK2XwPvq6rvLkb/36D7Ybq3AB+ku/+YBnynqu5N8ucttt3oliDdSJfE/RNdAtRbVd2U5Hmt/9fRfXXwA3TJxBcYSBIW9W9XUicjf220JEla0SXZhi6p+B8boyVpSXOPhSRJkqTeTCwkSZIk9WZiIUmSJKk391hIkiRJ6s0ZC0mSJEm9mVhIkiRJ6s3EQpIkSVJvJhaSJEmSejOxkCRJktSbiYUkSZKk3kwsJEmSJPVmYiFJkiSpNxMLSZIkSb2ZWEiSJEnqzcRCkiRJUm8mFpIkSZJ6M7GQJEmS1JuJhSRJkqTeTCwkSZIk9WZiIUmSJKk3EwtJkiRJvZlYSJIkSerNxEKSJElSbyYWkiRJknozsZAkSZLUm4mFJEmSpN5MLCRJkiT1ZmIhSZIkqTcTC0mSJEm9mVhIkiRJ6s3EQpIkSVJvJhaSJEmSejOxkCRJktSbiYUkSZKk3kwsJEmSJPVmYiFJkiSpNxMLSZIkSb2ZWEiSJEnqzcRCkiRJUm8mFpIkSZJ6M7GQJEmS1JuJhSRJkqTeTCwkSZIk9WZiIUmSJKk3EwtJkiRJvZlYSJIkSerNxEKSJElSbyYWkiRJknozsZAkSZLUm4mFJEmSpN5MLCRJkiT1ZmIhSZIkqTcTC0mSJEm9mVhIkiRJ6s3EQpIkSVJvJhaSJEmSeltjogOQVnUbb7xxbbPNNhMdhiRJ0piuvPLKO6tqk5HKTCykCbbNNttwxRVXTHQYkiRJY0oyfbQyl0JJkiRJ6s3EQpIkSVJvLoWSJtgjd8zhjq+dMtFhSJKkFdgm7zpgokNwxkKSJElSfyYWkiRJknozsZAkSZLUm4mFJEmSpN5MLCRJkiT1ZmIhSZIkqTcTC0mSJEm9mVhIkiRJ6s3EQpIkSVJvJhaSJEmSejOxWIkkuTjJIRMdx5KQ5Mgkp0x0HJIkSRofEwtNuCS7JbllouOQJEnS4jOxWE4lWWOiY1iYJKsPe71cxytJkqSla5VMLJJ8OMkNSeYluSbJ69vxg5NcluTzSe5KclOSPQfaHZzkxtbupiT7t+PTkzy/PT8gSSV5Vnt9SJKz2/PVBvqeneR7STZsZdu0dn+Z5A/ARe34O5Jc2+K5IMnWA/G8Ksl1Se5O8mUg4xz/O9s5h8b/vHZ8+7acam6Sq5PsM9DmpCRfS3JeknuBP0tyc5IPJfkNcG+SNZK8KMlP2zl+nWS3gXNsmORbSWa28ZydZD3gfGBqkvntMXVYvOcmec+wY79J8roxxvm465Nk2lhLxZI8LclF7f25M8mpSaa0sg8n+f6w+l9Kclx7/pQkl7Tr+m9JvuJyLkmStKpYJRML4AZgF2AycBRwSpInt7IXAv8FbAx8DvhmOusBxwF7VtX6wEuAq1qbacBu7fnLgRuBXQdeT2vP3wu8rpVNBe4CvjIstl2B7YHd243zPwBvADYBLgW+C5BkY+As4GMt1huAl4418CT7AUcCBwEbAPsAs5OsCZwD/ATYFHgPcGqSZww0fxtwNLA+cFk79lbgNcAU4EnAucCngA2B9wNnJdmk1f0OsC6wQ+vj2Kq6F9gTmFlVk9pj5rCwTwYOGBjDc4DNgfMWMs7Fuj50ydln6N6f7YEt6a4XdNd+ryQbtD5WB94EnNbKTwN+AWzU2hw4jv4kSZJWCqtkYlFVZ1bVzKp6rKrOAH4PvKAVT6+qE6vqUbob2ifT3TADPAbsmGSdqppVVVe349NYkEjsQndjOvR6VxYkFocBH62qW6rqQbqbzzcOW0Z0ZFXdW1X3t/qfqaprq+oR4NPAc9usxV7ANVX1/ap6GPgicNs4hn8I8Lmq+s/qXF9V04EXAZOAY6rqoaq6CPgxXeIw5F+r6j/adXugHTuuqma0eA8Azquq81qdC4Er6G7Gn0yXQBxeVXdV1cNVNY3x+VdguyTbtdcHAmdU1UMLabNY16ddjwur6sGqugP4J9p72a7TL+mSQ4A/B+6rqsuTbAXsDHy8Xb/LgB+N1k+SQ5NckeSK2fPvGSssSZKk5d4qmVgkOSjJVW25zlxgR7pPtWHg5rOq7mtPJ7VP1t8MHA7MastzntnKpwG7JNkMWB04A3hpkm3oZkWGZja2Bn440O+1wKMsSFwAZgw83xr40kD9OXSfqG9O94n6f9etqhrWdjRb0n16P9xUYEZVPTZwbHrra6TYRot3v6F4W8wvo0vOtgTmVNVd44jxcVoS9j3ggCSr0SU73xmj2WJdnySbJjk9ya1J7gFOYcHfBnSzEkPJ1ttYMFsxlW589w3UHbW/qjqhqnaqqp02mrTBWGFJkiQt91a5xKJ92n8i8NfARlU1Bfgd49ifUFUXVNWr6G6Ur2vnoaquB+6jW+p0SVXNo0tQDgUuG7hZn0G3lGrKwGPtqrp1sJuB5zOAw4bVX6eqfgrMortZHxpXBl8vxAzgaSMcnwls2W7ch2wFjBbbaPF+Z1i861XVMa1sw6H9Cgs5x2hOBvYHXkE3S/CzMeov7vX5TIvn2VW1Ad0szODfxpnAbkm2AF7PgsRiFt341h2oO57+JEmSVgqrXGIBrEd343gHQJK3081YLFSSJyXZp+21eBCYTzfbMGQaXbIytLzn4mGvAY4Hjh7agJ1kkyT7LqTb44GPJNmh1Z/c9khAt5dhhyRvaEup3gtsNtY4gG8A70/y/LZ3ZNsWz8+Be4EPJlmzbbreGzh9HOcccgqwd5Ldk6yeZO10XyW7RVXNotuk/dUkT2x9vLy1ux3YKMnk0U7cEonHgC8w9mwFLP71WZ/uvZ2bZHPgA8PiuIPuvf0WcFNVXduOT6db9nVkkrWSvJju+kmSJK0SVrnEoqquobs5/RndDe2fAP8xjqarAX9P98n+HLp19+8eKJ9Gd1N6ySivAb5Et+7+J0nmAZfTbRYfLdYfAp8FTm/Lcn5Ht0+BqroT2A84BpgNbDeecVTVmXQbsE8D5gFnAxu2/Qr7tPPfCXwVOKiqrhvrnAPnngHsS7fh/A66WYoPsODv7EDgYbrZnj8C72vtrqPbGH1jW0I1lZF9m+79GvOblhb3+tBt5n8ecDddcvKDEeqcBrySBbMVQ/YHXtz6+xTdkrgHx9GnJEnSCi/d0nNp+ZfkIODQqnrZYra/GDilqr6xRAMbvb8zgOuq6oiF1Xvu1k+tCz/8iWURkiRJWklt8q4Dxq60BCS5sqp2GqlslZux0Iqp7V14N3DCRMcymiQ7t9/BWC3JHnSzN2dPdFySJEnLgonFSijJ8VnwY3ODj+MnOrbFkWR3uqVVtzOw/CjJLqOMc/4Y51ta12czuv0X8+l+8+RdVfWrnueUJElaIbgUSppgLoWSJEl9uRRKkiRJ0krBxEKSJElSbyYWkiRJknozsZAkSZLUm4mFJEmSpN5MLCRJkiT1tsZEByCt6tbYZMNl9hVxkiRJS4szFpIkSZJ6M7GQJEmS1JuJhSRJkqTeTCwkSZIk9WZiIUmSJKk3EwtJkiRJvZlYSJIkSerN37GQJtjDf/wDt37lryY6DEmSVgmb/9VXJjqElZYzFpIkSZJ6M7GQJEmS1JuJhSRJkqTeTCwkSZIk9WZiIUmSJKk3EwtJkiRJvZlYSJIkSerNxEKSJElSbyYWkiRJknozsViJJLk4ySETHceSkOTIJKdMdBySJEkaHxMLTbgkuyW5ZaLjkCRJ0uIzsVhOJVljomNYmCSrD3u9XMcrSZKkpWuVTCySfDjJDUnmJbkmyevb8YOTXJbk80nuSnJTkj0H2h2c5MbW7qYk+7fj05M8vz0/IEkleVZ7fUiSs9vz1Qb6np3ke0k2bGXbtHZ/meQPwEXt+DuSXNviuSDJ1gPxvCrJdUnuTvJlIOMc/zvbOYfG/7x2fPu2nGpukquT7DPQ5qQkX0tyXpJ7gT9LcnOSDyX5DXBvkjWSvCjJT9s5fp1kt4FzbJjkW0lmtvGcnWQ94HxgapL57TF1WLznJnnPsGO/SfK6hYxx6HquMXBszKViSZ6W5KL2/tyZ5NQkU1rZh5N8f1j9LyU5rj1/SpJL2nX9tyRfcTmXJElaVaySiQVwA7ALMBk4CjglyZNb2QuB/wI2Bj4HfDOd9YDjgD2ran3gJcBVrc00YLf2/OXAjcCuA6+ntefvBV7XyqYCdwFfGRbbrsD2wO7txvkfgDcAmwCXAt8FSLIxcBbwsRbrDcBLxxp4kv2AI4GDgA2AfYDZSdYEzgF+AmwKvAc4NckzBpq/DTgaWB+4rB17K/AaYArwJOBc4FPAhsD7gbOSbNLqfgdYF9ih9XFsVd0L7AnMrKpJ7TFzWNgnAwcMjOE5wObAeWONdzEE+Azd+7M9sCXd9YLu2u+VZIMWx+rAm4DTWvlpwC+AjVqbA5dCfJIkSculVTKxqKozq2pmVT1WVWcAvwde0IqnV9WJVfUo3Q3tk+lumAEeA3ZMsk5Vzaqqq9vxaSxIJHahuzEder0rCxKLw4CPVtUtVfUg3c3nG4ctIzqyqu6tqvtb/c9U1bVV9QjwaeC5bdZiL+Caqvp+VT0MfBG4bRzDPwT4XFX9Z3Wur6rpwIuAScAxVfVQVV0E/JgucRjyr1X1H+26PdCOHVdVM1q8BwDnVdV5rc6FwBV0N+NPpksgDq+qu6rq4aqaxvj8K7Bdku3a6wOBM6rqoXG2H7d2PS6sqger6g7gn2jvZbtOv6RLDgH+HLivqi5PshWwM/Dxdv0uA340Wj9JDk1yRZIrZs+/f0kPQ5IkaZlbJROLJAcluaot15kL7Ej3qT8M3JxX1X3t6aT2yfqbgcOBWW15zjNb+TRglySbAasDZwAvTbIN3azI0MzG1sAPB/q9FniUBYkLwIyB51sDXxqoP4fuE/XN6T5R/++6VVXD2o5mS7rZjeGmAjOq6rGBY9NbXyPFNlq8+w3F22J+GV1ytiUwp6ruGkeMj9OSsO8BByRZjS7Z+c6inmc8kmya5PQktya5BziFBX8b0M1KDCVbb2PBbMVUuvHdN1B31Pejqk6oqp2qaqeNJq2zBEcgSZI0MVa5xKJ92n8i8NfARlU1Bfgd49ifUFUXVNWr6G6Ur2vnoaquB+6jW+p0SVXNo0tQDgUuG7hZn0G3lGrKwGPtqrp1sJuB5zOAw4bVX6eqfgrMortZHxpXBl8vxAzgaSMcnwls2W7ch2wFjBbbaPF+Z1i861XVMa1sw6H9Cgs5x2hOBvYHXkE3S/CzMerf2/5dd+DYZuPo5zMtnmdX1QZ0szCDfxtnArsl2QJ4PQsSi1l04xvsbzzvhyRJ0kphlUssgPXobhzvAEjydroZi4VK8qQk+7S9Fg8C8+lmG4ZMo0tWhpb3XDzsNcDxwNFDG7CTbJJk34V0ezzwkSQ7tPqT2x4J6PYy7JDkDW0p1XsZ343zN4D3J3l+2zuybYvn53Q34x9MsmbbdL03cPo4zjnkFGDvJLsnWT3J2um+SnaLqppFt0n7q0me2Pp4eWt3O7BRksmjnbglEo8BX2AcsxVtGdOtdLMcqyd5ByMnVMOtT/fezk2yOfCBEc57MfAt4KaqurYdn0637OvIJGsleTHd9ZMkSVolrHKJRVVdQ3dz+jO6G9o/Af5jHE1XA/6e7pP9OXTr7t89UD6N7qb0klFeA3yJbt39T5LMAy6n2yw+Wqw/BD4LnN6W5fyObp8CVXUnsB9wDDAb2G4846iqM+k2YJ8GzAPOBjZs+xX2aee/E/gqcFBVXTfWOQfOPQPYl27D+R10sxQfYMHf2YHAw3SzPX8E3tfaXUe3MfrGtoRqKiP7Nt37Nd5vWnpn63823Ybxn46jzVHA84C76ZK3H4xQ5zTglSyYrRiyP/Di1t+n6JbEPTjOWCVJklZo6ZbmS8u/JAcBh1bVyyY6lvFIcgZwXVUdsbB6z9lq0zrvQ/strIokSVpCNv+r4V/IqUWR5Mqq2mmkslVuxkIrprZ34d3ACRMdy2iS7Nx+B2O1JHvQzd6cPdFxSZIkLQsmFiuhJMcP/Njc4OP4iY5tcSTZnW5p1e0MLD9Kssso45w/xvmW1vXZjG7/xXy63zx5V1X9quc5JUmSVgguhZImmEuhJEladlwK1Y9LoSRJkiQtVSYWkiRJknozsZAkSZLUm4mFJEmSpN5MLCRJkiT1ZmIhSZIkqTcTC0mSJEm9rTHRAUirujU33crv1JYkSSs8ZywkSZIk9WZiIUmSJKk3EwtJkiRJvZlYSJIkSerNxEKSJElSbyYWkiRJknozsZAkSZLUm79jIU2w+Xdez6Unvnaiw5AkaaWzyzt/PNEhrFKcsZAkSZLUm4mFJEmSpN5MLCRJkiT1ZmIhSZIkqTcTC0mSJEm9mVhIkiRJ6s3EQpIkSVJvJhaSJEmSejOxkCRJktSbicViSLJVkvlJVp/oWCRJkqTlgYnFOCW5OckrAarqD1U1qaoeXYb975bklmXVnxZdkjcl+WmS+5JcPNHxSJIkLUtrTHQAUpI1quqRiY5jCZgDfBF4JvDnExyLJEnSMuWMxTgk+Q6wFXBOWwL1wSSVZI1WfnGST7VPq+cnOSfJRklOTXJPkv9Mss3A+Z6Z5MIkc5L8V5I3DZTtleSaJPOS3Jrk/UnWA84Hprbzz08yNckLkvwsydwks5J8OclaA+eqJO9O8vt2vk8meVprc0+S7w3VH5oRSfIPSe5sMzT7j+PaXJzkkIHXBye5bBztKslfJfk98Pt27LVJrmrj+WmSZw/U3zLJD5LckWR2ki8PlL0jybVJ7kpyQZKth/VzeLsGdyX5SpIMlL+ztZ3XrvvzknwgyVnD4v3nJF9c2Jiq6t+q6nvAzLHGL0mStLIxsRiHqjoQ+AOwd1VNAr43QrW3AAcCmwNPA34GfAvYELgWOAKgJQkXAqcBmwJvBb6aZId2nm8Ch1XV+sCOwEVVdS+wJzCzLcGaVFUzgUeBvwU2Bl4MvAJ497C49gCeD7wI+CBwArA/sGU7/1sH6m7WzrU58BfACUmesUgXa9G8Dngh8KwkzwP+BTgM2Aj4OvCjJE9oe1l+DEwHtmnxnQ6Q5HXAPwBvADYBLgW+O6yf1wI7A88B3gTs3truBxwJHARsAOwDzAZOAfZIMqXVWwN4M/CdJTx+SZKklYaJxZLzraq6oarupptduKF9gv0IcCbwp63ea4Gbq+pbVfVIVf0SOAt4Yyt/mO5Ge4OququVj6iqrqyqy9t5bqa7Gd91WLXPVtU9VXU18DvgJ1V140Ccfzqs/j9W1YNVNQ04l+5GfGn5TFXNqar7gXcCX6+qn1fVo1V1MvAgXUL0AmAq8IGqureqHqiqoVmRw9p5rm3X+tPAcwdnLYBjqmpuVf0B+L/Ac9vxQ4DPVdV/Vuf6qppeVbOAS4D9Wr09gDur6solNfAkhya5IskVc+c9tKROK0mSNGFMLJac2wee3z/C60nt+dbAC9tyn7lJ5tLNIGzWyv8XsBcwPcm0JC8ercMkT0/y4yS3JbmH7qZ648WMC+CuNjsyZDrdDf3SMmPg+dbA3w+7Llu2/rcEpo+yD2Nr4EsDbeYAoZvVGHLbwPP7WDDmLYEbRontZOCA9vwAlvBsRVWdUFU7VdVOU9Zfa+wGkiRJyzkTi/GrJXSeGcC0qpoy8JhUVe8CaJ+e70u3TOpsFiy7Gqn/rwHXAdtV1QZ0S4IyQr3xemJbqjVkK8beL3AvsO7A681GqziCwTHNAI4edl3WrarvtrKthva0DDODbunYYLt1quqn4+h/Bt2ytZGcDTw7yY50s0ynjn9YkiRJqx4Ti/G7HXjqEjjPj4GnJzkwyZrtsXOS7ZOslWT/JJOr6mHgHrp9FEP9b5Rk8sC51m915id5JvCuJRDfUS2OXehuqM8co/5VwBuSrJtkW+AvF7PfE4HDk7wwnfWSvCbJ+sAvgFnAMe342kle2todD3xkaI9Kkslt78R4fAN4f5Lntz63HVpCVVUPAN+n2wvzi7aMaqGSrJ5kbfUZujAAACAASURBVLpvW1utxbnmIlwDSZKkFZaJxfh9BvhYW27zxrEqj6aq5gGvptvsPZNumc5ngSe0KgcCN7elTYfTluNU1XV0m5JvbMt+pgLvB94GzKO7MT9jceNqbgPuanGdChze+l2YY4GH6BKfk1nMT/ar6gq6fRZfbjFcDxzcyh4F9ga2pdtEfwvdZmqq6od01+/0ds1+R7fRfTx9ngkcTZc8zKObpdhwoMrJwJ8w/mVQB9ItL/sasEt7fuI420qSJK3QUrWkVvhoRZZkN+CUqtpiomNZXiTZim6p2WZVdc/S6ueZ20ypEz/6sqV1ekmSVlm7vPPHEx3CSifJlVW100hlzlhII0iyGvB3wOlLM6mQJElaWfjL2xpTkqvpvn1puMOqasSlT22PxvkjlbXfAllutQ3st9N9K9Yew8rmj9Jsz6q6dGnHJkmStLwysRAAVXUxMOIyqKraYaTjY5zvUh7/VbYrjPaVuyPGvrwnRZIkSRPFpVCSJEmSejOxkCRJktSbiYUkSZKk3kwsJEmSJPVmYiFJkiSpNxMLSZIkSb2ZWEiSJEnqzd+xkCbYpI23ZZd3/niiw5AkSerFGQtJkiRJvZlYSJIkSerNxEKSJElSbyYWkiRJknozsZAkSZLUm4mFJEmSpN5MLCRJkiT15u9YSBNszuzf892Tdp/oMCRJWm699eALJjoEjYMzFpIkSZJ6M7GQJEmS1JuJhSRJkqTeTCwkSZIk9WZiIUmSJKk3EwtJkiRJvZlYSJIkSerNxEKSJElSbyYWkiRJknozsViOJTkyySkTHcegJJVk28Vod3WS3ZZCSJIkSVoOrDHRAWjlk+Qk4Jaq+tjQsaraYeIikiRJ0tLmjIUkSZKk3kwslgNJpiY5K8kdSW5K8t5R6p2Z5LYkdye5JMkOA2UnJTk+yYVJ5iWZlmTrVpYkxyb5Y2v7myQ7trInJPl8kj8kub2dY52B834gyawkM5O8YxxjORTYH/hgkvlJzmnHb07yyvb8yDaWU1qsv03y9CQfaTHOSPLqgXNOTvLNFsetST6VZPUx4li9jevOJDcm+au2jGuhs3RJ3p7k2hbXjUkOGyi7NslrB16v0c7/vPb6oCTTk8xO8o+DY5YkSVrZmVhMsCSrAecAvwY2B14BvC/J7iNUPx/YDtgU+CVw6rDy/YFPAhsDVw2Uvxp4OfB0YArwZmB2K/tsO/5cYNsWw8dbbHsA7wde1fod8ya5qk5o/X6uqiZV1d6jVN0b+A7wROBXwAV0f4+bA58Avj5Q92TgkRbfn7bxHDJGKO8EXtvq7wS8cazYmz+2dhsAbweOHUocgO8Cbx2ouztwZ1X9MsmzgK/SvQdPBia3sUiSJK0STCwm3s7AJlX1iap6qKpuBE4E3jK8YlX9S1XNq6oHgSOB5ySZPFDl3Kq6pJV/FHhxki2Bh4H1gWcCqaprq2pWktDdgP9tVc2pqnnApwf6fhPwrar6XVXd2/pcUi6tqguq6hHgTGAT4Jiqehg4HdgmyZQkTwL2BN5XVfdW1R+BY0e6PsO8CfhiVc2oqjnAZ8YTVFWdW1U3VGca8BNgl1Z8GrBPknXb67e1Y9AlLudU1WVV9RBdclaj9ZPk0CRXJLli3ryHxhOaJEnScs3N2xNva2BqkrkDx1YHLgWmDx1oS3+OBvajuwl/rBVtDNzdns8Yql9V85PMAaZW1UVJvgx8BdgqyQ/pZiLWBtYFruxyjK6r1j/AVODKgbims+TcPvD8frpP/h8deA0wqcWwJjBrIMbVGBjrKKYOqzOu2JPsCRxBN4uzGt31+S1AVV2f5Fpg77bEax+6GZH/0V9V3ZdkNqNoMzsnADz1KZNHTUAkSZJWFM5YTLwZwE1VNWXgsX5V7TWs3tuAfemWI00GtmnHM1Bny6EnSSYBGwIzAarquKp6PrAD3U3zB4A76W7idxjoe3JVTWqnmTV4TmCrcY5pSd4ozwAeBDYeiHGDcXzL1CLHnuQJwFnA54EnVdUU4Dwef42HlkPtC1xTVdcP9LfFwLnWATYaq09JkqSVhYnFxPsFcE+SDyVZp2063jHJzsPqrU93gz2b7lP0T49wrr2SvCzJWnR7LX5eVTOS7JzkhUnWBO4FHgAerarH6JZdHZtkU4Akmw/s7/gecHCSZ7XlP0eMc0y3A08d9xVYiKqaRbcc6QtJNkiyWpKnJdl1jKbfA96bZIskTwQ+PI7u1gKeANwBPNJmL149rM7p7di7WLAMCuD7dDMZL2nX/ygen5BIkiSt1EwsJlhb/rM33ebpm+hmEb5BNysx6Nt0y3luBa4BLh/hdKfR3fzPAZ5Pt5EYuo3IJwJ3tXPMpvtUHuBDwPXA5UnuAf4NeEaL7Xzgi8BFrc5F4xzWN4FnJZmb5OxxtlmYg+hu+q9pY/g+3QbphTmRbkP4r+k2uv9grE7aHpP30iUld9HNEv1oWJ1ZwM+AlwBnDBy/GngPXeIxC5hHtxH8wbH6lSRJWhmkyuXdK4OM8KN0WiDJNnSJ25ptw/jS7m8SMBfYrqpuWljdpz5lch19xIuWdkiSJK2w3nrwBRMdgpokV1bVTiOVOWMhLSFJ9k6ybpL16GaEfgvcPLFRSZIkLRsmFlosSa5uP4A3/LH/2K2XWAzHjxLD8WO0G6nN/CS7LKzdOOxLt1l+Jt3vfrylnBKUJEmrCL9udiVRVQcv4/7G+lamZRHD4cDh46x7Mws2U09aSNU+8RzC2D/cJ0mStFJyxkKSJElSbyYWkiRJknozsZAkSZLUm4mFJEmSpN5MLCRJkiT1ZmIhSZIkqTcTC0mSJEm9+TsW0gTbcKPteOvBF0x0GJIkSb04YyFJkiSpNxMLSZIkSb2ZWEiSJEnqzcRCkiRJUm8mFpIkSZJ6M7GQJEmS1JtfNytNsNvm/J7Pnr77RIchSdJy5UNv8avYVzTOWEiSJEnqzcRCkiRJUm8mFpIkSZJ6M7GQJEmS1JuJhSRJkqTeTCwkSZIk9WZiIUmSJKk3EwtJkiRJvZlYSJIkSerNxEKSJElSbyYWy7EkRyY5ZaLjGJSkkmy7GO2uTrLbUghJkiRJy4E1JjoArXySnATcUlUfGzpWVTtMXESSJEla2pyxkCRJktSbicVyIMnUJGcluSPJTUneO0q9M5PcluTuJJck2WGg7KQkxye5MMm8JNOSbN3KkuTYJH9sbX+TZMdW9oQkn0/yhyS3t3OsM3DeDySZlWRmkneMYyyHAvsDH0wyP8k57fjNSV7Znh/ZxnJKi/W3SZ6e5CMtxhlJXj1wzslJvtniuDXJp5KsPkYcj1tGlmSbtoxrobN0Sd6e5NoW141JDhsouzbJawder5HkziTPa68PSjI9yewk/zg4ZkmSpJWdicUES7IacA7wa2Bz4BXA+5LsPkL184HtgE2BXwKnDivfH/gksDFw1UD5q4GXA08HpgBvBma3ss+2488Ftm0xfLzFtgfwfuBVrd8xb5Kr6oTW7+eqalJV7T1K1b2B7wBPBH4FXED397g58Ang6wN1TwYeafH9aRvPIWPFspj+CLwW2AB4O3DsUOIAfBd460Dd3YE7q+qXSZ4FfJXuPXgyMLmNRZIkaZVgYjHxdgY2qapPVNVDVXUjcCLwluEVq+pfqmpeVT0IHAk8J8nkgSrnVtUlrfyjwIuTbAk8DKwPPBNIVV1bVbOSBHgn8LdVNaeq5gGfHuj7TcC3qup3VXVv63NJubSqLqiqR4AzgU2AY6rqYeB0YJskU5I8CdgTeF9V3VtVfwSOHen6LAlVdW5V3VCdacBPgF1a8WnAPknWba/f1o4BvBE4p6ouq6qH6JKzGq2fJIcmuSLJFffOe2hpDEWSJGmZcvP2xNsamJpk7sCx1YFLgelDB9rSn6OB/ehuwh9rRRsDd7fnM4bqV9X8JHOAqVV1UZIvA18BtkryQ7qZiLWBdYEruxyj66r1DzAVuHIgruksObcPPL+f7pP/RwdeA0xqMawJzBqIcTUGxrokJdkTOIJuFmc1uuvzW4Cquj7JtcDebYnXPnQzKLQ4B6//fUlmM4o2s3MCwBZPnTxqAiJJkrSicMZi4s0AbqqqKQOP9atqr2H13gbsS7ccaTKwTTuegTpbDj1JMgnYEJgJUFXHVdXzgR3obpo/ANxJdxO/w0Dfk6tqUjvNrMFzAluNc0xL8kZ5BvAgsPFAjBuM41um7qVLCoZsNlZHSZ4AnAV8HnhSVU0BzuPx13hoOdS+wDVVdX07PgvYYuBc6wAbjdWnJEnSysLEYuL9ArgnyYeSrJNk9SQ7Jtl5WL316W6wZ9PdMH96hHPtleRlSdai22vx86qakWTnJC9MsibdDfcDwKNV9Rjdsqtjk2wKkGTzgf0d3wMOTvKstvzniHGO6XbgqeO+AgtRVbPoliN9IckGSVZL8rQku47R9Crg5Um2asvFPjKO7tYCngDcATzSZi9ePazO6e3Yu1iwDArg+3QzGS9p1/8oHp+QSJIkrdRMLCZYW/6zN93m6ZvoZhG+QTcrMejbdEuRbgWuAS4f4XSn0d38zwGeT7eRGLqNyCcCd7VzzKb7VB7gQ8D1wOVJ7gH+DXhGi+184IvARa3OReMc1jeBZyWZm+TscbZZmIPobvqvaWP4Pt0G6VFV1YXAGcBv6JZz/XisTtoek/fSJVR30c0S/WhYnVnAz4CXtPMPHb8aeA9d4jELmEe3EfzBcYxPkiRphZcql3evDDLCj9Jp4rSlaHOB7arqpoXV3eKpk+s9n37RsglMkqQVxIfecsFEh6ARJLmyqnYaqcwZC2kJSbJ3knWTrEc3I/Rb4OaJjUqSJGnZMLHQYklydfsBvOGP/cduvcRiOH6UGI4fo91IbeYn2WVh7cZhX7rN8jPpfvfjLeWUoCRJWkX4dbMriao6eBn3N9a3Mi2LGA4HDl+MdpPGrrXoquoQlt4P90mSJC3XnLGQJEmS1JuJhSRJkqTeTCwkSZIk9WZiIUmSJKk3EwtJkiRJvZlYSJIkSerNr5uVJthmG27nr4tKkqQVnjMWkiRJknozsZAkSZLUm4mFJEmSpN5MLCRJkiT1ZmIhSZIkqTcTC0mSJEm9mVhIkiRJ6s3fsZAm2O/n3sye//oXEx2GJEkT6vx9T57oENSTMxaSJEmSejOxkCRJktSbiYUkSZKk3kwsJEmSJPVmYiFJkiSpNxMLSZIkSb2ZWEiSJEnqzcRCkiRJUm8mFpIkSZJ6M7GQJEmS1JuJxSouSSXZdgL63SXJfy3rfiVJkrR0mFhomRiewFTVpVX1jImMSZIkSUuOiYUkSZKk3kwsViBJpiY5K8kdSW5K8t4kGya5Jcnerc6kJNcnOai9PinJ8UkuTDIvybQkW49y/tck+VWSe5LMSHLkQNk2bdbhL5L8IcmdST46UP6CJD9LMjfJrCRfTrJWK7ukVft1kvlJ3pxktyS3DLTfPsnFrf3VSfYZKDspyVeSnNvG8PMkTxvH9XpVkuuS3N3imZbkkDHaPC3JRUlmtzGemmRKK/twku8Pq/+lJMe1509JckmL8d9azKeMFackSdLKwMRiBZFkNeAc4NfA5sArgPcBOwPvAE5MsilwLHBVVX17oPn+wCeBjYGrgFNH6eZe4CBgCvAa4F1JXjeszsuAZ7T+P55k+3b8UeBvWx8vbuXvBqiql7c6z6mqSVV1xrCxrdnG9hNgU+A9wKlJBpdKvRU4CngicD1w9ChjGDrnxsBZwMdaTDcAL11Ym6GmwGeAqcD2wJbAka3su8BeSTZofawOvAk4rZWfBvwC2Ki1OXAc/UmSJK0UTCxWHDsDm1TVJ6rqoaq6ETgReEtV/QQ4E/h3uoTgsGFtz62qS6rqQeCjwIuTbDm8g6q6uKp+W1WPVdVv6G6kdx1W7aiqur+qfk2X5Dyntb2yqi6vqkeq6mbg6yO0Hc2LgEnAMW1sFwE/pksmhvygqn5RVY/QJUbPHeOcewHXVNX3q+ph4IvAbWMFUlXXV9WFVfVgVd0B/NPQOKpqOvBLYCjZ+nPgvqq6PMlWdO/Rx9sYLgN+NFo/SQ5NckWSKx6654GxwpIkSVrumVisOLYGpralQnOTzAX+AXhSKz8B2BH4VlXNHtZ2xtCTqpoPzKH7RP5xkrwwyf9tS63uBg6n+7R/0ODN+X10CQFJnp7kx0luS3IP8OkR2o5mKjCjqh4bODadbmZmof2Odc6hF1VVg69Hk2TTJKcnubWN4xQeP47TWJDwvI0FsxVTgTlVdd9A3VH7q6oTqmqnqtpprQ3WHissSZKk5Z6JxYpjBnBTVU0ZeKxfVXu1JTlfB75Nt3xp+NfH/vfsRJJJwIbAzBH6OI3uU/Ytq2oycDzd0qDx+BpwHbBdVW1Al/SMt+1MYMu23GvIVsCt42w/klk8ftwZfL0QnwEKeHYbxwE8fhxnArsl2QJ4PQsSi1nAhknWHag7nv4kSZJWCiYWK45fAPck+VCSdZKsnmTHJDvT3cRDt9fi88C3W7IxZK8kL2ubqT8J/LyqRvo0fX26T90fSPICuk/kx2t94B5gfpJnAu8aVn478NRR2v6cbn/HB5OsmWQ3YG/g9EXof7hzgR2SvCHJGsB7gc3G0W59YD4wN8nmwAcGC9vyqIuBb9Elete249OBK4Ajk6yV5MVtDJIkSasEE4sVRFU9Snej+lzgJuBO4Bt06/z/Djio1fks3SfuHx5ofhpwBN0SqOfTbeYeybuBTySZB3wc+N4ihPh+ukRkHt3ejzOGlR8JnNyWcb1p2NgeAvYB9mzj+mobz3WL0P/jVNWdwH7AMcBsYDvgP8bR9CjgecDddMnJD0aocxrwShbMVgzZn27j+mzgU3TX4MHFCF+SJGmFk27puVZWSU4Cbqmqj010LBMtycXAKVX1jWXU3xnAdVV1xMLqTd5243rJF16zLEKSJGm5df6+J090CBqHJFdW1U4jlTljIS0hSXZuv4OxWpI9gH2Bsyc6LkmSpGXBxEIrrCS7tB/c+x+PMdodP0q743uGtBnd/ov5wHHAu6rqVz3PKUmStEJYY6ID0NJVVQdPdAxLS1VdythfOztYf7f29Bt0X6W7pOM5h+6H/iRJklY5zlhIkiRJ6s3EQpIkSVJvJhaSJEmSejOxkCRJktSbiYUkSZKk3kwsJEmSJPXm181KE2y7Kdv4a6OSJGmF54yFJEmSpN5MLCRJkiT1ZmIhSZIkqTcTC0mSJEm9mVhIkiRJ6s3EQpIkSVJvJhaSJEmSevN3LKQJ9vu5t/OaH3xxosOQJGmpOPcN75voELSMOGMhSZIkqTcTC0mSJEm9mVhIkiRJ6s3EQpIkSVJvJhaSJEmSelukxCLJLkn2HXi9cZLTklyV5AtJ1lzyIUqSJEla3i3qjMXngB0HXn8JeAVwOXAwcNSSCUuSJEnSimRRE4tnAFcCJFkXeD3wN1V1OPBB4M1LNjxJkiRJK4JFTSzWAh5oz19K9wN757bX/w948hKKS5IkSdIKZFETi+uAPdrz/YGfVdW89noqMGdJBSZJkiRpxbHGItb/BHBmkr8EJgP7DpTtAfxqSQW2PEuyFXANMLmqHp3oeCRJkqSJtkgzFlX1I2B74HBgx6o6f6D4Z8DRSzC25UqSm5O8EqCq/lBVk5ZlUpFktyS3/H/27jzMsqq89/j3J6PQzIPYCE0EEQJRVBS9ESXRiOAYRRwYRGRyvCZOEQ2DiqiJVzGoiHoBBQRxQAENOAREwQGMGhm8AtI0NCA03fTADO/9Y6+yD2VV16k+1V1dzffzPPX0OXvtvfa71jn9PPs9a629l9f5NH5JPp5kVpL5SWYmef9kxyRJkrS8jHfEgqq6DrhuhO0nTkhEesRJsmpVPTDZcUyALwFHV9WiJJsDFyS5qqq+OdmBSZIkLWvjfkBekiclOTPJtUnuTfLUtv2YJHtMfIiTL8lXgC2Bc5IsTPKeJJVk1VZ+YZIPJ7mklZ+TZKMkp7Vfr3+ZZKue+rZL8v0kdyT5fZK9e8r2THJlkgVJbkryriRrA98Dprf6FyaZnuQZSS5NMi/JzUmOT7J6T12V5M1J/tDq+1CSrdsx85N8bWj/oRGRJIcnub2N0OzTR99cmOSgnvcHJPlJH8dVkrck+QPwh7btxe2ZKPNaXz6pZ/8tknwzyW1J5iQ5vqfswCRXJZmb5PwkM4ad57DWB3OTfCZJesoPbscuaP3+1CTvTvKNYfH+R5JPLalNVfX7qlrUs+khYJux+kKSJGllMN4H5O1Bd7vZzYAvA70PxLsXeNvEhbbiqKr9gBuAl1TVNOBrI+z2GmA/YHNga7qpYScBGwJXAUcCtCTh+8DpwKbAa4HPJtmh1fMl4NCqWofumSE/aherewCz2xSsaVU1G3gQ+CdgY+BZdM8UefOwuF4IPA14Jt0tgU+kW3i/Rav/tT37btbq2hx4PXBikieOq7PG5+XALsBftwT1/wKHAhsBnwe+k2SNJKsA5wIzga1afGcAJHk5cDjwCmAT4GLgq8PO82Lg6cCTgb2B3duxrwKOAvYH1gVeCswBTgVemGT9tt+qdLdS/spYDUryL0kWAjcCa9N9zpIkSSu98Y5YHAucXFXP5S/XU/wa2GlCopqaTqqqa6vqTrrRhWur6gdtis9ZwFPafi8Grq+qk6rqgar6FfANYK9Wfj/dhfa6VTW3lY+oqi6vqp+1eq6nuxh/7rDdPlZV86vqCuB3wAVVdV1PnE8Ztv+/VtW9VXUR3a2E92bZObaq7qiqu4GDgc9X1c+r6sGqOoUuWX0m8Ay6u469u6oWVdU9VTU0KnJoq+eq1tcfAXbqHbUAPlpV86rqBuC/WPw9PQj4eFX9sjrXVNXMqroZ+DHwqrbfC4Hbq+rysRpUVR8F1gGeSpeI3DnSfkkOSXJZksvuu3PRSLtIkiRNKeNNLLYDzmyva1jZfLpf5x+pbu15ffcI76e11zOAXdp0n3lJ5tGNIGzWyl8J7AnMTHJRkmeNdsIk2yY5N8ktSebTXVRvvJRxAcwdNpVnJt0F/bIyq+f1DOCdw/pli3b+LYCZo6zDmAEc13PMHUDoRjWG3NLz+i4Wt3kL4NpRYjsF2Le93pc+RiuGtCTlv+n6d8Sn0VfViVW1c1XtvPp6a/dbtSRJ0gprvInFn4DHj1K2A910oZXV8ERqac0CLqqq9Xv+plXVmwDar+cvo5smdTaLp12NdP7P0T1b5AlVtS7dlKCMsF+/NmhTtYZsCcwe45hFwFo97zcbbccR9LZpFnDMsH5Zq6q+2sq2HFrTMswsuqljvcc9uqou6eP8s+imrY3kbOBJSXakG2U6rf9m/dmqS6hfkiRppTLexOIM4INJnt2zrZJsC7yXpbv4mipuZfSkajzOBbZNsl+S1drf05Nsn2T1JPskWa+q7qcbBRq6pe2twEZJ1uupa522z8Ik2wFvmoD4jm5x7Ep3QX3WGPv/GnhFkrWSbAO8cSnP+wXgsCS7pLN2khclWQf4BXAz8NG2fc0kf9uOOwF439AalSTrtbUT/fgi8K4kT2vn3GZoClVV3QN8nW6NxC/aNKpRJXlUkkOTbNDqegbwFuCH4+wHSZKkKWm8icW/ApcBF7F4dOLbdHP3f0s3FWdldSzwgTbdZq+xdh5Ne1L5C+gWe8+mm6bzMWCNtst+wPVtatNhtOk4VXU13aLk69q0n+nAu4DXAQvoLszPZDC3AHNbXKcBh7XzLskngfvoEp9TWMrksqouo1tncXyL4RrggFb2IPASujss3UC3MPrVrexbdP13Ruuz39EtdO/nnGfRrRU6na4Pz+bh0/lOAf6G/qdB/SPd1KoFdAvA/6P9SZIkrfRSNf4ZPkmeR3cHoo3p5rT/sKq+P8GxaTlKshtwalU9brJjWVGke8L61cBmVTV/WZ1nvW22qGd//J3LqnpJkibVea94x2SHoAmU5PKq2nmksr4fkJdkTbpRibdX1X/iFA+txJI8Cvhn4IxlmVRIkiStLPpOLKrqnnZf/4eWYTxaASW5gu7uS8MdWlUjTn1qazS+N1JZexbICqstYL+V7q5YLxxWtnCUw/aoqouXdWySJEkrqr4Ti+Y04A3ABcsgFk2iqroQGHEaVFXtMNL2Meq7mIffynbKaLfcHTH2FT0pkiRJmizjTSxuAPZOchnwXbpfdXsXaVRVfW6igpMkSZI0NYw3sfhE+/exdE8WHq7onq0gSZIk6RFkXIlFVY339rSSJEmSHgFMFCRJkiQNbFwjFkn2HGufqvru0ocjSZIkaSoa7xqLc+nWUWTY9t4F3KsMFJEkSZKkKWe8icVfjbBtQ+AFwAF0t6KVNA5PWP8xPpVUkiRNeeNdvD1zhM0zgf9O8iBwOPDSiQhMkiRJ0tQxkYu3/xv4+wmsT5IkSdIUMSGJRZLV6aZC3TwR9UmSJEmaWsZ7V6hf8vCF2gCrA1sB6+AaC0mSJOkRabyLt6/gLxOLe4CzgLOr6ooJiUqSJEnSlDLexdsHLKM4JEmSJE1h41pjkeRHSbYbpWzbJD+amLAkSZIkTSXjnQq1G7DuKGXrAs8ZKBrpEeiauXfw4q+fNtlhSJI0pnP32meyQ9AKbGnuCjV8jcXQXaH+Hrhl4IgkSZIkTTljjlgkORI4or0t4GdJRtv93yYoLkmSJElTSD9Tob4L3A4E+DTwCeD6YfvcB1xdVRdPaHSSJEmSpoQxE4uq+iXwS4AkC4Dzqur2ZR2YJEmSpKljvLebPWVZBSJJkiRp6hrvXaFI8mrgYGBbYM3h5VW16QTEJUmSJGkKGe9zLF4HnAJcAzwO+A5wbqtnPnD8RAcoSZIkacU33tvNvhv4EPCW9v6zVXUg8Fd0C7zvmsDYJEmSJE0R400sngD8tKoeBB6kPSyvqhYAHwPeOrHhSZIkSZoKxptY3Ams0V7fBGzfUxZgo/FUlmTLJAuTrDLOONS0/nv8ZMchSZKkR7bxJhaXAU9qr78DHJHk4CSvp3s43s/HqiDJOp4pXQAAIABJREFU9UmeD1BVN1TVtDYCslwk2S3JjcvrfMta67/rJjuOpdX7fZAkSdLUNd67Qh0LzGivj2ivPwusQvesi0MmLjStiJKsWlUPjLVtMuKQJEnS5BnXiEVV/ayqzmyv51XVy4BpwPpVtctYv5wn+QqwJXBOm8LzniSVZNVWfmGSDye5pJWfk2SjJKclmZ/kl0m26qlvuyTfT3JHkt8n2bunbM8kVyZZkOSmJO9KsjbwPWB6q39hkulJnpHk0iTzktyc5Pgkq/fUVUnenOQPrb4PJdm6HTM/ydeG9h8aEUlyeJLb2y/y+4zVt63tB/W8PyDJT/o4rpJs016fnOQzSc5rcf48ydY9++7Q01+3Jjm8bV8jyaeSzG5/n0qyxrD2vDfJLcBJSY5K8vUkpyaZDxyQZL0kX2r9d1P7HFfpOffBSa5qcV2Z5KkjfR+W0M6tWlvfmOQG4Edt+4Gt3rlJzk8yo+eY0dr7qCT/kuTaJHPa57fhsPO8PskN7TN8f0+dq7TP9trWlsuTbNH6/RPDYj4nyTvG+gwlSZJWBuOdCgVAOlsk+V/AqlU1v5/jqmo/4AbgJVU1DfjaCLu9BtgP2BzYGrgUOAnYELgKOLLFsDbwfeB0YFPgtcBnk+zQ6vkScGhVrQPsCPyoqhYBewCz2xSiaVU1m24h+j8BGwPPAp4HvHlYXC8EngY8E3gPcCKwD7BFq/+1Pftu1uraHHg9cGKSJ/bTRxPgtcDRwAZ0twU+BiDJOsAPgP8EpgPbAD9sx7yfrl07AU8GngF8oKfOzej6fwaLR6VeBnwdWB84je42xA+0ep8CvAA4qJ37VcBRwP50C/5fCswZ/n2oqo/30b7n0q3t2T3Jy4HDgVcAmwAXA1/to71vB17e6poOzAU+M+w8zwaeSPddOCLJ0Hqif6br4z1bWw6kuxvaKcBrkzyqnX/jduxX+2iTJEnSlDfuxCLJm+kWbs+ku5B7Ytv+zQn6dfakqrq2qu6kG124tqp+0Ka9nEV30QrwYuD6qjqpqh6oql8B3wD2auX3A3+dZN2qmtvKR1RVl7fRmAeq6nrg83QXnb0+VlXzq+oK4HfABVV1XU+cTxm2/79W1b1VdRFwHrA3y8c3q+oXrb9Oo0sWoOuvW6rqE1V1T1UtqKqhNTH7AB+sqj9V1W10icl+PXU+BBzZ2nN323ZpVZ1dVQ/RXWDvAbyjqhZV1Z+AT9IlidAlGB+vql9W55qqmrmU7TuqneNu4FDg2Kq6qrX3I8BObdRiSe09FHh/Vd1YVffSJT17pY2cNUdX1d1V9RvgN3QJ11BbPlBVv29t+U1VzamqX9Dd3OB5bb/XABdW1a0jNSLJIUkuS3LZffP7ysslSZJWaON9QN67gf8DfAH4e7o7QQ25EHj1BMTUeyF29wjvp7XXM4Bd2vSleUnm0V0gb9bKX0n3q/LMJBcledZoJ0yybZJzk9zSpvZ8hG7EYWniApjbRkeGzKT7ZXx5uKXn9V0sjmsL4NpRjplOF+OQ4fHeVlX3DDtmVs/rGcBqwM09n8Xn6UaSxjr3eA0/73E957yD7ju5+RjnnAF8q+e4q+hGrR7Ts8/S9OMpwL7t9b7AV0ZrRFWdWFU7V9XOq6+77mi7SZIkTRnjHbF4C3BEVR1JN1rR6/fAtn3UUeM852hmARdV1fo9f9Oq6k0A7dfxl9Fd3J7N4mlXI53/c8DVwBOqal266TUZYb9+bdCmag3ZEpg9xjGLgLV63m822o5LaRbd1LKRzGbxonz4y3hH6rPebbOAe4GNez6Ldatqh57y0c493u/D8PMeOuw78OiqumSMc84C9hh23JpVdVMf519SvacCL0vyZLrpWmf31yRJkqSpb7yJxWbA5aOUPQSs2UcdtwIT8dyFc4Ftk+yXZLX29/Qk2ydZPck+SdarqvuB+XS/SA+df6Mk6/XUtU7bZ2GS7YA3TUB8R7c4dqWblnPWGPv/GnhFkrXSLcZ+4wTE0OtcYLMk72iLtddJsksr+yrwgSSbtLUBR9BdJPelqm4GLgA+kWTdtjh66yRD08m+CLwrydPa+pxtehZZD/J9OAF439C6mraA/FV9tPcE4JihGFq7X9bnOb8IfCjJE1pbnpRko9YPN9LdHe0rwDd6po1JkiSt9MabWFzDX649GPIc4Mo+6jiW7iJ2HovXQ4xbe9r3C+jmss+mm7ryMRY/wG8/4Po2tekw2hSVqrqa7kL6ujYVZjrwLuB1wAK6aV5nLm1czS10C4Jn061zOKydd0k+CdxHd6F9SjtuwrT++gfgJS2+PwB/14o/TPeMkt8C/wP8qm0bj/2B1em+A3PpFnY/tp37LLpF5KfT9fHZdIvBoef7kORd42zTt+g+8zPa5/w7urUeY7X3OLrnsFyQZAHwM2AX+vN/6Ea/LqBLRr8EPLqn/BTgb1jCNChJkqSVUar6n4mS7naonwU+SHfheCXdOobNgU8DB1fV6csgzikjyW7AqVX1uMmORctfkufQjfZs1Ra2j2n9rR9fz/7Yh5ZtYJIkTYBz9xrzDvpaySW5vKp2HqlsXA/Iq6ovJtmAbqrM0W3zd+kWtx71SE8q9MiWZDXgfwNf7DepkCRJWlmMmVgkuQB4W1X9HqCq/i3J1XSLm6fR3Ynn0nbbVS1Bkit4+CLpIYdW1YhTn9oaje+NVNaeBbLSSPcgwc+PUDSzZyH4Cqk95+IyulvTvmGSw5EkSVru+hmxeD7w54XO6Z6mfDbw9CU9G+KRqqouBEacBrU0F8dVdTEPv5XtSqslVxO6tmR5qaqrgLXH3FGSJGkltVRP3mawW7FKkiRJWsksbWIhSZIkSX/Wb2Ix1gPSJEmSJD2C9XtXqPOTPDBs2w9H2EZVbTp4WJIkSZKmkn4Si6PH3kWSJEnSI9mYiUVVmVhIkiRJWiIXb0uSJEka2LievC1p4m2zwYacu9c+kx2GJEnSQByxkCRJkjQwEwtJkiRJAzOxkCRJkjQwEwtJkiRJAzOxkCRJkjQwEwtJkiRJAzOxkCRJkjQwn2MhTbJr5s7nZV8/f7LDkCRNUd/ea/fJDkECHLGQJEmSNAFMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLB7hklSSbSbhvLsm+f3yPq8kSZKWDRMLLRfDE5iquriqnjiZMUmSJGnimFhIkiRJGpiJxRSSZHqSbyS5Lckfk7w9yYZJbkzykrbPtCTXJNm/vT85yQlJvp9kQZKLkswYpf4XJfnvJPOTzEpyVE/ZVm3U4fVJbkhye5L395Q/I8mlSeYluTnJ8UlWb2U/brv9JsnCJK9OsluSG3uO3z7Jhe34K5K8tKfs5CSfSXJea8PPk2w9Rl8Nxbtqz7YLkxw0xnFbJ/lRkjmtjaclWb+V/UuSrw/b/7gkn26v/yrJj1uMP2gxn7qk80mSJK0sTCymiCSPAs4BfgNsDjwPeAfwdOBA4AtJNgU+Cfy6qr7cc/g+wIeAjYFfA6eNcppFwP7A+sCLgDclefmwfZ4NPLGd/4gk27ftDwL/1M7xrFb+ZoCqek7b58lVNa2qzhzWttVa2y4ANgXeBpyWpHeq1GuBo4ENgGuAY0Zpw6ACHAtMB7YHtgCOamVfBfZMsm6LexVgb+D0Vn468Atgo3bMfssoRkmSpBWOicXU8XRgk6r6YFXdV1XXAV8AXlNVFwBnAT+kSwgOHXbseVX146q6F3g/8KwkWww/QVVdWFX/U1UPVdVv6S6knztst6Or6u6q+g1dkvPkduzlVfWzqnqgqq4HPj/CsaN5JjAN+Ghr24+Ac+mSiSHfrKpfVNUDdInRTn3WPS5VdU1Vfb+q7q2q24D/Q2tHVc0EfgUMJVt/D9xVVT9LsiXdZ3REa8NPgO+Mdp4khyS5LMll982/c1k0RZIkabkysZg6ZgDT21SheUnmAYcDj2nlJwI7AidV1Zxhx84aelFVC4E76H6Rf5gkuyT5rzbV6k7gMLoRiF639Ly+iy4hIMm2Sc5NckuS+cBHRjh2NNOBWVX1UM+2mXQjM0s870RLsmmSM5Lc1NpxKg9vx+ksTnhex+LRiunAHVV1V8++sxhFVZ1YVTtX1c6rr7veBLZAkiRpcphYTB2zgD9W1fo9f+tU1Z5tSs7ngS/TTV8afvvYP49OJJkGbAjMHuEcp9P9yr5FVa0HnEA3NagfnwOuBp5QVevSJT39Hjsb2KJN9xqyJXBTn8ePZFH7d62ebZv1cdyxQAFPau3Yl4e34yxgtySPA/6RxYnFzcCGSXrP9xejQpIkSSsrE4up4xfA/CTvTfLoJKsk2THJ0+ku4qFba/HvwJdbsjFkzyTPboupPwT8vKpG+jV9Hbpf3e9J8gy6X+T7tQ4wH1iYZDvgTcPKbwUeP8qxP6dLBN6TZLUkuwEvAc4Yx/kfpk1jugnYt/XVgcASF3w36wALgXlJNgfePUK9FwIn0SV6V7XtM4HLgKOSrJ7kWa0NkiRJjwgmFlNEVT1Id6G6E/BH4Hbgi3Tz/P8Z2L/t8zG6X9z/pefw04Ej6aZAPY1uMfdI3gx8MMkC4Ajga+MI8V10icgCurUfZw4rPwo4pU3j2ntY2+4DXgrs0dr12daeq8dx/pEcTJcYzAF2AC7p45ijgacCdwLnAd8cYZ/TgeezeLRiyD50C9fnAB+m64N7lyZwSZKkqSZVNdkxaBlKcjJwY1V9YLJjeaRJciZwdVUduaT91t9623rux/5jOUUlSVrZfHuv3Sc7BD2CJLm8qnYeqcwRC2mCJHl6ew7Go5K8EHgZcPZkxyVJkrQ8mFhoykqya3vg3l/8jXHcCaMcd8KAIW1Gt/5iIfBp4E1V9d8D1ilJkjQlrDr2LprKquqAyY5hWamqi1mK285W1WF0t9Kd6HjOoXvQnyRJ0iOOIxaSJEmSBmZiIUmSJGlgJhaSJEmSBmZiIUmSJGlgJhaSJEmSBmZiIUmSJGlgJhaSJEmSBuZzLKRJts0G6/LtvXaf7DAkSZIG4oiFJEmSpIGZWEiSJEkamImFJEmSpIGZWEiSJEkamImFJEmSpIGZWEiSJEkamImFJEmSpIH5HAtpkl039x72/sbVkx2GJGkK+tort5vsEKQ/c8RCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sAeEYlFkl2T/H6y41hWklyRZLdJOvduSW6cjHOPJsmFSQ5aiuNOSPKvyyImSZKkld2qkx3A8lBVFwNPnOw4JkKSk4Ebq+oDQ9uqaofJi2hqSnIAcFBVPXtoW1UdNnkRSZIkTW0r/YhFkoGSpySrTFQskiRJ0spqyiYWSa5P8r4kVyaZm+SkJGsOTc1J8t4ktwAnDZ+uk2T7Nl1mXptG9NKespOTfC7Jd5MsAv5uCTFslOQ7SeYn+UWSDyX5SSvbKkn1JjbDp+gkOTDJVS3+85PMaNuT5JNJ/pTkziS/TbJjkkOAfYD3JFmY5Jyevnh+e71Gkk8lmd3+PpVkjVY21DfvbHXfnOQNffT1Gkn+PckNSW5tU4YePcq+/5Lk2iQL2mfzjz1lByT5aZL/aO26OsnzhpVf1479Y5J9xuqrVvYPra47kxwPZIz2bA+cADyr9eO8tv3kJB8e1lfv6emrlyfZM8n/S3JHksN76nxUT9vnJPlakg3H6ltJkqSVxZRNLJp9gN2BrYFtgaHpQZsBGwIzgEN6D0iyGnAOcAGwKfA24LQkvVOlXgccA6wD/GQJ5/8McA/wWODA9teXJC8HDgdeAWwCXAx8tRW/AHhOa9P6wKuBOVV1InAa8PGqmlZVLxmh6vcDzwR2Ap4MPIPF/QJd36wHbA68EfhMkg3GCPdjLZadgG3asUeMsu+1wK7tHEcDpyZ5bE/5LsB1wMbAkcA3k2yYZG3g08AeVbUO8L+AX8OS+yrJxsA3Whs3buf/2yU1pqquAg4DLm39uP4ou24GrNnT3i8A+wJPa208Isnj275vB14OPBeYDsyl+35IkiQ9Ikz1xOL4qppVVXfQJQKvbdsfAo6sqnur6u5hxzwTmAZ8tKruq6ofAef2HAvw7ar6aVU9VFX3jHTiNkXqlcARVbWoqn4HnDKO2A8Fjq2qq6rqAeAjwE7tl/j76ZKa7YC0fW7us959gA9W1Z+q6ja6i/v9esrvb+X3V9V3gYUsYf1JkgAHA/9UVXdU1YIW62tG2r+qzqqq2a3vzgT+QJfcDPkT8Kl2/jOB3wMvamUPATsmeXRV3VxVV/TRV3sCV1bV16vqfuBTwC199tVY7geOafWeQZe4HFdVC1psVwBP6onx/VV1Y1XdCxwF7JVRpuIlOSTJZUkuu3f+3AkKV5IkafJM9cRiVs/rmXS/FAPcNlpC0PaZVVUPDTt281HqHc0mdIvfh8fQrxnAcW061jzgDropPJu3ZOd4ul+8b01yYpJ1+6x3+rA4evsFupGPB3re30WXaI1mE2At4PKeWP+zbf8LSfZP8uuefXekuyAfclNV1fD4qmoR3cjMYcDNSc5Lsl3bZ9S+am3782fQ6u7n8+vHnKp6sL0eSlBv7Sm/m8V9NwP4Vk+MVwEPAo8ZqeKqOrGqdq6qnddYd6wBI0mSpBXfVE8stuh5vSUwu72uEfYdMhvYIklv27cEbup5v6Tjh9wGPDBCDEMWtX/X6tm2Wc/rWcChVbV+z9+jq+oSgKr6dFU9DdiBbhrSu/uMbTbdRW5vTLNH2bcft9NdQO/QE+d6VfUXyUgbQfgC8FZgozbF6Hc8fM3D5m0U5C/iq6rzq+of6KaWXd3qgiX31c30fAat7t7PZDT9fMbjMYtuGldvjGtW1U1jHilJkrQSmOqJxVuSPK4tkj0cOLOPY35Od9H/niSrpXv+w0voprr0rf2S/U3gqCRrJflr4PU95bfRJSv7JlklyYF0a0GGnAC8L8kOAEnWS/Kq9vrpSXZp60EW0a3jGPrl/Fbg8Yzuq8AHkmzS1h8cAZw6nrYNa+dDdBf4n0yyaYtv8yS7j7D72nQX7Le1/d5AN2LRa1Pg7a3vXwVsD3w3yWOSvLSttbiXborWUJtH7SvgPGCHJK9o047ezsMTuNHcCjwuyep97NuPE4BjsngB/iZJXjZBdUuSJK3wpnpicTrdIuzr2t+Hxzqgqu4DXgrsQfdr/GeB/avq6qU4/1vppsLcApwMnDSs/GC6kYY5dCMPl/TE8S26RdFnJJlP98v+Hq14XbqL+bl0U4XmAP/eyr4E/HWbcnP2CDF9GLgM+C3wP8Cv6KNfxvBe4BrgZy3WHzDCuoyquhL4BHAp3YX73wA/Hbbbz4En0PX9McBeVTWH7rv4TrrRizvoFkG/udU7al9V1e3Aq4CP0vXTE0Y450h+RLdG4pYkt/fTCWM4DvgOcEGSBcDP6BaqS5IkPSLk4dPdp44k19M94OwHkx3LkIzw0DUtZv+MbMOtd6znf/zrkx2GJGkK+tortxt7J2kCJbm8qnYeqWyqj1hIkiRJWgGYWIwh3QP0Fo7wt8/YR08dK2M70z3Ib6Q2nTDZsUmSJK1spuxUKGll4VQoSdLSciqUljenQkmSJElapkwsJEmSJA3MxEKSJEnSwEwsJEmSJA3MxEKSJEnSwEwsJEmSJA3MxEKSJEnSwFad7ACkR7rHb7Cm9yGXJElTniMWkiRJkgZmYiFJkiRpYCYWkiRJkgZmYiFJkiRpYCYWkiRJkgZmYiFJkiRpYN5uVppkc+Y9wCnfvG2yw5AkTRGvf8Umkx2CNCJHLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLJZCki2TLEyyymTHIkmSJK0ITCz6lOT6JM8HqKobqmpaVT24HM+/W5Ibl9f5NH5J/j3JH5IsSHJ1kv0nOyZJkqTlZdXJDkBKsmpVPTDZcUyARcBLgP8HPB34zyTXVNUlkxuWJEnSsueIRR+SfAXYEjinTYF6T5JKsmorvzDJh5Nc0srPSbJRktOSzE/yyyRb9dS3XZLvJ7kjye+T7N1TtmeSK9uv3jcleVeStYHvAdNb/QuTTE/yjCSXJpmX5OYkxydZvaeuSvLmnl/RP5Rk63bM/CRfG9p/aEQkyeFJbm8jNPv00TcXJjmo5/0BSX7Sx3GV5C1J/gD8oW17cZJft/ZckuRJPftvkeSbSW5LMifJ8T1lBya5KsncJOcnmTHsPIe1Ppib5DNJ0lN+cDt2Qev3pyZ5d5JvDIv3P5J8akltqqojq+rqqnqoqn4OXAw8a6y+kCRJWhmYWPShqvYDbgBeUlXTgK+NsNtrgP2AzYGtgUuBk4ANgauAIwFakvB94HRgU+C1wGeT7NDq+RJwaFWtA+wI/KiqFgF7ALPbFKxpVTUbeBD4J2BjugvY5wFvHhbXC4GnAc8E3gOcCOwDbNHqf23Pvpu1ujYHXg+cmOSJ4+qs8Xk5sAvw10meCvxf4FBgI+DzwHeSrNHWspwLzAS2avGdAZDk5cDhwCuATegu5r867DwvphtBeDKwN7B7O/ZVwFHA/sC6wEuBOcCpwAuTrN/2WxV4NfCVfhuW5NHtnFf0e4wkSdJUZmIxcU6qqmur6k660YVrq+oHbYrPWcBT2n4vBq6vqpOq6oGq+hXwDWCvVn4/3YX2ulU1t5WPqKour6qftXqup7sYf+6w3T5WVfOr6grgd8AFVXVdT5xPGbb/v1bVvVV1EXAe3YX4snJsVd1RVXcDBwOfr6qfV9WDVXUKcC9dQvQMYDrw7qpaVFX3VNXQqMihrZ6rWl9/BNipd9QC+GhVzauqG4D/AnZq2w8CPl5Vv6zONVU1s6puBn4MvKrt90Lg9qq6fBxtOwH4DXD+SIVJDklyWZLLFtw5ZxzVSpIkrZhMLCbOrT2v7x7h/bT2egawS5vuMy/JPLoRhM1a+SuBPYGZSS5KMupUmiTbJjk3yS1J5tNdVG+8lHEBzG2jI0Nm0l3QLyuzel7PAN45rF+2aOffApg5yjqMGcBxPcfcAYRuVGPILT2v72Jxm7cArh0ltlOAfdvrfRnfaMW/0Y0G7V1VNdI+VXViVe1cVTuvs95G/VYtSZK0wjKx6N+IF4hLYRZwUVWt3/M3rareBNB+PX8Z3TSps1k87Wqk838OuBp4QlWtSzclKCPs168N2lStIVsCs8c4ZhGwVs/7zUbbcQS9bZoFHDOsX9aqqq+2si2H1rQMM4tu6ljvcY/uc8H0LLppayM5G3hSkh3pRplO66dBSY6mm7b2gqqa388xkiRJKwMTi/7dCjx+Auo5F9g2yX5JVmt/T0+yfZLVk+yTZL2quh+YT7eOYuj8GyVZr6euddo+C5NsB7xpAuI7usWxK90F9Vlj7P9r4BVJ1kqyDfDGpTzvF4DDkuySztpJXpRkHeAXwM3AR9v2NZP8bTvuBOB9Q2tUkqzX1k7044vAu5I8rZ1zm6EpVFV1D/B1urUwv2jTqJYoyfuA1wH/UFXOb5IkSY8oJhb9Oxb4QJtus9dYO4+mqhYAL6Bb7D2bbprOx4A12i77Ade3qU2H0abjVNXVdIuSr2vTfqYD76K7kF1Ad2F+5tLG1dwCzG1xnQYc1s67JJ8E7qNLfE6hz1/2h6uqy+jWWRzfYrgGOKCVPUh3G9dt6BbR30i3mJqq+hZd/53R+ux3dCMG/ZzzLOAYuuRhAd0oxYY9u5wC/A39T4P6CN0ozx+y+O5dh/d5rCRJ0pSWUaaA6xEmyW7AqVX1uMmOZUWRZEu6qWabLctpTX+1zU511Me/v6yqlyStZF7/ik0mOwQ9giW5vKp2HqnMEQtpBEkeBfwzcIZrJSRJksbmk7c1piRX0N19abhDq2rEqU9tjcb3RiprzwJZYbUF7LfS3RXrhcPKFo5y2B5VdfGyjk2SJGlFZWIhAKrqQmDEaVBVtcNI28eo72IefivbKaPdcnfE2Ff0pEiSJGmyOBVKkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzMRCkiRJ0sBMLCRJkiQNzNvNSpNso/VX9SmqkiRpynPEQpIkSdLATCwkSZIkDczEQpIkSdLATCwkSZIkDczEQpIkSdLATCwkSZIkDczEQpIkSdLAfI6FNMkWzXmAn538p8kOQ8vAMw/YdLJDkCRpuXHEQpIkSdLATCwkSZIkDczEQpIkSdLATCwkSZIkDczEQpIkSdLATCwkSZIkDczEQpIkSdLATCwkSZIkDczEQpIkSdLATCwkSZIkDczEQpIkSdLATCykCZbkwiQHTXYckiRJy9Oqkx2AtLJIEiCTHYckSdJkcMRCApJcn+R9Sa5MMjfJSUnWTLJBknOT3Na2n5vkcT3HXZjkmCQ/Be4CvgLsChyfZGGS4yerTZIkScuTiYW02D7A7sDWwLbAB+j+j5wEzAC2BO4GhicL+wGHAOsABwAXA2+tqmlV9dblErkkSdIkM7GQFju+qmZV1R3AMcBrq2pOVX2jqu6qqgVt+3OHHXdyVV1RVQ9U1f39nCjJIUkuS3LZvAVzJrgZkiRJy5+JhbTYrJ7XM4HpSdZK8vkkM5PMB34MrJ9klVGO60tVnVhVO1fVzuuvs9GAYUuSJE0+EwtpsS16Xm8JzAbeCTwR2KWq1gWe08p7F2nXsHqGv5ckSVrpmVhIi70lyeOSbAgcDpxJt27ibmBe235kH/XcCjx+2YUpSZK04jGxkBY7HbgAuK79fRj4FPBo4HbgZ8B/9lHPccBe7S5Sn15GsUqSJK1QfI6FtNgvq+rYYdvuAnYbtu3zQy+qangZVXUp3V2lJEmSHjEcsZAkSZI0MBMLSZIkSQNzKpQEVNVWkx2DJEnSVOaIhSRJkqSBmVhIkiRJGpiJhSRJkqR9bR9OAAAbUUlEQVSBmVhIkiRJGpiJhSRJkqSBmVhIkiRJGpi3m5Um2dobrcozD9h0ssOQJEkaiCMWkiRJkgZmYiFJkiRpYCYWkiRJkgZmYiFJkiRpYCYWkiRJkgZmYiFJkiRpYCYWkiRJkgbmcyykSXb/Lfdxy7/NnOwwHnE2e/eMyQ5BkqSViiMWkiRJkgZmYiFJkiRpYCYWkiRJkgZmYiFJkiRpYCYWkiRJkgZmYiFJkiRpYCYWkiRJkgZmYiFJkiRpYCYWkiRJkgZmYiFJkiRpYCYWkyzJhUkOmuw4JkKSo5KcOtlxSJIkafkzsdBSSbJbkhsnO44V1cqUMEqSJPXDxGICJVl1smNYkiSrDHu/QscrSZKkqWPKJBZJ/iXJtUkWJLkyyT+27Qck+UmSf08yN8kfk+zRc9wBSa5rx/0xyT5t+8wkT2uv901SSf66vT8oydnt9aN6zj0nydeSbNjKtmrHvTHJDcCP2vYDk1zV4jk/yYyeeP4hydVJ7kxyPJA+239wq3Oo/U9t27dvv47PS3JFkpf2HHNyks8l+W6SRcDfJbk+yXuT/BZYlGTVJM9Mckmr4zdJduupY8MkJyWZ3dpzdpK1ge8B05MsbH/Th8V7XpK3Ddv22yQvX0Ibh/pz1Z5tf/7lP8kq7XO+vX2mbxm+/yj1/kUbhvXrNUnuSPKdoXb0Ecuo37skxwC7Ase3vjl+SfFJkiStDKZMYgFcS3exth5wNHBqkse2sl2A3wMbAx8HvpTO2sCngT2qah3gfwG/bsdcBOzWXj8HuA54bs/7i9rrtwMvb2XTgbnAZ4bF9lxge2D3duF8OPAKYBPgYuCrAEk2Br4BfKDFei3wt2M1PMmrgKOA/YF1gZcCc5KsBpwDXABsCrwNOC3JE3sOfx1wDLAO8JO27bXAi4D1gccA5wEfBjYE3gV8I8kmbd+vAGsBO7RzfLKqFgF7ALOralr7mz0s7FOAfXva8GRgc+C7Y7V3CQ4GXgw8BdgZ2KvP4/6iDS2mvweOBfYGHgvMBM4YRzwjfu+q6v10n/tbW9+8dRx1SpIkTUlTJrGoqrOqanZVPVRVZwJ/AJ7RimdW1Req6kG6C9rH0l0wAzwE7Jjk0VV1c1Vd0bZfxOJEYle6C8yh989lcWJxKPD+qrqxqu6lu8Dfa9iv5EdV1aKqurvtf2xVXVVVDwAfAXZqoxZ7AldW1der6n7gU8AtfTT/IODjVfXL6lxTVTOBZwLTgI9W1X1V9SPgXLrEYci3q+qnrd/uads+XVWzWrz7At+tqu+2fb4PXAbs2RK3PYDDqmpuVd1fVRfRn28DT0jyhPZ+P+DMqrqvz+NHsjfwqRb7HXSf2RKN0YZ9gP9bVb9qn+37gGcl2arPeJb0vRsrrkOSXJbksjmL7ujzdJIkSSuuKZNYJNk/ya/bdJ15wI50vxRDz8V5Vd3VXk5rv6y/GjgMuLlNz9mulV8E7JpkM2AV4Ezgb9tF5XosHtmYAXyr57xXAQ/y8AvIWT2vZwDH9ex/B910p83pRjz+vG9V1bBjR7MF3ejGcNOBWVX1UM+2me1cI8U2WryvGoq3xfxsuovkLYA7qmpuHzE+TLtQ/xqwb5JH0SU7XxlvPcM8rP/o2jqWJbVhem8dVbUQmMPD+29JRvze9XNgVZ1YVTtX1c4brb1hn6eTJElacU2JxKL92v8F4K3ARlW1PvA7+lifUFXnV9U/0F0oX93qoaquAe6im+r046paQHeheAjwk56L9Vl0U6nW7/lbs6pu6j1Nz+tZwKHD9n90VV0C3Ex3oTvUrvS+X4JZwNYjbJ8NbNEu3IdsCYwW22jxfmVYvGtX1Udb2YZJ1h+jjtGcQjcq8Dzgrqq6dIz9F7V/1+rZtlnP64f1H11bx7KkNsymS6wAaFPnNqLrv7FiGUs//SNJkrTSmBKJBbA23YXabQBJ3kA3YrFESR6T5KXtgvFeYCHdaMOQi+iSlaGpMRcOew9wAnBMS25IskmSly3htCcA70uyQ9t/vbZGArq1DDskeUWbSvV2+rtY/SLwriRPa2tHtmnx/JzuAvg9SVZri65fwvjWCZwKvCTJ7m1x9JrpbiX7uKq6mW6R9meTbNDO8Zx23K3ARknWG63ilkg8BHyCPkYrquo2uov6fVssB/LwhOprwNuTPC7JBsC/9FHnktpwOvCGJDslWYNu2trPq+r6PmIZy63A48exvyRJ0pQ2JRKLqrqS7uL0UroLtr8BftrHoY8C3kn3y/QddGsn3txTfhHdouYfj/Ie4DjgO8AFSRYAP6NbtDtarN8CPgackWQ+3cjKHq3sduBVwEfpptw8oZ92VNVZdAuwTwcWAGcDG7b1Ci9t9d8OfBbYv6quHqvOnrpnAS+jW3B+G90v/O9m8XdjP+B+utGePwHvaMddTbco/bo2hWo6I/sy3efV74PzDm7nn0O32PqSnrIvAOcDvwF+BXyzzzpHa8MPgX+lW1B/M13i8Jo+YxnLcXRrceYm+fQ4jpMkSZqS0k3zl5aNJPsDh1TVs5dB3VsBfwRWawvlp6QnP+5Jdf7/Pmeyw3jE2ezdM8beSZIkPUySy6tq55HKpsSIhaamJGvRjRCdONmxSJIkadkysVhBJDkhix821/t3wmTHtjSS7E43tepWuilcQ9t3HaWdCwc834h1Jtl1wKZIkiSpD0t8YrGWn6o6jO62uCuFqjqfbtH98O0X0+ctWfs4x/UsvjPYhNQpSZKkpeOIhSRJkqSBmVhIkiRJGpiJhSRJkqSBmVhIkiRJGpiJhSRJkqSBmVhIkiRJGpi3m5Um2Wqbre5ToCVJ0pTniIUkSZKkgZlYSJIkSRqYiYUkSZKkgZlYSJIkSRqYiYUkSZKkgZlYSJIkSRqYiYUkSZKkgfkcC2mS3f+nhdz66R8vk7of8/bnLJN6JUmShnPEQpIkSdLATCwkSZIkDczEQpIkSdLATCwkSZIkDczEQpIkSdLATCwkSZIkDczEQpIkSdLATCwkSZIkDczEQpIkSdLAVtjEIsmuSX4/2XEsK0muSLLbMqz/5CQfnqC6dkty40TUtZTnryTbTMJ5V+rvoCRJ0kRadbIDGE1VXQw8cbLjmAhJTgZurKoPDG2rqh0mLyKNJEkBT6iqa2Dl+g5KkiQtayvkiEWSgRKeJKtMVCySJEmSxrZcE4sk1yd5X5Irk8xNclKSNYem2iR5b5JbgJOGT79Jsn2SC5PMa9OIXtpTdnKSzyX5bpJFwN8tIYaNknwnyfwkv0jyoSQ/aWVbtWk3q/bsf2GSg3reH5jkqhb/+UlmtO1J8skkf0pyZ5LfJtkxySHAPsB7kixMck5PXzy/vV4jyaeSzG5/n0qyRisb6pt3trpvTvKGPrt8gyTnJVmQ5OdJtu5px3FJZrV+uDzJrj1lj259OjfJlcDT+zlZkulJvpHktiR/TPL2tn3D1oaXtPfTklyTZP/2/uQkJyT5fov1oqF+HeEcL0ry3y3uWUmO6ikb+vxen+SGJLcneX9P+TOSXNq+QzcnOT7J6q3sx22337TP6dVL8R38zGj9LUmStLKbjBGLfYDdga2BbYGh6UGbARsCM4BDeg9IshpwDnABsCnwNuC0JL3TVF4HHAOsA/xkCef/DHAP8FjgwPbXlyQvBw4HXgFsAlwMfLUVvwB4TmvT+sCrgTlVdSJwGvDxqppWVS8Zoer3A88EdgKeDDyDxf0CXd+sB2wOvBH4TJIN+gj5tcDRwAbANXT9M+SX7XwbAqcDZyVZs5UdSff5bE33Wb1+rBMleRTdZ/SbFufzgHck2b2q7qDr5y8k2RT4JPDrqvpyTxX7AB8CNgZ+TddnI1kE7E/Xxy8C3tQ+l17PppvC9DzgiCTbt+0PAv/UzvGsVv5mgKp6Ttvnye1zOnNY+/r5Di6pvyVJklZqk5FYHF9Vs9rF5jF0F2MADwFHVtW9VXX3sGOeCUwDPlpV91XVj4Bze44F+HZV/bSqHqqqe0Y6cbopUq8EjqiqRVX1O+CUccR+KHBsVV1VVQ8AHwF2ar+u30+X1GwHpO1zc5/17gN8sKr+VFW30V2c7tdTfn8rv7+qvgsspL+5/9+sql+0WE+jSyQAqKpTq2pOVT1QVZ8A1uipc2/gmKq6o6pmAZ/u41xPBzapqg+2z+g64AvAa9r5LgDOAn5IlxAcOuz486rqx1V1L12i9awkWww/SVVdWFX/0z7n39Ilds8dttvRVXV3Vf2GLtF5cjv28qr6WWvz9cDnRzh2NP18B0ft7+GSHJLksiSX3bFwXp8hSJIkrbgmI7GY1fN6JjC9vb5ttISg7TOrqh4aduzmo9Q7mk3oFqwPj6FfM4Dj2lSYecAdQIDN24Xm8XQjIrcmOTHJun3WO31YHL39At3IxwM97++iu8gdyy2jHdOmVl3Vpm3NoxsR2bgnnvH20Qxg+lDftDoPBx7Ts8+JwI7ASVU1Z9jxfz5fVS2k69vpw/YhyS5J/qtNt7oTOKwn7iEjtjvJtknOTXJLkvl0ieHwY0fTz3dw1P4erqpOrKqdq2rnDaet32cIkiRJK67JSCx6f4XeEpjdXtcSjpkNbNGm2/Qee1PP+yUdP+Q24IERYhiyqP27Vs+2zXpezwIOrar1e/4eXVWXAFTVp6vqacAOdFOi3t1nbLPpLsx7Y5o9yr4Da+sp3ks3MrFBVa0P3EmXJAHczOh9NJpZwB+H9c06VbVnO+cqdCMEX6abvjT89rF/Pl+SaXRTtEbqg9OB7/D/27v/YLnK+o7j74/cDCHkB8T8MAZIhioiQUEFRytQq5EOKVgpaiFIG1Qo2oztKFpaBSOh1gDVqY0aUeBiIIkJJUgHLQotLTQTIWhFM1yiCYQfEkISfuTXTRj49o/nuXKy3eTuzdm95+69n9fMmbv7PGfPec5nk9199jzPWTg8IsYACwrt7s23gC7SlZ9Gkzo+jT62kX+DZmZmZkNWFR2Lv5J0mKSxpA923+/tAcBPSR/6PydpmNLvP5wBLOnLjiPiJeAWYI6kEZKOoTB/IA9DehL4iKQDJH2UNM+gxwLg7yRNA5A0RtKH8u0T87fpw3Jbu0lj+gGeBo7cR9MWA1+QNF7SOOAy4Ma+HFsfjSJ1sJ4BOiRdBhTPriwlHeehkg4jzSfozX3AC0oT8A/K+R0rqWfi99/nvx8Frga+pz2v3jVD0kl5MvVc4Kd5GFa9tm+JiG5JbyfNrWnUKOAFYJuko4FP1NTv63lqyr9BMzMzs8Gqio7FItIE2HV56fVH3CJiN/B+4DRgE/BN4M8joms/9j+bNERlA9AJXF9TfwHpTMNm0pmHFYV2LAfmAUvyUJpf5TZB+mD+HeBZ0hCZzaQP0ADXAsfkIUK31mnTFcAq4EHgl8DPaCCXEu4AfgSsyW3tZs+hT1/K5Y+QnquFvW0wd9rOIM0reIT0PH0XGCPpbcCnSc/ZS6QMA7iksIlFpEnjW4C3kead1PNJ4HJJW0kdsKW9H+7vXEzqiGwlPVe1ndo5wA35efpwzfE189+gmZmZ2aCjiEZGEDVpZ9KjwMcj4s5+22kvJM0itemkqtsyVKnODwgOJccdcXT8+OJrWrLtiZ86pfeVzMzMzBok6YGIOKFe3YD8gTwzMzMzM2svg7JjkX+8bFudZW/Da9pSfx+npCP2sr9tkhqZ4G1mZmZmg1RH76s0T0RM7af9TOvDup2kuRZtpy/H2aT9PUZjl7nt63ZnNXubZmZmZta/BuUZCzMzMzMz61/uWJiZmZmZWWnuWJiZmZmZWWnuWJiZmZmZWWnuWJiZmZmZWWnuWJiZmZmZWWnuWJiZmZmZWWn9+jsWZvb/DZswkomfOqXqZpiZmZmV4jMWZmZmZmZWmjsWZmZmZmZWmjsWZmZmZmZWmiKi6jaYDWmStgIPV92OQWocsKnqRgxCzrV1nG3rONvWcbatMVBznRIR4+tVePK2WfUejogTqm7EYCRplbNtPufaOs62dZxt6zjb1mjHXD0UyszMzMzMSnPHwszMzMzMSnPHwqx611TdgEHM2baGc20dZ9s6zrZ1nG1rtF2unrxtZmZmZmal+YyFmZmZmZmV5o6FmZmZmZmV5o6FWUUkjZW0XNJ2Seslzay6TQORpAMlXZsz2irp55JOK9S/V1KXpB2S/lPSlEKdJM2TtDkvV0pSoX5qfsyOvI3p/X18A4Wk10vqlnRjoczZliTpbEkP5f/nayWdnMudbQk5gx9KelbSBknzJXXkOmfbIEmzJa2StEtSZ01dy3KUNDO/pm+XdKuksS0/2H62t2wlvUPSTyRtkfSMpGWSJhXq2zpbdyzMqvMNYDcwETgX+JakadU2aUDqAB4H/gAYA1wKLM0vruOAW3LZWGAV8P3CYy8EPgAcB7wZOB34y0L9YuDnwKuBzwM3S6r7oz9DwDeA+3vuONvyJL0PmAecD4wCTgHWOdum+CawEZgEHE96ffiks+2z3wJXANcVC1uZY36f+zZwHun9bwfp+Rxs6mYLHEqalD0VmAJsBa4v1Ld3thHhxYuXfl6Ag0mdiqMKZQuBr1TdtnZYgAeBs0gvwCtqct0JHJ3vrwAuLNR/DFiZbx8F7AJGFervAS6q+vgqyPNsYCkwB7gxlznb8rmuAD5Wp9zZls/2IWBG4f5VpA9Uznb/8rwC6Czcb1mOwJeBRYW63yO9H45q5jENlKU22zr1bwW2Fu63dbY+Y2FWjaOAlyJiTaHsF4DPWPRC0kRSfqtJef2ipy4itgNreSXHPerZM+NpwLqI2LqX+iFB0mjgcuAzNVXOtgRJBwAnAOMl/UbSE3m4zkE422b4Z+BsSSMkTQZOA/4dZ9ssrcyxdttryV+0NbH97eQU0vtZj7bO1h0Ls2qMBJ6vKXueNFzC9kLSMOAm4IaI6KL3HGvrnwdG5vGqfg6SucC1EfF4TbmzLWciMAz4IHAyabjOW4Av4Gyb4b9IH6JeAJ4gDdW5FWfbLK3M0Tlnkt4MXAZ8tlDc1tm6Y2FWjW3A6Jqy0aSxllaHpFeRhovtBmbn4t5yrK0fDWyLdI54yD8Hko4HpgNfq1PtbMvZmf/+S0Q8FRGbgK8CM3C2peTXgjtIcwAOBsaRxq3Pw9k2SytzdM6ApNcBPwL+OiLuKVS1dbbuWJhVYw3QIen1hbLj2PN0qGX5m5prSd8CnxURL+aq1aTcetY7mDSmdHW9evbMeDVwpKRRe6kfCt5NmkD4mKQNwMXAWZJ+hrMtJSKeJX2TXu9XaJ1tOWOBw4H5EbErIjaTJr/OwNk2SytzrN32kcCBpPfFISFfYetOYG5ELKypbu9sq57U4sXLUF2AJaSrOxwMvIt0unJa1e0aiAuwAFgJjKwpH59zOwsYTvrGcmWh/iLSJM/JwGtJL7oXFepXAlfnx54JPAeMr/p4+zHXEcBrCsvVwM05V2dbPt/LSVfamkD6Rv0e0tAzZ1s+23XAJaSrxh0CLCcNk3S2fcuxIx/rP5LOCA/PZS3LkVeGsJ1Mev+7EVhSdRb9mO1k0nyVz+7lcW2dbeXBe/EyVBfSt263AtuBx4CZVbdpIC6ky/EF0E06zduznJvrpwNdpKEndwNTC48VcCWwJS9XAirUT82P2Qk8DEyv+ngrznoO+apQzrYpeQ4jXerxOWAD8HVguLNtSrbH5wyeBTYBy4AJzrbPOc7Jr6/FZU6rcwRmkt73tgM/AMZWnUV/ZQt8Md8uvp9tGyzZKjfCzMzMzMxsv3mOhZmZmZmZleaOhZmZmZmZleaOhZmZmZmZleaOhZmZmZmZleaOhZmZmZmZleaOhZmZmZmZleaOhZmZWR9ImiNpU9XtaISkUyX9TdXtMLOhwR0LMzOzwetUwB0LM+sXHVU3wMzMzJpL0jDg5arbYWZDi89YmJmZ7SdJ75YUkt4r6QeStkv6dR6CdICkqyRtkvSkpE/XPLZT0ipJH5DUJalb0r2SjqlZb4Skr0vakNe5X9KpNevcLelmSRdKWgt0A9cAnwGm5DaGpM68/jsl3Sbpt7nN/yvp3JptzsqPeZOkn+T1uiT9aZ0czpR0n6SdkjZL+qGkKYX6YyXdLmlrXpZJek3J+M1sgHHHwszMrLxvA/cCZwLrgZuB+cAoYGa+/0+S3lHzuCnAV4G5eb0xwB2ShhfW+Q5wPvAPefuPA7dLOqlmW+8CPgH8LXAG8CVgEbABeGde5hb2+z/Ax/O6/wpcL+mcOse2CLgt7/vXwBJJh/VUSjoPuAVYC3w4t3UNMD7Xvy7vazhwHjALmAb8myTV2Z+ZtSkPhTIzMytvYURcBSDpCWA18IaIeE8uuxP4M9KH85WFx40D/iQiVuT1HiB9QJ8FLJD0RuAc4PyIuCGvcwfwIHAp8EeFbR0CvCUiNvQUSHoK2BURxX0SEUsK6wj4b+Aw4AJgcc2xfS0iriu072ng9Ny+VwFfAZZHRLFTclvh9hdJnZvTImJ33s6DQBcwA7i9Nkwza08+Y2FmZlbeXYXbv8l//6OnICJeBtYBk2set7GnU5HXWw88ALw9F50ICFhWs61lQO0ZiweKnYp9kXRoHl61HngxLxcCR9VZ/ceFfW8GNpI6IQBvAF4LXL+P3U0HlgMvS+qQ1AE8AjwKnNBIe82sPbhjYWZmVt5zPTd6vpUvlmW7ScOBijbW2dZGYFK+PQnYFhE7atZ5Ghgh6cCaskZ1ks6gXEW6ctSJwHV12gf7Po5X579P7WNf40jDs16sWY4EDu9Dm81sgPNQKDMzs+pM2EvZ6nz7KWCkpBE1nYuJwI6I2FUoi0Z2mOdv/DEwOyIWFMr358vGzfnvpH2ss4V0xuK7dera4vdAzKwxPmNhZmZWnQmSfr/njqQjgLcC9+Wi+0kdhg8W1lG+f28D2693luRA4ADgd50SSaOA9+9H+x8GngT+Yh/r3AUcSxqqtapmeXQ/9mlmA5TPWJiZmVVnE7BQ0qXATuBy0lCoToCIeEjSYmC+pNGk+RsXAEeTrgDVmy5goqRZwK+ATRHxqKT7gcskvUD6vYtLgOeB0X1pfES8LOlzwE2SbiJN/A7gPcDiiFgFzCF1lG6XdF0+5snA+4DOiLi7L/s0s4HLHQszM7PqrAe+TLqy0hRgFXBORHQX1rkAmEe6CtQhwC+B0yOikTMWS4E/BK4kXf71BtIVp2aSfufie6ThTPOBEcDsvh5ARCyS1A18nnRZ3e2kK189k+vX5MvsXpH3eRDpLMddvDLR3cwGAUU0NCTTzMzMmij/WN2xEeErI5nZoOA5FmZmZmZmVpo7FmZmZmZmVpqHQpmZmZmZWWk+Y2FmZmZmZqW5Y2FmZmZmZqW5Y2FmZmZmZqW5Y2FmZmZmZqW5Y2FmZmZmZqW5Y2FmZmZmZqX9H0YV4h8oinJfAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Save the best model of LightGBM.\n",
      "Job Done\n"
     ]
    }
   ],
   "source": [
    "# Random seed\n",
    "SEED = 123\n",
    "\n",
    "# Function to seed everything\n",
    "def seed_everything(seed):\n",
    "    random.seed(seed)\n",
    "    np.random.seed(seed)\n",
    "    os.environ['PYTHONHASHSEED'] = str(seed)\n",
    "    \n",
    "seed_everything(SEED)\n",
    "\n",
    "\n",
    "# Funcion for user stats with loops\n",
    "def add_features(df, answered_correctly_u_count, answered_correctly_u_sum, elapsed_time_u_sum, explanation_u_sum, timestamp_u, timestamp_u_incorrect, answered_correctly_q_count, answered_correctly_q_sum, elapsed_time_q_sum, explanation_q_sum, answered_correctly_uq, update = True):\n",
    "    # -----------------------------------------------------------------------\n",
    "    # Client features\n",
    "    answered_correctly_u_avg = np.zeros(len(df), dtype = np.float32)\n",
    "    elapsed_time_u_avg = np.zeros(len(df), dtype = np.float32)\n",
    "    explanation_u_avg = np.zeros(len(df), dtype = np.float32)\n",
    "    timestamp_u_recency_1 = np.zeros(len(df), dtype = np.float32)\n",
    "    timestamp_u_recency_2 = np.zeros(len(df), dtype = np.float32)\n",
    "    timestamp_u_recency_3 = np.zeros(len(df), dtype = np.float32)\n",
    "    timestamp_u_incorrect_recency = np.zeros(len(df), dtype = np.float32)\n",
    "    # -----------------------------------------------------------------------\n",
    "    # Question features\n",
    "    answered_correctly_q_avg = np.zeros(len(df), dtype = np.float32)\n",
    "    elapsed_time_q_avg = np.zeros(len(df), dtype = np.float32)\n",
    "    explanation_q_avg = np.zeros(len(df), dtype = np.float32)\n",
    "    # -----------------------------------------------------------------------\n",
    "    # User Question\n",
    "    answered_correctly_uq_count = np.zeros(len(df), dtype = np.int32)\n",
    "    # -----------------------------------------------------------------------\n",
    "    \n",
    "    for num, row in enumerate(df[['user_id', 'answered_correctly', 'content_id', 'prior_question_elapsed_time', 'prior_question_had_explanation', 'timestamp']].values):\n",
    "        \n",
    "        # Client features assignation\n",
    "        # ------------------------------------------------------------------\n",
    "        if answered_correctly_u_count[row[0]] != 0:\n",
    "            answered_correctly_u_avg[num] = answered_correctly_u_sum[row[0]] / answered_correctly_u_count[row[0]]\n",
    "            elapsed_time_u_avg[num] = elapsed_time_u_sum[row[0]] / answered_correctly_u_count[row[0]]\n",
    "            explanation_u_avg[num] = explanation_u_sum[row[0]] / answered_correctly_u_count[row[0]]\n",
    "        else:\n",
    "            answered_correctly_u_avg[num] = np.nan\n",
    "            elapsed_time_u_avg[num] = np.nan\n",
    "            explanation_u_avg[num] = np.nan\n",
    "            \n",
    "        if len(timestamp_u[row[0]]) == 0:\n",
    "            timestamp_u_recency_1[num] = np.nan\n",
    "            timestamp_u_recency_2[num] = np.nan\n",
    "            timestamp_u_recency_3[num] = np.nan\n",
    "        elif len(timestamp_u[row[0]]) == 1:\n",
    "            timestamp_u_recency_1[num] = row[5] - timestamp_u[row[0]][0]\n",
    "            timestamp_u_recency_2[num] = np.nan\n",
    "            timestamp_u_recency_3[num] = np.nan\n",
    "        elif len(timestamp_u[row[0]]) == 2:\n",
    "            timestamp_u_recency_1[num] = row[5] - timestamp_u[row[0]][1]\n",
    "            timestamp_u_recency_2[num] = row[5] - timestamp_u[row[0]][0]\n",
    "            timestamp_u_recency_3[num] = np.nan\n",
    "        elif len(timestamp_u[row[0]]) == 3:\n",
    "            timestamp_u_recency_1[num] = row[5] - timestamp_u[row[0]][2]\n",
    "            timestamp_u_recency_2[num] = row[5] - timestamp_u[row[0]][1]\n",
    "            timestamp_u_recency_3[num] = row[5] - timestamp_u[row[0]][0]\n",
    "        \n",
    "        if len(timestamp_u_incorrect[row[0]]) == 0:\n",
    "            timestamp_u_incorrect_recency[num] = np.nan\n",
    "        else:\n",
    "            timestamp_u_incorrect_recency[num] = row[5] - timestamp_u_incorrect[row[0]][0]\n",
    "            \n",
    "        # ------------------------------------------------------------------\n",
    "        # Question features assignation\n",
    "        if answered_correctly_q_count[row[2]] != 0:\n",
    "            answered_correctly_q_avg[num] = answered_correctly_q_sum[row[2]] / answered_correctly_q_count[row[2]]\n",
    "            elapsed_time_q_avg[num] = elapsed_time_q_sum[row[2]] / answered_correctly_q_count[row[2]]\n",
    "            explanation_q_avg[num] = explanation_q_sum[row[2]] / answered_correctly_q_count[row[2]]\n",
    "        else:\n",
    "            answered_correctly_q_avg[num] = np.nan\n",
    "            elapsed_time_q_avg[num] = np.nan\n",
    "            explanation_q_avg[num] = np.nan\n",
    "        # ------------------------------------------------------------------\n",
    "        # Client Question assignation\n",
    "        answered_correctly_uq_count[num] = answered_correctly_uq[row[0]][row[2]]\n",
    "        # ------------------------------------------------------------------\n",
    "        # ------------------------------------------------------------------\n",
    "        # Client features updates\n",
    "        answered_correctly_u_count[row[0]] += 1\n",
    "        elapsed_time_u_sum[row[0]] += row[3]\n",
    "        explanation_u_sum[row[0]] += int(row[4])\n",
    "        if len(timestamp_u[row[0]]) == 3:\n",
    "            timestamp_u[row[0]].pop(0)\n",
    "            timestamp_u[row[0]].append(row[5])\n",
    "        else:\n",
    "            timestamp_u[row[0]].append(row[5])\n",
    "        # ------------------------------------------------------------------\n",
    "        # Question features updates\n",
    "        answered_correctly_q_count[row[2]] += 1\n",
    "        elapsed_time_q_sum[row[2]] += row[3]\n",
    "        explanation_q_sum[row[2]] += int(row[4])\n",
    "        # ------------------------------------------------------------------\n",
    "        # Client Question updates\n",
    "        answered_correctly_uq[row[0]][row[2]] += 1\n",
    "        # ------------------------------------------------------------------\n",
    "        # Flag for training and inference\n",
    "        if update:\n",
    "            # ------------------------------------------------------------------\n",
    "            # Client features updates\n",
    "            answered_correctly_u_sum[row[0]] += row[1]\n",
    "            if row[1] == 0:\n",
    "                if len(timestamp_u_incorrect[row[0]]) == 1:\n",
    "                    timestamp_u_incorrect[row[0]].pop(0)\n",
    "                    timestamp_u_incorrect[row[0]].append(row[5])\n",
    "                else:\n",
    "                    timestamp_u_incorrect[row[0]].append(row[5])\n",
    "            \n",
    "            # ------------------------------------------------------------------\n",
    "            # Question features updates\n",
    "            answered_correctly_q_sum[row[2]] += row[1]\n",
    "            # ------------------------------------------------------------------\n",
    "             \n",
    "            \n",
    "    user_df = pd.DataFrame({'answered_correctly_u_avg': answered_correctly_u_avg, 'elapsed_time_u_avg': elapsed_time_u_avg, 'explanation_u_avg': explanation_u_avg, \n",
    "                            'answered_correctly_q_avg': answered_correctly_q_avg, 'elapsed_time_q_avg': elapsed_time_q_avg, 'explanation_q_avg': explanation_q_avg, \n",
    "                            'answered_correctly_uq_count': answered_correctly_uq_count, 'timestamp_u_recency_1': timestamp_u_recency_1, 'timestamp_u_recency_2': timestamp_u_recency_2,\n",
    "                            'timestamp_u_recency_3': timestamp_u_recency_3, 'timestamp_u_incorrect_recency': timestamp_u_incorrect_recency})\n",
    "    \n",
    "    df = pd.concat([df, user_df], axis = 1)\n",
    "    return df\n",
    "        \n",
    "def update_features(df, answered_correctly_u_sum, answered_correctly_q_sum, timestamp_u_incorrect):\n",
    "    for row in df[['user_id', 'answered_correctly', 'content_id', 'content_type_id', 'timestamp']].values:\n",
    "        if row[3] == 0:\n",
    "            # ------------------------------------------------------------------\n",
    "            # Client features updates\n",
    "            answered_correctly_u_sum[row[0]] += row[1]\n",
    "            if row[1] == 0:\n",
    "                if len(timestamp_u_incorrect[row[0]]) == 1:\n",
    "                    timestamp_u_incorrect[row[0]].pop(0)\n",
    "                    timestamp_u_incorrect[row[0]].append(row[4])\n",
    "                else:\n",
    "                    timestamp_u_incorrect[row[0]].append(row[4])\n",
    "            # ------------------------------------------------------------------\n",
    "            # Question features updates\n",
    "            answered_correctly_q_sum[row[2]] += row[1]\n",
    "            # ------------------------------------------------------------------\n",
    "            \n",
    "    return\n",
    "\n",
    "def read_and_preprocess(feature_engineering = False):\n",
    "    \n",
    "    train_pickle = '../input/riiid-cross-validation-files/cv1_train.pickle'\n",
    "    valid_pickle = '../input/riiid-cross-validation-files/cv1_valid.pickle'\n",
    "    question_file = '../input/riiid-test-answer-prediction/questions.csv'\n",
    "    \n",
    "    # Read data\n",
    "    feld_needed = ['timestamp', 'user_id', 'answered_correctly', 'content_id', 'content_type_id', 'prior_question_elapsed_time', 'prior_question_had_explanation']\n",
    "    train = pd.read_pickle(train_pickle)[feld_needed]\n",
    "    valid = pd.read_pickle(valid_pickle)[feld_needed]\n",
    "    # Delete some trianing data to don't have ram problems\n",
    "    if feature_engineering:\n",
    "        train = train.iloc[-40000000:]\n",
    "    \n",
    "    # Filter by content_type_id to discard lectures\n",
    "    train = train.loc[train.content_type_id == False].reset_index(drop = True)\n",
    "    valid = valid.loc[valid.content_type_id == False].reset_index(drop = True)\n",
    "    \n",
    "    # Changing dtype to avoid lightgbm error\n",
    "    train['prior_question_had_explanation'] = train.prior_question_had_explanation.fillna(False).astype('int8')\n",
    "    valid['prior_question_had_explanation'] = valid.prior_question_had_explanation.fillna(False).astype('int8')\n",
    "    \n",
    "    # Fill prior question elapsed time with the mean\n",
    "    prior_question_elapsed_time_mean = train['prior_question_elapsed_time'].dropna().mean()\n",
    "    train['prior_question_elapsed_time'].fillna(prior_question_elapsed_time_mean, inplace = True)\n",
    "    valid['prior_question_elapsed_time'].fillna(prior_question_elapsed_time_mean, inplace = True)\n",
    "    \n",
    "    # Merge with question dataframe\n",
    "    questions_df = pd.read_csv(question_file)\n",
    "    questions_df['part'] = questions_df['part'].astype(np.int32)\n",
    "    questions_df['bundle_id'] = questions_df['bundle_id'].astype(np.int32)\n",
    "    \n",
    "    train = pd.merge(train, questions_df[['question_id', 'part']], left_on = 'content_id', right_on = 'question_id', how = 'left')\n",
    "    valid = pd.merge(valid, questions_df[['question_id', 'part']], left_on = 'content_id', right_on = 'question_id', how = 'left')\n",
    "    \n",
    "    # Client dictionaries\n",
    "    answered_correctly_u_count = defaultdict(int)\n",
    "    answered_correctly_u_sum = defaultdict(int)\n",
    "    elapsed_time_u_sum = defaultdict(int)\n",
    "    explanation_u_sum = defaultdict(int)\n",
    "    timestamp_u = defaultdict(list)\n",
    "    timestamp_u_incorrect = defaultdict(list)\n",
    "    \n",
    "    # Question dictionaries\n",
    "    answered_correctly_q_count = defaultdict(int)\n",
    "    answered_correctly_q_sum = defaultdict(int)\n",
    "    elapsed_time_q_sum = defaultdict(int)\n",
    "    explanation_q_sum = defaultdict(int)\n",
    "    \n",
    "    # Client Question dictionary\n",
    "    answered_correctly_uq = defaultdict(lambda: defaultdict(int))\n",
    "    \n",
    "    print('User feature calculation started...')\n",
    "    print('\\n')\n",
    "    train = add_features(train, answered_correctly_u_count, answered_correctly_u_sum, elapsed_time_u_sum, explanation_u_sum, timestamp_u, timestamp_u_incorrect, answered_correctly_q_count, answered_correctly_q_sum, elapsed_time_q_sum, explanation_q_sum, answered_correctly_uq)\n",
    "    valid = add_features(valid, answered_correctly_u_count, answered_correctly_u_sum, elapsed_time_u_sum, explanation_u_sum, timestamp_u, timestamp_u_incorrect, answered_correctly_q_count, answered_correctly_q_sum, elapsed_time_q_sum, explanation_q_sum, answered_correctly_uq)\n",
    "    gc.collect()\n",
    "    print('User feature calculation completed...')\n",
    "    print('\\n')\n",
    "    \n",
    "    features_dicts = {\n",
    "        'answered_correctly_u_count': answered_correctly_u_count,\n",
    "        'answered_correctly_u_sum': answered_correctly_u_sum,\n",
    "        'elapsed_time_u_sum': elapsed_time_u_sum,\n",
    "        'explanation_u_sum': explanation_u_sum,\n",
    "        'answered_correctly_q_count': answered_correctly_q_count,\n",
    "        'answered_correctly_q_sum': answered_correctly_q_sum,\n",
    "        'elapsed_time_q_sum': elapsed_time_q_sum,\n",
    "        'explanation_q_sum': explanation_q_sum,\n",
    "        'answered_correctly_uq': answered_correctly_uq,\n",
    "        'timestamp_u': timestamp_u,\n",
    "        'timestamp_u_incorrect': timestamp_u_incorrect\n",
    "    }\n",
    "    \n",
    "    return train, valid, questions_df, prior_question_elapsed_time_mean, features_dicts\n",
    "\n",
    "# Function for training and evaluation\n",
    "def train_and_evaluate(train, valid, feature_engineering = False):\n",
    "    \n",
    "    TARGET = 'answered_correctly'\n",
    "    # Features to train and predict\n",
    "    FEATURES = ['prior_question_elapsed_time', 'prior_question_had_explanation', 'part', 'answered_correctly_u_avg', 'elapsed_time_u_avg', 'explanation_u_avg',\n",
    "                'answered_correctly_q_avg', 'elapsed_time_q_avg', 'explanation_q_avg', 'answered_correctly_uq_count', 'timestamp_u_recency_1', 'timestamp_u_recency_2', 'timestamp_u_recency_3', \n",
    "                'timestamp_u_incorrect_recency']\n",
    "    \n",
    "    # Delete some training data to experiment faster\n",
    "    if feature_engineering:\n",
    "        train = train.sample(15000000, random_state = SEED)\n",
    "    gc.collect()\n",
    "    print(f'Traning with {train.shape[0]} rows and {len(FEATURES)} features')    \n",
    "    drop_cols = list(set(train.columns) - set(FEATURES))\n",
    "    y_train = train[TARGET]\n",
    "    y_val = valid[TARGET]\n",
    "    # Drop unnecessary columns\n",
    "    train.drop(drop_cols, axis = 1, inplace = True)\n",
    "    valid.drop(drop_cols, axis = 1, inplace = True)\n",
    "    gc.collect()\n",
    "    \n",
    "    lgb_train = lgb.Dataset(train[FEATURES], y_train)\n",
    "    lgb_valid = lgb.Dataset(valid[FEATURES], y_val)\n",
    "    del train, y_train\n",
    "    gc.collect()\n",
    "    \n",
    "    params = pd.read_pickle('../input/the-best-of-all-parameters-on-lightgbm/The_best_of_all_parameters_on_LightGBM.pickle')\n",
    "    \n",
    "    model = lgb.train(\n",
    "        params = params,\n",
    "        train_set = lgb_train,\n",
    "        num_boost_round = 10000,\n",
    "        valid_sets = [lgb_train, lgb_valid],\n",
    "        early_stopping_rounds = 10,\n",
    "        verbose_eval = 50\n",
    "    )\n",
    "    \n",
    "    print('Our ROC AUC score for the validation data is:', roc_auc_score(y_val, model.predict(valid[FEATURES], num_iteration=model.best_iteration)))\n",
    "    \n",
    "    feature_importance = model.feature_importance()\n",
    "    feature_importance = pd.DataFrame({'Features': FEATURES, 'Importance': feature_importance}).sort_values('Importance', ascending = False)\n",
    "    \n",
    "    fig = plt.figure(figsize = (10, 10))\n",
    "    fig.suptitle('Feature Importance', fontsize = 20)\n",
    "    plt.tick_params(axis = 'x', labelsize = 12)\n",
    "    plt.tick_params(axis = 'y', labelsize = 12)\n",
    "    plt.xlabel('Importance', fontsize = 15)\n",
    "    plt.ylabel('Features', fontsize = 15)\n",
    "    sns.barplot(x = feature_importance['Importance'], y = feature_importance['Features'], orient = 'h')\n",
    "    plt.show()\n",
    "    \n",
    "    return TARGET, FEATURES, model\n",
    "\n",
    "# Using time series api that simulates production predictions\n",
    "def inference(TARGET, FEATURES, model, questions_df, prior_question_elapsed_time_mean, features_dicts):\n",
    "    \n",
    "    # Get feature dict\n",
    "    answered_correctly_u_count = features_dicts['answered_correctly_u_count']\n",
    "    answered_correctly_u_sum = features_dicts['answered_correctly_u_sum']\n",
    "    elapsed_time_u_sum = features_dicts['elapsed_time_u_sum']\n",
    "    explanation_u_sum = features_dicts['explanation_u_sum']\n",
    "    answered_correctly_q_count = features_dicts['answered_correctly_q_count']\n",
    "    answered_correctly_q_sum = features_dicts['answered_correctly_q_sum']\n",
    "    elapsed_time_q_sum = features_dicts['elapsed_time_q_sum']\n",
    "    explanation_q_sum = features_dicts['explanation_q_sum']\n",
    "    answered_correctly_uq = features_dicts['answered_correctly_uq']\n",
    "    timestamp_u = features_dicts['timestamp_u']\n",
    "    timestamp_u_incorrect = features_dicts['timestamp_u_incorrect']\n",
    "        \n",
    "    # Get api iterator and predictor\n",
    "    env = riiideducation.make_env()\n",
    "    iter_test = env.iter_test()\n",
    "    set_predict = env.predict\n",
    "    \n",
    "    previous_test_df = None\n",
    "    for (test_df, sample_prediction_df) in iter_test:\n",
    "        if previous_test_df is not None:\n",
    "            previous_test_df[TARGET] = eval(test_df[\"prior_group_answers_correct\"].iloc[0])\n",
    "            update_features(previous_test_df, answered_correctly_u_sum, answered_correctly_q_sum, timestamp_u_incorrect)\n",
    "        previous_test_df = test_df.copy()\n",
    "        test_df = test_df[test_df['content_type_id'] == 0].reset_index(drop = True)\n",
    "        test_df['prior_question_had_explanation'] = test_df.prior_question_had_explanation.fillna(False).astype('int8')\n",
    "        test_df['prior_question_elapsed_time'].fillna(prior_question_elapsed_time_mean, inplace = True)\n",
    "        test_df = pd.merge(test_df, questions_df[['question_id', 'part']], left_on = 'content_id', right_on = 'question_id', how = 'left')\n",
    "        test_df[TARGET] = 0\n",
    "        test_df = add_features(test_df, answered_correctly_u_count, answered_correctly_u_sum, elapsed_time_u_sum, explanation_u_sum, timestamp_u, timestamp_u_incorrect, answered_correctly_q_count, answered_correctly_q_sum, elapsed_time_q_sum, explanation_q_sum, answered_correctly_uq, update = False)\n",
    "        test_df[TARGET] =  model.predict(test_df[FEATURES], num_iteration=model.best_iteration)\n",
    "        set_predict(test_df[['row_id', TARGET]])\n",
    "        \n",
    "    # Save the model\n",
    "    pickle.dump(model, open('The_best_of_model_on_LightGBM.pickle', 'wb'))\n",
    "    print('Save the best model of LightGBM.')\n",
    "    \n",
    "    print('Job Done')\n",
    "    \n",
    "train, valid, questions_df, prior_question_elapsed_time_mean, features_dicts = read_and_preprocess(feature_engineering = True)\n",
    "TARGET, FEATURES, model = train_and_evaluate(train, valid, feature_engineering = True)\n",
    "inference(TARGET, FEATURES, model, questions_df, prior_question_elapsed_time_mean, features_dicts)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "papermill": {
     "duration": 0.019112,
     "end_time": "2020-12-24T23:27:45.842754",
     "exception": false,
     "start_time": "2020-12-24T23:27:45.823642",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## I would like to thank the original author, [@ragnar](https://www.kaggle.com/ragnar123) deeply."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  },
  "papermill": {
   "duration": 5240.036226,
   "end_time": "2020-12-24T23:27:46.179655",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2020-12-24T22:00:26.143429",
   "version": "2.1.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}