lgbm-time-series-of-cryptocurrency-prices.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f2e93f7b",
   "metadata": {
    "papermill": {
     "duration": 0.013186,
     "end_time": "2023-10-31T13:56:49.335724",
     "exception": false,
     "start_time": "2023-10-31T13:56:49.322538",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# LGBM on time series data of cryptocurrency prices\n",
    "The dataset used in this notebook can be found here: https://www.kaggle.com/competitions/g-research-crypto-forecasting/data\n",
    "\n",
    "I will compare two LGBM baselines,\n",
    "\n",
    "* one that considers each asset independently (i.e. For each asset a Model File will be saved) and,\n",
    "* one multiple inputs that models all assets together (i.e. single Model File will be saved).\n",
    "\n",
    "And in the competition, the predictions will be evaluated on a weighted version of the Pearson correlation coefficient, with weights given by the Weight column in the Asset Details file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "859ea668",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:56:49.363706Z",
     "iopub.status.busy": "2023-10-31T13:56:49.362948Z",
     "iopub.status.idle": "2023-10-31T13:56:53.831726Z",
     "shell.execute_reply": "2023-10-31T13:56:53.830387Z"
    },
    "papermill": {
     "duration": 4.486101,
     "end_time": "2023-10-31T13:56:53.835243",
     "exception": false,
     "start_time": "2023-10-31T13:56:49.349142",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_20/2106194749.py:17: MatplotlibDeprecationWarning: The seaborn styles shipped by Matplotlib are deprecated since 3.6, as they no longer correspond to the styles shipped by seaborn. However, they will remain available as 'seaborn-v0_8-<style>'. Alternatively, directly use the seaborn API instead.\n",
      "  style.use('seaborn-colorblind')\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from tqdm.auto import tqdm\n",
    "import joblib\n",
    "import time\n",
    "import datetime\n",
    "from scipy import stats\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.style as style\n",
    "from matplotlib_venn import venn2, venn3\n",
    "import seaborn as sns\n",
    "from matplotlib import pyplot\n",
    "from matplotlib.ticker import ScalarFormatter\n",
    "sns.set_context(\"talk\")\n",
    "style.use('seaborn-colorblind')\n",
    "from xgboost import XGBRegressor\n",
    "import lightgbm as lgb\n",
    "\n",
    "import warnings\n",
    "warnings.simplefilter('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7b1a3531",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:56:53.863180Z",
     "iopub.status.busy": "2023-10-31T13:56:53.862702Z",
     "iopub.status.idle": "2023-10-31T13:56:53.868321Z",
     "shell.execute_reply": "2023-10-31T13:56:53.866913Z"
    },
    "papermill": {
     "duration": 0.023384,
     "end_time": "2023-10-31T13:56:53.871079",
     "exception": false,
     "start_time": "2023-10-31T13:56:53.847695",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "ROOT_DIR = '/kaggle/input/g-research-crypto-forecasting'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fc85158",
   "metadata": {
    "papermill": {
     "duration": 0.011728,
     "end_time": "2023-10-31T13:56:53.895307",
     "exception": false,
     "start_time": "2023-10-31T13:56:53.883579",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Config"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "916cbfed",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:56:53.921719Z",
     "iopub.status.busy": "2023-10-31T13:56:53.921273Z",
     "iopub.status.idle": "2023-10-31T13:56:53.927123Z",
     "shell.execute_reply": "2023-10-31T13:56:53.925847Z"
    },
    "papermill": {
     "duration": 0.022581,
     "end_time": "2023-10-31T13:56:53.930046",
     "exception": false,
     "start_time": "2023-10-31T13:56:53.907465",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "class CFG:\n",
    "    INPUT_DIR = ROOT_DIR\n",
    "    OUTPUT_DIR = './'\n",
    "    SEED = 42"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b17fabe",
   "metadata": {
    "papermill": {
     "duration": 0.012183,
     "end_time": "2023-10-31T13:56:53.955214",
     "exception": false,
     "start_time": "2023-10-31T13:56:53.943031",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Reduce the memory footprint of the dataset\n",
    "Based on the data type and the range of values, the function below determines the smallest possible data type that can accommodate the data without losing information. For example, if the data type is an integer and the range of values fits within the bounds of an int8 data type, the function converts the column data type to int8."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e3bdc147",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:56:53.983697Z",
     "iopub.status.busy": "2023-10-31T13:56:53.982955Z",
     "iopub.status.idle": "2023-10-31T13:56:53.995498Z",
     "shell.execute_reply": "2023-10-31T13:56:53.994477Z"
    },
    "papermill": {
     "duration": 0.029055,
     "end_time": "2023-10-31T13:56:53.998109",
     "exception": false,
     "start_time": "2023-10-31T13:56:53.969054",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_optimal_numeric_type(c_min: float, c_max: float, col_type: str) -> str:\n",
    "    \"\"\"\n",
    "    Determines the optimal numeric data type for a given range of values.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    c_min : float\n",
    "        The minimum value of the data.\n",
    "    c_max : float\n",
    "        The maximum value of the data.\n",
    "    col_type : str\n",
    "        The current data type of the column ('int' or 'float').\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    optimal_type : str\n",
    "        The optimal data type for the given range of values.\n",
    "    \"\"\"\n",
    "    type_info = np.iinfo if col_type == 'int' else np.finfo\n",
    "    for dtype in [np.int8, np.int16, np.int32, np.int64, np.float16, np.float32, np.float64]:\n",
    "        if col_type in str(dtype):\n",
    "            if c_min > type_info(dtype).min and c_max < type_info(dtype).max:\n",
    "                return dtype\n",
    "    return None\n",
    "\n",
    "\"\"\" Based on the data type and the range of values, the function determines the smallest possible data type that can accommodate the data without losing information. For example, if the data type is an integer and the range of values fits within the bounds of an int8 data type, the function converts the column data type to int8: \"\"\"\n",
    "\n",
    "def reduce_memory_usage(df: pd.DataFrame) -> pd.DataFrame:\n",
    "    \"\"\"\n",
    "    Reduces memory usage of a pandas DataFrame by converting its columns to the most memory-efficient data types\n",
    "    without losing information.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    df : pd.DataFrame\n",
    "        The input pandas DataFrame that needs memory optimization.\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    df : pd.DataFrame\n",
    "        The optimized pandas DataFrame with reduced memory usage.\n",
    "    \"\"\"\n",
    "\n",
    "    # Iterate through each column in the DataFrame\n",
    "    df_copy = df.copy()\n",
    "    for col in df_copy.columns:\n",
    "        col_type = df_copy[col].dtype\n",
    "\n",
    "        # Check if the data type is not an object (i.e., numeric type)\n",
    "        if col_type != object:\n",
    "            c_min, c_max = df_copy[col].min(), df_copy[col].max()\n",
    "            col_type_str = 'int' if 'int' in str(col_type) else 'float'\n",
    "            optimal_type = get_optimal_numeric_type(c_min, c_max, col_type_str)\n",
    "            if optimal_type:\n",
    "                df_copy[col] = df_copy[col].astype(optimal_type)\n",
    "        # If the data type is an object, convert the column to a 'category' data type\n",
    "        else:\n",
    "            df_copy[col] = df_copy[col].astype('category')\n",
    "\n",
    "    # Return the optimized DataFrame with reduced memory usage\n",
    "    return df_copy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9d378c6",
   "metadata": {
    "papermill": {
     "duration": 0.011647,
     "end_time": "2023-10-31T13:56:54.022267",
     "exception": false,
     "start_time": "2023-10-31T13:56:54.010620",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## The dataset - train.csv\n",
    "* timestamp - A timestamp for the minute covered by the row.\n",
    "* Asset_ID - An ID code for the cryptoasset.\n",
    "* Count - The number of trades that took place this minute.\n",
    "* Open - The USD price at the beginning of the minute.\n",
    "* High - The highest USD price during the minute.\n",
    "* Low - The lowest USD price during the minute.\n",
    "* Close - The USD price at the end of the minute.\n",
    "* Volume - The number of cryptoasset units traded during the minute.\n",
    "* VWAP - The volume weighted average price for the minute.\n",
    "* Target - 15 minute residualized returns. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e176ea1a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:56:54.050355Z",
     "iopub.status.busy": "2023-10-31T13:56:54.049245Z",
     "iopub.status.idle": "2023-10-31T13:58:00.043812Z",
     "shell.execute_reply": "2023-10-31T13:58:00.042749Z"
    },
    "papermill": {
     "duration": 66.023225,
     "end_time": "2023-10-31T13:58:00.057704",
     "exception": false,
     "start_time": "2023-10-31T13:56:54.034479",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 24236806 entries, 0 to 24236805\n",
      "Data columns (total 10 columns):\n",
      " #   Column     Dtype  \n",
      "---  ------     -----  \n",
      " 0   timestamp  int64  \n",
      " 1   Asset_ID   int64  \n",
      " 2   Count      float64\n",
      " 3   Open       float64\n",
      " 4   High       float64\n",
      " 5   Low        float64\n",
      " 6   Close      float64\n",
      " 7   Volume     float64\n",
      " 8   VWAP       float64\n",
      " 9   Target     float64\n",
      "dtypes: float64(8), int64(2)\n",
      "memory usage: 1.8 GB\n"
     ]
    }
   ],
   "source": [
    "before = pd.read_csv(os.path.join(CFG.INPUT_DIR, 'train.csv'))\n",
    "before.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "46f629fc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:58:00.086105Z",
     "iopub.status.busy": "2023-10-31T13:58:00.085020Z",
     "iopub.status.idle": "2023-10-31T13:59:06.750667Z",
     "shell.execute_reply": "2023-10-31T13:59:06.748737Z"
    },
    "papermill": {
     "duration": 66.699362,
     "end_time": "2023-10-31T13:59:06.769158",
     "exception": false,
     "start_time": "2023-10-31T13:58:00.069796",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(24236806, 10)\n",
      "CPU times: user 49.7 s, sys: 10.8 s, total: 1min\n",
      "Wall time: 1min 6s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>Asset_ID</th>\n",
       "      <th>Count</th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "      <th>VWAP</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1514764860</td>\n",
       "      <td>2</td>\n",
       "      <td>40.0</td>\n",
       "      <td>2376.000000</td>\n",
       "      <td>2400.000000</td>\n",
       "      <td>2358.00000</td>\n",
       "      <td>2374.00000</td>\n",
       "      <td>19.233006</td>\n",
       "      <td>2373.116392</td>\n",
       "      <td>-0.004219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1514764860</td>\n",
       "      <td>0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>8.531250</td>\n",
       "      <td>8.531250</td>\n",
       "      <td>8.53125</td>\n",
       "      <td>8.53125</td>\n",
       "      <td>78.379997</td>\n",
       "      <td>8.530000</td>\n",
       "      <td>-0.014397</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1514764860</td>\n",
       "      <td>1</td>\n",
       "      <td>229.0</td>\n",
       "      <td>13832.000000</td>\n",
       "      <td>14016.000000</td>\n",
       "      <td>13664.00000</td>\n",
       "      <td>13848.00000</td>\n",
       "      <td>31.550062</td>\n",
       "      <td>13827.062093</td>\n",
       "      <td>-0.014641</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1514764860</td>\n",
       "      <td>5</td>\n",
       "      <td>32.0</td>\n",
       "      <td>7.660156</td>\n",
       "      <td>7.660156</td>\n",
       "      <td>7.65625</td>\n",
       "      <td>7.65625</td>\n",
       "      <td>6626.713379</td>\n",
       "      <td>7.657713</td>\n",
       "      <td>-0.013924</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1514764860</td>\n",
       "      <td>7</td>\n",
       "      <td>5.0</td>\n",
       "      <td>25.921875</td>\n",
       "      <td>25.921875</td>\n",
       "      <td>25.87500</td>\n",
       "      <td>25.87500</td>\n",
       "      <td>121.087311</td>\n",
       "      <td>25.891363</td>\n",
       "      <td>-0.008263</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    timestamp  Asset_ID  Count          Open          High          Low  \\\n",
       "0  1514764860         2   40.0   2376.000000   2400.000000   2358.00000   \n",
       "1  1514764860         0    5.0      8.531250      8.531250      8.53125   \n",
       "2  1514764860         1  229.0  13832.000000  14016.000000  13664.00000   \n",
       "3  1514764860         5   32.0      7.660156      7.660156      7.65625   \n",
       "4  1514764860         7    5.0     25.921875     25.921875     25.87500   \n",
       "\n",
       "         Close       Volume          VWAP    Target  \n",
       "0   2374.00000    19.233006   2373.116392 -0.004219  \n",
       "1      8.53125    78.379997      8.530000 -0.014397  \n",
       "2  13848.00000    31.550062  13827.062093 -0.014641  \n",
       "3      7.65625  6626.713379      7.657713 -0.013924  \n",
       "4     25.87500   121.087311     25.891363 -0.008263  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "train = pd.read_csv(os.path.join(CFG.INPUT_DIR, 'train.csv')).pipe(reduce_memory_usage)\n",
    "print(train.shape)\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "f20def20",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:06.801608Z",
     "iopub.status.busy": "2023-10-31T13:59:06.800925Z",
     "iopub.status.idle": "2023-10-31T13:59:06.826249Z",
     "shell.execute_reply": "2023-10-31T13:59:06.824260Z"
    },
    "papermill": {
     "duration": 0.045252,
     "end_time": "2023-10-31T13:59:06.829576",
     "exception": false,
     "start_time": "2023-10-31T13:59:06.784324",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>Asset_ID</th>\n",
       "      <th>Count</th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "      <th>VWAP</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>24236801</th>\n",
       "      <td>1632182400</td>\n",
       "      <td>9</td>\n",
       "      <td>775.0</td>\n",
       "      <td>157.125000</td>\n",
       "      <td>157.250000</td>\n",
       "      <td>156.750000</td>\n",
       "      <td>157.000000</td>\n",
       "      <td>4.663726e+03</td>\n",
       "      <td>156.994319</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24236802</th>\n",
       "      <td>1632182400</td>\n",
       "      <td>10</td>\n",
       "      <td>34.0</td>\n",
       "      <td>2438.000000</td>\n",
       "      <td>2438.000000</td>\n",
       "      <td>2430.000000</td>\n",
       "      <td>2432.000000</td>\n",
       "      <td>3.975460e+00</td>\n",
       "      <td>2434.818747</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24236803</th>\n",
       "      <td>1632182400</td>\n",
       "      <td>13</td>\n",
       "      <td>380.0</td>\n",
       "      <td>0.091370</td>\n",
       "      <td>0.091553</td>\n",
       "      <td>0.091248</td>\n",
       "      <td>0.091370</td>\n",
       "      <td>2.193732e+06</td>\n",
       "      <td>0.091388</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24236804</th>\n",
       "      <td>1632182400</td>\n",
       "      <td>12</td>\n",
       "      <td>177.0</td>\n",
       "      <td>0.282227</td>\n",
       "      <td>0.282471</td>\n",
       "      <td>0.281738</td>\n",
       "      <td>0.281982</td>\n",
       "      <td>1.828508e+05</td>\n",
       "      <td>0.282134</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24236805</th>\n",
       "      <td>1632182400</td>\n",
       "      <td>11</td>\n",
       "      <td>48.0</td>\n",
       "      <td>232.750000</td>\n",
       "      <td>232.750000</td>\n",
       "      <td>232.250000</td>\n",
       "      <td>232.250000</td>\n",
       "      <td>1.035123e+02</td>\n",
       "      <td>232.569697</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           timestamp  Asset_ID  Count         Open         High          Low  \\\n",
       "24236801  1632182400         9  775.0   157.125000   157.250000   156.750000   \n",
       "24236802  1632182400        10   34.0  2438.000000  2438.000000  2430.000000   \n",
       "24236803  1632182400        13  380.0     0.091370     0.091553     0.091248   \n",
       "24236804  1632182400        12  177.0     0.282227     0.282471     0.281738   \n",
       "24236805  1632182400        11   48.0   232.750000   232.750000   232.250000   \n",
       "\n",
       "                Close        Volume         VWAP  Target  \n",
       "24236801   157.000000  4.663726e+03   156.994319     NaN  \n",
       "24236802  2432.000000  3.975460e+00  2434.818747     NaN  \n",
       "24236803     0.091370  2.193732e+06     0.091388     NaN  \n",
       "24236804     0.281982  1.828508e+05     0.282134     NaN  \n",
       "24236805   232.250000  1.035123e+02   232.569697     NaN  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "9b6479df",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:06.860500Z",
     "iopub.status.busy": "2023-10-31T13:59:06.859300Z",
     "iopub.status.idle": "2023-10-31T13:59:06.876294Z",
     "shell.execute_reply": "2023-10-31T13:59:06.873992Z"
    },
    "papermill": {
     "duration": 0.03743,
     "end_time": "2023-10-31T13:59:06.880423",
     "exception": false,
     "start_time": "2023-10-31T13:59:06.842993",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 24236806 entries, 0 to 24236805\n",
      "Data columns (total 10 columns):\n",
      " #   Column     Dtype  \n",
      "---  ------     -----  \n",
      " 0   timestamp  int32  \n",
      " 1   Asset_ID   int8   \n",
      " 2   Count      float32\n",
      " 3   Open       float16\n",
      " 4   High       float16\n",
      " 5   Low        float16\n",
      " 6   Close      float16\n",
      " 7   Volume     float32\n",
      " 8   VWAP       float64\n",
      " 9   Target     float16\n",
      "dtypes: float16(5), float32(2), float64(1), int32(1), int8(1)\n",
      "memory usage: 716.5 MB\n"
     ]
    }
   ],
   "source": [
    "train.info() # A quite nice memory reduction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e4e5238e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:06.911504Z",
     "iopub.status.busy": "2023-10-31T13:59:06.911008Z",
     "iopub.status.idle": "2023-10-31T13:59:07.486855Z",
     "shell.execute_reply": "2023-10-31T13:59:07.485541Z"
    },
    "papermill": {
     "duration": 0.59431,
     "end_time": "2023-10-31T13:59:07.489755",
     "exception": false,
     "start_time": "2023-10-31T13:59:06.895445",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "timestamp         0\n",
       "Asset_ID          0\n",
       "Count             0\n",
       "Open              0\n",
       "High              0\n",
       "Low               0\n",
       "Close             0\n",
       "Volume            0\n",
       "VWAP              9\n",
       "Target       750338\n",
       "dtype: int64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# missing values?\n",
    "train.isna().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5546c41",
   "metadata": {
    "papermill": {
     "duration": 0.013788,
     "end_time": "2023-10-31T13:59:07.517608",
     "exception": false,
     "start_time": "2023-10-31T13:59:07.503820",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## asset_details.csv\n",
    "Provides the real name and of the cryptoasset for each Asset_ID and the weight each cryptoasset receives in the metric."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "166944f8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:07.548237Z",
     "iopub.status.busy": "2023-10-31T13:59:07.547739Z",
     "iopub.status.idle": "2023-10-31T13:59:07.571661Z",
     "shell.execute_reply": "2023-10-31T13:59:07.570448Z"
    },
    "papermill": {
     "duration": 0.042802,
     "end_time": "2023-10-31T13:59:07.574181",
     "exception": false,
     "start_time": "2023-10-31T13:59:07.531379",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(14, 3)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Asset_ID</th>\n",
       "      <th>Weight</th>\n",
       "      <th>Asset_Name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>2.397895</td>\n",
       "      <td>Bitcoin Cash</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>4.304065</td>\n",
       "      <td>Binance Coin</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>6.779922</td>\n",
       "      <td>Bitcoin</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5</td>\n",
       "      <td>1.386294</td>\n",
       "      <td>EOS.IO</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7</td>\n",
       "      <td>2.079442</td>\n",
       "      <td>Ethereum Classic</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>6</td>\n",
       "      <td>5.894403</td>\n",
       "      <td>Ethereum</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>9</td>\n",
       "      <td>2.397895</td>\n",
       "      <td>Litecoin</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>11</td>\n",
       "      <td>1.609438</td>\n",
       "      <td>Monero</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>13</td>\n",
       "      <td>1.791759</td>\n",
       "      <td>TRON</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>12</td>\n",
       "      <td>2.079442</td>\n",
       "      <td>Stellar</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>3</td>\n",
       "      <td>4.406719</td>\n",
       "      <td>Cardano</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>8</td>\n",
       "      <td>1.098612</td>\n",
       "      <td>IOTA</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>10</td>\n",
       "      <td>1.098612</td>\n",
       "      <td>Maker</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>4</td>\n",
       "      <td>3.555348</td>\n",
       "      <td>Dogecoin</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    Asset_ID    Weight        Asset_Name\n",
       "0          2  2.397895      Bitcoin Cash\n",
       "1          0  4.304065      Binance Coin\n",
       "2          1  6.779922           Bitcoin\n",
       "3          5  1.386294            EOS.IO\n",
       "4          7  2.079442  Ethereum Classic\n",
       "5          6  5.894403          Ethereum\n",
       "6          9  2.397895          Litecoin\n",
       "7         11  1.609438            Monero\n",
       "8         13  1.791759              TRON\n",
       "9         12  2.079442           Stellar\n",
       "10         3  4.406719           Cardano\n",
       "11         8  1.098612              IOTA\n",
       "12        10  1.098612             Maker\n",
       "13         4  3.555348          Dogecoin"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "asset_details = pd.read_csv(os.path.join(CFG.INPUT_DIR, 'asset_details.csv'))\n",
    "asset_details['Asset_ID'] = asset_details['Asset_ID'].astype(np.int8)\n",
    "print(asset_details.shape)\n",
    "asset_details"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1956a7e1",
   "metadata": {
    "papermill": {
     "duration": 0.013321,
     "end_time": "2023-10-31T13:59:07.602613",
     "exception": false,
     "start_time": "2023-10-31T13:59:07.589292",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Feature engineering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "1cea87e6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:07.633827Z",
     "iopub.status.busy": "2023-10-31T13:59:07.633097Z",
     "iopub.status.idle": "2023-10-31T13:59:11.704912Z",
     "shell.execute_reply": "2023-10-31T13:59:11.703009Z"
    },
    "papermill": {
     "duration": 4.092015,
     "end_time": "2023-10-31T13:59:11.708573",
     "exception": false,
     "start_time": "2023-10-31T13:59:07.616558",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ">> data goes from  2018-01-01T00:01:00 to  2021-09-21T00:00:00 shape= (24236806, 9)\n"
     ]
    }
   ],
   "source": [
    "# select train and validation period\n",
    "\n",
    "# auxiliary function, from datetime to timestamp\n",
    "totimestamp = lambda s: np.int32(time.mktime(datetime.datetime.strptime(s, \"%d/%m/%Y\").timetuple()))\n",
    "\n",
    "train_window = [totimestamp(\"01/01/2018\"), totimestamp(\"01/06/2021\")]\n",
    "\n",
    "valid_window = [totimestamp(\"02/06/2020\"), totimestamp(\"21/09/2021\")]\n",
    "\n",
    "train = train.set_index(\"timestamp\")\n",
    "\n",
    "beg_ = train.index[0].astype('datetime64[s]')\n",
    "end_ = train.index[-1].astype('datetime64[s]')\n",
    "\"\"\" taking the first and last index values and converting the retrieved index values into datetime objects with second precision. \"\"\"\n",
    "\n",
    "print('>> data goes from ', beg_, 'to ', end_, 'shape=', train.shape)\n",
    "\n",
    "# drop rows without target\n",
    "train.dropna(subset=['Target'], inplace=True)\n",
    "\n",
    "# add train flag\n",
    "train['train_flg'] = 1\n",
    "\n",
    "train.loc[valid_window[0]:valid_window[1], 'train_flg'] = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2a33d8f",
   "metadata": {
    "papermill": {
     "duration": 0.014244,
     "end_time": "2023-10-31T13:59:11.737284",
     "exception": false,
     "start_time": "2023-10-31T13:59:11.723040",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Merge the Asset Details .csv file to get the \"Weights\" for each of the Assets, which will be required for the evaluation parts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "c9c7ef53",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:11.767275Z",
     "iopub.status.busy": "2023-10-31T13:59:11.766626Z",
     "iopub.status.idle": "2023-10-31T13:59:15.428205Z",
     "shell.execute_reply": "2023-10-31T13:59:15.426625Z"
    },
    "papermill": {
     "duration": 3.681343,
     "end_time": "2023-10-31T13:59:15.432513",
     "exception": false,
     "start_time": "2023-10-31T13:59:11.751170",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def merge_asset_details(train, asset_details):\n",
    "    \"\"\"Add asset details to train df\n",
    "    \"\"\"\n",
    "    return train.merge(\n",
    "        asset_details\n",
    "        , how='left'\n",
    "        , on='Asset_ID'\n",
    "    )\n",
    "\n",
    "# merge asset_details\n",
    "train = merge_asset_details(train, asset_details)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "3deb591b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:15.467280Z",
     "iopub.status.busy": "2023-10-31T13:59:15.466869Z",
     "iopub.status.idle": "2023-10-31T13:59:15.478768Z",
     "shell.execute_reply": "2023-10-31T13:59:15.477237Z"
    },
    "papermill": {
     "duration": 0.03098,
     "end_time": "2023-10-31T13:59:15.481526",
     "exception": false,
     "start_time": "2023-10-31T13:59:15.450546",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_features(df: pd.DataFrame) -> pd.DataFrame:\n",
    "    \"\"\"\n",
    "    Extract additional features from the given DataFrame for financial analysis.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    df : pd.DataFrame\n",
    "        A pandas DataFrame containing the stock data with columns:\n",
    "        'Open', 'High', 'Low', 'Close', 'Volume', and 'Count'.\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    df_copy : pd.DataFrame\n",
    "        A new DataFrame with the extracted features as additional columns.\n",
    "    \"\"\"\n",
    "    # Create a copy of the input DataFrame to avoid modifying the original data\n",
    "    df_copy = df.copy()\n",
    "\n",
    "    # Calculate the upper and lower shadows for the candlestick chart\n",
    "    df_copy['upper_shadow'] = df_copy['High'] - np.maximum(df_copy['Close'], df_copy['Open'])\n",
    "    df_copy['lower_shadow'] = np.minimum(df_copy['Close'], df_copy['Open']) - df_copy['Low']\n",
    "\n",
    "    # Calculate the ratio of closing price to opening price\n",
    "    df_copy['open2close'] = df_copy['Close'] / df_copy['Open']\n",
    "\n",
    "    # Calculate the ratio of the highest price to the lowest price\n",
    "    df_copy['high2low'] = df_copy['High'] / df_copy['Low']\n",
    "\n",
    "    # Calculate the mean and median prices for each row\n",
    "    mean_price = df_copy[['Open', 'High', 'Low', 'Close']].mean(axis=1)\n",
    "    median_price = df_copy[['Open', 'High', 'Low', 'Close']].median(axis=1)\n",
    "\n",
    "    # Calculate the ratios of high and low prices to the mean and median prices\n",
    "    df_copy['high2mean'] = df_copy['High'] / mean_price\n",
    "    df_copy['low2mean'] = df_copy['Low'] / mean_price\n",
    "    df_copy['high2median'] = df_copy['High'] / median_price\n",
    "    df_copy['low2median'] = df_copy['Low'] / median_price\n",
    "\n",
    "    # Calculate the ratio of volume to the count, adding 1 to avoid division by zero\n",
    "    df_copy['volume2count'] = df_copy['Volume'] / (df_copy['Count'] + 1)\n",
    "\n",
    "    return df_copy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "91b7aa1c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:15.513179Z",
     "iopub.status.busy": "2023-10-31T13:59:15.512432Z",
     "iopub.status.idle": "2023-10-31T13:59:45.150766Z",
     "shell.execute_reply": "2023-10-31T13:59:45.149865Z"
    },
    "papermill": {
     "duration": 29.657573,
     "end_time": "2023-10-31T13:59:45.153396",
     "exception": false,
     "start_time": "2023-10-31T13:59:15.495823",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(23486468, 21)\n",
      "CPU times: user 26.3 s, sys: 3.27 s, total: 29.5 s\n",
      "Wall time: 29.6 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Asset_ID</th>\n",
       "      <th>Count</th>\n",
       "      <th>Open</th>\n",
       "      <th>High</th>\n",
       "      <th>Low</th>\n",
       "      <th>Close</th>\n",
       "      <th>Volume</th>\n",
       "      <th>VWAP</th>\n",
       "      <th>Target</th>\n",
       "      <th>train_flg</th>\n",
       "      <th>...</th>\n",
       "      <th>Asset_Name</th>\n",
       "      <th>upper_shadow</th>\n",
       "      <th>lower_shadow</th>\n",
       "      <th>open2close</th>\n",
       "      <th>high2low</th>\n",
       "      <th>high2mean</th>\n",
       "      <th>low2mean</th>\n",
       "      <th>high2median</th>\n",
       "      <th>low2median</th>\n",
       "      <th>volume2count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>23486463</th>\n",
       "      <td>9</td>\n",
       "      <td>163.0</td>\n",
       "      <td>156.500000</td>\n",
       "      <td>156.625000</td>\n",
       "      <td>156.000000</td>\n",
       "      <td>156.125000</td>\n",
       "      <td>7.381963e+02</td>\n",
       "      <td>156.260253</td>\n",
       "      <td>-0.001154</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Litecoin</td>\n",
       "      <td>0.125000</td>\n",
       "      <td>0.125000</td>\n",
       "      <td>0.997559</td>\n",
       "      <td>1.003906</td>\n",
       "      <td>1.001953</td>\n",
       "      <td>0.998535</td>\n",
       "      <td>1.001953</td>\n",
       "      <td>0.998535</td>\n",
       "      <td>4.501197</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23486464</th>\n",
       "      <td>10</td>\n",
       "      <td>23.0</td>\n",
       "      <td>2420.000000</td>\n",
       "      <td>2420.000000</td>\n",
       "      <td>2412.000000</td>\n",
       "      <td>2414.000000</td>\n",
       "      <td>1.315794e+00</td>\n",
       "      <td>2417.326038</td>\n",
       "      <td>0.009834</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Maker</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.997559</td>\n",
       "      <td>1.002930</td>\n",
       "      <td>1.001953</td>\n",
       "      <td>0.998535</td>\n",
       "      <td>1.001953</td>\n",
       "      <td>0.998535</td>\n",
       "      <td>0.054825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23486465</th>\n",
       "      <td>13</td>\n",
       "      <td>512.0</td>\n",
       "      <td>0.091003</td>\n",
       "      <td>0.091125</td>\n",
       "      <td>0.090820</td>\n",
       "      <td>0.090881</td>\n",
       "      <td>1.900696e+06</td>\n",
       "      <td>0.090943</td>\n",
       "      <td>0.004162</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>TRON</td>\n",
       "      <td>0.000122</td>\n",
       "      <td>0.000061</td>\n",
       "      <td>0.998535</td>\n",
       "      <td>1.002930</td>\n",
       "      <td>1.001953</td>\n",
       "      <td>0.998535</td>\n",
       "      <td>1.001953</td>\n",
       "      <td>0.998535</td>\n",
       "      <td>3705.059570</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23486466</th>\n",
       "      <td>12</td>\n",
       "      <td>193.0</td>\n",
       "      <td>0.281006</td>\n",
       "      <td>0.281250</td>\n",
       "      <td>0.280273</td>\n",
       "      <td>0.280518</td>\n",
       "      <td>1.628699e+05</td>\n",
       "      <td>0.280655</td>\n",
       "      <td>0.000449</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Stellar</td>\n",
       "      <td>0.000244</td>\n",
       "      <td>0.000244</td>\n",
       "      <td>0.998047</td>\n",
       "      <td>1.003906</td>\n",
       "      <td>1.001953</td>\n",
       "      <td>0.998047</td>\n",
       "      <td>1.001953</td>\n",
       "      <td>0.998047</td>\n",
       "      <td>839.535583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23486467</th>\n",
       "      <td>11</td>\n",
       "      <td>38.0</td>\n",
       "      <td>232.125000</td>\n",
       "      <td>232.250000</td>\n",
       "      <td>231.250000</td>\n",
       "      <td>231.625000</td>\n",
       "      <td>4.914804e+01</td>\n",
       "      <td>231.794078</td>\n",
       "      <td>0.002567</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>Monero</td>\n",
       "      <td>0.125000</td>\n",
       "      <td>0.375000</td>\n",
       "      <td>0.998047</td>\n",
       "      <td>1.003906</td>\n",
       "      <td>1.001953</td>\n",
       "      <td>0.997070</td>\n",
       "      <td>1.001953</td>\n",
       "      <td>0.997070</td>\n",
       "      <td>1.260206</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 21 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          Asset_ID  Count         Open         High          Low        Close  \\\n",
       "23486463         9  163.0   156.500000   156.625000   156.000000   156.125000   \n",
       "23486464        10   23.0  2420.000000  2420.000000  2412.000000  2414.000000   \n",
       "23486465        13  512.0     0.091003     0.091125     0.090820     0.090881   \n",
       "23486466        12  193.0     0.281006     0.281250     0.280273     0.280518   \n",
       "23486467        11   38.0   232.125000   232.250000   231.250000   231.625000   \n",
       "\n",
       "                Volume         VWAP    Target  train_flg  ...  Asset_Name  \\\n",
       "23486463  7.381963e+02   156.260253 -0.001154          0  ...    Litecoin   \n",
       "23486464  1.315794e+00  2417.326038  0.009834          0  ...       Maker   \n",
       "23486465  1.900696e+06     0.090943  0.004162          0  ...        TRON   \n",
       "23486466  1.628699e+05     0.280655  0.000449          0  ...     Stellar   \n",
       "23486467  4.914804e+01   231.794078  0.002567          0  ...      Monero   \n",
       "\n",
       "         upper_shadow  lower_shadow  open2close  high2low  high2mean  \\\n",
       "23486463     0.125000      0.125000    0.997559  1.003906   1.001953   \n",
       "23486464     0.000000      2.000000    0.997559  1.002930   1.001953   \n",
       "23486465     0.000122      0.000061    0.998535  1.002930   1.001953   \n",
       "23486466     0.000244      0.000244    0.998047  1.003906   1.001953   \n",
       "23486467     0.125000      0.375000    0.998047  1.003906   1.001953   \n",
       "\n",
       "          low2mean  high2median  low2median  volume2count  \n",
       "23486463  0.998535     1.001953    0.998535      4.501197  \n",
       "23486464  0.998535     1.001953    0.998535      0.054825  \n",
       "23486465  0.998535     1.001953    0.998535   3705.059570  \n",
       "23486466  0.998047     1.001953    0.998047    839.535583  \n",
       "23486467  0.997070     1.001953    0.997070      1.260206  \n",
       "\n",
       "[5 rows x 21 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "feature_df = get_features(train)\n",
    "\n",
    "print(feature_df.shape)\n",
    "feature_df.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "3f3e7a27",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:45.185865Z",
     "iopub.status.busy": "2023-10-31T13:59:45.184914Z",
     "iopub.status.idle": "2023-10-31T13:59:45.193156Z",
     "shell.execute_reply": "2023-10-31T13:59:45.191900Z"
    },
    "papermill": {
     "duration": 0.026547,
     "end_time": "2023-10-31T13:59:45.195758",
     "exception": false,
     "start_time": "2023-10-31T13:59:45.169211",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Asset_ID', 'Count', 'Open', 'High', 'Low', 'Close', 'Volume', 'VWAP',\n",
       "       'Target', 'train_flg', 'Weight', 'Asset_Name', 'upper_shadow',\n",
       "       'lower_shadow', 'open2close', 'high2low', 'high2mean', 'low2mean',\n",
       "       'high2median', 'low2median', 'volume2count'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "feature_df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89dd2d89",
   "metadata": {
    "papermill": {
     "duration": 0.014648,
     "end_time": "2023-10-31T13:59:45.310481",
     "exception": false,
     "start_time": "2023-10-31T13:59:45.295833",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "## Light GBM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "47179d3a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:45.343515Z",
     "iopub.status.busy": "2023-10-31T13:59:45.343025Z",
     "iopub.status.idle": "2023-10-31T13:59:45.350469Z",
     "shell.execute_reply": "2023-10-31T13:59:45.349052Z"
    },
    "papermill": {
     "duration": 0.027096,
     "end_time": "2023-10-31T13:59:45.353409",
     "exception": false,
     "start_time": "2023-10-31T13:59:45.326313",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10 features_selected_for_training: ['Asset_ID', 'Count', 'Open', 'High', 'Low', 'Close', 'Volume', 'VWAP', 'train_flg', 'Weight']\n"
     ]
    }
   ],
   "source": [
    "target = 'Target'\n",
    "\n",
    "# drops = ['timestamp', 'Asset_Name', 'Weight', 'train_flg', 'Open', 'High', 'Low', 'Close', 'Volume', 'VWAP']\n",
    "drops = ['Asset_Name']\n",
    "features_selected_for_training = [f for f in train.columns if f not in drops + [target]]\n",
    "\n",
    "categoricals = ['Asset_ID']\n",
    "\n",
    "print('{:,} features_selected_for_training: {}'.format(len(features_selected_for_training), features_selected_for_training))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "841197d0",
   "metadata": {
    "papermill": {
     "duration": 0.016665,
     "end_time": "2023-10-31T13:59:45.386461",
     "exception": false,
     "start_time": "2023-10-31T13:59:45.369796",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### LGBM Configurations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "650ac232",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:45.420127Z",
     "iopub.status.busy": "2023-10-31T13:59:45.419076Z",
     "iopub.status.idle": "2023-10-31T13:59:45.425583Z",
     "shell.execute_reply": "2023-10-31T13:59:45.424482Z"
    },
    "papermill": {
     "duration": 0.025774,
     "end_time": "2023-10-31T13:59:45.428784",
     "exception": false,
     "start_time": "2023-10-31T13:59:45.403010",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "params = {\n",
    "        'n_estimators': 10000,\n",
    "        'objective': 'regression',\n",
    "        'metric': 'rmse',\n",
    "        'boosting_type': 'gbdt',\n",
    "        'max_depth': -1,\n",
    "        'learning_rate': 0.01,\n",
    "        'subsample': 0.72,\n",
    "        'subsample_freq': 4,\n",
    "        'feature_fraction': 0.4,\n",
    "        'lambda_l1': 1,\n",
    "        'lambda_l2': 1,\n",
    "        'seed': 42,\n",
    "        }"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3adda715",
   "metadata": {
    "papermill": {
     "duration": 0.015203,
     "end_time": "2023-10-31T13:59:45.461073",
     "exception": false,
     "start_time": "2023-10-31T13:59:45.445870",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Train Full Model (i.e. a single model file will be saved)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "58c1ae40",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:45.492673Z",
     "iopub.status.busy": "2023-10-31T13:59:45.491896Z",
     "iopub.status.idle": "2023-10-31T13:59:45.499868Z",
     "shell.execute_reply": "2023-10-31T13:59:45.498879Z"
    },
    "papermill": {
     "duration": 0.026716,
     "end_time": "2023-10-31T13:59:45.502398",
     "exception": false,
     "start_time": "2023-10-31T13:59:45.475682",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def train_lgbm_model(feature_df, features_selected_for_training, target, categoricals, params):\n",
    "    \"\"\"\n",
    "    Train an LGBM model on the given feature DataFrame, save the model and return feature importances.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    feature_df : pd.DataFrame\n",
    "        DataFrame containing the features_selected_for_training and target variable\n",
    "    features_selected_for_training : list of str\n",
    "        List of feature names used for training the model\n",
    "    target : str\n",
    "        Name of the target variable\n",
    "    categoricals : list of str\n",
    "        List of categorical feature names\n",
    "    params : dict\n",
    "        Dictionary of parameters for the LGBM model\n",
    "    output_dir : str\n",
    "        Directory to save the trained model\n",
    "    \n",
    "    Returns\n",
    "    -------\n",
    "    fi_df : pd.DataFrame\n",
    "        DataFrame containing the feature importances\n",
    "    \"\"\"\n",
    "    \n",
    "    # Instantiate the LGBM model\n",
    "    model = lgb.LGBMRegressor(**params)\n",
    "\n",
    "    # Fit the model on the training data and validate on the validation data\n",
    "    model.fit(\n",
    "        feature_df.query('train_flg == 1')[features_selected_for_training],\n",
    "        feature_df.query('train_flg == 1')[target].values, \n",
    "        eval_set=[(feature_df.query('train_flg == 0')[features_selected_for_training]\n",
    "                   , feature_df.query('train_flg == 0')[target].values)],\n",
    "        verbose=-1, \n",
    "        early_stopping_rounds=100,\n",
    "        categorical_feature=categoricals,\n",
    "    )\n",
    "    \n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "765fc0a8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T13:59:45.536459Z",
     "iopub.status.busy": "2023-10-31T13:59:45.535663Z",
     "iopub.status.idle": "2023-10-31T14:02:45.578492Z",
     "shell.execute_reply": "2023-10-31T14:02:45.577013Z"
    },
    "papermill": {
     "duration": 180.077873,
     "end_time": "2023-10-31T14:02:45.596697",
     "exception": false,
     "start_time": "2023-10-31T13:59:45.518824",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[LightGBM] [Warning] feature_fraction is set=0.4, colsample_bytree=1.0 will be ignored. Current value: feature_fraction=0.4\n",
      "[LightGBM] [Warning] lambda_l1 is set=1, reg_alpha=0.0 will be ignored. Current value: lambda_l1=1\n",
      "[LightGBM] [Warning] lambda_l2 is set=1, reg_lambda=0.0 will be ignored. Current value: lambda_l2=1\n",
      "LGBM model saved to output dir\n"
     ]
    }
   ],
   "source": [
    "model = train_lgbm_model(feature_df, features_selected_for_training, target, categoricals, params)\n",
    "\n",
    "# save model\n",
    "joblib.dump(model, os.path.join(CFG.OUTPUT_DIR, 'lgb_model_val.pkl'))\n",
    "\n",
    "print('LGBM model saved to output dir')\n",
    "\n",
    "# feature importance\n",
    "fi_df = pd.DataFrame()\n",
    "fi_df['features_selected_for_training'] = features_selected_for_training\n",
    "fi_df['importance'] = model.booster_.feature_importance(importance_type=\"gain\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "820bd2e6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T14:02:45.629887Z",
     "iopub.status.busy": "2023-10-31T14:02:45.628994Z",
     "iopub.status.idle": "2023-10-31T14:02:46.088613Z",
     "shell.execute_reply": "2023-10-31T14:02:46.087190Z"
    },
    "papermill": {
     "duration": 0.48026,
     "end_time": "2023-10-31T14:02:46.091742",
     "exception": false,
     "start_time": "2023-10-31T14:02:45.611482",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Axes: xlabel='importance', ylabel='features_selected_for_training'>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsQAAATcCAYAAACDAJNPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAACerElEQVR4nOzdZ3gV1fr+8XvSe4GQ0LsUUYQEEAglIChFFEVA6U0BFSyIePSooB4LKILiT1CkHlGKqKgIgoTeCUWk19BDSYAQIG3/X/Df+xBTYTYJYb6f69qXycwzs56d8+ZmnTVrDJvNZhMAAABgUS4F3QAAAABQkAjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACzNraAbgHWUL19ecXFx8vLyUoUKFQq6HQAAcIc7ePCgrly5otDQUB06dCjbOoMXcyC/+Pj46PLlywXdBgAAsBhvb28lJSVle54ZYuQbLy8vXb58Wd7e3qpevXpBtwMAAO5wO3fu1OXLl+Xl5ZVjHYEY+aZChQqKj49X9erVtWnTpoJuBwAA3OEiIiIUExOT61JNHqoDAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApfGmOuS71NPndPrL/xZ0GwAAoAAVG9itoFtwYIYYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiJ1o6dKl6t27typVqiRfX1/5+/uratWqGjhwoGJiYgq6PQAAAGSBQOwESUlJ6tSpk5o1a6YpU6YoLi5OVapUUYUKFXTkyBGNHz9ederU0aBBg5SWllbQ7QIAAOA6BGKTUlNT1bp1a82ePVtBQUGaOHGizpw5o82bN2vbtm06ffq0Pv74Y3l4eGjcuHHq0aNHQbcMAACA6xCITXrvvfe0fPlyeXl5afHixerbt688PT0d5319fTVkyBDNnj1bhmFoxowZmjp1agF2DAAAgOsRiE24cOGCPv30U0nSa6+9poiIiGxr27Vr55gdfu+995Senu44ZxiGDMPQoUOHFB0drZYtW6pIkSLy9fVVgwYN9P333+fYx+HDhzVo0CBVrVpVPj4+CggI0P3336//+7//U2pqaqb6pUuXyjAMlS9fXpI0a9YsNWzYUP7+/goICFCLFi20atWqG/1zAAAAFEoEYhPmz5+vCxcuyNXVVQMHDsy1ftCgQZKkffv2acOGDZnOz549Wy1atNCGDRtUqVIlBQYGau3atXrqqaf02muvZXnPX3/9VTVq1NC4ceN05MgRVa5cWUWLFtWGDRv03HPPqW3btkpOTs62pzfffFOdO3fW4cOHVaVKFUnSn3/+qebNmxOKAQCAJRCITbAHxrvvvluhoaG51oeHhysgIECStHr16kzn33jjDQ0cOFCnTp3Shg0bdOzYMX355ZdycXHRRx99pD/++CND/c6dO9W5c2dduXJFI0eOVEJCgrZt26aDBw9q8+bNqlatmv744w+98847WfZz7Ngxffrpp5ozZ46OHTumTZs26dSpU3riiSeUnJysV199NdfvNGHCBEVEROTps3PnzlzvBwAAkN/cCrqBwuzo0aOSpEqVKuWp3jAMVaxYUVu2bHFce72qVavq888/l2EYjvoBAwZo7dq1mjp1qj788EM9+OCDjvrhw4crKSlJb731loYOHZrhXvfdd5++++47hYeH6/PPP9e///1veXl5ZahJTU3Ve++9pw4dOjiOeXt7a9y4cZo3b55Wr16t+Ph4BQcHZ/udTpw4wZZyAACgUCMQm3Dx4kVJkp+fX56v8fX1lXRt/fE/Pf/8844w/M/jU6dO1fLly5WUlCQfHx8lJyfrl19+kSQ988wzWY5Vq1YtlS9fXgcPHlRMTIwaNmyYqaZ///6ZjoWFhal8+fLas2ePDhw4kOPa6BIlSig8PDzb89fbuXOnLl++nKdaAACA/EIgNsHf31+SlJiYmOdrLl26JEmOpRPXu/vuu7O8xn48LS1N+/btU82aNbV3715dvnxZLi4u6ty5c7bjnTlzRpKynJEOCQlRUFBQlteFhYVpz549uX63/v37ZxmqsxIREcFsMgAAuO0QiE0oXbq0JGn//v15qrfZbDpw4ECGa6+X3TpkHx8f+fn5KTEx0TErnZCQIElKT0/P08NvSUlJmY7ZZ6uz4uLi4ugZAADgTsZDdSZERkZKknbs2KG4uLhc62NiYhxLJbJavpDdPZKSkhwztfZZafsyDR8fH9lstlw/vXr1uuHvBwAAYAUEYhPatGmjgIAApaWl6csvv8y1fty4cZKuPYRXt27dTOd37NiR5XX23RlcXV0dD/Dddddd8vDwUFJSkg4ePHizXwEAAMDyCMQmBAQE6IUXXpAkffjhh9q0aVO2tb/++qvjDXVvvPGGY0nC9b744ossr7UH6caNGzuWOfj4+Kht27aSpE8++eTmvwQAAIDFEYhNevPNNxUZGakrV66oRYsWmjRpkq5eveo4f+nSJX3yySd64oknZLPZ1KlTJ/Xu3TvLe+3atUsvvPCC43qbzaavv/5a06ZNkyQNGzYsQ/17770nX19fffHFF3r11VcVHx+f4XxSUpJ+/PFHPf300878ygAAAHcUArFJ7u7uWrhwoR577DElJCSob9++CgkJUXh4uO677z4VK1ZMr7zyipKTkzVgwAB9++232d7rP//5j8aNG6ewsDDVq1dPpUqV0jPPPKP09HS98soratWqVYb6u+++Wz/++KOCgoI0atQohYaG6p577lH9+vVVtWpVBQQE6PHHH9eiRYtu9Z8BAACg0GKXCSfw9fXV3LlztWTJEk2dOlUrVqzQrl275OLiotKlS6tZs2Z6+umnVadOnRzv07FjR9WpU0fvv/++Nm7cqKtXr6pevXp64YUX1KVLlyyvadmypXbt2qXPP/9c8+fP1759+3T58mUFBQUpMjJSrVu31uOPP34rvjYAAMAdwbCxr1aBs7+M4+DBgypfvnzBNnML2fchrlmmvBb/692CbgcAABSgYgO73fIx7NkjPDw8x2e9WDIBAAAASyMQAwAAwNIIxAAAALA0AjEAAAAsjV0mbgM81wgAAFBwmCEGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGluBd0ArMetWBEVG9itoNsAAACQxAwxAAAALI5ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEtzK+gGYD1XTu/Xri8eLeg2AOCWqvbczwXdAoA8YoYYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAtYVFSUDMPQ8OHDC7oVAAAASyIQAwAAwNIIxAAAALA0AjEAAAAsjUBcCP30009q1aqVQkJC5OnpqTJlyqhbt27atm1bptrXXntNhmHo3//+d6Zz/fr1k2EYcnFx0blz5zKcO3funFxcXOTt7a2rV6/esu8CAABQ0AjEhcyAAQP02GOPaeHChfL09FTNmjV18eJFffvtt6pTp46+++67DPVRUVGSpKVLl2a6V3R0tCTJZrNp2bJlGc4tW7ZMNptN9evXl6en5y35LgAAALcDt4JuAHk3ceJETZgwQR4eHpo8ebK6dOkiSbp69aqGDh2qzz//XH369FGtWrVUvXp1SVKjRo3k5uam9evX69KlS/L19ZUkxcbG6sCBAypVqpSOHTum6OhoPfbYY46x7AHaHqizM2HCBH311Vd56n/nzp03+I0BAABuPQJxIWGz2fT+++9LkoYMGeIIw5Lk6empzz77TGvWrNHGjRs1atQoTZo0SZLk5+enOnXqaO3atVq1apUefPBBSf+bHX7hhRc0YsQIx+929kDcrFmzHPs6ceKEYmJinPIdAQAACgKBuJDYvXu3Dh48KEl68cUXs6x5+eWX1aVLFy1YsCDD8aioKK1du1ZLly51BGJ74H3wwQe1ePFiLVq0SKdPn1axYsV09uxZ/fXXX/L29tb999+fY18lSpRQeHh4nr7Dzp07dfny5TzVAgAA5BcCcSGxe/duSVKxYsUUGhqaZc29994r6dqs7YULFxQQECDpWiD+8MMPM8wCR0dHq2jRoqpZs6aaNWumP/74Q0uXLlXHjh1vaP1w//791b9//zx9h4iICGaTAQDAbYeH6gqJixcvSpLCwsKyrSlevHimeunaOmJ3d3dt3LhRiYmJOnjwoA4fPqymTZvKMAzHOmF7YM7rcgkAAIA7AYG4kPD395cknTp1KtuakydPZqqXJF9fX9WpU0epqalauXJlpsBbp04d+fn5OQKx/b+5PVAHAABwJyAQFxJVq1aVJJ0+fTrbULx9+3ZJ19b12pdL2F0/C2wPvPZA7ObmpkaNGmnXrl3666+/9Pfff+dp/TAAAMCdgEBcSFStWlUVKlSQJI0dOzbLmtGjR0uSWrdunemcPfxGR0dr6dKlCg0NVY0aNTKdHzFihGw2mxo2bCgPDw+nfgcAAIDbEYG4kDAMQ6+//rok6ZNPPtH333/vOHf16lW99NJL2rBhg7y8vDR06NBM10dGRjrWER85ciTTcgj773Pnzs3wOwAAwJ2OQHybGDlypEJCQrL9DBo0SP369dMzzzyj5ORkPfXUUypTpozq1aunsLAwjRkzRm5ubvrmm29UrVq1TPf38fFR3bp1ZbPZJGV+YC4iIkIBAQGO8wRiAABgFQTi28Tly5d19uzZbD/2XSMmTJiguXPnqmXLlkpKStKWLVvk5+enrl27atOmTRle2PFP14fgfwZeV1dXNW7cWNK18FyvXj3nf0kAAIDbkGGzTwkCt5h9H+K7ywTqh2FNC7odALilqj33c0G3AFiePXuEh4dr06ZN2dYxQwwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACzNraAbgPV4Faukas/9XNBtAAAASGKGGAAAABZHIAYAAIClEYgBAABgaQRiAAAAWBqBGAAAAJZGIAYAAIClEYgBAABgaQRiAAAAWBqBGAAAAJZGIAYAAIClEYgBAABgaQRiAAAAWBqBGAAAAJZGIAYAAIClEYgBAABgaQRiAAAAWBqBGAAAAJZGIAYAAIClEYgBAABgaQRiAAAAWBqBGAAAAJZGIAYAAICluRV0A7Ce+LP7NGdyq4JuAwAcnui9oKBbAFCAmCEGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIncQwDBmGUdBtAAAA4AbdkYG4V69eMgxDrVu3zlP9qVOn5O7uLsMwtHjx4lvcHQAAAG4nd2Qg7tmzpyRp0aJFOnnyZK713377rVJTU1WmTBk1b978VrcHAACA28gdGYijoqJUvnx5paWl6dtvv821furUqZKkHj16yMXljvyTAAAAIBt3ZPozDEM9evSQJE2bNi3H2i1btmjbtm2S/jezDAAAAOu4IwOxdC3cGoahbdu2acuWLdnW2QNzw4YNddddd0m6tqZ4yJAhqlatmry9vRUQEKD7779fY8eOVXJy8g31Ub58eRmGoUOHDmV5fsqUKTIMQ7169cpw/NChQxke1Pvtt98UFRWlwMBABQcHq23bto4gL0nr1q3TI488omLFisnHx0cNGzbUkiVLcuxt9erVevLJJ1W6dGl5enoqJCREbdq00YIFC27oOwIAABRmd2wgrlixoho1aiTpf0si/ik1NdWxpMI+O7x161bVrFlTo0eP1oEDB1S9enWVLFlS69ev14svvqhmzZrpwoUL+fMl/r//+7//08MPP6y9e/eqUqVKunr1qubPn6+mTZtqz549mjt3rho3bqzVq1erXLlycnNz05o1a9SqVSstX748y3u+/fbbioyM1MyZM5WUlKQaNWrIxcVFv//+u1q3bq0RI0bk63cEAAAoKHdsIJbkmHWdMWOGUlNTM51fsGCB4uLi5OXlpc6dO+vKlSvq0KGD4uLi1Lx5cx05ckQxMTHatWuXNmzYoJIlS2r16tUaNGhQvn6PoUOHaurUqTp27JhiYmJ09OhR1a1bVwkJCXr22WfVp08fvfvuuzp16pQ2btyoU6dOqU2bNkpJSdFrr72W6X5Tp07VO++8o7CwMP344486d+6cYmJiFBcXpzlz5sjf31/Dhw/XokWLcu1twoQJioiIyNNn586dt+LPAwAAYModHYg7duwoHx8fxcXFZbkMwD5z3L59ewUGBmrmzJnav3+//Pz8NHv2bIWFhTlq69SpowkTJkiS/vvf/2a7BOJW6Nu3r2NNtCQVKVJE77zzjiTpzz//VJMmTTRs2DC5urpKkry9vfXxxx9LktasWaP4+HjHtSkpKfr3v/8tSfruu+/Uvn37DGN16NBB7733niRp1KhRufZ24sQJxcTE5Olz+fLlm/8jAAAA3CJ3dCD29/dXhw4dJGV+uC4+Pl6//PKLpP/NJP/++++SpO7du6tIkSKZ7vfwww+ratWqSk9P1x9//HELO8+oX79+mY6Fh4fneL569ery9vaWJB04cMBxfO3atTp69KgqV66sZs2aZTmePSSvXLlSaWlpOfZWokQJhYeH5+lj7wcAAOB24lbQDdxqPXv21PTp0zVv3jwlJCQoKChIkjRz5kxdvXpVJUuWVIsWLSRJu3fvliTdc8892d7vnnvu0e7du7Vr165b3rtdpUqVMh0rVqxYjuftNbGxsUpMTHQcsz+Id/bsWcca63+y2WySpMuXL+vs2bMKDQ3Ntrf+/furf//+uX8JSREREYqJiclTLQAAQH654wNx8+bNVbZsWcXGxmrmzJmO8GZfLtG9e3fHUoOLFy9KUoalEv9UvHjxDLX5wdfXN9Ox618TndX562vsAVeSEhISJF2bIV+1alWuYyclJd1IqwAAAIXOHb1kQsq4J7E9BO/Zs0dr166VlHHvYX9/f0nXtl3Ljv3Nd/bavIwvZQyl17t06VKe7uMsfn5+kqQ2bdrIZrPl+ilfvny+9gcAAJDf7vhALP0v9K5Zs0Z79+51rCeuV6+eqlev7qirWrWqJGn79u3Z3st+rlq1anka2z57m13I3rNnT57u4yz25SB///13vo4LAABwu7JEIK5cubIiIyMlXZslnj59uqTMb6Zr3bq1JGn69Ok6d+5cpvvMnz9fu3fvlouLix588ME8jW1/2ceaNWsynUtISNB3332X9y/iBI0aNVLx4sV1+PBhzZkzJ1/HBgAAuB1ZIhBL/9tJYvTo0YqNjZWnp6eeeuqpDDWdO3dWpUqVlJiYqM6dOysuLs5xLiYmxrH+uHv37nleSvDwww9LuraF2datWx3HT548qS5dujjW9OYXT09Pffjhh5Kk3r17a9KkSUpJSclQc+rUKY0fP95RBwAAcCezTCDu1KmTfHx8HHvhtmvXTsHBwRlqvLy89MMPPyg0NFSLFy9WmTJlFB4erurVqysiIkJHjx5VgwYN9Nlnn+V53B49eqhOnTo6ceKEwsPDVa1aNdWqVUtlypTRX3/9pbfeesup3zMvevbsqQ8//FBJSUnq27evihQpooiICNWrV09ly5ZV8eLFNXDgwHzdSQMAAKCgWCYQBwQE6LHHHnP8bp8x/qf77rtP27Zt08svv6xy5cppx44djjfDffrpp4qOjlZAQECex3V3d9eiRYs0ePBglSpVSgcOHNCZM2fUt29fxcTEqHTp0ma/2k0ZNmyYNm/erH79+iksLEw7duxwzGC3a9dOEydOdLzcAwAA4E5m2LLb/gBwMvs+xBXKBWjk2w0Kuh0AcHiid+a3mQIo/OzZIzw8XJs2bcq2zjIzxAAAAEBWCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALM2toBuA9QQXrawnei8o6DYAAAAkMUMMAAAAiyMQAwAAwNIIxAAAALA0AjEAAAAsjUAMAAAASyMQAwAAwNIIxAAAALA0AjEAAAAsjUAMAAAASyMQAwAAwNIIxAAAALA0AjEAAAAsjUAMAAAASyMQAwAAwNIIxAAAALA0AjEAAAAsjUAMAAAASyMQAwAAwNIIxAAAALA0AjEAAAAsjUAMAAAASyMQAwAAwNLcCroBWM+J+L0aPuuhgm4DhdDwTgsLugUAwB2IGWIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKW5mbm4YsWKN1Tv5eWloKAg1ahRQ61atVL79u3l6upqpgUAAADAFFOB+NChQ46fDcOQzWbLsu6f59atW6dJkyapevXqmj17tqpXr26mDQAAAOCmmVoyMXnyZH366acKDg6WzWZT48aN9eabb2r8+PEaP3683nzzTTVu3Fg2m01FihTRmDFjNHr0aD311FPy9PTUjh071KpVKyUkJDjp6xQ+w4cPl2EYGj58uNPuGRUVJcMwNGXKFKfdEwAA4E5laob4iSeeUL169eTi4qIlS5YoKioqy7rly5friSee0KRJk7RmzRq9+OKL2rNnj5o3b66jR4/qiy++0BtvvGGmlTw7cuSIypYtK0navn27atSokanmzJkzCg0Nlc1m08MPP6xffvkly3sNHTpUH3/8sZo2baqlS5feyrbzRUJCgsaMGSNJTg3oAAAAtzNTM8QffPCBdu3apQkTJmQbhiWpSZMmGj9+vLZt26aPPvpIklSlShV98sknstls2QbOW6FMmTKqUKGCpGtBPSvLly93LPFYtWqV0tPTs6xbtmyZJOX43XMTEhKiqlWrKiQk5Kbv4SwJCQkaMWKERowYUdCtAAAA5BtTgXjOnDny8PBQ+/btc6199NFH5enpqVmzZjmOtWvXTi4uLtqzZ4+ZNm5Y06ZNJeUciKVrDw3Gx8frr7/+ylSTmJiozZs3Z7jfzXj++ee1a9cuPf/88zd9DwAAANw8U4E4NjZW3t7ecnHJ/Taurq7y9vbW4cOHHcd8fHwUFBSkS5cumWnjhuUlELu5uenll1/Otm716tVKTU2Vh4eH6tevf+uaBQAAwC1lKhD7+vrq/Pnz2r9/f661+/btU0JCgry9vR3HbDabLl68qCJFiphp44bZA/Hx48e1b9++DOfOnz+vrVu3Kjw8XK1atZL0v6UR17Mfq1evXobvtHr1aj355JMqXbq0PD09FRISojZt2mjBggVZ9pLbQ3Xz589XVFSUAgICFBgYqCZNmmjevHmSru3eYRhGjt/11KlTGjhwoKOfihUr6vXXX9eVK1cy1PXq1cuxlOT6e9s/1+8oAgAAcCcxFYjtM6PPP/+8UlJSsq1LTU3VoEGDZBiGGjRo4Dh+9OhRpaSkqESJEmbauGEVKlRQ6dKlJWWe/V25cqXS09PVpEkTVapUSSVKlNCKFSsy3cN+3fXLJd5++21FRkZq5syZSkpKUo0aNeTi4qLff/9drVu3vuG1uWPHjlXbtm21bNkyeXp6qmrVqtq9e7ceffRRff7557lef+TIEYWHh+ubb75RsWLFVLp0aR06dEgffPCBOnbsmKG2SpUqqlOnjuP3yMjIDB8vL68b6h0AAKCwMBWIhw4dKkn6448/FB4erunTp+vw4cNKSUlRamqqYmNjNW3aNEVEROiPP/6QJL366quO6+0znQ0bNjTTxk3JbtmE/ffGjRs7/hsXF6ddu3Y5aq5cuaINGzZkuM/UqVP1zjvvKCwsTD/++KPOnTunmJgYxcXFac6cOfL399fw4cO1aNGiPPW3ZcsWDRkyRJL07rvv6uTJk1q/fr1OnDihkSNH6pVXXsn1Hu+++67Cw8N17Ngxbd68Wfv371d0dLR8fX3166+/Zujl9ddf1+zZsx2/r1y5MsOnePHiWY4xYcIERURE5Omzc+fOPH13AACA/GRq27UmTZpozJgxeumll/T333+rV69eWdbZbDYZhqHRo0c7gqYkxcXF6dFHH1Xnzp3NtHFTmjZtqm+//TbTcohly5bJMAw1atRI0rVAPGvWLC1btkzVqlWTJK1du1ZXr16Vm5ubGjZsqJSUFP373/+WJH333Xdq1qxZhnt26NBBx44d0wsvvKBRo0apZcuWufY3evRopaWl6bHHHnPcW5JcXFw0dOhQbdmyRTNmzMjxHsHBwZoxY4b8/f0zfO++ffvqs88+02+//ZanXnJy4sQJxcTEmLoHAABAQTI1QyxJgwYN0vLly9W8eXNJ18Lv9R9Jat68uZYvX64XXnghw7UjRozQjz/+mCEk5xf7zO6hQ4d09OhRSVJSUpJiYmJUo0YNx7pme2/XzyTbf65Tp458fX21du1aHT16VJUrV84Uhu3sO3GsXLlSaWlpufa3cOFCSVLfvn2zPN+vX79c7/HUU09lCMN29qUueVn7nZsSJUooPDw8T5/r11oDAADcLkzNENs1bNhQixcvVnx8vDZv3qzTp09LkooVK6batWsrODjYGcM4VZUqVVS8eHGdPHlSy5YtU9euXbV69WqlpKSoSZMmjrp7771XgYGBWQZie6jetm2bJOns2bOOmeV/sv/j4PLlyzp79qxCQ0Oz7S0hIUFxcXGSpPvuuy/Lmlq1auX6He+6664sj4eFhUm6tnWcWf3791f//v3zVBsREcFsMgAAuO04JRDbBQcHO2aKC4OmTZtq5syZjkBsXz5xfSB2cXFRZGSk5s+frwMHDqhMmTJas2aN43pJjldPx8fHa9WqVbmOm5SUlOP564NqVjO8OR2/nq+vb5bH7dvk2UM6AACAlZleMlGY2YOvfcb3nw/U2V2/bGLjxo1KSkqSq6urIiMjJUl+fn6SpDZt2mRaMpLVp3z58jn2Zb+fJF28eDHLmuyOAwAA4MY4dYb4ypUrio+Pz3ELNkkqW7asM4e9afYZ3t27dys2Nlbr169XpUqVVLJkyQx11wfiU6dOSZJq166tgIAASdI999wjSfr777+d0ldQUJBCQ0MVFxenbdu2ObaIu97WrVudMtb1ctvTGAAA4E5keoY4KSlJw4cPV9WqVeXr66vSpUurQoUK2X4qVqzojL6d4u6771ZISIgkadSoUbpy5UqWD/jVrVtXXl5eWrZsmWNZxfX7Dzdq1EjFixfX4cOHNWfOHKf09tBDD0mSJk2alOX5b775xinjXO/6h94uX77s9PsDAADcjkwF4oSEBNWvX1/vvvuu9u7dm6flAunp6c7q3TTDMBzLJiZOnCgp4/phOw8PD9WrV08HDhxQdHS0pIyB2NPTUx9++KEkqXfv3po0aVKmWfJTp05p/PjxjrrcvPzyy3J1ddUPP/ygDz74wLEzRXp6uj799FPNmjXrBr9t7ooVK+ZYm5zV2/kAAADuRKYC8bvvvqvt27fLzc1NL7/8spYtW6a9e/fq4MGDOX5uJ/Zga3+VcXZbwNmPX7lyRS4uLpnqevbsqQ8//FBJSUnq27evihQpooiICNWrV09ly5ZV8eLFNXDgwAwv+MhJrVq19PHHH0u69tKMEiVK6P7771eJEiX08ssva+TIkZL+94CcMxiGoSeffFKS9PDDDys8PFxRUVGKiorSyZMnnTYOAADA7cTUGuKffvpJhmFozJgxGjhwoLN6ylfXz/SWKFFClStXzrLu+gBcs2ZNBQUFZaoZNmyYWrdurc8//1zR0dHasWOH0tPTFRYWpnbt2unRRx/Vo48+mufeXnzxRVWpUkUfffSRYmJitHPnTtWsWVPjx4/Xgw8+qBdffNGxjtlZPv30U/n7++vnn3/W33//reTkZEn/+wcDAADAncawmdh7y8vLS2lpabp48aK8vLyc2RdysXHjRtWtW1e1atXS5s2bC7qdPLHvQ1yigr+e+bBBQbeDQmh4p4UF3QIAoBCxZ4/w8HBt2rQp2zpTM8RFihTRlStXCMMFwP6wXUG85Q8AAOBOYmoBaqNGjXT+/HkdO3bMWf3gOtOnT9eiRYsyPIh49epVffTRRxo/frzc3Nz0zDPPFGCHAAAAhZ+pQDxs2DC5ubnp3XffdVY/uM66dev04IMPKjg4WHXq1FFERISKFi2q1157TTabTaNGjXLsgQwAAICbY2rJREREhKZMmaI+ffooJSVFb7zxxm21z3Bh16VLFyUmJmr16tXau3evLl++rJCQED300EMaPHhwhgcCAQAAcHNMBWJ7+HV1ddWUKVM0ZcoUFSlSxLGXbVYMw9D+/fvNDGsZDRs2VMOGDQu6DQAAgDuaqUB86NChTMfOnj2rs2fPZnsNrwcGAADA7cRUIJ48ebKz+gAAAAAKhKlA3LNnT2f1AQAAABQI5733FwAAACiECMQAAACwNAIxAAAALC3PgdjV1VWurq6qUaNGpmM38nFzM7VsGQAAAHCqPKdTm82W4b///BkAAAAojPIciKOjoyVJPj4+mY4BAAAAhVWeA3FWrwnm1cEAAAAo7HioDgAAAJZGIAYAAIClOW3Lh/T0dO3du1fnzp1TSkpKjrVNmjRx1rAAAACAKaYD8YkTJ/Svf/1Lc+bM0eXLl3OtNwxDqampZocFAAAAnMJUID5+/Ljuv/9+HT9+PM9bsLFVGwAAAG4nptYQDx8+XMeOHZOfn58+++wzHT58WCkpKUpPT8/xAwAAANwuTM0Q//777zIMQ998842eeOIJZ/UEAAAA5BtTM8SnT5+Wm5ub2rdv76R2AAAAgPxlKhCHhobK29tbbm5O26wCAAAAyFemAnGLFi108eJF7d2711n9AAAAAPnKVCB+/fXX5evrq2HDhjmrHwAAACBfmQrElStX1rx587Rs2TK1bNlS0dHRunTpkrN6AwAAAG45U4t/XV1dHT8vWbJES5YsyfUaXswBAACA24mpQMxLNgAAAFDYmQrE0dHRzuoDAAAAKBCmAnHTpk2d1QcAAABQIEw9VAcAAAAUdgRiAAAAWFqel0zExsZKktzd3VWiRIkMx25U2bJlb+o6AAAAwNnyHIgrVKggSapWrZr+/vvvDMduBNuuAQAA4HaS50Bs32Lt+q3WbmbbNbZqAwAAwO0kz4H44MGDkq4tmfjnMeBGlAi+S8M7LSzoNgAAACTdQCAuV65cno4BAAAAhQm7TAAAAMDSCMQAAACwNFNvqvunuLg4HT16VJcuXcrx4bkmTZo4c1gAAADgpjklEI8bN06fffaZ9u/fn2st264BAADgdmI6ED/55JOaPXt2nrdTY9s1AAAA3E5MrSH+/vvvNWvWLAUEBGjOnDm6dOmSJKl48eJKTU3V0aNHNXnyZFWuXFkhISH6888/lZ6e7pTGAQAAAGcwFYinTJkiwzD07rvv6vHHH5e3t/f/buziopIlS6pnz56KiYlRmTJl1L59e+3bt8900wAAAICzmArEmzdvliR169Ytw/F/zgL7+flp3Lhxunjxoj766CMzQwIAAABOZSoQJyQkyN/fX0FBQY5j7u7ujqUT12vQoIF8fHy0ePFiM0MCAAAATmUqEBctWlSGYWQ4FhQUpKSkJCUkJGR5zcmTJ80MCQAAADiVqUBcqlQpXbhwQYmJiY5j1atXlyRFR0dnqI2JiVFSUpJ8fHzMDAkAAAA4lalAHB4eLknasGGD41jbtm1ls9n0yiuvaMOGDUpJSdHGjRvVs2dPGYahyMhIcx0DAAAATmQqENvD7+zZsx3HBg4cqFKlSungwYOqX7++vLy8dP/99+vvv/+Wm5ub3njjDdNNAwAAAM5iKhC3adNG0dHR6t27t+OYn5+flixZogYNGshmszk+ZcuW1dy5c3X//febbhoAAABwFlNvqnNzc1PTpk0zHb/rrru0atUqHT16VEeOHFFgYKCqV6+e6QE8AAAAoKCZCsSxsbGSpNDQUHl5eWU6X7p0aZUuXdrMEAAAAMAtZWrJRPny5VWxYkWdO3fOWf0AAAAA+crUDLGfn5/c3d1VsmRJZ/UDAAAA5CvTM8RJSUlKS0tzVj8AAABAvjIViNu3b6/k5GTNnz/fWf0AAAAA+cpUIB42bJgqV66sAQMGaNu2bc7qCQAAAMg3ptYQ//DDD+rfv7+GDx+uOnXqqFWrVoqMjFRoaKhcXV2zva5Hjx5mhkUhty/hpNr8+F5Bt4F8NP+xfxd0CwAAZMtUIO7Vq5djb2GbzabffvtNv/32W47XGIZBIAYAAMBtw1QgLlu2LC/bAAAAQKFmKhAfOnTISW0AAAAABcPUQ3UAAABAYWcqEC9fvlxr167Nc/369eu1fPlyM0MCAAAATmVqyURUVJRKlCihY8eO5am+c+fOOnLkiFJTU80MCwAAADiN6SUTNpvtltYDAAAAt1K+riG+dOmS3N3d83NIAAAAIEf5Foh3796tM2fOKDQ0NL+GBAAAAHJ1Q2uIf/75Z/38888Zjp0/f159+vTJ9hqbzaaEhAStWLFChmGocePGN9cpAAAAcAvcUCDesmWLpkyZIsMwHGuBL1++rClTpuTp+mLFiuntt9++4SYBAACAW+WGAnGtWrXUs2dPx+9Tp06Vt7e3OnXqlO01Li4uCggI0D333KMOHTooKCjoppsFAAAAnO2GAvGjjz6qRx991PH71KlTFRgYqMmTJzu9MQAAACA/mNqHODo6Wh4eHqabmD17ti5fvqwePXqYvhcAAABwI0wF4qZNmzqlicGDB+v06dMEYgAAAOS7fN2HOCe8sAMAAAAF4bYJxAAAAEBBIBADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0tzMXPzOO+9Iknr37q0yZcrc9H06deqkCxcumGkFAAAAuCmmAvGIESPk6uqqf/3rX6aaGDt2rKnrAQAAgJtlKhCHhIQoLS1N7u7uzuoHAAAAyFem1hDfd999SkhI0NmzZ53VDwAAAJCvTAXi/v37Kz09XaNHj3ZWPwAAAEC+MhWIO3TooJdfflkffvihXn31VZ05c8ZZfQEAAAD5wtQa4ubNm0uSfH199cknn+jTTz9V5cqVFRoaKldX1yyvMQxDf/75p5lhAQAAAKcxFYiXLl2a4fe0tDTt3r1bu3fvzvYawzDMDAkAAAA4lalA/PbbbzurDwAAAKBAEIgBAABgaby6GQAAAJZGIL7N9erVS4ZhaMqUKU67Z/ny5WUYRqY14AAAAFbktEC8efNmDR06VM2aNVONGjVUo0YNNWvWTK+++qo2b97srGEKnaioKBmGoeHDh+dYt3TpUhmGwUOHAAAA+czUGmJJunTpkp5++mnNnDlTkmSz2Rzndu7cqeXLl+uTTz7Rk08+qa+++kq+vr5mh7SUEiVKqGrVqgoMDCzoVgAAAO5IpgJxenq6Hn30UUVHR8tms6lEiRJq3ry5SpcuLUk6evSooqOjdfz4cX3//feKi4vTH3/8wSzoDfjggw/0wQcfFHQbAAAAdyxTgXjatGlasmSJ3N3d9cknn+jZZ5+Vi0vGVRjp6ekaP368XnrpJS1ZskTTp09Xjx49TDUNAAAAOIupNcT//e9/ZRiGRo0apeeffz5TGJYkFxcXPfvssxo1apRsNpumTZtmZkjLye2huunTp6tevXry9fVVkSJF1KpVK61cuVKHDh2SYRgqX758jvfft2+funbtquLFi8vLy0vVq1fXqFGjlJ6e7vwvAwAAcBsyFYi3bt0qV1dXPf3007nWPv3003Jzc9OWLVvMDInrvPTSS+rRo4c2bNigoKAgVapUSevWrVNUVJR+/PHHXK/fvHmzIiIi9OOPP6p06dIqVqyYdu3apVdffVWDBw/Oh28AAABQ8Ewtmbh48aL8/f3l7e2da623t7f8/f2VmJhoZkj8f7/++qvGjBkjV1dXffXVV+rdu7cMw1BycrJeffVVDRs2LNd7DBs2TD179tSnn34qPz8/SdKsWbP05JNP6v/+7/80ePBgValSJcd7TJgwQV999VWeet65c2ee6gAAAPKTqUAcEhKikydPKi4uTqGhoTnWxsXFKSEhQcWLFzczZKE1YsQIjRgxwmn3GzlypCRp8ODB6tOnj+O4h4eHxowZow0bNmj16tU53qNKlSoaP368XF1dHcc6deqkb7/9VvPmzdP8+fNzDcQnTpxQTEyMiW8CAABQsEwF4gYNGmju3LkaPny4/u///i/H2rfffls2m02RkZFmhiy0ypQpo7Jly2Z7/vz589q+fXue7pWYmKhVq1ZJkvr27ZtlTb9+/XINxP369csQhu3q16+vefPmaf/+/bn2UqJECYWHh+eh62szxJcvX85TLQAAQH4xFYife+45/fDDD5owYYIuXryot99+W5UrV85Qs2/fPg0fPlwzZsyQYRh67rnnTDVcWPXp0yfHl3MsXbpUzZo1y9O99u3bp/T0dLm7u6t69epZ1tSqVSvX+9x1111ZHg8LC5OkPC1v6d+/v/r3759rnSRFREQwmwwAAG47pgJxVFSUXnzxRY0ZM0YzZszQjBkzVKZMGZUqVUrStX2Ijx496qh/6aWX1LRpU3MdwxFUfX19s9zZQ5L8/f1zvU92L0mx3/P6l6wAAADcqUy/qW706NGqWLGihg8frnPnzik2NlaxsbEZaooWLarhw4dbdnbY2ewPwCUlJSk9PT3LUHzx4sX8bgsAAKBQMh2IJen5559Xv379tGjRIm3cuFFxcXGSpNDQUNWpU0ctW7aUl5eXM4aCpMqVK8vFxUXJycnatWuX7r777kw1W7duLYDOAAAACp88B+LPPvtMvr6+2T7E5eXlpXbt2qldu3ZOaw5Z8/PzU2RkpFasWKFJkybp448/zlQzadKkAugMAACg8MnzizlefPFFvfXWWxmOVaxYUfXr13d6U8jdq6++KkkaO3aspk6d6jienJysIUOGaO3atQXVGgAAQKFyQ2+q++frfA8dOpRpvTDyx8MPP6wXXnhBqamp6tWrl0qXLq169eqpePHiGjt2rD788ENJynJbNQAAAPxPngOxv7+/zp07p7S0tFvZD27AmDFjNHXqVNWpU0fnzp3T3r17VadOHf3555966KGHJEkBAQEF3CUAAMDtLc9riGvUqKF169Zp6NCh6tevn2Ong7S0NB05cuSGtujK6QUVd5qlS5fmqS4qKirLv+GUKVM0ZcqUbK/r0aOHevToken4nDlzJEnly5fPdO7QoUM59tKrVy/16tUrxxoAAIA7RZ4D8dNPP621a9dq7NixGjt2rOP4mTNnsgxd2TEMQ6mpqTfUJG6c/aG6xo0bF3AnAAAAt7c8L5no3bu3Ro0apdDQUNlsNsdspv3nvH7+uQ4ZN2/cuHGZHp67ePGiXnrpJf3+++8KCAhQt27dCqg7AACAwuGG9iEeMmSIhgwZojNnzujSpUuqUKGCihUrpvXr19+q/pCDBQsWaNCgQSpatKgqVqzo2Jf46tWrcnd31zfffKPQ0NCCbhMAAOC2dlMv5ggJCVFISIika7sYlCtXzqlNIW+effZZ+fr6asOGDdq5c6eSk5MVFhampk2b6pVXXtF9991X0C0CAADc9ky9qS46OloeHh6mm5g9e7YuX76c5cNhyF6bNm3Upk2bgm4DAACgUDMViJs2beqUJgYPHqzTp08TiAEAAJDvbujFHLfSjWzbBgAAADjLbROIAQAAgIJAIAYAAIClEYgBAABgaQRiAAAAWBqBGAAAAJZGIAYAAIClEYgBAABgaQRiAAAAWBqBGAAAAJZGIAYAAICluRV0A5LUqVMnXbhwoaDbAAAAgAXdFoF47NixBd0CAAAALCrPgXjatGlOG7RHjx5OuxcAAABgRp4Dca9evWQYhukBDcMgEAMAAOC2kedAXLZs2WwD8enTp5WUlHTthm5uKlq0qCTp7NmzSk1NlST5+voqJCTEbL8AAACAU+V5l4lDhw7p4MGDmT7Dhg1TSkqKGjVqpIULF+rixYs6ceKETpw4ocTERC1cuFCNGzdWSkqKhg0bpoMHD97K7wMAAADcEFPbri1ZskTPP/+82rVrp6VLl6ply5by9PR0nPfw8FDLli21dOlStWvXTs8//7yWLl1qtmcAAADAaUwF4k8++UQ2m02ffvqpXFyyv5VhGPrkk0+Unp6ujz/+2MyQAAAAgFOZCsQbN25UUFCQypQpk2tt2bJlFRQUpA0bNpgZEgAAAHAqU/sQX7x4UWlpaUpOTpaHh0eOtcnJybp06ZJcXV3NDAkAAAA4lakZ4goVKig1NTVPexRPmzZNKSkpqlChgpkhAQAAAKcyFYifeuop2Ww2DR48WFOnTs22btq0aRo8eLAMw9BTTz1lZkgAAADAqUwtmXjllVc0d+5cbdmyRX369NHbb7+tqKgolSpVSpJ07NgxLVu2TLGxsbLZbKpVq5ZeeeUVpzQOAAAAOIOpQOzl5aU///xTffv21U8//aTY2FhNnz49Q43NZpMkPfLII5o0aZK8vLzMDAkAAAA4lalALEnBwcGaO3euNmzYoO+//14bN25UXFycJCk0NFR16tRR586dVa9ePdPNAgAAAM5mOhDb1a1bV3Xr1nXW7QAAAIB8YeqhOgAAAKCwc9oMcXp6ujZt2qTDhw8rKSlJPXr0cNatAQAAgFvGKYH4888/13vvvaczZ844jl0fiOPj49W4cWOlpqZq2bJlCgsLc8awKKQqBxXX/Mf+XdBtAAAASHLCkonnnntOL774ok6fPi1/f38ZhpGpJjg4WOHh4dq7d69mz55tdkgAAADAaUwF4gULFujLL7+Un5+ffvzxRyUkJKhYsWJZ1nbp0kU2m02LFy82MyQAAADgVKYC8fjx42UYht555x09+uijOdY2aNBAkvTXX3+ZGRIAAABwKlOBeN26dZKkPn365FobGBiogIAAnTx50syQAAAAgFOZCsTnzp1TYGCg/P398zaYi4vS09PNDAkAAAA4lalAHBAQoAsXLiglJSXX2nPnzun8+fMKCQkxMyQAAADgVKYC8b333iubzeZYOpGT7777TjabTXXq1DEzJAAAAOBUpgLxE088IZvNpuHDh+e4FGLr1q3697//LcMw9NRTT5kZEgAAAHAqU4H46aef1t13363o6Gi1bNlSv/76q9LS0iRJe/fu1aJFizR48GA1bNhQ58+fV/369dWxY0enNA4AAAA4g6k31bm7u+u3335Tq1atFB0draVLlzrOVatWzfGzzWbTvffeqx9++CHLF3cAAAAABcX0m+rKlSunTZs2acSIESpbtqxsNluGT8mSJTV8+HCtXr1axYsXd0bPAAAAgNOYmiG28/Hx0Ztvvqk333xTx48f1/Hjx5WWlqbixYurXLlyzhgCAAAAuCWcEoivV7JkSZUsWdLZtwUAAABuCVNLJpo3b35DD8k99dRTeuCBB8wMCQAAADiVqRnipUuX3tC64LVr1yo2NtbMkAAAAIBTmX6o7kakp6ezywQAAABuK/kWiNPS0hQXFydfX9/8GhIAAADI1Q0tmbhw4YISEhIyHEtLS9ORI0dks9myvMZmsykhIUGTJ0/W1atXVbNmzZtuFgAAAHC2GwrEn376qd55550Mx86cOaPy5cvn6XrDMNS9e/cbGRIAAAC4pW74obrrZ4INw8h2ZvifSpUqpQEDBuj555+/0SEBAACAW+aGAvGLL76oXr16SboWjCtWrKhixYpp/fr12V7j4uKigIAABQYGmmoUAAAAuBVuKBAHBgZmCLZNmjRRSEgIb6MDAABAoWV6H2LgRu2LP6eH53xb0G0gF78+0bWgWwAAIF+Y3nbtwoULSkxMzLUuMTFRFy5cMDscAAAA4FSmAvHcuXMVHBysZ555Jtfabt26KTg4WPPmzTMzJAAAAOBUpgLx7NmzJUl9+/bNtfbpp5+WzWbTrFmzzAwJAAAAOJWpQLx582a5uLgoMjIy19rmzZvLxcVFMTExZoYEAAAAnMpUID527JiCgoLk5eWVa623t7eCgoJ07NgxM0MCAAAATmVqlwnDMJSUlJTn+suXL8swDDNDAgAAAE5laoa4TJkyunLliv76669ca7du3arLly+rVKlSZoYEAAAAnMpUII6KipLNZtPbb7+da+3w4cNlGIaaNWtmZkgAAADAqUwF4kGDBsnFxUU///yzunXrplOnTmWqOXXqlLp06aKff/5ZLi4uGjx4sJkhAQAAAKcytYa4WrVq+s9//qN//etf+u677zRnzhxFREQ4XuV8+PBhbdy4UampqZKk9957T3fffbf5rgEAAAAnMRWIJWnYsGEKCAjQa6+9posXL2rNmjVau3atJMlms0mSAgICNHLkyDy9wAMAAADIT6YDsSQNHDhQTz31lObMmaPVq1fr5MmTMgxDxYsXV8OGDdWxY0cFBAQ4YygAAADAqZwSiCUpKChI/fr1U79+/Zx1SwAAAOCWM/VQHQAAAFDYOW2G+MyZM4qOjtbhw4eVlJSkt956y1m3BgAAAG4Z04E4NTVVw4YN0//93/8pOTnZcfz6QBwfH6+KFSvq8uXL2rVrl8qXL292WAAAAMApTC+Z6Nixo8aMGaPk5GTVqFFDbm6ZM3ZwcLC6dOmi5ORkzZo1y+yQAAAAgNOYCsTff/+9fv75Z4WGhmrjxo3atm2bihQpkmVtx44dJUnR0dFmhgQAAACcylQgnjx5sgzD0KhRo1S7du0ca+vVqyfDMLRjxw4zQwIAAABOZSoQb968WZLUoUOHXGt9fHwUGBiouLg4M0MCAAAATmUqEJ8/f16BgYHy9vbOU316eroMwzAzJAAAAOBUpgJxcHCwzp8/rytXruRae+LECV24cEFhYWFmhgQAAACcylQgDg8Pl5S3B+UmTZokSWrQoIGZIQEAAACnMhWIu3btKpvNpjfffFOJiYnZ1i1YsEDvvvuuDMNQz549zQwJAAAAOJWpF3N06dJFX331lVasWKH69etrwIABjpdzLFq0SIcOHdIvv/yi+fPnKz09Xe3atdNDDz3klMYBAAAAZzAViA3D0E8//aTHHntMy5cv1wsvvOA416pVK8fPNptNLVq00LfffmtmOAAAAMDpTL+pLjg4WEuWLNHUqVPVuHFjeXh4yGazyWazydXVVQ0aNNCUKVO0YMEC+fn5OaNnAAAAwGlMzRDbubi4qHv37urevbvS09N17tw5paWlqWjRolm+yhkAAAC4XTg9rbq4uCgkJMTZtwUAAABuCdNLJgAAAIDCLM8zxLGxsU4btGzZsk67FwAAAGBGngNxhQoVnDKgYRhKTU11yr0AAAAAs/IciG02m1MGdNZ9AAAAAGfIcyA+ePDgrewDAAAAKBB5DsTlypW7lX0AAAAABYJdJgAAAGBpTg3ENptNZ86cceqOFAAAAMCt5JRAHBMTo8cff1yBgYEKCwtTxYoVM5yPj49X//79NWDAAF2+fNkZQxZ6hmHIMIyCbgMAAMDyTAfi6dOnq0GDBvrpp5+UmJgom82WaSeJ4OBg7d+/X19//bV+/vlns0PelrZv366XXnpJtWrVUtGiReXh4aGQkBA1atRIb731lvbv31/QLQIAACALpgLxjh079PTTTyslJUWDBw/Wxo0bs31tc8+ePWWz2fT777+bGfK2k5ycrAEDBqhmzZoaM2aM/vrrL4WEhKh27doKCgrSmjVr9O6776patWoaOXJkQbcLAACAf8jzLhNZGT16tJKTk/Xcc89pzJgxkiRXV9csax944AFJ0qZNm8wMeVtJT0/XI488ooULF8rHx0dvv/22+vXrpyJFijhqTp8+re+++04ffPCBVq9eXYDdAgAAICumAnF0dLQMw9CwYcNyrS1ZsqS8vb115MgRM0PeVj744AMtXLhQnp6e+vPPP1W/fv1MNcWKFdPgwYPVvXt3zZw5swC6BAAAQE5MLZk4fvy4fH19Vbp06TzV+/j43DEP1V28eFEff/yxJOmNN97IMgxfLzg4WAMGDMjTva9evapPP/1U9erVU0BAgLy9vVWtWjUNHTpUp0+fzvKa+Ph4vf7667rnnnvk4+MjLy8vlS5dWo0aNdLw4cOVkJCQ6Zr09HR9++23evDBBxUSEiIPDw+VLl1aPXv21K5du/LUKwAAQGFnaobY09NTV65ckc1my3XHhKtXryohIUHBwcFmhrxtzJ8/XwkJCXJ1ddXAgQOddt+EhAQ99NBDWr9+vSSpatWq8vHx0fbt2/Xxxx/rv//9r/744w/de++9jmsuXLig+vXra8+ePXJxcVHlypUVGBioEydOaO3atVq1apXat2+vWrVqOa5JSkrSE0884VjTXbJkSdWoUUP79u3TtGnTNHv2bP3www9q3bq1074bAADA7chUIK5YsaK2bt2qPXv2qGrVqjnWLly4UGlpaapRo4aZIW8bK1eulCTVqFEj2wcJb8bzzz+v9evXq0yZMvrpp58UHh4uSTp58qQ6d+6s5cuX64knntC2bdvk6ekpSfrmm2+0Z88e1axZU7/88ovKli3ruN/58+c1e/ZsFS1aNMM4zz33nH7//XfVq1dPX3/9tWrWrClJSklJ0Xvvvad33nlHXbt21e7du1WsWLFs+50wYYK++uqrPH23nTt33tDfAgAAID+YCsRt2rTRli1bNGbMGH355ZfZ1l28eFGvvfaaDMPQI488YmbI28axY8ckKdOey2YcPHhQ3333nSRp4sSJjjAsScWLF9fs2bNVoUIF7dmzRzNnzlSPHj0kSbt375Yk9enTJ0MYlqTAwED169cvw7EdO3Zo6tSpKlasmH755ReFhoY6zrm7u2vEiBH666+/9OOPP+rrr7/W66+/nm3PJ06cUExMjLkvDgAAUIBMrSF+8cUXFRgYqK+++kpvvvlmpnWqly9f1ty5c1WvXj3t2rVLxYsX1zPPPGNmyNvGxYsXJUl+fn5Ou+fChQuVnp6uu+++Ww8++GCm86GhoerWrZskacGCBY7j9hD866+/6tKlS7mO88MPP8hms+nRRx/NEIav1759e0nS0qVLc7xXiRIlFB4enqePt7d3rr0BAADkN1MzxCEhIZo9e7YeffRRvf/++xo5cqTS09MlXVuTeubMGaWlpclms8nPz09z5syRr6+vUxovaP7+/pKkxMREp93TPtOb07IS+9rh6x9669Onjz755BMtXrxYJUuW1EMPPaRGjRqpcePGqlWrVqb13du2bZMkLV68WI0aNcpyHPs/bo4ePZpjz/3791f//v1z/mL/X0REBLPJAADgtmMqEEtSixYttHbtWr344ouKjo52HD958qTj56ioKH3++ed3zPphSSpVqpSka8scnMU+6xwWFpZtTfHixTPU2o+tW7dOb731lubNm6fZs2dr9uzZkqTy5ctr+PDh6tmzp6PeHnYPHTqkQ4cO5dhTUlLSzXwVAACAQsN0IJauzVr++eefOnz4sFatWqXjx48rLS1NxYsXV2RkpCpXruyMYW4rjRo10rhx47R9+3adOXPGKQ/W2WedT506lW2N/R8a9lq7ypUra8aMGUpJSdGmTZu0YsUK/fjjj1qzZo169eolHx8fdezYUdL/lnmMHDlSQ4cONd03AABAYeaUQGxXrlw5lStXzpm3vG21adNGgYGBOn/+vL788ku9+eabpu9p36lj+/bt2dbYz1WrVi3L8+7u7qpfv77q16+voUOH6tlnn9WXX36p8ePHOwLxPffco59++inHcQAAAKzC1EN1Vubv768hQ4ZIkv7zn/9o7dq1OdbHx8dr/PjxOdY89NBDcnFx0c6dO/XHH39kOn/mzBlNnz5dkvK8P3DDhg0lXXuJit0TTzwhSZozZ45iY2PzdB8AAIA7lalAnJycrNjY2Azrhe0SExP1yiuv6L777lPt2rX15ptv3jFvqbN7/fXX1aJFC129elUtWrTQqFGjFB8fn6Hm3Llz+uKLL1SjRo0MO0NkpUKFCnrqqackSU8//bS2bNniOHfq1Cl16tRJSUlJqlKlijp16pShj6+++kpnz57NcL/jx49r3Lhxkq490GZ33333qVevXkpKStIDDzygZcuWZepl165devvtt/XLL7/k7Y8BAABQSJlaMjFx4kQNGjRIPXv21KRJkzKca9u2rVauXCmbzSbp2s4GK1asUHR0dK5vtSssXF1d9euvv+r555/XN998o1dffVX/+te/HG+KO3funA4ePKi0tDS5u7urcePGud5z3Lhx2rt3r9avX6/atWurevXq8vLy0vbt25WSkqKwsDDNmTPH8VIO6dq+wh988IEGDBig8uXLKzQ0VBcuXNCePXuUlpam0qVL6z//+U+Gcb788ktdvHhRP/zwg6KiohQWFqZy5copJSVFhw8f1rlz5yRJkydPdu4fDQAA4DZjaoZ44cKFkqQuXbpkOD5v3jytWLFChmGoa9eu6tevn9zd3bVixQrH/+V/p/D09NTXX3+tLVu2aPDgwapRo4ZOnTqlmJgYxcfHq379+nr77be1d+9exxKLnAQFBWn58uUaPXq06tSpoyNHjmjHjh2qUKGChgwZom3btmV4bbMkvfnmm3r99dfVoEEDJScna/PmzYqNjdU999yjN954Q1u3bs20ttvLy0tz5szRvHnz9Nhjj8nV1VWbN2/Wrl27FBISop49e+qnn37Sk08+6dS/FwAAwO3GsNmncG9ClSpVtH//fp05c0bBwcGO408++aRmz56t1157zTEzOX78eD377LN68MEHc106gDuTfR/igArl1XjkewXdDnLx6xNdC7oFAABMsWeP8PBwbdq0Kds6UzPEp0+flo+PT4YwLMmxH/H1rwzu3r27JGnr1q1mhgQAAACcylQgvnTpklxcMt7i0KFDOn36tMqUKaMKFSo4jvv6+iooKMixNhUAAAC4HZgKxEWKFFFiYqLjzWeStGTJEkn/2+7reqmpqY6XQgAAAAC3A1OBODw8XJL0zTffSJLS09P1zTffyDAMNWvWLEPt6dOnlZiY6Hj1MAAAAHA7MBWIe/bsKZvNptdee02tW7dWvXr1tGbNGvn5+Tneima3YsUKSVL16tXNDAkAAAA4lalA3LlzZ/Xq1UtpaWlauHChYmJi5OXlpfHjxysoKChD7cyZM7OcOQYAAAAKkqkXc0jSpEmT1LdvX61evVpBQUF64IEHVLFixQw1ycnJCgwMVI8ePdSmTRuzQwIAAABOYzoQS1JkZKQiIyOzPe/h4aGvvvoq2/Nr165VcnKymjRp4ox2AAAAgDxzSiA267HHHtPp06eVmppa0K0AAADAYkytIXYmEy/MAwAAAG7abROIAQAAgIJAIAYAAIClEYgBAABgaQRiAAAAWBqBGAAAAJZGIAYAAIClEYgBAABgaQRiAAAAWBqBGAAAAJZGIAYAAICluRV0A5LUoEEDxcfHF3QbAAAAsKBbGohTU1P1119/ycXFRTVr1pRhGFnWzZ0791a2AQAAAGTL1JKJ3bt365133tG0adMynVu6dKnKli2rOnXqKDw8XBUqVNDq1avNDAcAAAA4nalAPG3aNI0YMUKxsbEZjsfHx6tDhw46efKkbDabbDabYmNj1bZtW508edJUwwAAAIAzmQrES5YskSR16NAhw/FvvvlG8fHxKleunBYtWqSVK1fq3nvv1YULF/TZZ5+ZGRIAAABwKlOB+NixY5KkSpUqZTj+888/yzAMffDBB3rggQfUsGFDffnll7LZbFq4cKGZIQEAAACnMhWIT58+raCgIHl4eDiOpaSkaMOGDXJzc1O7du0cxxs2bCg3Nzft27fPzJAAAACAU5kKxC4uLrp06VKGY5s3b1ZycrLuu+8++fr6ZjgXGBioq1evmhkSAAAAcCpTgbh06dJKSUnRzp07Hcd+++03SVJkZGSGWpvNpgsXLigkJMTMkAAAAIBTmQrETZs2lc1m05AhQxQXF6ctW7Zo/PjxMgxDbdq0yVC7e/dupaSkqGTJkqYaBgAAAJzJVCAeMmSIPD09tXDhQpUoUUIRERE6ffq07rvvPrVs2TJD7YIFCyRJ9erVMzMkAAAA4FSmAnHVqlU1b948VahQQTabTYZhqGXLlvr5558z1U6ePFmS1KxZMzNDAgAAAE5l+tXNLVu21L59+3T69Gn5+/vLy8srU01KSopj/+G6deuaHRIAAABwGtOB2K5YsWLZnnN3d1fTpk2dNRQAAADgNKaWTAAAAACFnVMC8dGjR/Xyyy+rRo0a8vPzk5tbxonn+Ph4vf/++/rggw+UmprqjCEBAAAApzC9ZGLRokXq1KmTLly4IJvNJkkyDCNDTXBwsH766Sdt2rRJNWrU0COPPGJ2WBRilYOL6NcnuhZ0GwAAAJJMzhAfOXJETzzxhM6fP6927dppzpw5Cg4OzrK2T58+stlsjhd3AAAAALcDU4H4k08+0cWLF9WpUyf99NNPevzxx+Xh4ZFl7UMPPSRJ2rBhg5khAQAAAKcyFYgXLlwowzD07rvv5lpboUIFeXp66uDBg2aGBAAAAJzKVCCOjY2Vt7e37rrrrjzV+/n56dKlS2aGBAAAAJzKVCB2cXFRenp6nmpTU1N14cIFBQQEmBkSAAAAcCpTgbhcuXK6evWqYmNjc61dvny5UlJS8jybDAAAAOQHU4G4RYsWkqTx48fnWJeSkqI33nhDhmGodevWZoYEAAAAnMpUIH7ppZfk4eGhTz75RN98802WNTExMWrRooXWrVsnf39/Pfvss2aGBAAAAJzK9JKJiRMnKi0tTc8884zCwsIUHx8vSWrYsKFKlSqlunXrasWKFXJzc9O0adMUEhLilMYBAAAAZzD96uauXbvq999/V6VKlXT69GklJyfLZrNp7dq1OnHihGw2mypXrqwFCxbwhjoAAADcdky/ulmSWrZsqd27d2v58uVatWqVjh8/rrS0NBUvXlyRkZFq1qyZXF1dnTEUAAAA4FSmAvG0adMkXXsLXVhYmJo2baqmTZs6pTEAAAAgP5gKxL169ZKbm5sSEhKc1A4AAACQv0wF4iJFikiSfHx8nNIMAAAAkN9MPVRXrVo1nT9/XomJic7qBwAAAMhXpgJxr169lJaWpokTJzqrHwAAACBfmVoy0a9fPy1cuFDDhg2Th4eHnnnmGbm5OWXjCgAAACBfmEqvffr0kZ+fnzw9PTVo0CC99dZbqlu3rkJDQ7PdZs0wjGzfagcAAADkN1OBeMqUKTIMQzabTZJ07tw5LVy4MMtaex2BGAAAALcTU4G4R48eMgzDWb0AAAAA+c70DDEAAABQmJnaZQIAAAAo7NgSAvluf0KiHvthZUG3ccf7sUOjgm4BAIBCgRliAAAAWJqpGeKKFSve8DWGYWj//v1mhgUAAACcxlQgPnToUJ7qrt+ajV0pAAAAcDsxFYgnT56c4/nz589r3bp1mjt3rvz8/PTOO+/Ix8fHzJAAAACAU5kKxD179sxT3c6dO9WyZUt9++23Wrp0qZkhAQAAAKfKl4fqqlevri+++EKrV6/WmDFj8mNIAAAAIE/ybZeJtm3bysPDQ9OnT8+vIQEAAIBc5VsgdnNzk6enJztMAAAA4LaSb4F4z549unjxotzd3fNrSAAAACBX+RKIjx07pl69eskwDNWpUyc/hgQAAADyxNQuE3369Mnx/JUrV3TkyBFt2LBBKSkpMgxDr7zyipkhAQAAAKcyFYinTJmS4aUbOfHz89OYMWP00EMPmRkSAAAAcCpTgbhHjx45vnnOzc1NwcHBuu+++9SuXTsFBASYGQ4AAABwOtMzxAAAAEBhlm+7TAAAAAC3I1OBuE+fPnr55ZfzXP/qq6+qb9++ZoYEAAAAnMpUIJ4yZYq+//77PNfPnj2bZRYAAAC4reTrkom87EYBAAAA5Kd8DcRnzpyRj49Pfg4JAAAA5MjULhN5df78eU2cOFFJSUmqWbNmfgwJAAAA5MkNBeIRI0bonXfeyXDs1KlTcnV1zdP1hmGoQ4cONzIkAAAAcEvd8Azx9euA8/qWOkny8PBQ9+7d9dprr93okAAAAMAtc0OBuFevXoqKipJ0LRg3b95cRYoU0Q8//JDtNS4uLgoICFCVKlXk7e1tqlkAAADA2W4oEJcrV07lypVz/F62bFmFhYWpadOmTm8MAAAAyA+mHqo7dOiQk9oAAAAACgavbgYAAIClOXXbtbi4OB09elSXLl3K8WG7Jk2aOHNYAAAA4KY5JRCPGzdOn332mfbv359rrWEYSk1NdcawAAAAgGmmA/GTTz6p2bNn53n7NV7fDAAAgNuJqTXE33//vWbNmqWAgADNmTNHly5dkiQVL15cqampOnr0qCZPnqzKlSsrJCREf/75p9LT053SOAAAAOAMpgLxlClTZBiG3n33XT3++OMZ9hl2cXFRyZIl1bNnT8XExKhMmTJq37699u3bZ7ppAAAAwFlMBeLNmzdLkrp165bh+D9ngf38/DRu3DhdvHhRH330kZkhAQAAAKcyFYgTEhLk7++voKAgxzF3d3fH0onrNWjQQD4+Plq8eLGZIQEAAACnMhWIixYtKsMwMhwLCgpSUlKSEhISsrzm5MmTZoYEAAAAnMpUIC5VqpQuXLigxMREx7Hq1atLkqKjozPUxsTEKCkpST4+PmaGBAAAAJzKVCAODw+XJG3YsMFxrG3btrLZbHrllVe0YcMGpaSkaOPGjerZs6cMw1BkZKS5jgEAAAAnMhWI7eF39uzZjmMDBw5UqVKldPDgQdWvX19eXl66//779ffff8vNzU1vvPGG6aYBAAAAZzEViNu0aaPo6Gj17t3bcczPz09LlixRgwYNZLPZHJ+yZctq7ty5uv/++003DQAAADiLqUDs5uampk2bqm7duhmO33XXXVq1apViY2O1atUqbd++XQcPHlTbtm1NNVsQevXqJcMw1Lp16zzVnzp1Su7u7o79mQ3DyPT3ud4nn3wiwzBkGIZ++OGHbOtCQ0NlGIamTJmS5flu3bo57vPbb7/l2OOhQ4cctdd/vL29ValSJfXq1cuxpR4AAMCdzlQgzk3p0qXVoEED3X333Zl2oygsevbsKUlatGhRnnbI+Pbbb5WamqoyZcqoadOmkq7t13z9g4fXW7ZsmePn5cuXZ1mzY8cOnT59WpIUFRWV6fyFCxf0448/On7PLjRnpU6dOoqMjFRkZKTKli2r2NhYTZ06VfXq1buh+wAAABRWtzQQ3wmioqJUvnx5paWl6dtvv821furUqZKkHj166P7775eXl5fS0tK0atWqTLU2m00rV65UiRIl5O3tnW0gth8vW7asypcvn+n8rFmzlJSU5NgP+pdfflF8fHyevt/s2bO1cuVKrVy5Urt379aBAwfUtGlTpaamasCAATp69Gie7gMAAFBYOSUQJyUl6bPPPlPbtm11zz33qFKlShnOnz9/XjNmzNB3333njOHylWEY6tGjhyRp2rRpOdZu2bJF27Ztk3RtZtnT09OxZjqrsPvXX38pPj5eUVFRqlevnrZt26bz589nqrNfa59x/if7TO7QoUNVpUoVXb169ab/1mXKlNGMGTPk4eGhq1evas6cOTd1HwAAgMLCdCDesmWLqlevrpdeekm///67duzYoUOHDmWoCQgI0Hvvvadu3bppyZIlZofMd/Yt47Zt26YtW7ZkW2cPzA0bNtRdd90l6X8h9vqlEXb2Y40bN1bjxo2Vnp6uFStWZKrLKRDv27dPq1atkmEY6tq1q7p37y7pfzPVN6NkyZKqUqWKJGnPnj03fR8AAIDCwFQgPnv2rNq2basjR44oPDxcH3/8sQICAjLVGYahvn37ymazad68eWaGLBAVK1ZUo0aNJGUfNFNTUx1LKuzrjiWpSZMmkq7t1XzlypUM19iDbpMmTdS4ceMMx+z279+vY8eOSco6ENv7ady4scqVK+d4uG79+vXauXPnjX3R66Snp9/0tQAAAIWJqUD86aef6sSJE3rggQe0bt06vfzyy/L29s6y1r7DxJo1a8wMWWB69eolSZoxY4ZSU1MznV+wYIHi4uLk5eWlzp07O443aNBA7u7uSk5O1tq1azNcs2LFChUtWlR33323GjRoIFdX10yB2P57yZIlVbly5QznbDabY1baPjNcvnx5R7i+2Vni48ePa9++fZLkmCnOzoQJExQREZGnj5mADgAAcKuYCsS//PKLDMPQyJEj5eKS862qVq0qd3d37d+/38yQBaZjx47y8fFRXFycFixYkOm8PXy2b99egYGBjuM+Pj6ObdeuXzaxa9cunTp1So0aNZJhGPL391etWrW0adMmXbp0yVFnvyar2eElS5YoNjZWXl5e6tixo+O4PRxPnz5daWlpN/Q9jxw5oi5duig5OVmenp7q0KFDjvUnTpxQTExMnj6XL1++oV4AAADyg5uZiw8cOCAPDw/VqlUr11rDMBQQEJDlQ2OFgb+/vzp06KDp06dr2rRpevjhhx3n4uPj9csvv0j630zy9Zo2barVq1dnmP21/2yfzbX/vGnTJq1evVotW7bMUJfTcomHH344Qwjv2LGjBg0apOPHj2vx4sV66KGHsv1eHTt2lKenpyTpzJkz2r9/v1JTU+Xq6qovvvhCZcqUyfHvUqJECccrvHOzc+dOQjEAALjtmJohTk9Pl5ubW572GLbZbEpMTJSvr6+ZIQuUfW3wvHnzlJCQ4Dg+c+ZMXb16VSVLllSLFi0yXWcPs2vXrlVKSoqkjOuH7f65jvjo0aM6ePBghnvYJSYmau7cuZL+NyNsFxgYqEceeURS7nsSb9y4UatWrdKqVat06NAhlSlTRt27d9e6devUt2/fHK+VpP79+2vTpk15+lSvXj3X+wEAAOQ3U4G4VKlSSkpKUlxcXK61GzZs0NWrV1WhQgUzQxao5s2bq2zZsrp69apmzpzpOG6fqe3evbtcXV0zXRcZGSk3NzclJSVpw4YNkq6FXj8/vwyzq/YH9+zLJOz/DQsLU7Vq1TLcc9asWbp06ZKKFi2a5Vv07FvF/fTTTznOyh88eNDxeu0rV67owIEDmjZtmiIiInL/gwAAANwBTAVi+1vTJk+enGvtiBEjZBiGYylAYXT9nsT2ELxnzx7Hw3LX7y5xPT8/P9WuXVvStSB88OBBHTlyxPEgnV1oaKiqVq2q9evX6+rVq1nOItvZZ37Pnj0rDw+PTK9hti/puHLlSobwDgAAgIxMBeIXXnhBhmHo/fff1+LFi7OsOXXqlLp27arff/9dHh4eeu6558wMWeDsoXfNmjXau3evY5eHevXq5bgkwL7kYfny5TkG3caNG+vq1atat25dtuuHDxw4oJUrV0q6FqLDwsKy/Ni3wDOzJzEAAMCdzlQgrlGjht5//31dvHhRDz30kOrUqeP4v+e7dOmiyMhIlStXTt9//70kaezYsSpbtqz5rgtQ5cqVFRkZKela0Jw+fbqk7GeH7eyhdtWqVY6Xk1z/QJ2d/djs2bO1a9euDNfaTZ06VTabTXfddZdOnTqlkydPZvmxL7lYvXq19u7de7NfGQAA4I5m+k11r776qr7++msFBAQoJiZGV65ckc1m08yZM7VmzRolJycrMDBQU6ZM0TPPPOOMngucfSeJ0aNHKzY2Vp6ennrqqadyvKZRo0ZycXHRhQsXNGvWLHl4eDhe63w9eyCeOHGiJKlo0aKqUaOG4/z1ew/bl29kp1atWqpZs6YkZokBAACyk+dAPG/ePP3xxx9Znuvbt6+OHDmiyZMnq1+/fmrTpo0efPBB9ezZU1999ZUOHjyYa3grTDp16iQfHx/HFmLt2rVTcHBwjtcEBQXpvvvuk3RtXW/dunXl5eWVqa5ChQoqVaqU4612TZo0ybCLx9KlS3Xo0CEZhpFpd4ms2Geup0+fztvnAAAAspDnQNy+fXv17t07w7HmzZs7Xgjh5+fnCMC//vqrfv/9d0dAvn6P3DtBQECAHnvsMcfvWe09nJXrlz5ktX7Y7vqlFFktl7AfL1euXK5jdu3aVW5uboqNjXUs1QAAAMD/GDabzZaXQhcXFxUvXlzHjx/P8RiQnYiICMXExCiwYhU1G/lNQbdzx/uxQ6OCbgEAgAJlzx7h4eHatGlTtnV5niH28fFRfHy8U5oDAAAAbhd5DsRVqlRRcnKyRo8eraSkpFvZEwAAAJBv8hyIu3XrJpvNpqFDh8rf39/xQolTp07J1dU1zx83N7db9mUAAACAG5XnQPziiy/queeek5ubm+NVv5IcP9/IBwAAALhd5Hm61sXFRZ9//rk++OAD7dq1S5cuXVKzZs1UpEgR/fDDD7eyRwAAAOCWueH1C35+fqpTp47jdw8Pj0xbgwEAAACFhakFvZMnT5a3t7ezegEAAADynalAbH8LGgAAAFBY5fmhOgAAAOBORCAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGluBd0ArKdSkJ9+7NCooNsAAACQxAwxAAAALI5ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEtzK+gGYD2nE1L1xY+nCrqNO9Jzj4UVdAsAABQ6zBADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsLRCHYi7desmwzBkGIZ+++23gm7nhm3ZskXDhw/XlClTnHZP+99j6dKlGY4vXbrUcc7+cXNzU3BwsCpWrKhHHnlE77//vg4fPuy0XgAAAAqDQhuIL1y4oB9//NHxuzNDZX7ZsmWLRowYke+9R0ZGKjIyUvXr11fZsmV16dIl/fLLL3rjjTdUsWJF9e3bVxcvXszXngAAAAqKW0E3cLNmzZqlpKQkBQUFKSEhQb/88ovi4+MVHBxc0K3d9lauXJnp2P79+zVx4kSNHj1akyZN0ubNm7VixQr5+voWQIcAAAD5p9DOENtnVYcOHaoqVaro6tWr+u677wq2qUKsUqVK+uCDDxQdHS0vLy9t3rxZr7zySkG3BQAAcMsVykC8b98+rVq1SoZhqGvXrurevbskaerUqdles3DhQrVr105hYWFyd3dXcHCwqlatqu7du2vhwoWZ6r/77js98MADKlq0qNzd3RUSEqIaNWromWee0bp167IcY/v27erTp48qVKggLy8vBQUFKSoqSjNmzMhUW758efXu3VuStGzZsgxre8uXL38TfxXnaNiwoYYPHy5JmjRpko4fP15gvQAAAOSHQhmI7cG3cePGKleunOPhuvXr12vnzp2Z6sePH69WrVrp119/VWpqqmrWrKlSpUrp5MmT+u9//6svv/wyQ/2//vUvdenSRUuWLJG7u7vuu+8+hYSE6NChQ/r666+znIn++uuvVbt2bU2ePFlnz55VtWrV5Ovrq2XLlqlr166O8GtXt25d3XXXXZKkgIAAx7reyMhI1a1b11l/qpvSv39/ubm5KTk5WX/88UeB9gIAAHCrFbo1xDabTdOmTZMkx8xw+fLl1bhxYy1fvlxTp07Vhx9+6KhPS0vTG2+8IUkaN26cBgwYIFdXV8f59evXa+/evY7fz5w5o1GjRsnNzU3ff/+9Hn/8cRmGIUlKT0/X0qVLdenSpQw9RUdHa8CAAfL29taECRPUq1cvubhc+7fGsmXL9NRTT2nKlCmKjIxUv379JEmzZ8/WlClT1Lt3b9WuXTvTrhAFKSgoSPfee682b96sdevWqVevXtnWTpgwQV999VWe7pvVP1YAAAAKWqELxEuWLFFsbKy8vLzUsWNHx/Hu3btr+fLlmj59uv7zn/84Qu/p06d17tw5BQUF6bnnnst0v3r16qlevXqO3/ft26e0tDTVqlVLHTp0yFDr4uKi5s2bZ7rHa6+9pvT0dH3++eeZZoKbNm2q8ePH69FHH9XIkSMdgfh2V7ZsWW3evFmnTp3Kse7EiROKiYnJp64AAACcr9AFYvtyiYcffliBgYGO4x07dtSgQYN0/PhxLV68WA899JAkqVixYvLy8tL58+e1cOFCx/HslC1bVpK0Z88ebd68WbVr186x/ujRo1q/fr18fHzUtWvXLGvatGkjd3d37d27V8ePH1fJkiXz/H0Lip+fnyTluv1aiRIlFB4enqd77ty5U5cvXzbdGwAAgDMVqkCcmJiouXPnSvrfcgm7wMBAPfLII5o1a5amTJniCL6urq56+eWX9f7776tVq1aqXbu2HnjgATVs2FBRUVGZtmkrWbKkunTpohkzZigiIkINGjRQs2bNFBkZqSZNmmTahmzr1q2On7OaPbazL7s4evRooQjE9iAcEBCQY13//v3Vv3//PN0zIiKC2WQAAHDbKVSBeNasWbp06ZKKFi2q1q1bZzrfo0cPzZo1Sz/99JPOnz/vmEF+7733VLp0aX3xxRfavHmzNm/eLElyd3fX448/rtGjR2cIqZMnT1aNGjU0ceJErV69WqtXr5YkeXt7q0ePHvroo48c905ISJAkJSUladWqVbl+h6SkJFN/g/xif2NdWFhYAXcCAABwaxWqXSbsew+fPXtWHh4emV5F/PDDD0uSrly5opkzZzquMwxDAwcO1Pbt23Xs2DHNnDlTzzzzjHx8fDRz5ky1bdtWKSkpjnoPDw+9/vrrOnDggA4cOKBp06apW7dukq49RNalSxdHrX1pwd133y2bzZbrJyoq6hb/lcyLj4/X9u3bJUn3339/AXcDAABwaxWaQHzgwAHHG9ZCQ0MVFhaW5cf+f/FntydxyZIl1alTJ02YMEFbt26Vt7e3tmzZorVr12ZZX6FCBXXv3l3Tp0937AQxf/58HTlyRJJ0zz33OPq70fWx9mUUt5vx48crLS1Nnp6eua65BgAAKOwKTSCeOnWqbDab7rrrLp06dUonT57M8rNs2TJJ0urVqzNsp5aVcuXKOZZK5OUFFBEREfL09MxQX6lSJdWuXVtXrlzRF198cUPfydvbW5JuqwfNVq9erXfeeUeS1K9fPxUvXryAOwIAALi1CkUgvn7v4R49euRYW6tWLdWsWVPStRC9Y8cOPf3001qzZo1sNpujLj09XZMnT9aBAwdkGIZjp4Q///xTQ4YM0bZt2zLcNyUlRe+//76uXr0qHx8fVa9e3XHu448/lqurq/71r39p1KhRmdYJJyQk6L///a+GDh2a4XilSpUkSTt27NDp06dv5E/idAcOHNDrr7+uZs2a6cqVK4qIiNDIkSMLtCcAAID8UCgeqlu6dKkOHTokwzAy7S6RlZ49e2rIkCGaPn26Hn/8cU2cOFETJ06Uv7+/KlWqJDc3Nx0+fNgRQt98803HW+MuXryo0aNHa/To0QoODlaFChVks9l08OBBJSQkyDAMjR07NsPuC82bN9fkyZP1zDPP6NVXX9Wbb76patWqydPTU6dPn9ahQ4dks9nUtGnTDH3Wrl1bVatW1e7du1WpUiXdfffd8vLyUvHixfX999878S+YUaNGjRw/JyYm6sSJE4qLi5N0ba/lvn37asyYMfLx8bllPQAAANwuCkUgtq8Hbtq0qcqVK5drfdeuXTVs2DDFxsbq6NGjmjhxohYvXqzNmzfr4MGDunTpkkJCQvTII49o4MCBatWqlePaxo0b6/PPP9eff/6pv/76S3v27NHVq1dVvHhxtWrVSi+88ILq16+faczu3bsrMjJSn332mRYtWqR9+/bp6tWrCgkJUYsWLfTwww/r8ccfz3CNi4uL5s+fr9dff13Lli3Txo0blZaWlqfvaIZ9NwwXFxf5+/urSJEiateunerXr69u3bo59mIGAACwAsN2/ToC4Bay70NcpmJNDfv4j4Ju54703GNskwcAgJ09e4SHh2vTpk3Z1hWKNcQAAADArUIgBgAAgKUVijXEVtSxY0edOHEiz/X2PZoBAABwYwjEt6kNGzY4Xp8MAACAW4dAfJs6dOhQQbcAAABgCawhBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYmltBNwDrKRbkpuceCyvoNgAAACQxQwwAAACLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0twKugFYT9KZVG2eGFfQbdwStfuFFnQLAADgBjFDDAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQ3aPv27XrppZdUq1YtFS1aVB4eHgoJCVGjRo301ltvaf/+/QXd4k2bMmWKhg8fri1bthR0KwAAAPmGQJxHycnJGjBggGrWrKkxY8bor7/+UkhIiGrXrq2goCCtWbNG7777rqpVq6aRI0cWdLs3ZcqUKRoxYgSBGAAAWAqBOA/S09P1yCOPaMKECfL29tZHH32k06dPa/fu3Vq3bp327dunkydPauzYsQoJCdHq1asLumUAAADkkVtBN1AYfPDBB1q4cKE8PT31559/qn79+plqihUrpsGDB6t79+6aOXNmAXQJAACAm8EMcS4uXryojz/+WJL0xhtvZBmGrxccHKwBAwZkOHbq1CkNGTJE1apVk7e3twICAnT//fdr7NixSk5OznSPQ4cOyTAMlS9fPttxevXqJcMwNGXKlAzHp0yZIsMwFBUVJZvNpvHjx6t27dry8fFRkSJF1L59e/39998Zrlm6dKkMw9CyZcskSb1795ZhGI7P8OHDc/zOAAAAhRkzxLmYP3++EhIS5OrqqoEDB97w9Vu3btWDDz6ouLg4ubu765577lFSUpLWr1+v9evXa9asWfr9998VEBDg9N579uyp6dOnq3z58qpatap27dqln3/+WUuXLtXGjRtVuXJlSVJgYKAiIyP1119/6cKFC7rrrrsUGhrquE/ZsmWd3hsAAMDtgkCci5UrV0qSatSooZCQkBu69sqVK+rQoYPi4uLUvHlzzZgxQ2FhYZKkjRs36tFHH9Xq1as1aNAgTZ061al9r169Wjt37tSyZcvUpEkTSdK5c+fUvn17rVixQm+//ba+/fZbSVLt2rW1cuVKRUVFadmyZXr99dfVq1evPI0zYcIEffXVV3mq3blz5019FwAAgFuJQJyLY8eOSZIqVqx4w9fOnDlT+/fvl5+fn2bPnq0iRYo4ztWpU0cTJkxQu3bt9N///lcjRozIcYnEjUpJSdFnn33mCMOSVKRIEY0dO1bh4eGaP3++U8Y5ceKEYmJinHIvAACAgkAgzsXFixclSX5+fjd87e+//y5J6t69e4YwbPfwww+ratWq2r17t/744w8988wz5pq9TlBQkDp37pzpeO3ateXp6amEhASdPXtWRYsWNTVOiRIlFB4enqfanTt36vLly6bGAwAAcDYCcS78/f0lSYmJiTd87e7duyVJ99xzT7Y199xzj3bv3q1du3bdXIPZsK8PzkpoaKiOHDmixMRE04G4f//+6t+/f55qIyIimE0GAAC3HXaZyEWpUqUkSQcPHrzha+2zy/Z1w1kpXrx4hlpn8fX1zfaci8u1/9ltNptTxwQAACiMCMS5aNSokaRrr2w+c+bMDV1rn10+depUtjUnT57MUCtJhmFIyjmwXrp06YZ6AQAAQNYIxLlo06aNAgMDlZaWpi+//PKGrq1ataqka2E6O/Zz1apVcxyzz+7GxcVle92ePXtuqJe8sAdxAAAAKyEQ58Lf319DhgyRJP3nP//R2rVrc6yPj4/X+PHjJUmtW7eWJE2fPl3nzp3LVDt//nzt3r1bLi4uevDBBx3HQ0JCFBwcrCtXrmjz5s2Zrlu1apW2bdt2098pO97e3pLEg28AAMBSCMR58Prrr6tFixa6evWqWrRooVGjRik+Pj5Dzblz5/TFF1+oRo0aWrBggSSpc+fOqlSpkhITE9W5c+cMM74xMTGOh9G6d++eacu1tm3bSpJeeOEFnT171nF88+bN6tGjh9zd3Z3+PStVqiRJWr58OeuLAQCAZRCI88DV1VW//vqr+vXrp6SkJL366qsqVqyYqlWrpvvvv9/xZrfnn39eZ86cUePGjfX/2rvz6KjqPP3jT2XfIQkh7AkEQlhDgqARNBFQFmkEGgYQD0Kr0B49DCMj09iyqGkdRAemFVtQJHSPgCyOgmuzSYNpRRahUQIR2deWLIaE7N/fH/xSQ0wCWapIJff9OifnhPu9y+d+qFQ9ubmLJPn4+GjDhg1q3ry5tmzZorZt2yo+Pl5dunRR7969debMGSUkJOiPf/xjhW0+99xzCg4O1s6dO9WmTRv16tVL0dHRio+PV4cOHTR27FiH7+f48eNls9m0Zs0atW/fXnfffbeSkpIqPB4aAACgMSEQV5O3t7feeustffvtt5o+fbq6deumixcvat++fcrMzNQdd9yhefPmKT093X6KhSTFxsbq4MGDeuqppxQREaHvv/9eZ86cUZ8+fbRo0SJt37690sc2d+jQQampqRozZoz8/PyUlpYmd3d3vfjii/r000+dcoS4X79+WrNmjRISEpSRkaFdu3Zpx44dOnHihMO3BQAA4Cpshr+N4xYpuw9xTLueWjVnc32X4xRxjzav7xIAAMD/V5Y94uPjtXfv3irn4wgxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSCMQAAACwNAIxAAAALI1ADAAAAEsjEAMAAMDSPOq7AFiPXzMPxT3avL7LAAAAkMQRYgAAAFgcgRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkE4jqYP3++bDab5s+fX691pKSk6LbbbpO/v79sNptsNpuysrL0xRdfyGazKSkpqV7rAwAAcGUe9V1AXS1evFhZWVmaPHmyIiMj67ucWy4lJUVTpkyRJHXu3FnNmjWTJHl4NPj/WgAAgFuiwaemxYsX6+TJk0pKSrrlgbhZs2blQmh9eP311yVJr7zyimbOnFlvdQAAADRUDT4Q16cnn3xSTz75ZL3W8P3330uShg0bVq91AAAANFScQ9zAXb16VZLk6+tbz5UAAAA0TA02EKekpMhms+nkyZOSpHvuucd+QZnNZlNKSook2f994sQJ7dixQ/fff7/CwsLk5uamDz74QJKUlZWld955R6NHj1Z0dLT8/f0VEBCg2NhYzZ07V9nZ2ZXWUNVFdWUXs5WdwrF27VrdeeedCgwMVFBQkAYNGqQvv/yyTvsfGRkpm81m/3f79u3t+1rdi/zS0tL0L//yLwoLC5Ofn5+6d++uV199VSUlJUpKSpLNZtMXX3xRpzoBAABcXYM9ZSI8PFz9+vXTnj17VFBQoO7du6tJkyblxq+3Zs0a/f73v1dQUJA6duwoPz8/+9hHH32kRx55RJ6enmrRooW6du2q7OxspaWl6eDBg1q7dq1SU1MVEhJS4zrnzJmj5ORktWrVStHR0UpPT9fWrVu1c+dObdu2Tf369avV/vfp00dt2rSxB+vbbrtN3t7ekqR27drddPkvv/xSgwcPVm5urnx9fdW1a1dlZmbq3//93/X3v/+9VjUBAAA0RA02EA8dOlRDhw5VZGSkTp48qddee+2Gtxd79tln9cwzz2ju3Lny9PSUJOXn50uSevbsqY0bN2rQoEHlTj24fPmyfv/732vp0qWaPXu2li5dWqMaz549q0WLFmn9+vX69a9/LenaKQ6TJk3S+vXrNWvWrFofKV63bp0k2Y8Sr1u3rtoXFebl5WnChAnKzc3VyJEjtWLFCjVt2lSStGXLFo0ePdrem5tZunSpli1bVq15Dx8+XK35AAAAbqUGG4hrasiQIXrhhRfKTfPx8ZF0LRD37NmzwjKhoaF688039fHHH+vdd9/VG2+8IXd392pvs7i4WMnJyfYwLF071/f111/Xxo0blZqaqszMTAUHB9dyr2pnzZo1On36tFq3bq1Vq1aV+yVg0KBBevXVVzV16tRqrev8+fPat2+fs0oFAABwOssE4smTJ99wvLi4WBs3btSWLVt0/Phx5ebmqrS0VJL0888/Kzc3V+np6YqJianRdqdNm1ZhWnh4uCIjI3X06FH9+OOP6t27d43WWVefffaZJGnChAmVXoz30EMPafr06dU6StyyZUvFx8dXa7uHDx+2XwQIAADgKiwTiLt06VLl2Llz5zRs2DAdOHDghuu4fPlyjbbZrFkz+6kIvxQeHq6jR4/qypUrNVqnIxw9elSSFBsbW+m4r6+voqOjdfDgwZuua9q0aZWG/sr07t2bo8kAAMDlNNi7TNSUv79/lWOTJ0/WgQMHFBcXp40bN+rs2bMqKCiQMUbGGN11112SpKKiIodt083tWuuNMTVapyOUhfDAwMAq57nRGAAAQGNimSPEVTl//rw2b94sX19f/fWvf630qXM1PTLs6gICAiRJOTk5Vc5zozEAAIDGpMEfIb7+Xry1ceLECUnXTqmoLAxnZmbaTzFoLKKjoyWpylMi8vPzG90+AwAAVKXBB+Kyi8Jqe7FW2f2IL168WOnpC4sWLVJxcXHtC3RBQ4YMkXTtbhOVXTj37rvvVvu2awAAAA1dgw/EUVFRkqQdO3bUavmuXbsqNDRUZ8+e1dy5c1VSUiJJKi0t1ZIlS/Tiiy/ab8/WWIwfP17t2rXT6dOnNXHixHJP4tu2bZtmzpxpv1czAABAY9fgA/GECRMkSQsWLFDnzp2VmJiopKQk+63FbsbT01MvvfSSJCk5OVktW7ZUnz591KJFCz355JOaNGmSbr/9dqfVXx/8/Py0atUq+fn56f3331erVq3Up08fdezYUQMHDtTAgQOVkJAgSTW67zIAAEBD1OAD8YMPPqglS5YoLi5OZ86c0d/+9jft2LFDFy5cqPY6HnvsMa1fv159+/ZVTk6Ojhw5osjISL355ptavny5E6uvP2WPvR47dqx8fX116NAheXt7a8GCBVq7dq3y8vIkSUFBQfVcKQAAgHPZTH3c9wsuraSkRKGhocrOzlZmZmaV91KuqbL7EMfHx2vv3r0OWScAAEBVqps9GvwRYjjehg0blJ2drR49ejgsDAMAALgqArFFpaamKiUlxX5qRJmNGzfqt7/9rSTpiSeeqI/SAAAAbinLP5jDVfTv37/a87Zs2VLr1q2r0/bOnTunKVOmaNq0aercubN8fX114sQJXbp0SZI0evRoPfbYY3XaBgAAQENAIHYRX375ZbXnjYiIqPP2+vbtq6eeekpbt27VmTNnlJ2drcDAQCUlJenhhx/WpEmT7I+XBgAAaMwIxC7iVl/b2K5dO7366qu3dJsAAACuiEOAAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDQCMQAAACyNQAwAAABLIxADAADA0gjEAAAAsDSbMcbUdxGwhpCQEGVmZsrX11ddunSp73IAAEAjd/jwYV29elXBwcHKyMiocj4CMW4ZLy8vFRUV1XcZAADAYnx9fZWXl1fluMctrAUWZ7PZJEnu7u6KjY2t52oaj7Lffjny7lj01fHoqXPQV+egr45XHz09fvy48vPz1bx58xvORyDGLdO9e3ft27dPsbGx2rt3b32X02j07t1b+/btU5cuXeirA9FXx6OnzkFfnYO+Op4r95SL6gAAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGoEYAAAAlkYgBgAAgKURiAEAAGBpBGIAAABYGo9uxi0zdepUnT9/Xi1btqzvUhoV+uoc9NXx6Klz0FfnoK+O58o9tRljTH0XAQAAANQXTpkAAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiBGrWzfvl3Dhw9XWFiYfH19FRMTozlz5ig3N7fW69ywYYPuueceBQcHy9/fX7169dIrr7yioqIiB1bu2hzV15KSEm3evFkzZsxQ37591bRpU3l5eally5Z64IEH9PHHHztpD1yTM16v13vjjTdks9lks9mUlJTkkHW6Omf01Bij1atXa8iQIQoPD5e3t7datWqlgQMH6pVXXnFg9a7L0X09deqUpk+frpiYGPn5+cnHx0ft27fXlClT9I9//MPB1bueCxcu6C9/+YumT5+uhIQE+fr6Ouzn1NnvK67M0X01xig1NVW/+93v1L9/f4WGhsrT01NhYWG677779O6778rp94AwQA398Y9/NDabzUgybdq0MXFxccbb29tIMl26dDGXL1+u8TpnzpxpJBlJJioqyvTs2dO4u7sbSebuu+82+fn5TtgT1+LIvr799tv2frq5uZno6GgTHx9vgoKC7NOnTp1qSktLnbhHrsEZr9frnTlzplxfExMTHVO4C3NGT3NycsygQYPsfezQoYPp06ePadeunXF3dzehoaFO2BPX4ui+pqammsDAQCPJeHp6ms6dO5sePXoYX19fI8l4eHiYtWvXOmlvXMOiRYvsr6nrv+r6c+rs9xVX5+i+btmypdx6OnToYHr37m1CQkLs0+6//36nZgECMWpkz549xs3NzdhsNrN06VJ7oDp79qzp3bu3kWRGjx5do3W+//77RpLx9vY2H374oX364cOHTfv27Y0k89RTTzl0P1yNo/v61ltvmZ49e5q3337bZGVl2acXFRWZhQsX2t/I33jjDYfviytxxuv1l0aMGGHc3d3N8OHDLRGIndHT0tJSc++99xpJZsiQIeaHH34oN56ZmVnuvaExcnRfS0tLTceOHY0kk5CQYE6ePGkfy8rKMg8++KCRZIKCgsq9RzQ2y5cvN4MGDTKzZ88277//vpkzZ06df05vxfuKq3N0Xzdv3mzat29v/vu//9tcvHix3Nif//xn+y8bs2bNckD1lSMQo0YeeOABI8lMmjSpwtjRo0eNm5ubkWQOHDhQ7XXGxsYaSWbu3LkVxrZu3WoPy5cuXapT7a7M0X29fPnyDY/+PvbYY0aSiY2NrW3JDYIzXq/Xe++994wk86//+q9m3rx5lgjEzujpO++8YySZ22+/3RQVFTmy3AbD0X09dOiQ/cjat99+W2E8Pz/f+Pv7G0lm06ZNda6/oXjttdfq/HPq7PeVhqiufc3OzjaFhYVVjv/hD38wkkxISIgpKSmpZZU3xjnEqLYrV67os88+k3TtaTO/1KlTJw0YMECStG7dumqtMz09XQcOHKhynQMGDFDHjh1VUFCgjRs31rZ0l+aMvoaEhMhms1U5PnToUEnSkSNHalpug+GMvl4vMzNT06dPV5s2bZScnFy3YhsIZ/X0v/7rvyRJzz77rDw8rPcAVWf09erVq/bvo6KiKox7e3urTZs2kmSp6zTqytnvK1YVFBQkT0/PKsfLPrMyMjL0z3/+0yk1EIhRbfv371dBQYG8vb3Vt2/fSue56667JElfffVVtdZZNl+HDh3UunVrh6yzoXFGX2+m7MPSz8/PIetzRc7u68yZM3Xx4kW99tprCggIqFOtDYUzenrs2DEdOnRIbm5uuueee/T111/rt7/9rQYNGqSRI0fqP//zP/XTTz85bB9ckTP62rlzZ/n6+kqSUlNTK4yfP39eP/74o9zd3RUfH1/Lyq2nPt6vUf4XvLLXtaMRiFFtR48elSS1a9euyt/kyo5EVPfIY9k6KzuCUdt1NjTO6OvNrF69WtL/vXE3Rs7s67Zt27RixQqNGDFCI0eOrFOdDYkzerpnzx5JUmhoqJYsWaKEhAQtXbpUW7du1YcffqjZs2erU6dO2r59uwP2wDU5o6+BgYGaM2eOJGnKlClav369Ll++rJycHG3fvl3Dhg1TUVGRZs+erYiICAfshTXUx/s1/u8zKzY2VkFBQU7ZBoEY1ZaRkSHp2p/jq1I2lpmZWW/rbGhudQ8+/PBDffTRR7LZbJo1a1ad1+eqnNXXq1evaurUqQoICNDrr79etyIbGGf09Pz58/b5/+M//kPDhg3Td999p4KCAh08eFADBgxQVlaWRo8erTNnztRxD1yTs16rs2fP1ooVK9S0aVONHTtWzZo1U1BQkAYMGKC8vDytWbNGL7zwQt2Ktxg+s269vXv36s0335Qk/e53v3PadgjEqLb8/HxJkpeXV5XzeHt7Syr/541bvc6G5lb2IC0tTQ8//LAkacaMGbrzzjvrtD5X5qy+zps3T8eOHdPzzz+vtm3b1q3IBsYZPb1y5Yokqbi4WFFRUfrf//1fde3aVV5eXurRo4c2bdqkFi1aKCsrS4sXL67bDrgoZ71Wi4qK9OOPPyojI0MeHh7q1KmTunXrJm9vb6Wnp2v58uWN9pcMZ+Ez69a6ePGiRo8ereLiYo0aNUrjx4932rYIxKg2Hx8fSVJhYWGV8xQUFEiq/jk+zlhnQ3OrenD69GkNHjxY2dnZGjZsmBYsWFDrdTUEzujr/v37tWjRIsXFxWn69Ol1L7KBceZ7gCQ9+eSTFf4M7efnp8cff1yS7BczNTbOeg8YNWqUXnjhBcXFxenkyZM6evSoDh06pPPnz2vixInavHmzEhISlJOTU7cdsBA+s26d7OxsDR06VKdOnVLv3r2VkpLi1O0RiFFtwcHBkv7vT0aVKRsrm7c+1tnQ3IoeXLhwQQMHDtSpU6eUlJSkDRs23PCK3sbAGX195JFHVFpaqmXLlsnd3b3uRTYwznwPkKQuXbpUOk/Z9OPHj1drnQ2NM/q6adMmffzxx2rWrJlWr16tVq1aldveO++8o5iYGJ05c0ZLliypQ/XWwmfWrXHlyhUNGTJE+/fvV7du3fT555877dzhMta7vw1qLTo6WtK1R4EWFRVVGqiOHTtWbt7qrvOHH36ocp6arrOhcUZfr3fp0iUNGDBA6enpSkhI0KZNm8odlWusnNHX/fv3y93dXcOHD68wVvan/9TUVLVo0UKS9M033zSq0yqc0dOYmBj791X9Gbrs9VpaWlqjehsKZ/R1586dkqS+ffuqSZMmFcY9PT11zz33KC0tzX5hI27O2e/XkPLy8nT//ffrq6++UqdOnbRlyxaFhoY6fbscIUa1xcXFycvLSwUFBdq9e3el85S9CSckJFRrnXfccYeka0d+zp4965B1NjTO6GuZjIwM3XvvvTp8+LDi4+P16aefWuYWYc7qa0lJiS5evFjhKzc3V9K18zbLppWUlNR9R1yIM3oaFxdn/9Pyjz/+WOk8ZQGj7L65jY0z+lqT0yDKzovFzTnz/RrXXosjRozQ3/72N0VERGjr1q32AwzORiBGtQUGBmrw4MGSpGXLllUYT09P17Zt2yRJY8aMqdY6o6Oj1aNHjyrXuW3bNv3www/y8vLSiBEjalu6S3NGXyXp559/1n333aeDBw+qe/fu+utf/1rpkaLGyhl9Ndee7lnp17x58yRJiYmJ9mmRkZGO2RkX4Yye+vn56Ve/+pUkaeXKlRXGjTH2cwcHDhxYm7JdnrPeWyVp9+7dys7OrjBeVFRkv5Vd586da1W3FTnr/RrXXpO//vWvtXXrVrVu3Vrbtm27tX9hc8rz79Bo7d6929hstgrPcD937pz9Ge4jR46ssFxERISJiIgw69atqzC2fv16++OZN27caJ+elpZm2rdvbySZGTNmOG+nXICj+5qbm2v69+9vJJmYmJgKz4a3Cme8XqtilUc3O6On3333nfHw8DCSzPPPP2+Ki4uNMcYUFRWZp59+2kgyPj4+Ji0tzbk7V48c3dfTp08bb29vI8kMHTrUnD171j6WkZFhHnroISPJ2Gw2s2/fPufunAup7iOG+/XrZyIiIsyiRYsqjNX2/6oxq2tfi4uLzZgxY4wk06JFC3PkyBHnFVsFAjFqbNGiRcZmsxlJpm3btiYuLs7+xtu5c2fzz3/+s8Iykowks2LFikrXOWPGDPs8UVFRJjY21ri7uxtJpn///ubq1atO3qv658i+vvjii/axmJgY069fvyq/zp8/f4v2sH444/VaGasEYmOc09OUlBT7z3xYWJjp27evCQ0NNZKMp6enWbVqlZP3qv45uq8rV660/6Lh4eFhoqOjTbdu3ezrtNls5uWXX74Fe1Z/Tp06ZUJDQ+1f/v7+9n5cP33BggXllouIiDCSzLx58ypdb23+rxoTR/d11apV9tdyZGTkDT+znPULHIEYtbJlyxYzdOhQExISYry9vU10dLR55plnTE5OTqXzVydgrF271iQmJpomTZoYX19f06NHD7NgwQJTWFjopL1wPY7qa1k4q87X8ePHnb9j9cwZr9dfslIgNsY5Pf3mm2/M2LFjTXh4uPH09DQtWrQwEyZMMN9++62T9sL1OLqv3377rZkyZYqJiooy3t7exsvLy7Rr186MHz/e7Nq1y4l74hqOHz9erffBXwa0mwViY2r+f9WYOLqvK1asqPZn1vbt252yTzZjjKnqdAoAAACgseOiOgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGAACApRGIAQAAYGkEYgAAAFgagRgAAACWRiAGABczefJk2Ww2TZ48ub5LAQBL8KjvAgAAqInFixcrKytLI0eOVK9eveq7HACNAIEYAFxMy5Yt1blzZ7Vs2bK+S3FJixcv1smTJxUZGUkgBuAQBGIAcDEvvfSSXnrppfouAwAsg3OIAQAAYGkEYgBwMVVdVJeUlCSbzab58+eruLhYixYtUlxcnAICAtS8eXONHDlSBw4csM+fl5en5ORkde/eXf7+/goNDdW4ceN07NixSrebkpIim82myMhISdLmzZs1dOhQhYWFydfXV926dVNycrLy8/NvWP+xY8f0+OOPq1OnTvL19VVQUJDi4+P1/PPP6+eff650mS+++EI2m002m02StH//fk2cOFFt2rSRp6enkpKSNH/+fNlsNp08eVKSNGXKFPsy1y9b5tChQ5o/f74GDBigqKgoey1xcXF69tln9dNPP1W5D5GRkbLZbEpJSVFhYaEWLlyo2NhY+fv7q0mTJhowYIA+++yzG/ZBkr7++mtNmTJFHTt2lJ+fn4KCgtS1a1f95je/0eeff17pMqWlpXr33Xc1bNgwhYeHy8vLS2FhYbrvvvu0evVqGWNuul0ANWQAAC7l4YcfNpLMww8/XG56YmKikWSeeeYZM3DgQCPJeHl5GX9/fyPJSDIBAQHmm2++MT/99JOJi4szkoyPj4/x9fW1z9O8eXNz8uTJCttdsWKFkWQiIiLMkiVLjM1mM5JM06ZNjYeHh335uLg4k5GRUWnt7733nvH29rbPGxgYWO7fbdu2Nd9//32F5bZv326fZ/369cbT09NIMkFBQcbHx8ckJiaahQsXmvDwcOPm5mYfCw8PL/d1vYiICPs6fXx8TEhIiH2fJJnWrVubtLS0SvejbNnXXnvN3H777UaS8fT0NAEBAfblbTabWb58eaXLFxcXm+nTp9vnlWT8/f1NcHCwvYYmTZpUWO7y5cvm7rvvLrdckyZNyv17xIgRpqCgoNLtAqgdAjEAuJibBeKmTZua0NBQs27dOlNYWGhKS0vN7t27TYcOHYwkc+edd5pRo0aZyMhI8/nnn5uSkhJTUlJitmzZYsLCwowkM3HixArbLQvEfn5+xtPT04wdO9acOnXKGGNMXl6e+dOf/mQPt6NGjaqw/N69e+1Btl+/fubgwYPGGGNKSkrMxo0bTcuWLY0kExUVZXJycsote30gDggIMMOGDTOHDx+2jx89etT+fVlYXbFixQ37OGnSJJOSklIu/BcUFJgtW7aYvn37GkkmPj6+0mXLthEcHGxat25tPvjgA1NYWGiMMSYtLc3ccccd9lqzsrIqLD9r1iz7/vzmN78xR44csY9lZWWZDz74wIwbN67cMsXFxfb/4169eplNmzaZ3NxcY4wxV65cMStXrjTNmzc3ksyMGTNuuO8AaoZADAAu5maBWJLZuXNnheW2bt1qH/f19TXp6ekV5lm+fLl9vCzglSkLxJJMYmKiKSkpqbD822+/bZ9n9+7d5caGDBliJJmOHTvag9z19u3bZz/SvHDhwnJj1wfivn37muLi4ir7U91AfCM5OTkmPDy8yl6WbcPb27tcMC9z6dIl4+PjYySZ//mf/yk3duTIEftR7FmzZlW7pj//+c9GkomJiak0ZBtjzJ49e4zNZjNeXl7m4sWL1V43gBvjHGIAaGD69++v/v37V5iemJgob29vSdKYMWPUsWPHCvMMHjxYknT16lWlp6dXuY1nn31Wbm4VPyKmTJmiNm3aSJLWrFljn56VlWU/J/bpp5+Wn59fhWXj4uI0evRoSdLq1aur3PbTTz8td3f3KscdISAgQImJiZKkXbt2VTnfmDFjFBMTU2F6WFiYEhISJEkHDx4sN7Zy5UqVlpYqNDRUzz33XLVrWr58uSTp8ccfV5MmTSqdp3fv3urWrZsKCwu1ffv2aq8bwI0RiAGggenbt2+l093d3dWsWTNJUp8+fSqdJzw83P59ZmZmpfN4eHjorrvuqnTMzc1NSUlJkqQ9e/bYp+/bt89+sdegQYOqrP3ee++VdC1EFhUVVTpPv379qly+pj766CONGzdOHTp0kL+/f7kL8NauXStJOnPmTJXL33777VWOtWrVSpKUkZFRbnpqaqqka/vq4+NTrTpLSkr01VdfSZLmz5+vFi1aVPl15MgRSbJfXAig7rgPMQA0MIGBgVWOeXh43HCesnFJVQbSZs2a2Y80V6Z169aSpEuXLtmnXf992Xhlyo4uFxcXKyMjo1xAL9O8efMql6+u0tJSPfTQQ+WORHt4eCg4OFheXl6SpOzsbOXn5ys3N7fK9VSn17/s44ULFyRJERER1a43IyNDBQUFkqr+ReWX8vLyqr1+ADfGEWIAgEtxxOkSy5cv1+rVq+Xu7q65c+cqPT1dBQUFysjI0IULF3ThwgWNGTNGkhx+G7Nf3v6tOkpKSuzff/rppzLXrvG54df8+fMdWDVgbQRiAEA5P/30kwoLC6scP3v2rKTyR3Kv//5GpyCUjXl4eCgkJKSupVap7PzmRx99VM8995w6duxY4ZzosiO5jtaiRQtJNTulITQ01H7EmVMhgFuPQAwAKKe4uFg7d+6sdMwYox07dkiSbrvtNvv0+Ph4e+DcunVrlevesmWLJCk2Nlaenp61qq9sOzc6snv69GlJ1y7kq8yVK1f09ddf12r7N3PnnXdKuvZgk5s9xKSMp6en/dzwTZs2OaUuAFUjEAMAKvjDH/6g0tLSCtNXrlxpD5vjxo2zT2/atKn9DhYLFy6s9PzWAwcOaMOGDZKkCRMm1Lq2oKAgSdfubFGVsrs0XP/kvuu98MILysnJqXUNNzJ58mS5u7vr8uXLmjdvXrWXmzp1qiTpk08+0SeffHLDeX95IR+AuiEQAwDK8fPz065du/Tggw/aT3HIz8/XsmXL9Pjjj0uSHnjggQp3u0hOTpanp6d++OEHDR48WP/4xz8kXbvA7ZNPPtGwYcNUXFysqKgoTZs2rdb1de/eXZK0fv36Ki9AGzJkiCTprbfe0rJly+yngFy4cEH/9m//ppdfflmhoaG1ruFGOnbsqKefflqS9PLLL+vRRx8td4u7n3/+We+9955GjRpVbrmHHnpIgwYNkjFGo0aNUnJyss6dO2cfz83N1fbt2/XEE0+oQ4cOTqkdsCoCMQCgnLCwMC1atEhr165V27ZtFRISoqCgIE2bNk35+fmKjY213zP3evHx8frLX/4iLy8v7dq1Sz179lSTJk3k7++v+++/X+fOnVPbtm21adMmBQQE1Lq+qVOnymazKTU1VWFhYWrVqpUiIyMVGRlpn2fmzJmKiYlRcXGxpk2bJl9fXwUHB6tVq1ZavHixpk2bpuHDh9e6hptJTk7WE088IenaBX7R0dEKDAxUSEiImjZtqvHjx1e4j7C7u7s2bNig4cOHq7CwUHPmzFHr1q3VpEkTBQcHKzAwUAMGDNAbb7xxwztjAKg5AjEAoIInnnhCn3/+uYYMGSI3Nze5ubkpJiZGzz//vP7+979XeXR13Lhx+u677zRt2jRFRUWpoKBAHh4e6tWrl5577jkdOnRIXbp0qVNtd999tz7++GMNGjRITZs21cWLF3Xy5MlyF6M1bdpUqampmjFjhiIjI+Xu7i4PDw8lJSVp9erVevPNN+tUw824u7vr9ddf165duzRx4kS1a9dORUVFMsaoa9eueuSRR+ynj1wvKChImzZt0ieffKJx48apXbt2KigoUF5enlq3bq377rtPL730kv1exAAcw2Ycfb8ZAECDlJKSoilTpigiIkInTpyo73IA4JbhCDEAAAAsjUAMAAAASyMQAwAAwNIIxAAAALA0LqoDAACApXGEGAAAAJZGIAYAAIClEYgBAABgaQRiAAAAWBqBGAAAAJZGIAYAAIClEYgBAABgaQRiAAAAWNr/AzmEAc8X/pFvAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 700x1500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot feature importance\n",
    "fig, ax = plt.subplots(1, 1, figsize=(7, 15))\n",
    "sns.barplot(\n",
    "    x='importance'\n",
    "    , y='features_selected_for_training'\n",
    "    , data=fi_df.sort_values(by=['importance'], ascending=False)\n",
    "    , ax=ax\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5647541c",
   "metadata": {
    "papermill": {
     "duration": 0.01586,
     "end_time": "2023-10-31T14:02:46.123489",
     "exception": false,
     "start_time": "2023-10-31T14:02:46.107629",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Train individual model (i.e. for each asset a model file will be saved)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "dc6454af",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T14:02:46.158175Z",
     "iopub.status.busy": "2023-10-31T14:02:46.157777Z",
     "iopub.status.idle": "2023-10-31T14:05:12.016989Z",
     "shell.execute_reply": "2023-10-31T14:05:12.015506Z"
    },
    "papermill": {
     "duration": 145.881249,
     "end_time": "2023-10-31T14:05:12.020805",
     "exception": false,
     "start_time": "2023-10-31T14:02:46.139556",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lgb model for Bitcoin Cash saved!\n",
      "lgb model for Binance Coin saved!\n",
      "lgb model for Bitcoin saved!\n",
      "lgb model for EOS.IO saved!\n",
      "lgb model for Ethereum Classic saved!\n",
      "lgb model for Ethereum saved!\n",
      "lgb model for Litecoin saved!\n",
      "lgb model for Monero saved!\n",
      "lgb model for TRON saved!\n",
      "lgb model for Stellar saved!\n",
      "lgb model for Cardano saved!\n",
      "lgb model for IOTA saved!\n",
      "lgb model for Maker saved!\n",
      "lgb model for Dogecoin saved!\n"
     ]
    }
   ],
   "source": [
    "def train_lgbm_model_for_asset(asset, feature_df, features_selected_for_training, target, params, output_dir):\n",
    "    \"\"\"\n",
    "    Train an LGBM model on the given feature DataFrame for a specific asset, and save the model.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    asset : int\n",
    "        Asset_ID for which the model is trained\n",
    "    feature_df : pd.DataFrame\n",
    "        DataFrame containing the features and target variable\n",
    "    features_selected_for_training : list of str\n",
    "        List of selected feature names used for training the model\n",
    "    target : str\n",
    "        Name of the target variable\n",
    "    params : dict\n",
    "        Dictionary of parameters for the LGBM model\n",
    "    output_dir : str\n",
    "        Directory to save the trained model\n",
    "    \"\"\"\n",
    "\n",
    "    # Instantiate the LGBM model\n",
    "    model = lgb.LGBMRegressor(**params)\n",
    "\n",
    "    # Train the model on the specific asset and validate\n",
    "    train_data = feature_df.query('train_flg == 1 and Asset_ID == @asset')\n",
    "    val_data = feature_df.query('train_flg == 0 and Asset_ID == @asset')\n",
    "    \n",
    "    model.fit(\n",
    "        train_data[features_selected_for_training],\n",
    "        train_data[target].values, \n",
    "        eval_set=[(val_data[features_selected_for_training], val_data[target].values)],\n",
    "        verbose=-1, \n",
    "        early_stopping_rounds=100,\n",
    "    )\n",
    "    \n",
    "    # Save the model\n",
    "    asset_name = train_data['Asset_Name'].values[0]\n",
    "    model_path = os.path.join(output_dir, f'lgb_model_{asset_name}_val.pkl')\n",
    "    joblib.dump(model, model_path)\n",
    "    print(f'lgb model for {asset_name} saved!')\n",
    "\n",
    "# Train LGBM models for each asset\n",
    "for asset in feature_df['Asset_ID'].unique():\n",
    "    train_lgbm_model_for_asset(asset, feature_df, features_selected_for_training, target, params, CFG.OUTPUT_DIR)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "210351f5",
   "metadata": {
    "papermill": {
     "duration": 0.01679,
     "end_time": "2023-10-31T14:05:12.056574",
     "exception": false,
     "start_time": "2023-10-31T14:05:12.039784",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Evaluation score - weighted correlation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "2ffef8dc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T14:05:12.094065Z",
     "iopub.status.busy": "2023-10-31T14:05:12.093634Z",
     "iopub.status.idle": "2023-10-31T14:05:12.101582Z",
     "shell.execute_reply": "2023-10-31T14:05:12.100594Z"
    },
    "papermill": {
     "duration": 0.028934,
     "end_time": "2023-10-31T14:05:12.103796",
     "exception": false,
     "start_time": "2023-10-31T14:05:12.074862",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def weighted_mean(x, w):\n",
    "    \"\"\"Weighted Mean\"\"\"\n",
    "    return np.sum(x * w) / np.sum(w)\n",
    "\n",
    "def cov(prediction, target, w):\n",
    "    \"\"\"Weighted Covariance\"\"\"\n",
    "    return np.sum(w * (prediction - weighted_mean(prediction, w)) * (target - weighted_mean(target, w))) / np.sum(w)\n",
    "\n",
    "\"\"\" (prediction - weighted_mean(prediction, w)) * (target - weighted_mean(target, w))\n",
    "\n",
    "This calculate the differences between each data point and their respective weighted means. \n",
    "These differences are then element-wise multiplied by the weight array w. \"\"\"\n",
    "\n",
    "def corr(prediction, target, w):\n",
    "    \"\"\"Weighted Correlation\"\"\"\n",
    "    return cov(prediction, target, w) / np.sqrt(cov(prediction, prediction, w) * cov(target, target, w))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d83feb13",
   "metadata": {
    "papermill": {
     "duration": 0.016685,
     "end_time": "2023-10-31T14:05:12.138078",
     "exception": false,
     "start_time": "2023-10-31T14:05:12.121393",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Evaluating the full model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "35718036",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T14:05:12.176611Z",
     "iopub.status.busy": "2023-10-31T14:05:12.175407Z",
     "iopub.status.idle": "2023-10-31T14:05:29.313405Z",
     "shell.execute_reply": "2023-10-31T14:05:29.312487Z"
    },
    "papermill": {
     "duration": 17.160572,
     "end_time": "2023-10-31T14:05:29.316036",
     "exception": false,
     "start_time": "2023-10-31T14:05:12.155464",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Load the FULL MODEL *******************************************\n",
      "\n",
      "- Cardano: Validation Score (weighted correlation) = 0.0245\n",
      "\n",
      "- Bitcoin Cash: Validation Score (weighted correlation) = 0.0127\n",
      "\n",
      "- Binance Coin: Validation Score (weighted correlation) = 0.0240\n",
      "\n",
      "- Bitcoin: Validation Score (weighted correlation) = -0.0107\n",
      "\n",
      "- Dogecoin: Validation Score (weighted correlation) = 0.0147\n",
      "\n",
      "- EOS.IO: Validation Score (weighted correlation) = 0.0071\n",
      "\n",
      "- Ethereum Classic: Validation Score (weighted correlation) = 0.0124\n",
      "\n",
      "- Ethereum: Validation Score (weighted correlation) = 0.0148\n",
      "\n",
      "- Litecoin: Validation Score (weighted correlation) = -0.0064\n",
      "\n",
      "- TRON: Validation Score (weighted correlation) = -0.0026\n",
      "\n",
      "- Stellar: Validation Score (weighted correlation) = 0.0079\n",
      "\n",
      "- Monero: Validation Score (weighted correlation) = -0.0010\n",
      "\n",
      "- IOTA: Validation Score (weighted correlation) = 0.0055\n",
      "\n",
      "- Maker: Validation Score (weighted correlation) = 0.0012\n",
      "=> Overall Validation Score (weighted correlation) = 0.0118\n"
     ]
    }
   ],
   "source": [
    "# Compute the correlation\n",
    "print('Load the FULL MODEL *******************************************')\n",
    "\n",
    "model = joblib.load(os.path.join(CFG.OUTPUT_DIR, 'lgb_model_val.pkl'))\n",
    "\n",
    "val_df = train.query('train_flg == 0').copy()\n",
    "\n",
    "val_df['Prediction'] = model.predict(val_df[features_selected_for_training])\n",
    "\n",
    "for asset in val_df['Asset_ID'].unique():\n",
    "    tmp = val_df.query('Asset_ID == @asset')\n",
    "    coin = tmp['Asset_Name'].values[0]\n",
    "    \n",
    "    r = corr(tmp['Prediction'], tmp['Target'], tmp['Weight'])\n",
    "    \n",
    "    print('')\n",
    "    \n",
    "    print('- {}: Validation Score (weighted correlation) = {:.4f}'.format(coin, r))\n",
    "\n",
    "r = corr(val_df['Prediction'], val_df['Target'], val_df['Weight'])\n",
    "\n",
    "print('=> Overall Validation Score (weighted correlation) = {:.4f}'.format(r))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9035a57",
   "metadata": {
    "papermill": {
     "duration": 0.01787,
     "end_time": "2023-10-31T14:05:29.352860",
     "exception": false,
     "start_time": "2023-10-31T14:05:29.334990",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "### Evaluating the individual models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "cdcb4527",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T14:05:29.393091Z",
     "iopub.status.busy": "2023-10-31T14:05:29.392661Z",
     "iopub.status.idle": "2023-10-31T14:05:29.708429Z",
     "shell.execute_reply": "2023-10-31T14:05:29.707006Z"
    },
    "papermill": {
     "duration": 0.340708,
     "end_time": "2023-10-31T14:05:29.711809",
     "exception": false,
     "start_time": "2023-10-31T14:05:29.371101",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "asset_name = feature_df.query('Asset_ID == @asset')['Asset_Name'].values[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "52afaeea",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-10-31T14:05:29.752227Z",
     "iopub.status.busy": "2023-10-31T14:05:29.751799Z",
     "iopub.status.idle": "2023-10-31T14:05:49.959137Z",
     "shell.execute_reply": "2023-10-31T14:05:49.957863Z"
    },
    "papermill": {
     "duration": 20.2307,
     "end_time": "2023-10-31T14:05:49.961698",
     "exception": false,
     "start_time": "2023-10-31T14:05:29.730998",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INDIVIDUAL MODEL *******************************************\n",
      "\n",
      "- Cardano: Validation Score (weighted correlation) = 0.0065\n",
      "\n",
      "- Bitcoin Cash: Validation Score (weighted correlation) = 0.0115\n",
      "\n",
      "- Binance Coin: Validation Score (weighted correlation) = 0.0085\n",
      "\n",
      "- Bitcoin: Validation Score (weighted correlation) = 0.0049\n",
      "\n",
      "- Dogecoin: Validation Score (weighted correlation) = 0.0015\n",
      "\n",
      "- EOS.IO: Validation Score (weighted correlation) = 0.0063\n",
      "\n",
      "- Ethereum Classic: Validation Score (weighted correlation) = 0.0042\n",
      "\n",
      "- Ethereum: Validation Score (weighted correlation) = 0.0121\n",
      "\n",
      "- Litecoin: Validation Score (weighted correlation) = 0.0074\n",
      "\n",
      "- TRON: Validation Score (weighted correlation) = -0.0028\n",
      "\n",
      "- Stellar: Validation Score (weighted correlation) = 0.0081\n",
      "\n",
      "- Monero: Validation Score (weighted correlation) = 0.0143\n",
      "\n",
      "- IOTA: Validation Score (weighted correlation) = 0.0138\n",
      "\n",
      "- Maker: Validation Score (weighted correlation) = 0.0086\n",
      "=> Overall Validation Score (weighted correlation) = 0.0051\n"
     ]
    }
   ],
   "source": [
    "print('INDIVIDUAL MODEL *******************************************')\n",
    "val_df['Prediction'] = 0\n",
    "for asset in val_df['Asset_ID'].unique():\n",
    "    asset_name = feature_df.query('Asset_ID == @asset')['Asset_Name'].values[0]\n",
    "    # load model\n",
    "    model = joblib.load(os.path.join(CFG.OUTPUT_DIR, 'lgb_model_{}_val.pkl'.format(asset_name)))\n",
    "    \n",
    "    # inference\n",
    "    val_df.loc[val_df['Asset_ID'] == asset, 'Prediction'] = model.predict(val_df.loc[val_df['Asset_ID'] == asset, features_selected_for_training])\n",
    "    # Explanation of => vmodel.predict(val_df.loc[val_df['Asset_ID'] == asset, features_selected_for_training])\n",
    "    # The loc function is a label-based indexer that allows you to select rows and columns by their labels. \n",
    "    # Essentially, this line selects all the rows where the 'Asset_ID' matches the 'asset' variable.\n",
    "    # And then => ..., features_selected_for_training: => This part specifies the columns to be selected from the filtered DataFrame. \n",
    "    # So 'features_selected_for_training\" is a list of column names that you want to keep in the output. So overall, The loc function will return only these columns for the selected rows.\n",
    "    \n",
    "    tmp = val_df.query('Asset_ID == @asset')\n",
    "    \n",
    "    asset_name = tmp['Asset_Name'].values[0]\n",
    "    \n",
    "    r = corr(tmp['Prediction'], tmp['Target'], tmp['Weight'])\n",
    "    \n",
    "    print('')\n",
    "    \n",
    "    print('- {}: Validation Score (weighted correlation) = {:.4f}'.format(asset_name, r))\n",
    "    \n",
    "r = corr(val_df['Prediction'], val_df['Target'], val_df['Weight'])\n",
    "print('=> Overall Validation Score (weighted correlation) = {:.4f}'.format(r))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1237a5a9",
   "metadata": {
    "papermill": {
     "duration": 0.021034,
     "end_time": "2023-10-31T14:05:50.004134",
     "exception": false,
     "start_time": "2023-10-31T14:05:49.983100",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The full model seems to be performing better than the individual models."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 546.208921,
   "end_time": "2023-10-31T14:05:51.353391",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2023-10-31T13:56:45.144470",
   "version": "2.4.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}