Dota2_ml.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Предсказание результатов игры"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Введение\n",
    " Это задание - финальный проект курса\n",
    "Coursera [Введение в машинное обучение](https://www.coursera.org/learn/vvedenie-mashinnoe-obuchenie). На [Kaggle](https://kaggle.com/join/coursera_ml_dota2_contest) есть соотвествующее этому заданию соревнование.\n",
    "\n",
    " В данной работе с помощью методов машнного обучения *градиентного бустинга* и *логистической регресии* предсказывается результат игры Dota 2 по первым 5 минутам игры. \n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#общие библиотеки работы с данными\n",
    "import numpy as np\n",
    "import pandas\n",
    "\n",
    "#визуализация\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "from sklearn.cross_validation import cross_val_score\n",
    "from sklearn.cross_validation import KFold\n",
    "\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "from sklearn.linear_model import LogisticRegression \n",
    "\n",
    "import time\n",
    "import datetime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Загрузка данных\n",
    " Данные для обучения и проверки находятся в файлах `features.csv` и `features_test.csv`. В качестве индексного поля используется индентификатор матча - `match_id`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97230, 108)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train = pandas.read_csv('features.csv',index_col='match_id')\n",
    "\n",
    "test = pandas.read_csv('features_test.csv',index_col='match_id')\n",
    "#test = pandas.read_csv('test_items_bag.csv',index_col='match_id')\n",
    "\n",
    "train.shape #Размер выборки для обучения"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17177, 102)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.shape #Размер выборки для предсказания"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>start_time</th>\n",
       "      <th>lobby_type</th>\n",
       "      <th>r1_hero</th>\n",
       "      <th>r1_level</th>\n",
       "      <th>r1_xp</th>\n",
       "      <th>r1_gold</th>\n",
       "      <th>r1_lh</th>\n",
       "      <th>r1_kills</th>\n",
       "      <th>r1_deaths</th>\n",
       "      <th>r1_items</th>\n",
       "      <th>...</th>\n",
       "      <th>dire_boots_count</th>\n",
       "      <th>dire_ward_observer_count</th>\n",
       "      <th>dire_ward_sentry_count</th>\n",
       "      <th>dire_first_ward_time</th>\n",
       "      <th>duration</th>\n",
       "      <th>radiant_win</th>\n",
       "      <th>tower_status_radiant</th>\n",
       "      <th>tower_status_dire</th>\n",
       "      <th>barracks_status_radiant</th>\n",
       "      <th>barracks_status_dire</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>match_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1430198770</td>\n",
       "      <td>7</td>\n",
       "      <td>11</td>\n",
       "      <td>5</td>\n",
       "      <td>2098</td>\n",
       "      <td>1489</td>\n",
       "      <td>20</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>-52</td>\n",
       "      <td>2874</td>\n",
       "      <td>1</td>\n",
       "      <td>1796</td>\n",
       "      <td>0</td>\n",
       "      <td>51</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1430220345</td>\n",
       "      <td>0</td>\n",
       "      <td>42</td>\n",
       "      <td>4</td>\n",
       "      <td>1188</td>\n",
       "      <td>1033</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>-5</td>\n",
       "      <td>2463</td>\n",
       "      <td>1</td>\n",
       "      <td>1974</td>\n",
       "      <td>0</td>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1430227081</td>\n",
       "      <td>7</td>\n",
       "      <td>33</td>\n",
       "      <td>4</td>\n",
       "      <td>1319</td>\n",
       "      <td>1270</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>12</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>13</td>\n",
       "      <td>2130</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1830</td>\n",
       "      <td>0</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1430263531</td>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>4</td>\n",
       "      <td>1779</td>\n",
       "      <td>1056</td>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>27</td>\n",
       "      <td>1459</td>\n",
       "      <td>0</td>\n",
       "      <td>1920</td>\n",
       "      <td>2047</td>\n",
       "      <td>50</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1430282290</td>\n",
       "      <td>7</td>\n",
       "      <td>13</td>\n",
       "      <td>4</td>\n",
       "      <td>1431</td>\n",
       "      <td>1090</td>\n",
       "      <td>8</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>8</td>\n",
       "      <td>...</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>-16</td>\n",
       "      <td>2449</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1974</td>\n",
       "      <td>3</td>\n",
       "      <td>63</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 108 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          start_time  lobby_type  r1_hero  r1_level  r1_xp  r1_gold  r1_lh  \\\n",
       "match_id                                                                     \n",
       "0         1430198770           7       11         5   2098     1489     20   \n",
       "1         1430220345           0       42         4   1188     1033      9   \n",
       "2         1430227081           7       33         4   1319     1270     22   \n",
       "3         1430263531           1       29         4   1779     1056     14   \n",
       "4         1430282290           7       13         4   1431     1090      8   \n",
       "\n",
       "          r1_kills  r1_deaths  r1_items          ...           \\\n",
       "match_id                                         ...            \n",
       "0                0          0         7          ...            \n",
       "1                0          1        12          ...            \n",
       "2                0          0        12          ...            \n",
       "3                0          0         5          ...            \n",
       "4                1          0         8          ...            \n",
       "\n",
       "          dire_boots_count  dire_ward_observer_count  dire_ward_sentry_count  \\\n",
       "match_id                                                                       \n",
       "0                        4                         2                       2   \n",
       "1                        4                         3                       1   \n",
       "2                        4                         3                       1   \n",
       "3                        4                         2                       0   \n",
       "4                        3                         3                       0   \n",
       "\n",
       "          dire_first_ward_time  duration  radiant_win  tower_status_radiant  \\\n",
       "match_id                                                                      \n",
       "0                          -52      2874            1                  1796   \n",
       "1                           -5      2463            1                  1974   \n",
       "2                           13      2130            0                     0   \n",
       "3                           27      1459            0                  1920   \n",
       "4                          -16      2449            0                     4   \n",
       "\n",
       "          tower_status_dire  barracks_status_radiant  barracks_status_dire  \n",
       "match_id                                                                    \n",
       "0                         0                       51                     0  \n",
       "1                         0                       63                     1  \n",
       "2                      1830                        0                    63  \n",
       "3                      2047                       50                    63  \n",
       "4                      1974                        3                    63  \n",
       "\n",
       "[5 rows x 108 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Подготовка данных к обучению\n",
    "### Удаление колонок с итоговыми данными\n",
    "В исходной выборке присутствуют итоговые данные, название колонок с итоговыми данными объявлены в списке `resultColumns`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "resultColumns = ['duration', 'radiant_win', 'tower_status_radiant',\n",
    "                 'tower_status_dire', 'barracks_status_radiant', 'barracks_status_dire']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Создадим новую выборку, без колоной с результатами матча, с помощью фунции `filter`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "trainReduced=train[filter(lambda x: x not in resultColumns, train.columns)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97230, 102)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainReduced.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17177, 102)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Проверка данных на пропуски"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "colCounts = (97230 -  trainReduced.count())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "first_blood_time               19553\n",
       "first_blood_team               19553\n",
       "first_blood_player1            19553\n",
       "first_blood_player2            43987\n",
       "radiant_bottle_time            15691\n",
       "radiant_courier_time             692\n",
       "radiant_flying_courier_time    27479\n",
       "radiant_first_ward_time         1836\n",
       "dire_bottle_time               16143\n",
       "dire_courier_time                676\n",
       "dire_flying_courier_time       26098\n",
       "dire_first_ward_time            1826\n",
       "dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colCounts[colCounts > 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Среди пропущенных данных, много полей которые относят к \"первой крови\"*. Это можно объяснить, тем что событие первая кровь не всегда наступает в первые 5 минут. Если первая кровь не наступила, то и другие данные связанные с этим событием отсуствуют.\n",
    "\n",
    "Тем же (отсуствием события в первые 5 мин), обхясняется отсуствие полей `radiant_` , `dire_`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "first_blood_time               3552\n",
       "first_blood_team               3552\n",
       "first_blood_player1            3552\n",
       "first_blood_player2            7766\n",
       "radiant_bottle_time            2895\n",
       "radiant_courier_time            127\n",
       "radiant_flying_courier_time    4885\n",
       "radiant_first_ward_time         330\n",
       "dire_bottle_time               2842\n",
       "dire_courier_time               130\n",
       "dire_flying_courier_time       4524\n",
       "dire_first_ward_time            263\n",
       "dtype: int64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "colCountsTest = (17177 -  test.count())\n",
    "colCountsTest[colCountsTest > 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Заполнение пропущенных данных\n",
    "Пропущенные данные заполняются нулями."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "trainReduced=trainReduced.fillna(0)\n",
    "test = test.fillna(0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Series([], dtype: int64)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Проверка пропусков после заполнения\n",
    "colCounts = (trainReduced.count() - 97230 )\n",
    "colCounts[colCounts < 0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Series([], dtype: int64)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Проверка пропусков после заполнения для тестовых данных\n",
    "colCountsTest = (17177 -  test.count())\n",
    "colCountsTest[colCountsTest > 0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Целевая переменная\n",
    "Целевая перемнная содержится в колонке `radiant_win`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97230,)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y=train['radiant_win']\n",
    "Y.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Модификация `GradientBoostingClassifier` и `LogisticRegression` для использования оценок с `predict_proba`\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class GradientBoostingClassifier2(GradientBoostingClassifier):\n",
    "    def predict(X):\n",
    "        return GradientBoostingClassifier.predict_proba(self, X)\n",
    "    \n",
    "class LogisticRegression2(LogisticRegression):\n",
    "    def predict(X):\n",
    "        return LogisticRegression.predict_proba(self, X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Подход 1.  градиентный бустинг \"в лоб\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Выбор параметров обучения\n",
    "Для подбора параметров обучения, проведем кросс валидацию для разного числа деревьев."
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "roc_auc1=[]\n",
    "numOfTrees=[]\n",
    "timesElapsed1=[]\n",
    "for i in range(10,40,5):\n",
    "    start_time = datetime.datetime.now()\n",
    "    scores = cross_val_score(GradientBoostingClassifier2(n_estimators=i,random_state=1),\n",
    "                            trainReduced, Y, cv=KFold(len(trainReduced),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')\n",
    "    timeElapsed = datetime.datetime.now() - start_time\n",
    "    timesElapsed1.append(timeElapsed)\n",
    "    roc_auc1.append(scores.mean())\n",
    "    numOfTrees.append(i)\n",
    "    print 'i %d  score %f, time elapsed:%s' % (i, scores.mean(),timeElapsed)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97230, 102)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainReduced.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "trainRSample=trainReduced.sample(n=10000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 102)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainRSample.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Int64Index([ 46355,  58109,  87306,  99282,  73245,  97237,  49527, 104399,\n",
       "             28609,  24673,\n",
       "            ...\n",
       "             23503,  42826,  38027,  13351,  97547,  70076,  23257,  43413,\n",
       "             25014,  29944],\n",
       "           dtype='int64', name=u'match_id', length=10000)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainRSample.index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "YSample=Y.ix[trainRSample.index]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000,)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "YSample.shape"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "roc_auc1=[]\n",
    "numOfTrees=[]\n",
    "timesElapsed1=[]\n",
    "for i in range(10,200,30):\n",
    "    start_time = datetime.datetime.now()\n",
    "    scores = cross_val_score(GradientBoostingClassifier2(n_estimators=i,random_state=1,max_depth=7),\n",
    "                            trainRSample, YSample, cv=KFold(len(trainRSample),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')\n",
    "    timeElapsed = datetime.datetime.now() - start_time\n",
    "    timesElapsed1.append(timeElapsed)\n",
    "    roc_auc1.append(scores.mean())\n",
    "    numOfTrees.append(i)\n",
    "    print 'i %d  score %f, time elapsed:%s' % (i, scores.mean(),timeElapsed)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#from sklearn.neural_network import MLPClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Улучшение качества при изменении кол-ва деревьев"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def plotAll(roc_auc,time_elapsed,par):\n",
    "    timesElapsed1Sec = [ x.total_seconds() for x in time_elapsed ]\n",
    "    fig=plt.figure()\n",
    "    ax1 = fig.add_subplot(111)\n",
    "    ax1.plot(par,roc_auc, 'ro-')\n",
    "    #ax1.set_xlabel('number of trees')\n",
    "    ax1.set_ylabel('ROC AUC value')\n",
    "    ax2 = ax1.twinx()\n",
    "    ax2.plot(par,timesElapsed1Sec, 'g.-')\n",
    "    ax2.set_ylabel('Elapsed time, s')\n",
    "    plt.show"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "plotAll(roc_auc1,timesElapsed1,numOfTrees)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Как видно из графика, рост сткорости качества ROC AUC замедляется, а время необходимое для вычислений растет.\n",
    "Однако, рост качества при увеличении количества деревьев продалажается."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Подход 2. логистическая регрессия\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Масштабирование признаков"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Scale data\n",
    "standard_scaler = StandardScaler()\n",
    "Xtr_s = standard_scaler.fit_transform(trainReduced)\n",
    "#Xte_s = standard_scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def optimize_reg(X,Y, C_xrange):\n",
    "    roc_auc2=[]\n",
    "    Coeff2=[]\n",
    "    timesElapsed2=[]\n",
    "    for i in C_xrange:    \n",
    "        start_time = datetime.datetime.now()\n",
    "        scores = cross_val_score(LogisticRegression2(C=i,penalty='l2',random_state=1, n_jobs=-1),\n",
    "                            X, Y, cv=KFold(len(X),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')\n",
    "        timeElapsed = datetime.datetime.now() - start_time\n",
    "        timesElapsed2.append(timeElapsed)\n",
    "        roc_auc2.append(scores.mean())\n",
    "        Coeff2.append(i)\n",
    "        print 'i %f  score %f, time elapsed:%s' % (i, scores.mean(),timeElapsed)\n",
    "    return [roc_auc2,Coeff2,timesElapsed2]      "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "[roc_auc2,Coeff2,timesElapsed2] = optimize_reg(Xtr_s, Y, np.arange(0.001,0.01,0.001))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "plotAll(roc_auc2,timesElapsed2,Coeff2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Исключение из выборки категориальных признаков"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "catFeatures = ['lobby_type', 'r1_hero','r2_hero','r3_hero','r4_hero',\n",
    "              'r5_hero','d1_hero','d2_hero','d3_hero','d4_hero','d5_hero']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97230, 91)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "trainReduced2=trainReduced[filter(lambda x: x not in catFeatures, trainReduced.columns)]\n",
    "testReduced2=test[filter(lambda x: x not in catFeatures, test.columns)]\n",
    "\n",
    "standard_scaler2 = StandardScaler()\n",
    "Xtr_s2 = standard_scaler2.fit_transform(trainReduced2)\n",
    "Xts_s2 = standard_scaler2.transform(testReduced2)\n",
    "trainReduced2.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17177, 91)"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "testReduced2.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "[roc_auc3,Coeff3,timesElapsed3] = optimize_reg(Xtr_s2, Y, np.arange(0.001,0.01,0.001))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "plotAll(roc_auc3,timesElapsed3,Coeff3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Результат исключения категориальных признаков\n",
    "После исключения категориальных признаков, результат стал лучше, ROC AUC был: \n",
    "\n",
    "|#|До исключения|После|\n",
    "|---|---|---|\n",
    "|ROC_AUC|0.716360|0.716414|"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Формирование \"мешка слов\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "catFeatures2 = ['r1_hero','r2_hero','r3_hero','r4_hero',\n",
    "              'r5_hero','d1_hero','d2_hero','d3_hero','d4_hero','d5_hero']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Найдем количество уникальных героев"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 11,  67,  29,  20, 105,   4,  42,  21,  37,  84,  49,  26,  39,\n",
       "        88,  79,   7,  12,  33,  98,  27,  22,  66,  86,  80,  30,  75,\n",
       "        41,  96,  48,  15, 102,  13,  72,  93,  69,  25,   8,  28,  65,\n",
       "        55,  52,   3,  73,  57,  36, 101,  47,  35,  83, 100,  44,  17,\n",
       "        91,  53,  90,  19,  74,  76,  99,   1,  94,  82,  60, 112,  31,\n",
       "        85,  61,  70,  68,  51,   2,  50,  71,  23,  77,  16,  63,  92,\n",
       "        81,  18,  59,  95,  64,  58,  43, 104,  40,  87,  10,  14, 110,\n",
       "        38,  46,  54,  97,  89,   5,  62,   6,  78, 106,  34,  56, 103,\n",
       "       109,  45,   9,  32])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "heros = pandas.Series(trainReduced[catFeatures2].values.ravel()).unique()\n",
    "heros"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "108"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "S=heros.shape\n",
    "N=S[0]\n",
    "N"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "112"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "max(heros)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "112"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "herosTest = pandas.Series(test[catFeatures2].values.ravel()).unique()\n",
    "max(herosTest)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Проверим, совпадают ли списки герои в тренировочной и тестовой выборке"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(set(heros) - set(herosTest))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(set(herosTest) - set(heros))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "## Функция формирования мешка слов из задания\n",
    "def gen_bag(data,maxHeros):\n",
    "    X_pick = np.zeros((data.shape[0], maxHeros+1))\n",
    "    X_pick_columns = [];\n",
    "\n",
    "    for i, match_id in enumerate(data.index):\n",
    "        for p in xrange(5):\n",
    "            X_pick[i, data.ix[match_id, 'r%d_hero' % (p+1)]] = 1\n",
    "            X_pick[i, data.ix[match_id, 'd%d_hero' % (p+1)]] = -1\n",
    "    return X_pick"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Сформируем матрицы со словами\n",
    "X_pick = gen_bag(trainReduced, max(heros))\n",
    "X_pick_test = gen_bag(test,max(heros))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_pick_columns =[]\n",
    "for i in range(1,114):\n",
    "    X_pick_columns.append('r%d_hero' % i)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "113"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(X_pick_columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17177, 113)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_pick_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97230, 113)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_pick.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17177, 91)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Xts_s2.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Приклеиваем категориальные признаки героев"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Склеиваем матрицу с масшатбированной матрицей без категориальных признаков\n",
    "X_concat = np.hstack((trainReduced2,X_pick))\n",
    "X_concat_test = np.hstack((testReduced2,X_pick_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97230, 204)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_concat.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_concat_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "[roc_auc4,Coeff4,timesElapsed4] = optimize_reg(X_concat, Y, np.arange(0.005,0.1,0.005))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "plotAll(roc_auc4,timesElapsed4,Coeff4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "max(roc_auc4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "C_opt = Coeff4[roc_auc4.index(max(roc_auc4))]\n",
    "C_opt\n",
    "#0.049999999999999996"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Оптимальная модель\n",
    "\n",
    "Как видно из результата кросс-валидации качество модели улучшилось. \n",
    "\n",
    "|Метод| ROC_AUC|\n",
    "|---|---|---|\n",
    "| только масштабированная|0.716360|\n",
    "|+ без категориальных признаков|0.716414|\n",
    "|+ \"мешок слов\"|0.751944|\n",
    "|+ \"мешок слов комната\"|0.752097|\n",
    "|+ \"нелиненые параметры\"|0.7524|\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Максимальное и минимальное качество на тестовой модели"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores = cross_val_score(LogisticRegression2(C=C_opt,penalty='l2',random_state=1, n_jobs=-1),\n",
    "                            X_concat, Y, cv=KFold(len(X_concat),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores.mean()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores.max()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores.min()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "(scores.max()-scores.min())/scores.max()*100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Таким образом, разница между минимальным и максимальным качеством на тестовой выборке для оптимального алгоритма 0.7%"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "trainReduced['lobby_type'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1    9857\n",
       "7    5029\n",
       "0    2291\n",
       "Name: lobby_type, dtype: int64"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test['lobby_type'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Функция формирования мешка слов из задания\n",
    "def gen_bag2(data,maxLobbies):\n",
    "    X_pick = np.zeros((data.shape[0], maxLobbies+1))\n",
    "\n",
    "    for i, match_id in enumerate(data.index):\n",
    "        for p in xrange(3):\n",
    "            X_pick[i, data.ix[match_id, 'lobby_type']] = 1\n",
    "    return X_pick"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_pick2 = gen_bag2(trainReduced,7)\n",
    "X_pick2_test = gen_bag2(test,7)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "X_pick2_columns =[]\n",
    "for i in range(1,9):\n",
    "    X_pick2_columns.append('lobby_type%d' % i)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "len(X_pick2_columns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97230, 8)"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_pick2.shape"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_pick2[:,7].max()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "type(X_pick2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X_pick3 = np.delete(X_pick2,[2,3,4,5,6],1)\n",
    "X_pick3_test = np.delete(X_pick2_test,[2,3,4,5,6],1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97230, 3)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_pick3.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_pick2_columns = ['lobby_type_0', 'lobby_type_1', 'lobby_type_7']"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_pick3[:,2].max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Приклеиваем тип комнаты"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_concat = np.hstack((X_concat,X_pick3))\n",
    "X_concat_test = np.hstack((X_concat_test,X_pick3_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "scores2 = cross_val_score(LogisticRegression2(C=0.049999999999999996,penalty='l2',random_state=1, n_jobs=-1),\n",
    "                            X_concat, Y, cv=KFold(len(X_concat),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.516677350142284"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scores2.max()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "[roc_auc5,Coeff5,timesElapsed5] = optimize_reg(X_concat, Y, np.arange(0.005,0.1,0.005))"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "C_opt2 = Coeff4[roc_auc5.index(max(roc_auc5))]\n",
    "C_opt2\n",
    "# 0.049999999999999996"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Предсказание на тестовых данных"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "clf=LogisticRegression(C=0.05,penalty='l2',random_state=1, n_jobs=-1)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "clf.fit(X_concat,Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_concat_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_concat.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Y_pred = clf.predict_proba(X_concat_test)[:, 1]"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# Резльтирующие коэффициенты логистической регрессии\n",
    "plt.plot(clf.coef_[0],'ro-')"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Y_pred"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "type(Y_pred)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "test['radiant_win'] = Y_pred"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "test['radiant_win'].to_csv('prediction_result.csv')"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "result = pandas.read_csv('prediction_result.csv', header=None)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "result.head()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "result[1].min()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "result[1].max()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Предсказание с учетом комнаты"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "standard_scaler3 = StandardScaler()\n",
    "X_concat = standard_scaler2.fit_transform(X_concat)\n",
    "X_concat_test = standard_scaler2.transform(X_concat_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=0.005, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=-1,\n",
       "          penalty='l2', random_state=1, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf2=LogisticRegression(C=0.005,penalty='l2',random_state=1, n_jobs=-1)\n",
    "clf2.fit(X_concat,Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fd88ff2c150>]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEACAYAAABRQBpkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXl8VNXd/z93Mtl3trCFmyEsorhRF6xWg4rWBSjuCLaD\n1K06blDbKnAz0lqx4/J7YuvS6jOtgtRHRa31UbE0bo8odUGRRUlmBghrSCDJZJ3M9/fHmXvn3pl7\nZ+5kJjKB8369eJFkzpx77nY+57uccwQiAofD4XCObiyHuwEcDofDOfxwMeBwOBwOFwMOh8PhcDHg\ncDgcDrgYcDgcDgdcDDgcDoeDFImBIAg/FgRhiyAI3wqC8Cudz2cKgrBBEIQvBEH4jyAI56biuBwO\nh8NJDUKy8wwEQbAA+BbAeQB2AVgP4Boi2qIqk0dE7aGfjwewmojGJXVgDofD4aSMVFgGpwH4joh8\nRNQDYBWAWeoCshCEKADQmILjcjgcDidFpEIMRgHYofp9Z+hvGgRB+IkgCJsBvAng9hQcl8PhcDgp\n4nsLIBPRq0Q0CcAMAM99X8flcDgcTnysKaijAcAY1e+jQ3/ThYg+FATBKgjCYCI6EPm5IAh8sSQO\nh8NJECISkvl+KiyD9QDGCYIgCoKQBeAaAK+rCwiCUKn6eQoA6AmBDBHxf0SQJOmwtyEd/vHrwK8F\nvxax/6WCpC0DIuoVBOE2AO+AicszRLRZEISb2Mf0NIDLBUH4KYBuAH4AVyd7XA6Hw+GkjlS4iUBE\nbwGYGPG3p1Q/PwTgoVQci8PhcDiph89ATmOqqqoOdxPSAn4dwvBrEYZfi9SS9KSzVCMIAqVbmzgc\nDiedEQQBlAYBZA6Hw+EMcLgYcDgcDoeLAYfD4XC4GHA4HA4HXAw4HA6HAy4GHA6HwwEXAw6Hw+GA\niwGHw+FwwMWAw+FwOOBiwOFwOBxwMeBwOBwOuBhwOBwOB1wMOBwOhwMuBhwOh8MBFwMOh8PhgIsB\nh8PhcMDFgMPhcDjgYsDhcDgccDHgcDgcDrgYcDgcDgdcDDgcDocDLgYcDofDARcDDofD4QCwHu4G\nfF/4PB64lyxBsKEBllGjYF+2DKLNlnAZDofDORIRiOhwt0GDIAiUSJvMdvI106fDWVeHfAB+AFJl\nJRxr1ihlzZThcDicdEQQBBCRkFQlRJRW/1iTzOGtr6eFlZXUBhAB1AbQwspK8tbXa8pVz52rlCFV\n2eq5cxMqw+FwOOlIqN9Mqu9NScxAEIQfC4KwRRCEbwVB+JXO59cKgrAh9O9DQRCON1Ovz+OBc948\nSNOmwTlvHnwej+Zz95IlykgeAPIBOOvq4F6yRFMu2NCglIGqbHDXroTKcDgczpFK0jEDQRAsAB4H\ncB6AXQDWC4LwGhFtURWrB3A2ER0SBOHHAP4MYGqsenXdNuvWadw2Zjtwy6hR8Ic+k/EDsIwcmVAZ\nDofDOVJJhWVwGoDviMhHRD0AVgGYpS5AROuI6FDo13UARsWr1MyoX+7A1eh14PZlyyCNHauUleMB\n9mXLtGUqK2OW4XA4nCOVVGQTjQKwQ/X7TjCBMOLnAP43XqVmRv32ZcsgrVsXHfSN6MBFmw2Ol16C\na8oUBKdMgWXSJDgiAs2izQbHmjVwnXUWgkSwnHtuVBmAZxxxOJwjk+81tVQQhGkA5gM4K17Z9qKi\nuG4b0WaD45134Dr2WAQFAZbLL9ftwAFALCyEBAAPPwxUVekeU7TZIIkiMGEC4HZHfW7GdcXhcDgD\nkVSIQQOAMarfR4f+pkEQhBMAPA3gx0TUHKvCO++4Ax/U1uJDAEsBXATW8d47ZgzuVo36fR4P3AsX\nItjVBUt+fuxRelMT+z8QiH02TU2GZYxcV64lSyA9/3zsejkcDidF1NbWora2NqV1pkIM1gMYJwiC\nCGA3gGsAzFEXEARhDICXAVxHRHXxKiw9cADvt7SgEYAbLMgQBGA5+WTjeQF+P6Tp041H6SkQA55x\nxOFw0oGqqipUqTwcTqcz6TqTDiATUS+A2wC8A+AbAKuIaLMgCDcJgnBjqNgSAIMA/EkQhC8EQfg0\nVp1ypysCkAA4ASwDUNTSopQxm1aqYEYMiGKKgdmANYfD4Qw0UhIzIKK3AEyM+NtTqp9vAHCD2frM\npHkmPEo3IwatrUBvr2EZswFrDofDGWik5UJ1ZlJBEx6lmxGDOGWUjKOJEyEBcM2dy4PHHA7niCAt\nF6oTbTY4Xn8drhNPRLCiApbMTDjefFPT6SY8Sk+BGMhtk8aPB7ZuBf72N8CSlnrK4XA4CZGWYgAA\n4vDhkIqLgVtvBXw+IGL0rYzSp01DMDcXlm+/jT1KNysGghA/yFxfH64rK8vkGXE4HE76krZigI4O\nICcHyMwEenp0i4g2G6QpU4BrrgHmzo0SDA1NTcDgwfHFYPBgFjcwggjweJhFwMWAw+EcIaS3GOTm\nAlZr7A68qwvIz2dliNjIXo+mJmDYsPhiEKOMMq8hEIDFYoF92zaIJ5yQwElxOBxOepK+YtDZySyD\neGIgl8vIAIJB9r8eTU3A8OHmxKCzM+oj3dnHs2bBsXYtDyBzOJwBT/pGP81aBmZFI0nLQHdeg9dr\nPK+BwzlMxFv6ncPRI70tA7NuouzscLnsbM3HPo8H7sWLEdy/H5Yvv4R9wgSIOtX4PB64V69mS1u0\nt8Pu8WhG/Hz2MWcgwNfP4vSV9LYM4gSQAcS0DOQXY9HKlXACWPTdd6h54omokZJSrr4ezoYGLGpu\nRs306ZpyfPYxZyCQ8Mx8DidEeotBXywDFbovRlNT1Ith5gXS3e9gzBi+3wEnreAWLKevpLebKJEA\nsk45sy+GmXLKvIaLL0awpweWffvgeOYZbnpz0gq+Yx+nrwx8y6CzM2wZRLiTzLp2zJYTbTZI55wD\n58KFkMrLIQ4fntApcTj9Dd+xj9NX0lcM5AByZmZ8N5GBZaD7YpSURL0YCb1A7e1AXl58keJwDgOK\nBXvqqZBKSvj6WRzTpK+bSA4g64z4NcRwEykvxs9+huDmzbCMGgXH1KlRL4Zos8Hx9ttwjRuH4Bln\nwPLFF8YvEBcDTpoj2myQ5s0DXnwR4JsucUyS3mIQz00UDLLPMjMNy4k2G6QFC4C1a4Hjjwf27NGt\nShw6FFJhIfDaa8CxxxovbSGLQTyLhcM5nPT0xB5EcTgRpK8YmAkgy5lEghC7XFsbUFAQu8yhQ0Bx\ncfwRf3s7W/6CWwb9hs/jgXvJEgQbGmAZNSr2dqYcfXp6gO7uw90KzgAifcWgowMoKord6crBYyA1\nYhDveIDWTcRHXimHT5pKEdwy4CTIwAggGz3UcvAY+H4tAx4z6Df4pKkUwcWAkyDpKwbqAHIqLIPC\nwtSKAY8Z9At80lSK4GLASZD0FoN4AeT+sAwyMsLLYevBLYN+hS/7kSJ4zICTIOkrBmYCyHIZIHa5\n1lbzYmCxsH/BoH45HjPoV/ikqRTBLQNOgqR3ADmeZZBoALmtLb4YqOuK3BuBCPD7zc2M5vQJZW7I\nKacgWFoKy9SpcBzF2UR9zqzSEQOepcWJRfqLQSoDyJ2d5sRAdhVFLIeN7m72WYx5DZzkEW02SCNH\nAldcAUjS4W7OYSOpzKoIMeBZWpx4DHw3USpTSyMtg0hkFxHAA8j9TWcns8KOYpLKrIqIGfAsLU48\n0lcMDlcAOVZdajHglkH/0tHB7ttRTFKZVRGWAc/S4sQjfcUglQFkM2LQ0pK4GPAAXf/BLYPkMqt6\neliMq7c3+bo4RwXpKwbqmMHhcBOFXiIN3DL4/ujoOOrFIKnMKnmgEvrfvmwZpIoKnqXFMST9A8ix\nRuBm3ERETAzirSfEYwbpAxG3DKDKrDr5ZARHjoRlyhTzmVVqMcjJYXX99a9wnXMOW5l37NijOkuL\nE036ikGqAshdXeyzeBlAPGaQPgQCbJ7HABSDVKdvijYbpKFDgUWLgOuvN/9FWQxUQWSxpAQSADz7\nLHDMMX1uE+fIJCViIAjCjwE8BuZ2eoaIlkd8PhHAfwOYAuBeInokbqWyZWCxsJFiMMh+VmPGMpBd\nRLHKAOGF6mKV4zGD74eODvb/ABODfkvf7OpKfDZxhJsIQDgg39nZ97ZwjliSjhkIgmAB8DiACwEc\nB2COIAiRw44DABwA/mCqUnk5CKs19vLUZiyDOGLg83jgnDsXUnMznA4HfB4PtwwON3JnNcDEoN/S\nN7u6Eh94yM+m+nutrex/LgYcHVJhGZwG4Dsi8gGAIAirAMwCsEUuQESNABoFQbjUVI2yVSAI7HfZ\nP5+VpS1nJpsohhhEjeReeAHSp5/CkZEBkYvB4aOjg937ASYG/Za+2d2dGsuAiwEnBqnIJhoFYIfq\n952hv/UdWQxkjFwySbqJDEdy+/fzAPLhpLMTKC0dcGLQb+mbfbEMdGIG3E3EiUVaBpCrf/979gJU\nV6OqqgpV/eQmMhzJBQLcTXQ46ewEhgwBvN7D3ZKEsC9bBmndOm3MoLISjmTTN1MVM5AtAzkmwxmw\n1NbWora2NqV1pkIMGgCMUf0+OvS3PiNs24ZgIADLtm2w/exnsTOF4lkGra1sLwOdMvJITi0IfgCW\n7GweQD6cdHSwzK5AgF3jzMx+PVyqMoCUVNDjjkNwwgRYJk9OPn1Tzqzqi2WQlcUDyEcoVVVVqKqq\nUn53Op1J15kKMVgPYJwgCCKA3QCuATAnRnkhXoWL3niDjaxWrGDZGIC+Dz9Jy8BwJFdaaiwGgwbF\nPh4neeRd7vLzmauopKTfDpXqDCDRZoOUnw84ncCsWck3ULYI+mIZ5OXxmAHHNEnHDIioF8BtAN4B\n8A2AVUS0WRCEmwRBuBEABEEoEwRhB4C7ANwnCMJ2QRAKjOqM8uH7/cZuohiWgc/jgbOmBtLatXDO\nmwffrl2aMspIbto0SKWlcM2dyzqBwkIeMzicyDGjgoJ+jxv0SwZQdzezWlOBXE9fLIO8PPh8Pjjn\nzYM0bRqcL78MH8DFgKNLSmIGRPQWgIkRf3tK9fNeAOV9qTsfQNDITO7q0loGqoc8asS3YgWkjz6C\no7MToqoK0WaDdNttwPPPs39yXTxmcPiQRV62DPqRfskA6g8x6INl4LNaUXPXXXDu2hW2egQBjp07\nNe8AhwOk89pEIfwALEaj8BiWge6Iz+uFWzaVNQfxs45HxsjK+Ne/ID3yCLMympt5zKC/+B7FIOUZ\nQESsA08DMXC3tChCAITeASK4//nP1LSNc0SRltlEclBX8eELQsIBZMMRn94CdH5/eMSvU5diZfh8\nrF2bNkEaNgyOH/6Qj7D6A9lNlCIxiBUgTnkGUG9vWBBSgSwCfXATBYNB/Xfg4MFUtIxzhJGWYuCa\nOhVBjweW889n2RizZyecWmqYKaR3wDiWga6VsW8fXF98gaN3H65+JIWWQbwAsRI3OussBAFYpk1L\nLgNI7rzTwDKwFBTA39IS/Q6oBz4cToi0FANp3jzgm2+AP/2J/cHAbePeuhXB22+H5ZhjYB83TpNx\npDviGzsWju3bow/Y3h5TDAytjPb2pM6TY0AKLQOjALFryRJIoRiRaLNBOuYYoLwccLuTOl6/iUEf\nLAP7ccdBAuDcuzf8DhQWwnHqqalpG+eIIi3FQBOoBaJy+pXRXnMz8j/7DP7PPoM0ZAgc06crbhtl\nxPejHyE4aBAsJ5wAhyRB1FutMTJ9MZH5CJzUk0LLwHSAuL09NZOx0sgyEIcOheOee+B64AFm9XR2\nwjFnDkSi1LStH0j1qq8c86SnGPTFbdPYCNenn2rcNqLNBmniRODee4Hzzguvfhq5AqrfD4xSraCh\nNx/h44/hrK8Pj7CGD4dj/PgUnjRHIXKeQRIYCnlkgLijgwlCsqSDZSDHxXJyIJaWQho1CsjIAHbu\nBMaPBzZsSE3bUky/rfrKMUV6ZhPpWQZm3DZ6HYd60pm8AmpkEDlOAFm02eBYtQqurCxI06ax+Qi/\n+x1Ebhn0Dx0dKbMMTO8W1tGRGstA7rxTlcvf3c2e20QsA3nWdmYm+7m9nQlBayswdGjazjPot1Vf\nOaZIX8tgjGqFi4jUUsPRnpxZpEYtBkC4o1cvcRAZM8jIiIpRiFlZkCZMANauZX9Ys4bPM+gvUmgZ\nKO7Cc85BMBiEpapKP0CcKjHoD8sgPz95Mdi/n1nDpaVpKwb9tuorxxTpKQZxYgb2ZcsgffQRnF5v\n2JwsK4PjuOOi6zISAzUm5hlgzx5g+PDYZTipoaMDvtZWuP/xDwSbmmD59tukfMeizQZpyhS2lMiz\nzxoeM23dRAUFibmJ1GLQ3c3Oq6CAiUFubtqJgRwn+HrTJnMuPU6/kJ5iEKdzFm02OP78Z7guuwzB\nU06BZeRIOM44A+K//x1dV2tr6sRgxAhtGT7prF/wNTai5o034Ny3T7tGVTK+456e8EJteqSzZVBY\n2DfLQF6orqMDOPZYYO9e5n5LIzFQxwkaASwBsAxIzZwPTkKkpxjEiRkAgJiTA2ny5LDb5tVX4y9U\nZ1BXvJgBAGD3bm4ZfE+4P/9cEQJAPx00YQIBYzEgYs9cOloG3d1MDPpqGcjLX1dWsvPLzYWvuRnu\nefPSImNHHSfIB3AHgAcB+DIyUHnNNcmv+soxTXqKgZmR+oEDwODBscvIL6Y60JuMZaCOY4TKDNRU\nuHRud7C9Pcp33Ahgw7vvQpo2rW/tlUfIenR3M0E4TJZBzHshu4n27TP/HbUYHDoEX1YW3F9/jWBD\nA1p++UsEt27FA998kxYZO5FxAhHMMpCs1r4LP6dPpKcYRFoGemsTHTjANkCR0evAI60CVTnNy+T1\nwt7UFF5aQi/jaPdu4PTTNW3y+f0DMhUu3VP4LJmZGt+xD8D/A/Dc3r3I37u3b+3t6TEORnd0sKQB\nlWXQZ7GURcCkGMS9F7IYNDSY/45KDHw7dqCmpwfOrVuRD2BJbS1+jeiVgROxulI5kDBMBtFbNobT\nvxBRWv0DQNXFxeR9801SuO46IrebNCxfTrRwYfj3NWuIzjtPW8bnIyov1/6tooK8771HCysrqY2N\nB6kNoIUVFeStr2dlFi8muv9+7ffOPpto7drw719/TdXFxUodpKqreu5cSmeq585N63Z7TzqJFo4a\npbRxcah9SbX39NOJxo/X/2zXLqKhQ4kEgSgYJG99ffTzUVkZfj5i8dprRAUFROecY6pZce/Fgw8S\nXXWV5jmO+52NG4kmTSJavpyqx4zRlF0a8T3l79OmmWpvUtfGbH2FheQFiLq64n63eu5cWlpVRdVz\n5/a5DUcCrCtPru9Ny3kGiw4dQs3NN8Pn8bA/9NVNZGAZuF0u/RVN5XxmVV0+j4etB79+PZyPPKJp\nU7C7O24qnPL9adPYaqfy9w8j/Z3Cl+w5iwAcTz4J1yWXQMrLg2/YsOTbGyuA3NHB3IQhH3tS+e6y\nj9+kZRD3Xsj1qQLIcb+jsgyCra2ashYgqVVaUz0XQEn9Pe88SILA5vBMm8as9KYmw+/J1tGiFSvg\nrK3FohUrUDN9elq8XwOVtHQT5QNwbt8eNl31XDvbt8O+YIHWtWNSDIKhtVoij6m8TFYr0N4ebY6/\n8QakzZuZOW61wpKRETMVLl3dMaZn5fYBM+cc183Q0QGxshLSk08Cp58O57Rp8K9YkVx744lBbi5z\nTXZ0JCyWmvPp7oY9OxuiSTGIey90UkvjfkclBhaLRVPWjuQydvoykIh3v0WbDdLChSwZ5LnngCuv\nZB8cOKBN2lBhZs0pToIka1qk+h/0TNdf/IK81dXR5mRZWdg0/Phj5gpQ869/EUWav5MnU/XFF8c3\nze+5hxbNnGlczusl74gRMU3mRNwxqTR549WVqKmfSNvinbOpY4siUX090aFDRAUFqXFNTJqkuIGi\n+PRToh/8gGjECKKGhoTvW1TbsrLIW1lpqllxz23RIqL77mOuJ7Pf+egjoqlTiZ58krxjx9LCnBxN\n2RsAWjRmDC3NzDT1rKnv/2UVFXGvjbr8nbNm0e0qV5XhvVu9mtXX3U00Ywb7+b33DNu0tKoqKXfX\nkQZS4CY67J1/VIP0HrDbb6fqH/wg9kO4fj17odW89hp7sNScdBJ5//GP2C+Ty0XeBQvo2pwc4wdu\n506ikSPZg//jH9NSgKpnzNA85GYf2FT6Yc3W5a2vp+qiIlo6YkTMDiHRthmd8x1Tp1L13Lk0e9iw\n+B1tWRnR7t1Evb1EFgtRIMDae+yxtDQvr29iOW4cO57fH/3Ze+8RnXUWUWUl0XffJXTOhsKh6rzj\n4a2vp+q8PFpaUhJ9bg4HG5xkZ0d/59JL2XN36aXa79TWEv3oR0TPPks0YgR5zzyTdc7TprH6LRai\nH/9Yv84I0Y+8FpsA+pnVanhtIsubjvesWsU+b2sjuvBCoqwsoldeMbxm6R73+r45YsUg6uW7+25a\nWlkZu2P94guiE0/UXqEVK4jmzNH+7ZRTiD79lD34VistHTSIqvPytC/TY49R9cSJsR/kPXtY0JGI\ndSYAEyQVZh9Ys+XMjNATekkGDya69trov/e1PoPymwCaX1BAbYgdwFTOLyODqq+4gp1fURFRczOr\n/JZbmNXQF0SRHWvfvujP3nqLaPp0osmTib76iohUHbTB6Flu63XFxfrnk5Vl2JSo+7h1KxO9SMuW\niOimm4gef5xZNZF8/rnuc0dr1hCdey7Rc88R5eURzZql/byggOjYY9l3OzqUNukJoJ51vAmgy0pL\naWlWVtS1ibz/pgPWbjf77MABZs2LItGf/xzzGqYykD3QSYUYpGXMwJWVpfWrW61sow7E8JMmmFoq\nVlRA6u0FiouBsjJA7bO2WhFsbcXPAbYePML+VUduLqRly7THk1MSW1o0hzK7i5YZP6zZ+INpn25P\nD/PJxplolaiPWO+cf1VQgBfa2pCPcAAz8j42ZGTAecIJqAmV87/0EqQvvoAjNxdiSwtbYry5mc0o\n7wuyH72tjS3WpkYdMwjFitz33INgezssmZlRPm71vXAZnI9FEHSboXsfP/gAjmAQot696OoKp1n3\n9rIUWHW7gehrop5nEJpopiEnB/D52M+HDgE5OYY++FsiAtAAMAnA5MJCODMywvuGh4h8XtphcH0i\n4z3qlNyuLjbb/8CB6OsRQrTZ4PjnP+E65hgER4+G5ZxzlPfKmSYT6gYaaSkG0tChUZ2z/dxzIbW1\nYUFdHV4E0APgq4IC/OrGG5UyUUHmlhbYTzpJuzWl3Il3drIxhdcLTJmibYDVCkteHoYAcABwAQiG\n/hVfcAF7uA4dihaDiJdSyZQ480wEd++G5dJL4fiv/4p6ONuLiuK+MGYDZqaDw/v3a9tugLo+HwA3\n2LXf7PHA5/FEnYvmJR07FpYzzsCkujrkr1sHgAUwIwX2rvJy7P2//8Mq1WQz5fyKiiDJInvwYHJi\nUFqqH0SWxSA3F776etTMnRvurHt6IE2frhFd9b3QOx+pqAgOg9npuvdx+3a4cnMh6c2D6Opiy0pk\nZbGMInXHbkYMAO2cHYCJQWMj63APHQLKygxFv00Q9J+nrCzdbJ/I5+UQogPW944Zg7sjA9byEhnd\n3exfeXlMMQAAsaSELVl/9tnA88+nbcLGQCEtU0s1s4EBIDMTYlERZj/7LP6QkYFFAH4L4IW2Nqy+\n/nqWTma1wtferk03+/xz1KxerU03k8XA72c/E0Ufz2qF/eSTIVVWYgjYy34PgI7KStz56KNKm+JZ\nBj6PB+7FixFsbIQlOxv2n/886qH0eTw49MUXWIJwyp/8wqiXWTY7QrcvWwZp0KD4Szbv2cP+jzPr\nVl4CejOAGkC59n/zeg1T+eSX1HnFFZCefx55qiWkRTCBfRDAT4uK4Jo7F8Unn4wTdWYd54MJMA4d\nYn84eJB1dHEydXRTW82IQV4e3E8/HTN10ufxoG7NGuVz+XxcAH6amQnX8cfDcc01ml331Bjex5wc\n/UlxXV1sBr0sBpHtBqLFQF6V10gMQsKHUaPYNUW4E1fjByBOnaq/BPiECbrPjnrJ8McBPAq2xIQL\n7D16EIDl5JOjO2e1ZdDdHdcyABAWo8ZGAHwJ7GRJTzGIfHhDHfi7Tz+Nmt5e/ZtttcLd1BT9MDQ3\nax8GtRiMGMGOpSMGYm4uG9WPHw9p/HiW/xzhulLEQH4pVGKg5EGvXAlnTw8WdXWh5he/iOo83UuW\n4NEdO+K+MEYvq3rEL1tFgUAAPwVwz6RJ0e2W2buXreIZxzIQbTY43nwTixEe/QL6naTSAS9YAB+g\ndOKRewoMAdABYNnixUwsWlqQCYP89/z88HVtbmb/x7AODPPPOzuZq0lPDOQZ77m5aI8hunLd4r59\nmraKYCJZOWwYpBkzII4eHV7iIoLI++gDGzl7OjvhPHAgWlxlMZCXo1YTyzKwWpmAAPqWQVkZc5Gq\n75HNFtXp3/noo+w9qKiAZLHAdeqp7Hnq6WFuq8g1w0LW8OKJE+ELXTsRYetpGYCiiEGTcp4A0N0N\nX1sbnB9+COn112PPU2lqYtcmZOXyJbCTIy3dRHqdM3p6Yt9sqxXBnp74D4NaDPLzw/8ijxcIsPzn\nM84Apk0D7HbdNoFI102kO0rZtSvKrSOfUz6g2aVNSjD+oGsiNzcb+0z37mWuOBOLs4lDhmAyELeT\n1BwbgKOhASJU7rIf/hDBPXtgOf98ON59F2JpKQDWQV4FnfhMQQGkk04Ki8HBg+zlb23VLkWi4vE7\n79R3p1kskOJYBr7eXmzevt3QzSbf00adtkrFxXCMGcNEoKQkvHx0xAZI5994IxyvvIKajg40gi2z\nsQxAfkcHqyfCJaXUkYhlYMZNlJfH2hkSA9Fmg+PFF+E69VQEKythmTpVs0icJIrwjR8Pt8eDZ6+/\nHpavvoIdgNjZGRWXE202lAAogMl4AaC4iXz19ajZvh3OQIBd21gr1h44wHZuC4lBf86fORoYUJZB\nzNGx1apMsNH9PKIu+P3wZWTAefAgpA8+0I5A1KN+vz86CA2wteEFgW2h2d7OflZ14GZHKWZG/ICq\nQx08GFJmZtSIX1d89uwxNpETEAO0tMScuap7bADuzz7TtF8aNgxOANLMmSyOExrp2pctwzOVlVgA\nZh0tBjBzQ9kkAAAgAElEQVSnoAAL/vlPiKNGad1E5eWGloHP48Gud97Rv+7BYFw3kXvjRizv6oIE\nrcvOkZ8P+7Jlyj1Vu4YkANeVlcExcybbW1geyWdnR7mzfB4PVl9/PX7Z0QEXgDsR9qUr1y3SrSHH\nDBK1DNRioBdAHjaMWQYhNxEAiEOHMpGbORPS889rg+Z1daj58kss2raNWVxNTagB4Nu6Nfp6Agg2\nNysJGJprmZUV7bKUzxOA+/HHFSGQr8mCujrcfe650TPam5qAiROZGBCZ39WOo8vAsAxC/nn7smWQ\nVq6Ekyh6dGy1wp6dDWnUKO0ItaxMm70jB5q3bUNNXR2cnZ3RIxC1GLS1Rbcnol1ob2cZKioxMDtK\n0d1f2WBGqFhRASkQYJ1DnCwOII6JvGcPE4P339f/XE1rKwuUlpbC2dwc1c5nr79e/9iRPvC9e5mf\nev169ntopCsLnfv88xHMykLmD36AGnlU+tpr7LrKHd/QoYZi4F6yBGM7O/WvOwAUFRmLQWEhgp2d\nmARt0oAFQNGECRBtNuWeNoIF0uWkAnHqVIh5eazu7m52f7Kz2Wi3qEjTPvnZlEL/4t6zkLj4ALgd\nDjTv24cde/di3PDhyOvuZqPzBCwDn8cD93ffISgIsBQXw15WFk6wkK9x5DMTDMK9axecwWCU4Lt+\n+1tIq1dHXVKLxaKbgGGZNEl/NnJIDIL792uuiQ/AM2AxqnyvVxsUbmoCRo9mGVZtbeEB0/TpCNbV\nwXL11XD8/vc8eGyS9BQDA8tArKiAgwiuOXOYu2HkyLApe/AgG7GtWQPXtdeied067AAwrrQU7iVL\nwg9dqC73U08pQgBEZOdceWV8y0DVLrS3Mx+sSgx03TqDBkV18qLNBsfrr8N14okITpoEy8GDxtkP\n+/Yxi6S9nVkklrBhl4iJ7PN44F69GsHsbFiammDXyQrS0NLCru1ll8G1dSuCH34Iy7RpcDzzjKaT\njDx2S29vOM1vxAjY9++HePHFwH/+wwqpRrqizQbpuOOAn/8cmDkzXFFxMbuuBw8yt0ZhIdDaqrvE\nQbChQRmNLgCUrLP/WCxYQsS+ayQGw4bBUlwMf8i1Jbvs/ABco0cDYPf0rtpaFDQ0aLNjvvgCvqlT\nIfr9WjGIsAwiBdsozVZzz7q62GY/e/ZgwT/+wTpGINwxWixw7NqlzZhTb24DKO+T4s7bs4e1ffdu\nSE8+CccNN7D7L1uJkWKwfz+CFgvyg0HNn/MBBHfvjr6eAOyDB0OyWuHcuVOxDu7KyUH+zp1YtGFD\ndLZPyE1kKS3VXBM39GNVriVLIFVUsPXJhg5l1kFhYXhXu7o6YNkybVYiJybpKQYGPny0t0PMzYW0\ncmX0d1R+fvv556Pmyy/xt85O5G/ZAv+WLVGj/sgRCKCNP5iyDOS4QXs7W0NFNUJTRilXXYXgrl2w\nDBoEx7nn6na6Ynk5pLw84NFHgQcf1H2AfR4P3DfeiGBvLywWC+wbN0I84QTlc13xKS2NEh+lQ/D5\nwqmd55wDYcIE7P/6axQAKJs6Fbc99li4rSGRE61WSPPnAx9+CMybp7Tz/BtvhGP1atSEMoL8AO7K\ny0NuRwcWhdYUUjqu0lKIW7awlzjS7dHYGB0LKCoCPB6NGPi2bUPNbbdFpRAKxx2HIQBmA/gDWPZT\nPgB/MAhJEDC6uxtiDDeRffp0SB4PnB0d4XozM+G45RblnhZPmIBqVaeeD+CB7dvhslpZamgMMYgU\nTTt01gkqL9fes+5uNnDp6YELOh1jMAjXunWaeJORZaDrzmtuDsexOjrY9Y8Ug507YSkshD9kFcr4\nAVgGD9Zfe6i1FY4XXoDrySfZ8z9yJIo//BDVoedOOb7csYfmT9ivuQbSv/+tnGcPYlhPhYXA5Mlh\nMRg7lhWQ3Yo7d7KYAscUKREDQRB+DOAxsMHOM0S0XKfMfwG4COwZshPRl4YV6lkGPT2ssy0s1P+O\nqgN3v/aa8ahfjj/Eyu2PFAOzlkHESyTabJCuu46NUsrL2Z4IegQCrK6CAt2Rq26AduZMOP79b6XD\nVsRn/nwEv/4aljFj4DjttCjxiewQGgHQjh0o2LEDj4Suh//113Hvl1/i7tpa9n1Z5NrawtZPaH19\nxQ/e3g4XQvM/rFaMGT4cD4RcX8o9CAbh+uorSERso6DIgKiRGLS0sEyi0lKgsBDu55/XDRIvnjwZ\nUnk5cnfsUIRA+ZwIrg8+gHTuudHXPzQxSxw9Go68PLiGDWNzJEaOhGPDBoiqNuVFdIhy/UG/X+sm\nysmJEoNIwR4CoG3wYFR3diLvtNNg+fJLOB5/XHvPuroQbGxU0mx1jx35zBjEDOK6Etvb2Y5oX33F\nEiMEgXX0d9+N5mAQjoKC8KRAAFJ2NmbPmKGf2793L8STT9YkS0iiaHz8YcMAAOKgQcy1NGcOgn//\nOzaPHg3/9u3672lTE8uIGzJESS8FwJ4Xm42JAcc0SYuBIAgWsJTi8wDsArBeEITXiGiLqsxFACqJ\naLwgCKcDeBLAVKM6pdWrYWlsjHLtmBWDoNELu2sXIIos/nDhhZA2bYKzpSXaV19fn7ibqKIC2LIl\nuozc0RcWAt9+q19PHDHQHdH5fFGZSaLNBumuu9im79OnA5s3R9UV2SG4AQwHojY8eUC9amxLSzj4\n2trKfg6JQaQfHAD8gQBu0bG8GgFs8HggAbDs28fcRpoCOmKg4yYKHjige3+LWlpwvcuFJXPnIj8i\n5VHusH27dkVv+djRoaSWigcOQLrlFuZiAICzztLsGWw4Ez4nh8VEOjuBrCzm41+0CMHOTo1v3LFm\nDVyTJyN4wgmw1NfjviuugLhpE1ux80c/YueopqsLluHDlbiH7rHVs5IBQ8sgriuxvZ11ylYr2yGt\nuVnT0W8GMCcrC8dkZSHfaoWjshLul182zt6KGNRZSkuNO/a2NiZa7e0QBQHSihXAe+/ho4cfhmP+\nfNS0taERwMMANlssGPbWW/ilIOAnp5yCd7duRXDRIlheeIFd50OH2J7PXAwSIhWWwWkAviMiHwAI\ngrAKwCwA6p5xFpirE0T0iSAIxYIglBHRXr0Kndu3awO6cqA21ig9I4PlPRPBkpUVd9Qv5ubCcfXV\ncLW3K2asEn/w+RIPIEe4iRTUHb1RfnyEGESa3e2ql03GMDisqsu3Z09UxxfZIchB0nyEZxjLf2uu\nq2OFWlvZnAy/n/08aRJ8334L97x5qHvjDd22tUUs763sViaLb0MDpBUr4Fi4kF1zWewjO8OiImb2\nNzcrYmDJy4uq+y8AfN98A/fjj6Ns8GD4I5Yp3wzg0337sOPvf0dNKBCqjGLLyyHKy1EAbHQsEzHC\nt597LqSvvooeREyYwJ6bgwdZJ1pfD+fmzWgMte3el1/GyAsuwG2PPQapsBB45RXg9tuB7dtZUB1Q\nlsPQ0NUF+333QXr7bSzo7NSf7aze1wMwFANdV2JOTtgt1dEBX28vewYuuAAb9+9ngdtQtZMAvNDd\nDVcgAOlHP2L7JRi5W61WlmGnwn755cwNpzcA+8UvwgH+rCxmleTkYPXdd+OXbW1YDKABwGgArwaD\nyD9wAJsBPPib3+BPoblHiju4tRXijBlcDBIkFWIwCsAO1e87wQQiVpmG0N90xQCIcO3MnBnfMhAE\nRRDsEyZA8vvhDHUImofuoYeU1FJx1ChIkhRdlzziJ0LU/sh65XQCyApqy8BoPX11B37wYJTZPSfe\nukw6dfn8ftS8/Tacfr+m45v97LOaDkEOCW4Gy9qQO5rNAO76z3/wqzPOQJ7fD3txMfO3t7TAN3o0\nal59Fc7ubsO1ecTjjoP0ySdKmuBfoJNGefBg2PpoamIWhyUi21l2Ex08qLiJ7FOnsvsbyvlXcvX3\n7YN/3z7clZeHe8eMwQOhUehmAMsBnHLwoP6Wj21tkORZuQAwblz4+Dk5GstADAbhmD8frsZGJrK1\ntWzAsnAhK9DUxNxYXV1oBItbOAHkd3bC//rrkL75Bo72dojZ2cwKevNN4OqrQw3Kj56F3N0Ncfx4\nOKZMgXvfPgT278dPAVQWFiI/IwOOWbMgvvGG9js9PeG5CYAiBoplsmQJGwBlZ8Ph84X3mfD5UPPB\nB3C2tiJ//Xosjriv8jULBoNMmHt7WdBd5/5bdN4Zcfx4OM46C661axHMyoJlxozwAEzOvGptVdrt\nPngQzqYm5APKvAX1/XsRwJ96e9GIcMZSbl0dlgEof/99BL/7DpZDh/j6RCZJywBytepnzzffAJdd\nFj9mAIRH/T09cCxfDteaNdGj/sgZyDHqQWcnG11ZDS5TZAA5nhjEsgwyMoCCArhbW+E8dEjTYS1v\na4v21w4frr8hSeh47pdeUoRArsdZVwfX00+zDmHSJARPOQVtGzbgUE4OftXYiBcQthCeAfByIID8\ndevY8QoK4Bg5EmJrKx7fuBHO0C5vduhMwAJwZ3U1cNttcLW2ovngQWw3Wm5Ctm50XEQ+jwfu3/4W\nzV99hR2LF2NcXh7yRo6EfeJEdg5XXIENn3+O56Dt4B9tb8fik09m7pjaWmwsLcXfGhrwEAw6t44O\n+Jqb4X7iCWYVPfoo7KNGseclQgywbx/zhd98MxssWCzMRSjv2dvcjGBWFvIB/YBvXR1cGRmQZDHY\nvh2QRV1PDELzDMSSEkhlZawTPuYYFjDdto39vGqV9js9PczK1EktFW02xbXoq62F+9JLEZw2DZZR\no9D21VdMCEJl5VnhkRbWRgDSJ5/AkpWF8+12SA0NWmtjxAg49DrfkhKIfj+kzk4WQ1OnR3d1RYlB\nUJXKqmfB1gFawQ217/cAFq1bx9oTa9LaAKa2tha1tbUprTMVYtAAYIzq99Ghv0WWKY9TRqE69L8f\ngOu448zFDIBwuUOHIE6aBOlnPzMuY2bEH8stpS4nzzPo6IheWVJeJ6awEL7Gxmh/tewisVqB7GzN\nCyAzCUDR5MlwNTQgWFQES1sbHLffrv9wh+oKqgRFRu58RZuNjYT/8Q/goovgu/lm/Gb+fKW8Gzqd\nWFsbXA0NsI8ejV3ffad8pp6A9VVhIZCZiXFNTXA/8QTshYWwNzWhJi8Px7W3x7ZuIsRADpovqKtj\n6ZRdXcg/eBD+XbsgbdkCx7JlkK64AtLnnxvGD6SnngIuugjS4MHIb2jQ9blvBvCp348d8+ahpquL\ndSCvvAJpwwbWgcjzBWT27lWCnWprVHErNjfDcvzx8CNGwLe3l3V48vnKbiKVGCjrWnV2wvLzn8Pe\n0wPx4EEWGC0qYkkJoZRYo0lnvoYG5vKx22EJTb5S7zZXM38+GzDU1sIP4EarVdNeO7RCvxnAcqsV\nfwsEkL9nD+v4n3wSs//nf+C69VYEN25Ey8SJsBYU4Nlt22CZN087Ki8pAb75hrVfHfAFmBgUF7Nz\nCc3athQUwH/wIPLBhCAArQXrArM41c/qiwCegI4AH2E7oFVVVaGqqkr53el0Jl1nKsRgPYBxgiCI\nAHYDuAbAnIgyrwO4FcDfBUGYCuCgUbxARuPa2bw5YTFAcXHsMqkQA3XMoKCA1dfWpj12IMCWOjh4\nEDVbt8L5zTdaf/WaNWxRs5CP1ZKZCX/Eshp+AKWVlZCGDweuuw54++3oWaXq41mtStDRsPOVj5mb\nC7G3FxPz8+EPWRKGnVh3N9wbN2JsT4+mbhHAVQB8RKgJmfX+V16BlJcHob0dTkB/CYfCwrB1EyEG\ncmBad3Td1sZe7rFjY+fqh9I7LUOGsBQ26HdupwQC+LXOmleuJUsgRWYF7dvHXIIy8rOiWqfKfttt\nkD78ELl+v/EEOKs1vJS2fE9US2hrXIUvvAApPx+OQYMgXnRReE2hjg6WTdPVBd9338HtdLKBxq5d\nOP+CC7D6Jz9h57puHfzr1mlGyO4lS+BUxQPyAYwNBKLa2w1gtsWC4UOH4pDVipURabXOffvgevpp\nSGeeiY+sVjzz9dfK+mFRo/KSEnafzzwTWLcu/AwCTHDl2eUhy8B+6qmQurrg3L8fdrAFEn8FKBas\nHcC9Ee01fHb5+kRxSXo5CiLqBXAbgHcAfANgFRFtFgThJkEQbgyVeROARxCEbQCeAvCLWHVK06Yp\nyy0AgPPhh5mf+8kn4ZPNcT3kF1POPInA5/HA+eabkJ54As7aWviMfPhmBENdTl4zXvZvq5HdNn/8\nY9Q0e2XpAdVLYS8thRTqyIGIKfW9veZcTlYr7AsXQrJajafmy8fMywN27oS9okKZym+49EQwiGB7\nu+4yA3cBihtLOb/2duwN/Ry1hENRkXbeRYQYyFlPMV/ujg7WwauW9JDdFPZlyxQXj/2mmyBlZSkz\nYh8EMDc7G78eNQp/DAQU94PuMXTcRIplAISfAdVzKY4dC8d556ENgCMzU9s2mw32rCz4vF44//IX\nJk4PPcSWWAhZBrrZY34/3Lt3s85ffs5CWVC+3FzUXHhheHG+b7/F8j//GU6PR/95g36a6c8BOELP\njBzw/z2Ad4JBPLF3LwoMsrjk9amWb9pkvJAkEH4nKypY/Ee9BLaOm0gcNQqOm2+Ga/RoPDt5Mgpm\nzUJeSYlmEDIS2mc11rIpnNikJGZARG8BmBjxt6cifr/NbH3OtWsB6OTXf/45JI8HDqMZs7IPX8cy\n0M3V/+1v4bjooui6rFb2cifgJvLt3w93ayuCl18Oy4QJUS4go3TI4K5dGjEQS0tZltP99yN4+umw\njBsXjneYjT9YrRAnTYKjtBSusjIEv/0Wliuv1Cw8FikG4siRcDz1FFxLlqD5pZfgyMjQTCKTRo6E\nY9cuuAHdZQYGWa2adE7Zr7sH4ZG7PLPXD8B1/PHMBSMTIQZy1lPMkX9HBxOZhQvhuvtuFpTMz4fj\nqafCe050dUEsK4Nj4kS4TjgBwV27YP38czywciWe/cMfDN1HyjFCYqBkeHk8sNx3H+zLl7NjZGSE\nLQP5vmRlQRw8GI8C8K1ahcVuN3xvvIGCwYNRdswx2LlnD1arn0U5sDx7NsTubuP5AIEA60TVYpCb\ni8cDgaiO/4SQyyuqjtAIWS/NdAiA4okT4RozBhveeQfPRXTshkt9jBwJ99tv44RQHMnomIoYjBkT\nnhsgC6ssBvv3hwPfRUUQs7Mh2WzA/fcDVVVwzpsHf2giI8BGoeqJe1eBuSD+CJUFarC8CyeCZLdK\nS/U/1iRGwhvK5+bS0tNPp2qLJWr7u4S2lpwxg5bm5FD1ueeS94wzoo6llBs8mJZOmUJ3AnR7ebn+\nFnwOB9F//RdVz5ljfPxPP2XbcRKxfZxvv52V+fpr7UGnTyd6+22ihx8muuMO3XbRY4+x7x86RFRY\nSHTDDUSDBmnLBIOs/t5eop/+lO2He/XV4c/POYe8K1ZQdX4+22O3qIi8b7xBlJdH3txcWiiK2nMt\nK6NFI0Yof/MCtDD0mRegu1TbhyrX5k9/Ipo9O3wtjzmGllZWRu29u0lVl7zl4gxBoHumTqXqsWPJ\nC7BN4ysq2J6+gwYRbd/OzqOzkygzk+iDD4h++MPw+f3gB0Tr1yvPhDfiGJr7d++95L37buMtFgcN\nImpsJDrzTKLycnZdfT6im29m12L9+qjvzoDBdqo/+AHRrbcaP6sA26Lzyy+Jjj+eqLKSPnz+ebpS\nVc4bKjfb6Bih511328jCQvL+9rdETU20NCMjaqtKL0Dzc3O13yktJW99PXtfTByz2mKhpRMmUPXQ\noeR94YXwPRkyhOg3v2HXUd7L/Pe/J7rnHrZFp2o70sh231BeTotmzqSlw4dTdXk5fXjqqdo9n4+C\nrTCRgm0vD3vnH9UglRikckN5M3Xp1hO5P7JOuZh7Jd9yC9Gf/sS+Iwj6bfy//yOaOpVVfs45RJdc\nwuqJ3Nt22jSif/2L6OmnyXvllfr7IbtcRHffTRQIsH11Z84kys3V1iN/RsT22J08mbVT5uqriZ5/\nnm1KPnIkUUYG0bZtbM9ngLwbN4ZftsxM8i5cSN45c5RrEtkpeEPX6LqysnBbX3+d6JJLYt47ea/g\nO6ZOpcsqKuimk06i+Xl52rIZGeT94Q+Jzj6b6Jhj2DFD+/oqovfuu+y6ypxxBtFHH7Fj22yKICwG\n6NqcHFo0c2b4ejqdVD15svH9HTaM7Yd9+unsOgJEu3cT3XknUWEhVV97bdR379N5DgmgpRMnEs2f\nr39NioqY8H3yCZHHQySK5B06lGbk5yvPnykRVj3L3vp6uvPMM2m21UrXDRtGi0aPJq/LRRQMUnVG\nhu453zlrVvjen3gieW+6iYiIqq3WKOFuA9v7Wi3umk588GDWiVdVUbXVSt5Fi4hOOIHdHyKiP/6R\niWpZGVFDg6bdup39ffcRTZpEdNVVdLRxxItBKjeUT1UZvXIxN/2+4Qaip54iIiLv4MFUDdDS/Hzt\nQ/z++0RnncV+vuQSookTWR0ffKC94z/6EdF775H3scdoYX6+/ov+4INsNEVElJNDdOKJbDP1YDBc\nT2cn6+iJiO66i6i4mGjx4vDnd95JdO+9RCUlzBoB2EbyNhsTEXVdEyYQXXstG0HLm8SXlMQX8f/9\nX6ILLkjI+jMsm51N3hkzqHrkSFpqsWivbXY20WuvEZ1/friic84h+ve/2T359FOqzs42HkUuX05L\n5RG/3vmMHEm0cyez7H74QyZ+n39O1cceS0vz8+m6YcOivmc4gj7zTPJefDETwNNPp8uGD6dfFhSw\ndl1/PSv73Xds0/iSEqrOyqL7VCIQOSjRFWEV3vp6Wjh2bJSV562vJ29lJS0sKoq94fwjjzALtaOD\nvFarYslVhwRvRkEBffjee7r3Tleshgwh7+jRYeF+7jn2bGVmsmc2Hk8+yQYuN9wQv+wRRirEID33\nMwihuz55WVnU+uRmlm/WrUsU+7S1pNHqk2oUn3NEPEAC4Bw+XLtevDqroqCALcwGRG8rKAejV6/W\nnUMQGYxGQQFbWoNIuw6QukxeHvOtDxoU/nzECODzz9n/chsLC1mAs7BQO7O0rIylCw4ZouSwV15y\nSfwgXlZW/A2LIjAq29zVhZr338ei0DLLyu5mHk94Mxw5514+trx8dmkppJEj4Vy7NmoNfwBATg7a\nVYH4qPNRxwxKSuADUHP55Vi0aROcfn/UrmgA82s7QhMJ5bqkykqcf/HFqAntzvbYJ5/gb3v2INjb\ny+JPcuZRaOIdWloQ7OlBJsIxHHlnMRkRzJduO/ZY3XNzL1miLJ0uX0vn3r1wL1kCcdw4OIqK4Joy\nRZPQoalDDq7v2wdx+HA41qzBi3PnIjhtGjLnzkXNV1/hzLPP1r13buhMQmxshFsdMygsZOt9yQv/\nxaO8nMX6VMuGc8yT1mKgzJicOxdSRQVcABwuV9RDbWaDGKWua69l2/dZLHC8+mrCW0vqlbMDUXsY\nK5k78oxgjwfOAwcg5ebC2dCg3cpPnnQGsBegu5t1XuosFrmc1YpgaDq/Gr1gNPLzWUeYkaGtK1IM\ngGgx+Owz+EpK4Pz0UyZgCxbAJwev1ZSVsTWZVMFfU5uMhHYCM3vNAeP7swOImqiniGNODksEUE8c\nVO8a1toas/Pwtbbi0L59xntUy8kGgQBQXMzmaKiCuT9H9LPxTFYWFvzzn+y5VnW0765dC6dqcl4+\nAGdHBzsPWczkXdSys2EhUnaIGwKgEoll0hiKa10dnNu24dmdO4GSElz/zDP6Qpmbq4gBhg1TBgN6\nwhp572KlL6sDyPB42Aq3ZigPTWUySivnxCZZ0yLV/6ByE6nxPvUUc7FMmRJl8pqJGWg45RRmrnZ1\naY+hV08oQBav3A3l5bRo4kRaKora9s2ZQ95HH43dvjffJLrwQvbznXeytlVWEr34orbdJ59M9Nln\nVH3RRcaulcWLie6/n5U/7jjmKho2jPmxZRobw0HlRx5hdbz+evjzd95hroeCAm2bc3PJW1mpbdOt\nt7Lvv/lm1DWKGcT7+GOi005L6N7pXveSEro2lpuuvJzoD38guuyycEWzZxO9/DL7+b33wi46Haqn\nTlV88NUhl+BiMN85ERGNH0+0dSvzVd90k67L0AvQ7OJiWjp4MFVPn07ek07SPdbSKVOMz+P++5k7\nT2b4cPJmZmpcM7cBdK1RXErv3HTcbpvA/Pym6njhBebWqqqipYMGxQzWqu+dF6CfGLnKgPC9Wr+e\nuSVPPtnw/miO8eWX7B6NG3fUBI5lkAI30WHv/KMapCMG3vr66AwWvWDYpZfSbICuiwwCRtRVPXYs\ny5Ix8KNWz51LS8eMoWpBYEEtHbz19VR98cWsnkGDWD2PPsqyh9RceSVVn3VWbL94KJhKRKwzB4jO\nO4/or3/V1nX88UQbNpB37VrjYPSvf030u9+x8qefTjRmDJEosqCjzJ49TCCImJ8VIProo/DnX39t\n7NcePFjbpvvvZ59/8onudTLks8+Ul9y7eTNVC0J0MFwHb309VRcU0NLTT6c7Cwro9qFDYwfwx48n\nkiRtttTVVxPJmSz/+AfRRRcZHm/psccad9BETAS++YYd51e/omqLRb8tVVVEF1xAtHatNpitOq/L\nRo40Po/f/57FbGQmTCAqKYkS3Q/fe890Jo2euM5QZQvFi+F4n3iCFkYG9GOIj7e+nu6cNYvm5+bq\nBpsXjhjBguTXXMO+sHUra4M63pPAucQcEB5hHDViYCbIaOZhSOiB+ctf2LFqaozvQGcnC1jJL/ez\nz7JUTTWzZ9NSOcvEqEN55RUieaT54IMs4Dt/vhJ4VpA7nj17wiPVoUO1L/2iRUTLl7Ofzz2X6NRT\nWUB606ZwPTt3ssAnEdHf/sbas3lz+Dp99hldp9NeAmhpaam2TbKY1NUZXyc9vvqKZd/I7Rkxwvx3\nTzyR6PPPqTo0gvXqdSzyfZ08mWjhQqJ588Lfv+66sNCuXKkVigiqzz479rM3eTLRhg2so37gAZZ6\nq/eMrVzJEgBCgXM1Rmm0smX64XvvUfWUKbRUDibX1zPrNpFrZoC3vp6qzz6bjewLCuiek06K/ayq\nr6Uph8MAAB5JSURBVE1VlWnhUL6jepflZ/g+gC7LyCDv88+zeuR3aPdu9nuM+6NXr9m2HEmkQgzS\ncqG6SMwEGXVnbUasSWKmjIK8Q1KMGci+Xbvgzstjm8nMmwf7mWeytdTVBAKwDBpkamkIn8cD90sv\nIZiRActHH8E+erR2vX+53IEDbM2ZoiK265l6MllolrLP44F761YEOzpgIYK9rg7ipEmaegBExQx8\nHg9qrroKIrQTsZRlov1+OENrzgCAe+VKtojYb34D+4MPml8MLBQzAKC/j0EsQpO7gqFJTvnQToL7\nuqwMj8rBTjlmYBBAjrfEiX3+fEjr12t3P1NPYpJjBqHApZibq10ZVF4kcf9+NlM9tKexGvVzKZ9H\nD4DNhYW4e84crL7++vAENXmJh2HD2LLbSSLabJAeegi47TZg5044bTb4v/zS+FlVEQytG6Qm3tIP\n6ndZRHgPDCkzE2JFBftFHUAGTMUMEt4DnBPFgBADM/v7mnkYEnlgfDk5rMN9+GFY/vWvqGVwlRnN\noVUe/StWQPr3v9na+OqKAgHYf/az6JUdx4wJdyiBAHydndoZ0t9+C+mPf4Rj/nxN1pFv927ULFjA\n1pxpaYG/pQXS9OnhTI9AILwMduh8/QCkm2+G44MPNDOZfR4P3I8/zjrzO++E/Xe/Uzom9VpCmmWi\nu7vhX7ECd73/PnIFQVkm2v/ii5A++8z86pCZmeFtL/soBuqZw3LH4gfgOv/8cBvkbCL1THK1ELW0\nxBQDsaICjuOPh+u779juZ8cco53JHZqF7uvoYMLY2gqLes9tmbY2lh2mIwaGHWR+Pt797DP9AUxX\nF6RUBUqHD2cL8HV0wL50KduvQf2sGszgtZSVmV9aXf6O0busXnJbvj55eWxVWBPPRiJ7gHMMSNa0\nSPU/6LiJzLh3UjmPIKnjRbpRQrOGNb7doiLyvvVWuMxzz1F1RUX8tpWXU/WsWbHL3XILVZ9ySuwy\nW7aQt6JC9xzvmTpV+Y4XxrNZY/rpzbB9O9Ho0eznVauIrrzS3PeI2KSiF14gb3a2MmnM0O133nls\n4t2tt4b/dvvtLL5DRLR0KYspGPHxxyz2Mno0kdcb/fnUqeR96SVaqIoV6LZj2zbmSnruOSKz82TK\ny41jFsOHh2etJ0tHB5t3Esrnjxv8D+F94w1amJmZkJ9e990SBPIedxxztwFs7kuobHVmpqmAMI8Z\nHCVuoqhNOdT7E4TQ3cUpYkRjpgxgzp1kaGUYzA1QryOP007TLqQXCCAYckNE1ae2WgIBBPfti10u\nEIidehoq425qUnacUp/jT3t7o0bbepucxFzczQzJuolaWiB2d8PxzjtwVVcbPhfIyWGWQazUUnkJ\naT1yctiIfv/+8CqjaqxWuJ98Ek7V0uO67sfc3LBlII+AQ+g+l3l5cIgi3FYr/Js2RY94i4qMV65N\nlJyc8HXKytI+qzEQx42DY+RIuPbsQfCUU2CpqIi+/pHfiXyXc3PhePNNiMXFYYsgKytseff0IH/b\nNvi3bYu5L4GZPoITmwEhBgDiPqBmHgazD4wZd5KhWRoMar+o9s8rleVrNzEJBNja7RFbNUaZuYEA\nLCNGxI0/WEpL45YJqhYhU59j+fDhkDIyNB3TVwUF8KtWJAWYbz4pszxZN9GBA0xkx42L3XFlZ7M8\neKOYQUsL2yDGiJwcJgSZmZpNYhQyMpQN69VECaNaDCLcRFHPZUYGHM3NEAMB2O+4A9KOHZr7cVd5\nOYqJIG3cGL1nQF8pK2Nxj4itKmOSmwuxrQ2SIADvvx+9S50Bmnd51y4mxjk5YZHMykosvqdXLydh\nBowYmMHMw2CmjBn/o+5obuxYODweZqjKL5WeGBQURImB/dRTmY9eXV9RkdZqCQTYfrgbNsSMP9iv\nuAJSY6O2zLBhmjKWrCxl/wL1OZZWVsK+cqVGMH91442Q1EFMAHvLy3GvOmaABFeHDM1ABsDEQA7Y\nm6GwkHXwZkbGRgFk+dhxJp0hJwfYs4ctu6yH1Ro/QQBQNnvXEwMg4rn85hvgqquAzk6I8q5uofvR\nUlSE3C++QHVog6GU7eRVVsb2mU6EnBwmyuPHmxaCKORBQHa2xjLgAeHvnyNKDFKFGXeSoZVx7LHs\npZezkIwsA/VeCoEAxEGDtPV1dLBtJtUveCAAsbJSW27DBjgee0wTZBZHjtSW2b4djgULNGXs5eWQ\nBg3SPUc9wRwdca73ha5Fn83ySDfRGWeY+x7AxKCuzpwYZGfri4G870S8DZOys5m4q/cwUGO1wn7N\nNZDWrtVu3BMpjNnZ4b0v4i2tMGgQ62SzsliGkigq98M5bx4WhQQYSOFOXsOHsy04EyEnh/0virHL\nxSIri80YVotBdjYPCB8GuBjoYNadpGtllJSwtX7iiUGEZRAVV1i1Cnj1Ve33QmmjmnIXX6ztXPTq\nuu02bTZNIACxoACOV1813ZkbWVR97oCSdRPt32/eMoixNlG8bCKlw4shBuKQIXBYLHBddRWCe/fq\nX0tBYO09dChcpxHyxi/yhkkq+m3EXFaWeAxCPo8xY2KXi8fQoVFuIrPxPU7q4GJgQJ/9j/KWhJHb\nS6opKIiyDKLKyIFLNXrlhgxhHWOsMvIaMhFlDquPVV6Lqbc3cTEoKDAvBkaWgcm1ieKKQWgPZDEY\nhLRiRWx3iSwG8VJCc3JYe3WEqj9GzD6PB+4PPmBbZiYSg8jMZCKXjGUAMDGIcBPxgPD3DxeDVCOL\ngUwCloGGyA7cqJy8Y1SsMpHColfmMOCzWuGeOxfBTZtguf9+2B991NzLLscMhg+PXzYnJ/p8E5h0\npnRQeplEAKu3u5t1ivH85rm5zC9vJj4iT7SKuE+pHjFH7QCYQAzC5/XCbbEg+MILsHz7bZ8C2T6P\nB+7t2xH0emGZPx92AKK87SUPCH+vHP4e4UijuJjtwSwj71usxqxloBaDYJD9U3U4Po8H7rVrEWxu\nhuWLL9jLaCQs8Y73PePzeFATCMD597+zTui11yBt3GguECqLgZmOR+7M++omsljYd2O4idDZGbZ0\nYpGby56NiNRSXQYN0i47HiLVI+a+ZO0AKhHp7UX+li3wb9mScCBbqUOeHLlyJSQAjkOHkKStwekD\nab2E9YDErGWQqJtIFpVQlpL8Ii3asAHO7dvDa/i3tAwIy8C9ZAmcRPrLTsejsJCdQzzfOxAu00c3\nkc/jgZMI0p//DOe8edqlxwF2Hbu6zF3PvDwmBmbW5pc3vtch1lLRidLXGISRiJi6f7HqAOBevdp0\nHZzUwcUg1fTFTaRnPRj4+WUMX8Zt2xKu63CQVCBUHsmbjRkA+mLQ1cWsLYPOWRHcQADOzZu1m+bI\nZGSYFwPZMkhSDFJJIvtJqElFINuwDrVlzfne4GKQQnweD5wffQTpoYfCo0iT8wziuokiyhi+SJ2d\nCdd1OOhrJwQgMTGIZRnI8QKDiVamRr99cRPFEQOfxwPnl19CqqvTt0ZSiKmNiHRI6v7FqyORZAJO\nyuBikCKUUeSWLXDW1YVHkXqdc1/cRBFlDF+kzEx9yyDN3ET2ZcsgqbaTNNsJAUiJZeA7eBDOBQsg\ntbcbdrimRr+yGKTIMlCeo7o6OA8e1LdGUohmN0Gj7S116KuIxKpjM4A5goD2vXv7XQQ50fAAcoow\nDMRlZUEyk00UObKM49oxzCrJy4tvGfT0HHYxEG02OMrL4RoyBMFvv4Xl0kvNB0LlOROJWAaq8/Ud\nOIAa9bLUBhk0ptI4E40Z9PTEFIO+BnSToS9ZO6kIZKvraK6rQ8vGjXihrQ35mzfDv3lzamZWc0zD\nLYMUYTiKTFU2UUQZ5UWaMQNSbm54RGe1DgjLAADE/HxIs2fDeeaZiQVCLRYmqH20DNx//asiBIBx\n8NPU6FeOGZh1E6nbpMNAWoYhFYFsuY7SykrUqNa/6ktAmpMch79HOEIwHEUCfZtnYMLPL9pskB5/\nHDjrLEAe2fWxrsNCZiabaRsrtdOIwsI+xwyCTU2mOlxTo99E3URATDE4WpdhGEgieKSSBj3CkYGh\n28bn07ooPB6477sPwfr68GxPvc45M5OtiSN/ZtSBR3b0RplJaWgZIDOTrcHTn2KgYxkksilLXBeK\n1Rq9RLYRcntjzDM4WpdhOFpFMJ3gbqIUoYwip02DVFoadtsEg0pHoQQHX3sNzq6ucHCwuTl+R29W\nDAaSZZCV1SfLwOfxwNnYCOmvf40faNSxDOx33AEpMzOp4KdCotlEQEzLoK8B3YFOKgLSnORIqkcQ\nBKEUwN/B9kHxAriKiA7plHsGwKUA9hLRCckcM50RbTZI1dXAffcxt00wyEb3oVnDhsHB3l5IM2dG\nVyh34gUFxh24mTkE6WwZNDUBJ55o+ivKrNXmZuQ3N8NfVxc70KhjGYhjx8IxYgRcw4YhuHs3LFVV\nfZ/Fm8g8A3k/hDippUfjMgx8LaLDT7I9wq8BvEtEDwmC8CsAvwn9LZL/BlAD4G9JHi/9UY/CI1w2\nhn7R9nbjjl7uxI06cPWG7BkZA8syyMxke+8mYBkknG2jN88gM5Pt4nb55SzV88EH+3oGiWUTmbAM\njmaORhFMJ5J1E80C8NfQz38F8BO9QkT0IYAEd84YoMTowA3nBuTk9N1NJAjxy6WrZdAHN1HCgUa5\n49VbqK61Vbu0d19IsZuIwzlcJCsGw4hoLwAQ0R4ABqt5HUXE6JgN/aLHHNN3MdArF9kxpbNlkKAY\nJDzzNdYM5La21IkBtww4A5y4T7AgCGsAlKn/BIDA9kmPhFLRqOrqauXnqqoqVFVVpaLa74dIy0DV\nMWv8ov/zP7BcfDEcjzwC8Y479EeWMeqKKhdLNOTZzPJ2nOkkBt3dCYlBwtk2sdYmSoUY9EPMgMOJ\nR21tLWpra1NaZ9wnmIimG30mCMJeQRDKiGivIAjDAexLRaPUYjDgiDOaV/yi69YBDz3ElmE2kymk\nlzKqLhcrtmC1siB2IMA6xUAgPTokOcUyATFIONBoZBn09KTOMujqMreCqonUUg7HDJGDZKfTmXSd\nyQ4PXwdgB7AcwM8AvBajrBD6d2QTb5QuE0c0fB4P3N99h+Dtt8MyeTLsl1zCZhf3tS4AwfPOg2XM\nGNgLCiAmuztVKpA76ARTSxMKNOpZBrJFEm9jGzPIbiIzosLFgJPGJCsGywG8KAjC9QB8AK4CAEEQ\nRgD4MxFdGvp9JYAqAIMFQdgOQCKi/07y2OlJdnbYJRNLDGIEmpX0yT17kL9nD/wbNkBauxaOigr9\nTT9iiIFSVyCA/A8+YG6V4mI4br758G8g0kcxSATfrl1MCK+9FpaxY8O7cVksbKnxVFkGZmMGWVmG\nq6RyOIeTpALIRNREROcT0UQiuoCIDob+vlsWgtDv1xLRSCLKJqIxR6wQAKxTyMhgboh4YmDQgeum\nT+7eDbfXm5q6Dh2Ce82aPp5gCulnMfB5PKi54AIsAuD8+GPtCqBZWWz28/eUTeTzeOB84AFIgQBf\nkZOTlvAZyP2BPOqP5yYysAwM0ye7umLXpbM1pmFdra2JnFH/0IeYQSLE3I9ATmv9HgLIyszzt96C\nMxjs92WpOZy+wMWgP5DdNn20DAzTJ+VsFKPjRWyNGbOukpJEzqh/yMxkbc2PlKvUEHNOQqrEwISb\nKBVbRHI4/Q0Xg/4gUcsgIlNIdz5CWRnskycb12UgPrp1FRbCPnt2X88udWRmss64n3zoMeckZGWx\n6/49uIn4ipycgUAaJJsfgZi1DAzcREr6pMOB4Nq1sFx2GRynngrx44+N6zI4nlLXhRciCMBy2mlw\ndHRAHDEiBSeaJFlZ/Ro8jjkn4bzzWKFUiEF3d0zLgK/IyRkIcDHoD8xaBvHmI7z8MlBcDDz7LPDi\ni/GtDIPjiTYbpKuvZiPxpUuBefPSZ9JZP4pBzDkJWVnsPplZRiIW8vdjXM+jdVlqzsAiDXqEIxB1\nR2/WMtDrlLKzgTFjgLo6c8ISq8yQIUB9ffh4R4EYADHmJGRlJW8VAOHrGENU+IqcnIFAGvQIRyBy\n56y3BWVkGcCwc/Z5PHC3tyN41VWw5OfDLoqx5xnE6uQHDwY+/TTm8b5PfB4P3KtWIVhXF97k5/vs\nHFMtBnGuJ1+Rk5PucDHoD+RRf25un2IGgGqyWEMD8hv+f3v3GmNFecdx/PvfrhcQXTFVNAjIohFb\nbdAXTZSaQiOENK0YXzQmS1JM+kpFjZKKqXX3+KJqQps0JL6QtkENxTQm3pqoiHQx1igkgFpAqtmz\np0VlvVBBvEXZf1/M7Nmz7JwbM7Nnds7vkxDH2Wfn8uTZ+c9znfeCpoV9+1hVLI5/aDYaDD75pOr5\nJlL53kaaTap8kD5VExwMRLJOo4nSEHNoKVQZjnjkSPRwxJHAUmv9ogwFg0wMtUyq83qkeShu34NI\niykYpCHmpDNocjjiJKsZZGKopWoGImMoGKQhgZpBU+v2T7Jg0PQ3CdKgYCAyhoJBGhKoGUROFjvz\nzOgPhDcSDLq64Isv6q+ZNAEy8fHzkQlvcTUwmkhkMtDrTBqarRlEtPWXhyPeeivDL71Ex0UXserq\nq6M7WBsJPmYwfXqwBEOLg0EmhlomVTNoYJ6ByGSgEpyGGjOCy+p9kIZwOOJDD8GVV8L11wfLYlc7\nVr3zwWhTUQaGlrZyqGWpWGTD9u0MHztGx+HD8Ya1qplIckIlOA11ZgQD4/sMqjUzTJ0aNO/U+jpZ\nA8GgVCyy4eOPGV6xgo5SiZVDQ63/nkELlIe1Dg4mM6xVzUSSE+ozSEMjb+qV3yU+dqyxYFAtTZ3z\nlZdQ/vhjCrt2sfrQIdbdcktbLqGc+LBW1QwkJxQM0tBIG37l3ICOjjHfIBjj1FODhdBqLZNc53yR\nD8ADB9pyCeXEh7Wqz0ByQsEgDY3WDBpp5zcLageffXbCx8rEuP6MSHxYq5qJJCcUDNLQTM2gkc7c\nqVPhyJHGgkHEQykT4/ozIvFhrWomkpxQCU5DM0NLkw4GEWkil1CePbstl1BOfFiragaSEwoGaWhm\n0lkSwaDO+coPwJ4ehgcG6DhyhFWbNrXtEsqJDmtVn4HkhEpwGtKoGRw+HKv/Yc7cufSuWQPr18OO\nHdDd3cQNSVVqJpKcUJ9BGo7/QH2Uk04KHt51PpkIBB+Mj9FMVNbVFQSVDEw6yw01E0lO6ImQhkaX\nh5gyBY4erf8gqdNMVDpwgA3ffMNwoUDHV1+xMuqbB6BgkAbVDCQnVDNIQ53RPWPS1RoyOqLG0NJS\nsci6pUtZDRTefpvVg4OsW7IkekKZgkHy1GcgOaFgkLBSsUhhzRp6BwcpPPkkpcOHqyceqRk0EgyG\nhxufUFZtRq2CQfLUTCQ5oSdCgsZ9znHvXnoPHmTVTTdFN9s0UzOA+BPKzjgjaG6qciw5AaoZSE6o\nZpCgyLf0Q4eqL/vQTM0AItM1NaGsszMIQMPDepNNilmQlwoGMsnFCgZmNt3MNpvZfjN7wcy6ItKc\nb2ZbzWyPmb1lZrfGOWeWNb3sw5QpsWsGTc+o7eoKHl5mtc8pjevsVHCVSS9uzWANsMXdLwa2AndH\npPkWuMPdvw9cCdxsZvNjnjeTmnlLLxWLFAYH6V27lkKpVHsF0dPCEFNrQtlll9Frxtr582svx9zV\npbfYpKlmIDkQNxgsBx4Jtx8Brjs+gbsfdPfd4fZRYB8wM+Z5MynyLf3ss8e9pZeXlP7oIwoDA8GS\n0tVGAEHNmgGEE8puv52CO73Ll9eeWaxgkLzOTuWpTHpxg8E57j4EwUMfOKdWYjO7AFgAvB7zvJlU\nuexDb3c3a4FVq1ePezg3vaZ+nWAAwOzZ9dOAgkEa1EwkOVD3qWBmLwIzKncBDtwTkbzKdxnBzKYB\nTwC3hTWEXCqve/Pss3DttRDRRNR030IjwWDWrPppQMEgDaoZSA7ULcHuvqTaz8xsyMxmuPuQmZ0L\nfFglXSdBIHjM3Z+ud86+vr7y9qJFi1i0aFG9X8mc0umnswEYvv9+Op5/fsx3dkf6FioDQs0lpRUM\nsk19BjLB+vv76e/vT/ag7n7C/4AHgbvC7buAB6qkexT4Q4PH9MlucGDA7+zu9qPBRy39KPid8+b5\n4MDA6M/nzfO94H3gvwH/+bRp/sq2bdEHfO45d3B/442a5+w75RS/t7vb+3p6yuca58473WfOjHuL\nUmnWLPdXX231VUgbC5+b8Z7nsX4ZzgK2APuBzcCZ4f7zgL+H2wuBY8BuYBewE1hW45jp5dgE6evp\nKQcCrwgIfT095TSvbNvmN06bVjVgjPHyy8Fx9uyJPN9IcGnoWPfd5z5nTkJ3KoMDA9532ml+7xVX\n1A7CIilKIhjE6kB290Pufo27X+zuS93903D/B+7+s3D7n+7+HXdf4O6Xu/sV7v58nPNmXSN9Alse\nfph1R4821olcp5mo6SUp1KSRiPKosM8/p7BzJ6s3bqw9KkwkwzQDOQWNzDdoqhO5TjBo9FilYpHC\npk30vv8+hRUr9NCKqelRYSIZpmCQgkZmBTe1jESdYNDIscpvsa+9RuHLL/UWm4CmR4WJZJiCQQrG\nzDdYvJi1PT3jZgU3tYxEnWDQyLH0Fpu8pgK6SMZZ0PeQHWbmWbumtJSKRTZUfJh9ZZUPs5f27GHD\npZcyfNVVdMydG5mu3rF6Fy+mEDEUrXfxYgpbtyZ+b+1g3Cq1BEG45nIgIikwM9w91oJjCgYZVyoW\nWXfNNRQGBmI9cAorVrB648ZxcxvW9vQk93H4NtRoQBdJk4JBG0jqIa63WJH8SiIYaIxhxiXVSVnu\nx6h4i12lt1gRCSkYZFzTS1fUUF43SUTkOBpNlHFNf7xGROQEqM9gElAnpYjUog5kERFJJBiomUhE\nRBQMREREwUBERFAwEBERFAxERAQFAxERQcFARERQMBARERQMREQEBQMREUHBQEREUDAQEREUDERE\nBAUDERFBwUBERFAwEBERFAxERISYwcDMppvZZjPbb2YvmFlXRJpTzOx1M9tlZnvM7HdxzikiIsmL\nWzNYA2xx94uBrcDdxydw96+Bxe5+OfAD4CdmtjDmedtCf39/qy8hE5QPo5QXo5QXyYobDJYDj4Tb\njwDXRSVy9y/CzVPCc/4v5nnbggp7QPkwSnkxSnmRrLjB4Bx3HwJw94PAOVGJzKzDzHYBB4F+d98b\n87wiIpKgznoJzOxFYEblLsCBeyKSe9Qx3H0YuNzMzgA2m9mP3X3bCVyviIikwNwjn9+N/bLZPmCR\nuw+Z2bnAP9z9kjq/81vgC3f/fZWfn/gFiYi0KXe3OL9ft2ZQxzPASuBB4JfA08cnMLPvAt+4+2Ez\nmwIsAQrVDhj3hkREpHlxawZnAX8DZgEl4Bfu/qmZnQesd/efmdllBJ3LRtBH8Zi7r41/6SIikpRY\nwUBERPIhMzOQzWyZmb1tZv82s7tafT0TzcwGzeyNcHLe9nBf3Ul9eWBmfzazITN7s2Jf1Xs3s7vN\n7B0z22dmS1tz1emokhe9ZnbAzHaG/5ZV/CzPeXG+mW0NJ6u+ZWa3hvvbrmxE5MWqcH9yZcPdW/6P\nICi9C8wBTgJ2A/NbfV0TnAcDwPTj9j0I/Drcvgt4oNXXmdK9/whYALxZ796B7wG7CPq7LgjLjbX6\nHlLOi17gjoi0l+Q8L84FFoTb04D9wPx2LBs18iKxspGVmsEPgXfcveTu3wCPE0xoaycjfSqVGprU\nN9m5+yuMn4hY7d6vBR5392/dfRB4h6D85EKVvICgfBxvOfnOi4PuvjvcPgrsA86nDctGlbyYGf44\nkbKRlWAwE/hvxf8fYPRG24UDL5rZDjP7VbhvhjcwqS+nqk1oPL6svEd7lJVbzGy3mf2polmkbfLC\nzC4gqDG9RvW/i7bIj4q8eD3clUjZyEowEFjo7lcAPwVuNrOrGT+Jr517+9v53h8Cut19AcEs/sg5\nOnllZtOAJ4Dbwrfitv27iMiLxMpGVoLBe8Dsiv8/P9zXNtz9g/C/HwFPEVTphsxsBkA4qe/D1l3h\nhKt27+8RDGUekfuy4u4fedgQDKxntLqf+7wws06Ch99j7j4yj6kty0ZUXiRZNrISDHYAF5rZHDM7\nGbiBYEJbWzCzqWHEx8xOA5YCbzE6qQ+qTOrLEWNs22e1e38GuMHMTjazucCFwPaJusgJMiYvwgfe\niOuBf4Xb7ZAXfwH2uvsfK/a1a9kYlxeJlo1W95JX9H4vI+ghfwdY0+rrmeB7n0swgmoXQRBYE+4/\nC9gS5stm4MxWX2tK9/9X4H3ga+A/wI3A9Gr3TrBU+rsEnWhLW339E5AXjwJvhmXkKYI283bIi4XA\nsYq/jZ3hc6Lq30Ve86NGXiRWNjTpTEREMtNMJCIiLaRgICIiCgYiIqJgICIiKBiIiAgKBiIigoKB\niIigYCAiIsD/AcThVwC/XgzBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd890cbd890>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Резльтирующие коэффициенты логистической регрессии\n",
    "plt.plot(clf2.coef_[0],'ro-')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Анализируем какие из коэффициентов имеют больший вес"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(207,)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf2.coef_[0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "allColumns = list(trainReduced2.columns.values) + X_pick_columns + X_pick2_columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "207"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(allColumns)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "<style>\n",
       "\n",
       "</style>\n",
       "\n",
       "<div id=\"fig_el26961405681012546082682948170\"></div>\n",
       "<script>\n",
       "function mpld3_load_lib(url, callback){\n",
       "  var s = document.createElement('script');\n",
       "  s.src = url;\n",
       "  s.async = true;\n",
       "  s.onreadystatechange = s.onload = callback;\n",
       "  s.onerror = function(){console.warn(\"failed to load library \" + url);};\n",
       "  document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "}\n",
       "\n",
       "if(typeof(mpld3) !== \"undefined\" && mpld3._mpld3IsLoaded){\n",
       "   // already loaded: just create the figure\n",
       "   !function(mpld3){\n",
       "       \n",
       "       mpld3.draw_figure(\"fig_el26961405681012546082682948170\", {\"axes\": [{\"xlim\": [0.0, 250.0], \"yscale\": \"linear\", \"axesbg\": \"#EEEEEE\", \"texts\": [], \"zoomable\": true, \"images\": [], \"xdomain\": [0.0, 250.0], \"ylim\": [-0.30000000000000004, 0.30000000000000004], \"paths\": [], \"sharey\": [], \"sharex\": [], \"axesbgalpha\": null, \"axes\": [{\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"bottom\", \"nticks\": 6, \"tickvalues\": null}, {\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"left\", \"nticks\": 9, \"tickvalues\": null}], \"lines\": [{\"color\": \"#FF0000\", \"yindex\": 1, \"coordinates\": \"data\", \"dasharray\": \"10,0\", \"zorder\": 2, \"alpha\": 1, \"xindex\": 0, \"linewidth\": 1.0, \"data\": \"data01\", \"id\": \"el2696140568118811856\"}], \"markers\": [{\"edgecolor\": \"#000000\", \"facecolor\": \"#FF0000\", \"edgewidth\": 0.5, \"yindex\": 1, \"coordinates\": \"data\", \"zorder\": 2, \"markerpath\": [[[0.0, 3.0], [0.7956093000000001, 3.0], [1.5587396123545605, 2.683901074764725], [2.121320343559643, 2.121320343559643], [2.683901074764725, 1.5587396123545605], [3.0, 0.7956093000000001], [3.0, 0.0], [3.0, -0.7956093000000001], [2.683901074764725, -1.5587396123545605], [2.121320343559643, -2.121320343559643], [1.5587396123545605, -2.683901074764725], [0.7956093000000001, -3.0], [0.0, -3.0], [-0.7956093000000001, -3.0], [-1.5587396123545605, -2.683901074764725], [-2.121320343559643, -2.121320343559643], [-2.683901074764725, -1.5587396123545605], [-3.0, -0.7956093000000001], [-3.0, 0.0], [-3.0, 0.7956093000000001], [-2.683901074764725, 1.5587396123545605], [-2.121320343559643, 2.121320343559643], [-1.5587396123545605, 2.683901074764725], [-0.7956093000000001, 3.0], [0.0, 3.0]], [\"M\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"Z\"]], \"alpha\": 1, \"xindex\": 0, \"data\": \"data01\", \"id\": \"el2696140568118811856pts\"}], \"id\": \"el2696140568101256720\", \"ydomain\": [-0.30000000000000004, 0.30000000000000004], \"collections\": [], \"xscale\": \"linear\", \"bbox\": [0.125, 0.125, 0.77500000000000002, 0.77500000000000002]}], \"height\": 320.0, \"width\": 480.0, \"plugins\": [{\"type\": \"reset\"}, {\"enabled\": false, \"button\": true, \"type\": \"zoom\"}, {\"enabled\": false, \"button\": true, \"type\": \"boxzoom\"}, {\"voffset\": 10, \"labels\": [\"start_time\", \"r1_level\", \"r1_xp\", \"r1_gold\", \"r1_lh\", \"r1_kills\", \"r1_deaths\", \"r1_items\", \"r2_level\", \"r2_xp\", \"r2_gold\", \"r2_lh\", \"r2_kills\", \"r2_deaths\", \"r2_items\", \"r3_level\", \"r3_xp\", \"r3_gold\", \"r3_lh\", \"r3_kills\", \"r3_deaths\", \"r3_items\", \"r4_level\", \"r4_xp\", \"r4_gold\", \"r4_lh\", \"r4_kills\", \"r4_deaths\", \"r4_items\", \"r5_level\", \"r5_xp\", \"r5_gold\", \"r5_lh\", \"r5_kills\", \"r5_deaths\", \"r5_items\", \"d1_level\", \"d1_xp\", \"d1_gold\", \"d1_lh\", \"d1_kills\", \"d1_deaths\", \"d1_items\", \"d2_level\", \"d2_xp\", \"d2_gold\", \"d2_lh\", \"d2_kills\", \"d2_deaths\", \"d2_items\", \"d3_level\", \"d3_xp\", \"d3_gold\", \"d3_lh\", \"d3_kills\", \"d3_deaths\", \"d3_items\", \"d4_level\", \"d4_xp\", \"d4_gold\", \"d4_lh\", \"d4_kills\", \"d4_deaths\", \"d4_items\", \"d5_level\", \"d5_xp\", \"d5_gold\", \"d5_lh\", \"d5_kills\", \"d5_deaths\", \"d5_items\", \"first_blood_time\", \"first_blood_team\", \"first_blood_player1\", \"first_blood_player2\", \"radiant_bottle_time\", \"radiant_courier_time\", \"radiant_flying_courier_time\", \"radiant_tpscroll_count\", \"radiant_boots_count\", \"radiant_ward_observer_count\", \"radiant_ward_sentry_count\", \"radiant_first_ward_time\", \"dire_bottle_time\", \"dire_courier_time\", \"dire_flying_courier_time\", \"dire_tpscroll_count\", \"dire_boots_count\", \"dire_ward_observer_count\", \"dire_ward_sentry_count\", \"dire_first_ward_time\", \"r1_hero\", \"r2_hero\", \"r3_hero\", \"r4_hero\", \"r5_hero\", \"r6_hero\", \"r7_hero\", \"r8_hero\", \"r9_hero\", \"r10_hero\", \"r11_hero\", \"r12_hero\", \"r13_hero\", \"r14_hero\", \"r15_hero\", \"r16_hero\", \"r17_hero\", \"r18_hero\", \"r19_hero\", \"r20_hero\", \"r21_hero\", \"r22_hero\", \"r23_hero\", \"r24_hero\", \"r25_hero\", \"r26_hero\", \"r27_hero\", \"r28_hero\", \"r29_hero\", \"r30_hero\", \"r31_hero\", \"r32_hero\", \"r33_hero\", \"r34_hero\", \"r35_hero\", \"r36_hero\", \"r37_hero\", \"r38_hero\", \"r39_hero\", \"r40_hero\", \"r41_hero\", \"r42_hero\", \"r43_hero\", \"r44_hero\", \"r45_hero\", \"r46_hero\", \"r47_hero\", \"r48_hero\", \"r49_hero\", \"r50_hero\", \"r51_hero\", \"r52_hero\", \"r53_hero\", \"r54_hero\", \"r55_hero\", \"r56_hero\", \"r57_hero\", \"r58_hero\", \"r59_hero\", \"r60_hero\", \"r61_hero\", \"r62_hero\", \"r63_hero\", \"r64_hero\", \"r65_hero\", \"r66_hero\", \"r67_hero\", \"r68_hero\", \"r69_hero\", \"r70_hero\", \"r71_hero\", \"r72_hero\", \"r73_hero\", \"r74_hero\", \"r75_hero\", \"r76_hero\", \"r77_hero\", \"r78_hero\", \"r79_hero\", \"r80_hero\", \"r81_hero\", \"r82_hero\", \"r83_hero\", \"r84_hero\", \"r85_hero\", \"r86_hero\", \"r87_hero\", \"r88_hero\", \"r89_hero\", \"r90_hero\", \"r91_hero\", \"r92_hero\", \"r93_hero\", \"r94_hero\", \"r95_hero\", \"r96_hero\", \"r97_hero\", \"r98_hero\", \"r99_hero\", \"r100_hero\", \"r101_hero\", \"r102_hero\", \"r103_hero\", \"r104_hero\", \"r105_hero\", \"r106_hero\", \"r107_hero\", \"r108_hero\", \"r109_hero\", \"r110_hero\", \"r111_hero\", \"r112_hero\", \"r113_hero\", \"lobby_type_0\", \"lobby_type_1\", \"lobby_type_7\"], \"hoffset\": 0, \"location\": \"mouse\", \"type\": \"tooltip\", \"id\": \"el2696140568118811856pts\"}], \"data\": {\"data01\": [[0.0, 0.04434526773715801], [1.0, 0.00031368539573230214], [2.0, 0.0881214654892514], [3.0, 0.1952039716574155], [4.0, 0.20356579163046745], [5.0, -0.015057543759404525], [6.0, -0.033147437700291486], [7.0, 0.03885908721594302], [8.0, 0.023423409827562858], [9.0, 0.07099383607115549], [10.0, 0.19242730591242957], [11.0, 0.21011356846730733], [12.0, -0.020172021083554648], [13.0, -0.03592521355405989], [14.0, 0.023854408389860582], [15.0, 0.009156654722882792], [16.0, 0.08470135422365789], [17.0, 0.1617927230658365], [18.0, 0.21310779248460068], [19.0, -0.016712054115350063], [20.0, -0.037816642562018], [21.0, 0.04579993430793115], [22.0, 0.014871756007665037], [23.0, 0.10036662219808189], [24.0, 0.16217380705188716], [25.0, 0.21320429282427875], [26.0, -0.019860266098183785], [27.0, -0.04423167054092045], [28.0, 0.03506301458422], [29.0, 0.04074039878780397], [30.0, 0.07840752666518515], [31.0, 0.1701808417604143], [32.0, 0.19725436496376733], [33.0, -0.02131555945552661], [34.0, -0.04287871917974667], [35.0, 0.049731606471849314], [36.0, -0.017515812545314247], [37.0, -0.0783908895893313], [38.0, -0.18774498946583584], [39.0, -0.2093816675577557], [40.0, 0.022540869127036626], [41.0, 0.04063596510989026], [42.0, -0.031591787378169554], [43.0, -0.010386593491209605], [44.0, -0.09137855623350795], [45.0, -0.19920317336540158], [46.0, -0.21140279964389058], [47.0, 0.024974292931394054], [48.0, 0.034408039454648105], [49.0, -0.04211248839161042], [50.0, -0.0028971681586984353], [51.0, -0.11369280464424351], [52.0, -0.13669703472914616], [53.0, -0.22755131970127696], [54.0, 0.006047759205860915], [55.0, 0.029645463686358415], [56.0, -0.041154735719724445], [57.0, -0.024378547440030283], [58.0, -0.07015021400164888], [59.0, -0.18193668560662038], [60.0, -0.21298636956589403], [61.0, 0.0200254802974236], [62.0, 0.0353991831099745], [63.0, -0.04358631908951941], [64.0, -0.002676498538319917], [65.0, -0.11111066728538858], [66.0, -0.1593594203503882], [67.0, -0.22078111089251412], [68.0, 0.008899785827634425], [69.0, 0.048501537672035154], [70.0, -0.04576790256337048], [71.0, 0.004614396550990048], [72.0, 0.004701457561899595], [73.0, -0.0026096455089122546], [74.0, -0.03994835564440743], [75.0, -0.004040796672453123], [76.0, -0.03136276391325785], [77.0, 0.03671382410093577], [78.0, -0.04323709730139952], [79.0, 0.02548578982749939], [80.0, -0.008421493056793405], [81.0, -0.0013482209147979661], [82.0, -0.04160042915574285], [83.0, 0.0051464747952569305], [84.0, 0.01967639542620876], [85.0, -0.01046586266991197], [86.0, 0.04172105255018676], [87.0, -0.018779698658115525], [88.0, 0.00781888043093597], [89.0, -0.006618835387104046], [90.0, 0.03295633632671959], [91.0, 0.0], [92.0, 0.03761027445388548], [93.0, -0.13622088942794855], [94.0, -0.06537456194652506], [95.0, -0.02110475447066249], [96.0, 0.026734273857000848], [97.0, 0.029848370052074743], [98.0, 0.139426900732017], [99.0, 0.05624996842612775], [100.0, -0.007975277522306418], [101.0, -0.008760616672128374], [102.0, 0.047658030196167836], [103.0, -0.021467458980984087], [104.0, -0.09473299579802405], [105.0, 0.008935678729757637], [106.0, -0.03508449600588794], [107.0, -0.00146325294948813], [108.0, -0.027441528416010174], [109.0, 0.01625973245134511], [110.0, 0.02256488251172928], [111.0, 0.008653154575602789], [112.0, -0.11411625851888858], [113.0, 0.07628385660351057], [114.0, -0.0011660665457999281], [115.0, 0.0], [116.0, -0.1550022603398335], [117.0, -0.07924880182839586], [118.0, -0.010124652712125221], [119.0, 0.09045727633255214], [120.0, -0.00682521026681741], [121.0, 0.019023114887581148], [122.0, 0.027142417488044395], [123.0, 0.046366264245994486], [124.0, -0.07229193035987205], [125.0, -0.02989138607633793], [126.0, -0.01907461171857886], [127.0, 0.022270397134145244], [128.0, 0.011174113248279431], [129.0, 0.030655942671099005], [130.0, -0.2146060578231865], [131.0, 0.03714803874386294], [132.0, -0.04446168779324445], [133.0, 0.10482831960753829], [134.0, -0.02504562785586077], [135.0, 0.00128066178005565], [136.0, -0.0015096033516444614], [137.0, -0.017194582065809854], [138.0, -0.11727936010127166], [139.0, 0.017225693869235052], [140.0, 0.031030629580444953], [141.0, -0.014743706792277064], [142.0, 0.020244025857577886], [143.0, 0.04949859695665608], [144.0, -0.036298957167045516], [145.0, -0.02751653148160491], [146.0, -0.07798641298963874], [147.0, -0.011059869031438503], [148.0, 0.1252287731552354], [149.0, -0.04040407410554968], [150.0, 0.012451377870360015], [151.0, 0.011298279161220951], [152.0, -0.05342804751256813], [153.0, 0.04893137031155099], [154.0, -0.022117621883236023], [155.0, -0.01740582808967065], [156.0, -0.013124084037420193], [157.0, -0.020251050002644995], [158.0, 0.16544419725067647], [159.0, -0.011154250985053809], [160.0, 0.0832078263840872], [161.0, 0.04967304391360482], [162.0, 0.12318389764420803], [163.0, 0.010799228411824824], [164.0, 0.051414284372136534], [165.0, -0.008590509940163717], [166.0, -0.023599819480780805], [167.0, -0.06326206268095282], [168.0, 0.033683777286717835], [169.0, -0.0017087213758050434], [170.0, -0.03283923314655126], [171.0, -0.012505213035174588], [172.0, 0.003300949761810418], [173.0, 0.035296561621183976], [174.0, 0.016123840939027228], [175.0, -0.008382448791772096], [176.0, 0.04638035491224528], [177.0, -0.0656954837799374], [178.0, -0.03568221571703649], [179.0, 0.005497111283608512], [180.0, -0.01202258498881999], [181.0, -0.009776317569690895], [182.0, 0.036195946978530444], [183.0, 0.022082386277074924], [184.0, 0.027240610933083886], [185.0, 0.03744860350566008], [186.0, -0.019772173279806435], [187.0, 0.011097601473824992], [188.0, -0.03488592248091415], [189.0, -0.022705658213736688], [190.0, -0.06678782318331415], [191.0, 0.05831486436433333], [192.0, -0.0802510998365847], [193.0, 0.06773578765861926], [194.0, 0.018952617936938174], [195.0, -0.04869486783630778], [196.0, -0.0004070606014293968], [197.0, 0.012672277514761325], [198.0, 0.0], [199.0, 0.0], [200.0, -0.006285880407519225], [201.0, 0.04418379725593357], [202.0, 0.0], [203.0, -0.003599809381262498], [204.0, -0.0354996653887285], [205.0, 0.028390815926553514], [206.0, -0.004528527799266339]]}, \"id\": \"el2696140568101254608\"});\n",
       "   }(mpld3);\n",
       "}else if(typeof define === \"function\" && define.amd){\n",
       "   // require.js is available: use it to load d3/mpld3\n",
       "   require.config({paths: {d3: \"https://mpld3.github.io/js/d3.v3.min\"}});\n",
       "   require([\"d3\"], function(d3){\n",
       "      window.d3 = d3;\n",
       "      mpld3_load_lib(\"https://mpld3.github.io/js/mpld3.v0.2.js\", function(){\n",
       "         \n",
       "         mpld3.draw_figure(\"fig_el26961405681012546082682948170\", {\"axes\": [{\"xlim\": [0.0, 250.0], \"yscale\": \"linear\", \"axesbg\": \"#EEEEEE\", \"texts\": [], \"zoomable\": true, \"images\": [], \"xdomain\": [0.0, 250.0], \"ylim\": [-0.30000000000000004, 0.30000000000000004], \"paths\": [], \"sharey\": [], \"sharex\": [], \"axesbgalpha\": null, \"axes\": [{\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"bottom\", \"nticks\": 6, \"tickvalues\": null}, {\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"left\", \"nticks\": 9, \"tickvalues\": null}], \"lines\": [{\"color\": \"#FF0000\", \"yindex\": 1, \"coordinates\": \"data\", \"dasharray\": \"10,0\", \"zorder\": 2, \"alpha\": 1, \"xindex\": 0, \"linewidth\": 1.0, \"data\": \"data01\", \"id\": \"el2696140568118811856\"}], \"markers\": [{\"edgecolor\": \"#000000\", \"facecolor\": \"#FF0000\", \"edgewidth\": 0.5, \"yindex\": 1, \"coordinates\": \"data\", \"zorder\": 2, \"markerpath\": [[[0.0, 3.0], [0.7956093000000001, 3.0], [1.5587396123545605, 2.683901074764725], [2.121320343559643, 2.121320343559643], [2.683901074764725, 1.5587396123545605], [3.0, 0.7956093000000001], [3.0, 0.0], [3.0, -0.7956093000000001], [2.683901074764725, -1.5587396123545605], [2.121320343559643, -2.121320343559643], [1.5587396123545605, -2.683901074764725], [0.7956093000000001, -3.0], [0.0, -3.0], [-0.7956093000000001, -3.0], [-1.5587396123545605, -2.683901074764725], [-2.121320343559643, -2.121320343559643], [-2.683901074764725, -1.5587396123545605], [-3.0, -0.7956093000000001], [-3.0, 0.0], [-3.0, 0.7956093000000001], [-2.683901074764725, 1.5587396123545605], [-2.121320343559643, 2.121320343559643], [-1.5587396123545605, 2.683901074764725], [-0.7956093000000001, 3.0], [0.0, 3.0]], [\"M\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"Z\"]], \"alpha\": 1, \"xindex\": 0, \"data\": \"data01\", \"id\": \"el2696140568118811856pts\"}], \"id\": \"el2696140568101256720\", \"ydomain\": [-0.30000000000000004, 0.30000000000000004], \"collections\": [], \"xscale\": \"linear\", \"bbox\": [0.125, 0.125, 0.77500000000000002, 0.77500000000000002]}], \"height\": 320.0, \"width\": 480.0, \"plugins\": [{\"type\": \"reset\"}, {\"enabled\": false, \"button\": true, \"type\": \"zoom\"}, {\"enabled\": false, \"button\": true, \"type\": \"boxzoom\"}, {\"voffset\": 10, \"labels\": [\"start_time\", \"r1_level\", \"r1_xp\", \"r1_gold\", \"r1_lh\", \"r1_kills\", \"r1_deaths\", \"r1_items\", \"r2_level\", \"r2_xp\", \"r2_gold\", \"r2_lh\", \"r2_kills\", \"r2_deaths\", \"r2_items\", \"r3_level\", \"r3_xp\", \"r3_gold\", \"r3_lh\", \"r3_kills\", \"r3_deaths\", \"r3_items\", \"r4_level\", \"r4_xp\", \"r4_gold\", \"r4_lh\", \"r4_kills\", \"r4_deaths\", \"r4_items\", \"r5_level\", \"r5_xp\", \"r5_gold\", \"r5_lh\", \"r5_kills\", \"r5_deaths\", \"r5_items\", \"d1_level\", \"d1_xp\", \"d1_gold\", \"d1_lh\", \"d1_kills\", \"d1_deaths\", \"d1_items\", \"d2_level\", \"d2_xp\", \"d2_gold\", \"d2_lh\", \"d2_kills\", \"d2_deaths\", \"d2_items\", \"d3_level\", \"d3_xp\", \"d3_gold\", \"d3_lh\", \"d3_kills\", \"d3_deaths\", \"d3_items\", \"d4_level\", \"d4_xp\", \"d4_gold\", \"d4_lh\", \"d4_kills\", \"d4_deaths\", \"d4_items\", \"d5_level\", \"d5_xp\", \"d5_gold\", \"d5_lh\", \"d5_kills\", \"d5_deaths\", \"d5_items\", \"first_blood_time\", \"first_blood_team\", \"first_blood_player1\", \"first_blood_player2\", \"radiant_bottle_time\", \"radiant_courier_time\", \"radiant_flying_courier_time\", \"radiant_tpscroll_count\", \"radiant_boots_count\", \"radiant_ward_observer_count\", \"radiant_ward_sentry_count\", \"radiant_first_ward_time\", \"dire_bottle_time\", \"dire_courier_time\", \"dire_flying_courier_time\", \"dire_tpscroll_count\", \"dire_boots_count\", \"dire_ward_observer_count\", \"dire_ward_sentry_count\", \"dire_first_ward_time\", \"r1_hero\", \"r2_hero\", \"r3_hero\", \"r4_hero\", \"r5_hero\", \"r6_hero\", \"r7_hero\", \"r8_hero\", \"r9_hero\", \"r10_hero\", \"r11_hero\", \"r12_hero\", \"r13_hero\", \"r14_hero\", \"r15_hero\", \"r16_hero\", \"r17_hero\", \"r18_hero\", \"r19_hero\", \"r20_hero\", \"r21_hero\", \"r22_hero\", \"r23_hero\", \"r24_hero\", \"r25_hero\", \"r26_hero\", \"r27_hero\", \"r28_hero\", \"r29_hero\", \"r30_hero\", \"r31_hero\", \"r32_hero\", \"r33_hero\", \"r34_hero\", \"r35_hero\", \"r36_hero\", \"r37_hero\", \"r38_hero\", \"r39_hero\", \"r40_hero\", \"r41_hero\", \"r42_hero\", \"r43_hero\", \"r44_hero\", \"r45_hero\", \"r46_hero\", \"r47_hero\", \"r48_hero\", \"r49_hero\", \"r50_hero\", \"r51_hero\", \"r52_hero\", \"r53_hero\", \"r54_hero\", \"r55_hero\", \"r56_hero\", \"r57_hero\", \"r58_hero\", \"r59_hero\", \"r60_hero\", \"r61_hero\", \"r62_hero\", \"r63_hero\", \"r64_hero\", \"r65_hero\", \"r66_hero\", \"r67_hero\", \"r68_hero\", \"r69_hero\", \"r70_hero\", \"r71_hero\", \"r72_hero\", \"r73_hero\", \"r74_hero\", \"r75_hero\", \"r76_hero\", \"r77_hero\", \"r78_hero\", \"r79_hero\", \"r80_hero\", \"r81_hero\", \"r82_hero\", \"r83_hero\", \"r84_hero\", \"r85_hero\", \"r86_hero\", \"r87_hero\", \"r88_hero\", \"r89_hero\", \"r90_hero\", \"r91_hero\", \"r92_hero\", \"r93_hero\", \"r94_hero\", \"r95_hero\", \"r96_hero\", \"r97_hero\", \"r98_hero\", \"r99_hero\", \"r100_hero\", \"r101_hero\", \"r102_hero\", \"r103_hero\", \"r104_hero\", \"r105_hero\", \"r106_hero\", \"r107_hero\", \"r108_hero\", \"r109_hero\", \"r110_hero\", \"r111_hero\", \"r112_hero\", \"r113_hero\", \"lobby_type_0\", \"lobby_type_1\", \"lobby_type_7\"], \"hoffset\": 0, \"location\": \"mouse\", \"type\": \"tooltip\", \"id\": \"el2696140568118811856pts\"}], \"data\": {\"data01\": [[0.0, 0.04434526773715801], [1.0, 0.00031368539573230214], [2.0, 0.0881214654892514], [3.0, 0.1952039716574155], [4.0, 0.20356579163046745], [5.0, -0.015057543759404525], [6.0, -0.033147437700291486], [7.0, 0.03885908721594302], [8.0, 0.023423409827562858], [9.0, 0.07099383607115549], [10.0, 0.19242730591242957], [11.0, 0.21011356846730733], [12.0, -0.020172021083554648], [13.0, -0.03592521355405989], [14.0, 0.023854408389860582], [15.0, 0.009156654722882792], [16.0, 0.08470135422365789], [17.0, 0.1617927230658365], [18.0, 0.21310779248460068], [19.0, -0.016712054115350063], [20.0, -0.037816642562018], [21.0, 0.04579993430793115], [22.0, 0.014871756007665037], [23.0, 0.10036662219808189], [24.0, 0.16217380705188716], [25.0, 0.21320429282427875], [26.0, -0.019860266098183785], [27.0, -0.04423167054092045], [28.0, 0.03506301458422], [29.0, 0.04074039878780397], [30.0, 0.07840752666518515], [31.0, 0.1701808417604143], [32.0, 0.19725436496376733], [33.0, -0.02131555945552661], [34.0, -0.04287871917974667], [35.0, 0.049731606471849314], [36.0, -0.017515812545314247], [37.0, -0.0783908895893313], [38.0, -0.18774498946583584], [39.0, -0.2093816675577557], [40.0, 0.022540869127036626], [41.0, 0.04063596510989026], [42.0, -0.031591787378169554], [43.0, -0.010386593491209605], [44.0, -0.09137855623350795], [45.0, -0.19920317336540158], [46.0, -0.21140279964389058], [47.0, 0.024974292931394054], [48.0, 0.034408039454648105], [49.0, -0.04211248839161042], [50.0, -0.0028971681586984353], [51.0, -0.11369280464424351], [52.0, -0.13669703472914616], [53.0, -0.22755131970127696], [54.0, 0.006047759205860915], [55.0, 0.029645463686358415], [56.0, -0.041154735719724445], [57.0, -0.024378547440030283], [58.0, -0.07015021400164888], [59.0, -0.18193668560662038], [60.0, -0.21298636956589403], [61.0, 0.0200254802974236], [62.0, 0.0353991831099745], [63.0, -0.04358631908951941], [64.0, -0.002676498538319917], [65.0, -0.11111066728538858], [66.0, -0.1593594203503882], [67.0, -0.22078111089251412], [68.0, 0.008899785827634425], [69.0, 0.048501537672035154], [70.0, -0.04576790256337048], [71.0, 0.004614396550990048], [72.0, 0.004701457561899595], [73.0, -0.0026096455089122546], [74.0, -0.03994835564440743], [75.0, -0.004040796672453123], [76.0, -0.03136276391325785], [77.0, 0.03671382410093577], [78.0, -0.04323709730139952], [79.0, 0.02548578982749939], [80.0, -0.008421493056793405], [81.0, -0.0013482209147979661], [82.0, -0.04160042915574285], [83.0, 0.0051464747952569305], [84.0, 0.01967639542620876], [85.0, -0.01046586266991197], [86.0, 0.04172105255018676], [87.0, -0.018779698658115525], [88.0, 0.00781888043093597], [89.0, -0.006618835387104046], [90.0, 0.03295633632671959], [91.0, 0.0], [92.0, 0.03761027445388548], [93.0, -0.13622088942794855], [94.0, -0.06537456194652506], [95.0, -0.02110475447066249], [96.0, 0.026734273857000848], [97.0, 0.029848370052074743], [98.0, 0.139426900732017], [99.0, 0.05624996842612775], [100.0, -0.007975277522306418], [101.0, -0.008760616672128374], [102.0, 0.047658030196167836], [103.0, -0.021467458980984087], [104.0, -0.09473299579802405], [105.0, 0.008935678729757637], [106.0, -0.03508449600588794], [107.0, -0.00146325294948813], [108.0, -0.027441528416010174], [109.0, 0.01625973245134511], [110.0, 0.02256488251172928], [111.0, 0.008653154575602789], [112.0, -0.11411625851888858], [113.0, 0.07628385660351057], [114.0, -0.0011660665457999281], [115.0, 0.0], [116.0, -0.1550022603398335], [117.0, -0.07924880182839586], [118.0, -0.010124652712125221], [119.0, 0.09045727633255214], [120.0, -0.00682521026681741], [121.0, 0.019023114887581148], [122.0, 0.027142417488044395], [123.0, 0.046366264245994486], [124.0, -0.07229193035987205], [125.0, -0.02989138607633793], [126.0, -0.01907461171857886], [127.0, 0.022270397134145244], [128.0, 0.011174113248279431], [129.0, 0.030655942671099005], [130.0, -0.2146060578231865], [131.0, 0.03714803874386294], [132.0, -0.04446168779324445], [133.0, 0.10482831960753829], [134.0, -0.02504562785586077], [135.0, 0.00128066178005565], [136.0, -0.0015096033516444614], [137.0, -0.017194582065809854], [138.0, -0.11727936010127166], [139.0, 0.017225693869235052], [140.0, 0.031030629580444953], [141.0, -0.014743706792277064], [142.0, 0.020244025857577886], [143.0, 0.04949859695665608], [144.0, -0.036298957167045516], [145.0, -0.02751653148160491], [146.0, -0.07798641298963874], [147.0, -0.011059869031438503], [148.0, 0.1252287731552354], [149.0, -0.04040407410554968], [150.0, 0.012451377870360015], [151.0, 0.011298279161220951], [152.0, -0.05342804751256813], [153.0, 0.04893137031155099], [154.0, -0.022117621883236023], [155.0, -0.01740582808967065], [156.0, -0.013124084037420193], [157.0, -0.020251050002644995], [158.0, 0.16544419725067647], [159.0, -0.011154250985053809], [160.0, 0.0832078263840872], [161.0, 0.04967304391360482], [162.0, 0.12318389764420803], [163.0, 0.010799228411824824], [164.0, 0.051414284372136534], [165.0, -0.008590509940163717], [166.0, -0.023599819480780805], [167.0, -0.06326206268095282], [168.0, 0.033683777286717835], [169.0, -0.0017087213758050434], [170.0, -0.03283923314655126], [171.0, -0.012505213035174588], [172.0, 0.003300949761810418], [173.0, 0.035296561621183976], [174.0, 0.016123840939027228], [175.0, -0.008382448791772096], [176.0, 0.04638035491224528], [177.0, -0.0656954837799374], [178.0, -0.03568221571703649], [179.0, 0.005497111283608512], [180.0, -0.01202258498881999], [181.0, -0.009776317569690895], [182.0, 0.036195946978530444], [183.0, 0.022082386277074924], [184.0, 0.027240610933083886], [185.0, 0.03744860350566008], [186.0, -0.019772173279806435], [187.0, 0.011097601473824992], [188.0, -0.03488592248091415], [189.0, -0.022705658213736688], [190.0, -0.06678782318331415], [191.0, 0.05831486436433333], [192.0, -0.0802510998365847], [193.0, 0.06773578765861926], [194.0, 0.018952617936938174], [195.0, -0.04869486783630778], [196.0, -0.0004070606014293968], [197.0, 0.012672277514761325], [198.0, 0.0], [199.0, 0.0], [200.0, -0.006285880407519225], [201.0, 0.04418379725593357], [202.0, 0.0], [203.0, -0.003599809381262498], [204.0, -0.0354996653887285], [205.0, 0.028390815926553514], [206.0, -0.004528527799266339]]}, \"id\": \"el2696140568101254608\"});\n",
       "      });\n",
       "    });\n",
       "}else{\n",
       "    // require.js not available: dynamically load d3 & mpld3\n",
       "    mpld3_load_lib(\"https://mpld3.github.io/js/d3.v3.min.js\", function(){\n",
       "         mpld3_load_lib(\"https://mpld3.github.io/js/mpld3.v0.2.js\", function(){\n",
       "                 \n",
       "                 mpld3.draw_figure(\"fig_el26961405681012546082682948170\", {\"axes\": [{\"xlim\": [0.0, 250.0], \"yscale\": \"linear\", \"axesbg\": \"#EEEEEE\", \"texts\": [], \"zoomable\": true, \"images\": [], \"xdomain\": [0.0, 250.0], \"ylim\": [-0.30000000000000004, 0.30000000000000004], \"paths\": [], \"sharey\": [], \"sharex\": [], \"axesbgalpha\": null, \"axes\": [{\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"bottom\", \"nticks\": 6, \"tickvalues\": null}, {\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"left\", \"nticks\": 9, \"tickvalues\": null}], \"lines\": [{\"color\": \"#FF0000\", \"yindex\": 1, \"coordinates\": \"data\", \"dasharray\": \"10,0\", \"zorder\": 2, \"alpha\": 1, \"xindex\": 0, \"linewidth\": 1.0, \"data\": \"data01\", \"id\": \"el2696140568118811856\"}], \"markers\": [{\"edgecolor\": \"#000000\", \"facecolor\": \"#FF0000\", \"edgewidth\": 0.5, \"yindex\": 1, \"coordinates\": \"data\", \"zorder\": 2, \"markerpath\": [[[0.0, 3.0], [0.7956093000000001, 3.0], [1.5587396123545605, 2.683901074764725], [2.121320343559643, 2.121320343559643], [2.683901074764725, 1.5587396123545605], [3.0, 0.7956093000000001], [3.0, 0.0], [3.0, -0.7956093000000001], [2.683901074764725, -1.5587396123545605], [2.121320343559643, -2.121320343559643], [1.5587396123545605, -2.683901074764725], [0.7956093000000001, -3.0], [0.0, -3.0], [-0.7956093000000001, -3.0], [-1.5587396123545605, -2.683901074764725], [-2.121320343559643, -2.121320343559643], [-2.683901074764725, -1.5587396123545605], [-3.0, -0.7956093000000001], [-3.0, 0.0], [-3.0, 0.7956093000000001], [-2.683901074764725, 1.5587396123545605], [-2.121320343559643, 2.121320343559643], [-1.5587396123545605, 2.683901074764725], [-0.7956093000000001, 3.0], [0.0, 3.0]], [\"M\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"Z\"]], \"alpha\": 1, \"xindex\": 0, \"data\": \"data01\", \"id\": \"el2696140568118811856pts\"}], \"id\": \"el2696140568101256720\", \"ydomain\": [-0.30000000000000004, 0.30000000000000004], \"collections\": [], \"xscale\": \"linear\", \"bbox\": [0.125, 0.125, 0.77500000000000002, 0.77500000000000002]}], \"height\": 320.0, \"width\": 480.0, \"plugins\": [{\"type\": \"reset\"}, {\"enabled\": false, \"button\": true, \"type\": \"zoom\"}, {\"enabled\": false, \"button\": true, \"type\": \"boxzoom\"}, {\"voffset\": 10, \"labels\": [\"start_time\", \"r1_level\", \"r1_xp\", \"r1_gold\", \"r1_lh\", \"r1_kills\", \"r1_deaths\", \"r1_items\", \"r2_level\", \"r2_xp\", \"r2_gold\", \"r2_lh\", \"r2_kills\", \"r2_deaths\", \"r2_items\", \"r3_level\", \"r3_xp\", \"r3_gold\", \"r3_lh\", \"r3_kills\", \"r3_deaths\", \"r3_items\", \"r4_level\", \"r4_xp\", \"r4_gold\", \"r4_lh\", \"r4_kills\", \"r4_deaths\", \"r4_items\", \"r5_level\", \"r5_xp\", \"r5_gold\", \"r5_lh\", \"r5_kills\", \"r5_deaths\", \"r5_items\", \"d1_level\", \"d1_xp\", \"d1_gold\", \"d1_lh\", \"d1_kills\", \"d1_deaths\", \"d1_items\", \"d2_level\", \"d2_xp\", \"d2_gold\", \"d2_lh\", \"d2_kills\", \"d2_deaths\", \"d2_items\", \"d3_level\", \"d3_xp\", \"d3_gold\", \"d3_lh\", \"d3_kills\", \"d3_deaths\", \"d3_items\", \"d4_level\", \"d4_xp\", \"d4_gold\", \"d4_lh\", \"d4_kills\", \"d4_deaths\", \"d4_items\", \"d5_level\", \"d5_xp\", \"d5_gold\", \"d5_lh\", \"d5_kills\", \"d5_deaths\", \"d5_items\", \"first_blood_time\", \"first_blood_team\", \"first_blood_player1\", \"first_blood_player2\", \"radiant_bottle_time\", \"radiant_courier_time\", \"radiant_flying_courier_time\", \"radiant_tpscroll_count\", \"radiant_boots_count\", \"radiant_ward_observer_count\", \"radiant_ward_sentry_count\", \"radiant_first_ward_time\", \"dire_bottle_time\", \"dire_courier_time\", \"dire_flying_courier_time\", \"dire_tpscroll_count\", \"dire_boots_count\", \"dire_ward_observer_count\", \"dire_ward_sentry_count\", \"dire_first_ward_time\", \"r1_hero\", \"r2_hero\", \"r3_hero\", \"r4_hero\", \"r5_hero\", \"r6_hero\", \"r7_hero\", \"r8_hero\", \"r9_hero\", \"r10_hero\", \"r11_hero\", \"r12_hero\", \"r13_hero\", \"r14_hero\", \"r15_hero\", \"r16_hero\", \"r17_hero\", \"r18_hero\", \"r19_hero\", \"r20_hero\", \"r21_hero\", \"r22_hero\", \"r23_hero\", \"r24_hero\", \"r25_hero\", \"r26_hero\", \"r27_hero\", \"r28_hero\", \"r29_hero\", \"r30_hero\", \"r31_hero\", \"r32_hero\", \"r33_hero\", \"r34_hero\", \"r35_hero\", \"r36_hero\", \"r37_hero\", \"r38_hero\", \"r39_hero\", \"r40_hero\", \"r41_hero\", \"r42_hero\", \"r43_hero\", \"r44_hero\", \"r45_hero\", \"r46_hero\", \"r47_hero\", \"r48_hero\", \"r49_hero\", \"r50_hero\", \"r51_hero\", \"r52_hero\", \"r53_hero\", \"r54_hero\", \"r55_hero\", \"r56_hero\", \"r57_hero\", \"r58_hero\", \"r59_hero\", \"r60_hero\", \"r61_hero\", \"r62_hero\", \"r63_hero\", \"r64_hero\", \"r65_hero\", \"r66_hero\", \"r67_hero\", \"r68_hero\", \"r69_hero\", \"r70_hero\", \"r71_hero\", \"r72_hero\", \"r73_hero\", \"r74_hero\", \"r75_hero\", \"r76_hero\", \"r77_hero\", \"r78_hero\", \"r79_hero\", \"r80_hero\", \"r81_hero\", \"r82_hero\", \"r83_hero\", \"r84_hero\", \"r85_hero\", \"r86_hero\", \"r87_hero\", \"r88_hero\", \"r89_hero\", \"r90_hero\", \"r91_hero\", \"r92_hero\", \"r93_hero\", \"r94_hero\", \"r95_hero\", \"r96_hero\", \"r97_hero\", \"r98_hero\", \"r99_hero\", \"r100_hero\", \"r101_hero\", \"r102_hero\", \"r103_hero\", \"r104_hero\", \"r105_hero\", \"r106_hero\", \"r107_hero\", \"r108_hero\", \"r109_hero\", \"r110_hero\", \"r111_hero\", \"r112_hero\", \"r113_hero\", \"lobby_type_0\", \"lobby_type_1\", \"lobby_type_7\"], \"hoffset\": 0, \"location\": \"mouse\", \"type\": \"tooltip\", \"id\": \"el2696140568118811856pts\"}], \"data\": {\"data01\": [[0.0, 0.04434526773715801], [1.0, 0.00031368539573230214], [2.0, 0.0881214654892514], [3.0, 0.1952039716574155], [4.0, 0.20356579163046745], [5.0, -0.015057543759404525], [6.0, -0.033147437700291486], [7.0, 0.03885908721594302], [8.0, 0.023423409827562858], [9.0, 0.07099383607115549], [10.0, 0.19242730591242957], [11.0, 0.21011356846730733], [12.0, -0.020172021083554648], [13.0, -0.03592521355405989], [14.0, 0.023854408389860582], [15.0, 0.009156654722882792], [16.0, 0.08470135422365789], [17.0, 0.1617927230658365], [18.0, 0.21310779248460068], [19.0, -0.016712054115350063], [20.0, -0.037816642562018], [21.0, 0.04579993430793115], [22.0, 0.014871756007665037], [23.0, 0.10036662219808189], [24.0, 0.16217380705188716], [25.0, 0.21320429282427875], [26.0, -0.019860266098183785], [27.0, -0.04423167054092045], [28.0, 0.03506301458422], [29.0, 0.04074039878780397], [30.0, 0.07840752666518515], [31.0, 0.1701808417604143], [32.0, 0.19725436496376733], [33.0, -0.02131555945552661], [34.0, -0.04287871917974667], [35.0, 0.049731606471849314], [36.0, -0.017515812545314247], [37.0, -0.0783908895893313], [38.0, -0.18774498946583584], [39.0, -0.2093816675577557], [40.0, 0.022540869127036626], [41.0, 0.04063596510989026], [42.0, -0.031591787378169554], [43.0, -0.010386593491209605], [44.0, -0.09137855623350795], [45.0, -0.19920317336540158], [46.0, -0.21140279964389058], [47.0, 0.024974292931394054], [48.0, 0.034408039454648105], [49.0, -0.04211248839161042], [50.0, -0.0028971681586984353], [51.0, -0.11369280464424351], [52.0, -0.13669703472914616], [53.0, -0.22755131970127696], [54.0, 0.006047759205860915], [55.0, 0.029645463686358415], [56.0, -0.041154735719724445], [57.0, -0.024378547440030283], [58.0, -0.07015021400164888], [59.0, -0.18193668560662038], [60.0, -0.21298636956589403], [61.0, 0.0200254802974236], [62.0, 0.0353991831099745], [63.0, -0.04358631908951941], [64.0, -0.002676498538319917], [65.0, -0.11111066728538858], [66.0, -0.1593594203503882], [67.0, -0.22078111089251412], [68.0, 0.008899785827634425], [69.0, 0.048501537672035154], [70.0, -0.04576790256337048], [71.0, 0.004614396550990048], [72.0, 0.004701457561899595], [73.0, -0.0026096455089122546], [74.0, -0.03994835564440743], [75.0, -0.004040796672453123], [76.0, -0.03136276391325785], [77.0, 0.03671382410093577], [78.0, -0.04323709730139952], [79.0, 0.02548578982749939], [80.0, -0.008421493056793405], [81.0, -0.0013482209147979661], [82.0, -0.04160042915574285], [83.0, 0.0051464747952569305], [84.0, 0.01967639542620876], [85.0, -0.01046586266991197], [86.0, 0.04172105255018676], [87.0, -0.018779698658115525], [88.0, 0.00781888043093597], [89.0, -0.006618835387104046], [90.0, 0.03295633632671959], [91.0, 0.0], [92.0, 0.03761027445388548], [93.0, -0.13622088942794855], [94.0, -0.06537456194652506], [95.0, -0.02110475447066249], [96.0, 0.026734273857000848], [97.0, 0.029848370052074743], [98.0, 0.139426900732017], [99.0, 0.05624996842612775], [100.0, -0.007975277522306418], [101.0, -0.008760616672128374], [102.0, 0.047658030196167836], [103.0, -0.021467458980984087], [104.0, -0.09473299579802405], [105.0, 0.008935678729757637], [106.0, -0.03508449600588794], [107.0, -0.00146325294948813], [108.0, -0.027441528416010174], [109.0, 0.01625973245134511], [110.0, 0.02256488251172928], [111.0, 0.008653154575602789], [112.0, -0.11411625851888858], [113.0, 0.07628385660351057], [114.0, -0.0011660665457999281], [115.0, 0.0], [116.0, -0.1550022603398335], [117.0, -0.07924880182839586], [118.0, -0.010124652712125221], [119.0, 0.09045727633255214], [120.0, -0.00682521026681741], [121.0, 0.019023114887581148], [122.0, 0.027142417488044395], [123.0, 0.046366264245994486], [124.0, -0.07229193035987205], [125.0, -0.02989138607633793], [126.0, -0.01907461171857886], [127.0, 0.022270397134145244], [128.0, 0.011174113248279431], [129.0, 0.030655942671099005], [130.0, -0.2146060578231865], [131.0, 0.03714803874386294], [132.0, -0.04446168779324445], [133.0, 0.10482831960753829], [134.0, -0.02504562785586077], [135.0, 0.00128066178005565], [136.0, -0.0015096033516444614], [137.0, -0.017194582065809854], [138.0, -0.11727936010127166], [139.0, 0.017225693869235052], [140.0, 0.031030629580444953], [141.0, -0.014743706792277064], [142.0, 0.020244025857577886], [143.0, 0.04949859695665608], [144.0, -0.036298957167045516], [145.0, -0.02751653148160491], [146.0, -0.07798641298963874], [147.0, -0.011059869031438503], [148.0, 0.1252287731552354], [149.0, -0.04040407410554968], [150.0, 0.012451377870360015], [151.0, 0.011298279161220951], [152.0, -0.05342804751256813], [153.0, 0.04893137031155099], [154.0, -0.022117621883236023], [155.0, -0.01740582808967065], [156.0, -0.013124084037420193], [157.0, -0.020251050002644995], [158.0, 0.16544419725067647], [159.0, -0.011154250985053809], [160.0, 0.0832078263840872], [161.0, 0.04967304391360482], [162.0, 0.12318389764420803], [163.0, 0.010799228411824824], [164.0, 0.051414284372136534], [165.0, -0.008590509940163717], [166.0, -0.023599819480780805], [167.0, -0.06326206268095282], [168.0, 0.033683777286717835], [169.0, -0.0017087213758050434], [170.0, -0.03283923314655126], [171.0, -0.012505213035174588], [172.0, 0.003300949761810418], [173.0, 0.035296561621183976], [174.0, 0.016123840939027228], [175.0, -0.008382448791772096], [176.0, 0.04638035491224528], [177.0, -0.0656954837799374], [178.0, -0.03568221571703649], [179.0, 0.005497111283608512], [180.0, -0.01202258498881999], [181.0, -0.009776317569690895], [182.0, 0.036195946978530444], [183.0, 0.022082386277074924], [184.0, 0.027240610933083886], [185.0, 0.03744860350566008], [186.0, -0.019772173279806435], [187.0, 0.011097601473824992], [188.0, -0.03488592248091415], [189.0, -0.022705658213736688], [190.0, -0.06678782318331415], [191.0, 0.05831486436433333], [192.0, -0.0802510998365847], [193.0, 0.06773578765861926], [194.0, 0.018952617936938174], [195.0, -0.04869486783630778], [196.0, -0.0004070606014293968], [197.0, 0.012672277514761325], [198.0, 0.0], [199.0, 0.0], [200.0, -0.006285880407519225], [201.0, 0.04418379725593357], [202.0, 0.0], [203.0, -0.003599809381262498], [204.0, -0.0354996653887285], [205.0, 0.028390815926553514], [206.0, -0.004528527799266339]]}, \"id\": \"el2696140568101254608\"});\n",
       "            })\n",
       "         });\n",
       "}\n",
       "</script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import mpld3\n",
    "\n",
    "mpld3.enable_notebook()\n",
    "fig, ax = plt.subplots(subplot_kw=dict(axisbg='#EEEEEE'))\n",
    "\n",
    "somefig=ax.plot(clf2.coef_[0],'ro-')\n",
    "#x=range(len(allColumns))\n",
    "#plt.xticks(clf2.coef_[0],allColumns, rotation='vertical')\n",
    "\n",
    "#labels = ['point {0}'.format(i + 1) for i in range(N)]\n",
    "tooltip = mpld3.plugins.PointLabelTooltip(somefig[0], labels=allColumns)\n",
    "mpld3.plugins.connect(fig, tooltip)\n",
    "mpld3.display()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.75213195753995232"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scores2 = cross_val_score(LogisticRegression2(C=0.005,penalty='l2',random_state=1, n_jobs=-1),\n",
    "                            X_concat, Y, cv=KFold(len(X_concat),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')\n",
    "scores2.mean()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(97230, 207)"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Reduced4train = pandas.DataFrame(X_concat,columns=allColumns)\n",
    "Reduced4train.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17177, 207)"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Reduced4test = pandas.DataFrame(X_concat_test,columns=allColumns)\n",
    "Reduced4test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mostImportantColumnsR = ['r1_lh','r2_lh','r3_lh','r4_lh','r5_lh']\n",
    "mostImportantColumnsD = ['d1_lh','d2_lh','d3_lh','d4_lh','d5_lh']\n",
    "mostImportantHerosR = ['r8_hero','r23_hero','r43_hero','r58_hero','r68_hero']\n",
    "mostImportantHerosD = ['r3_hero','r26_hero','r40_hero','r48_hero']\n",
    "GoldR=['r1_gold','r2_gold','r3_gold','r4_gold','r5_gold']\n",
    "GoldD=['d1_gold','d2_gold','d3_gold','d4_gold','d5_gold']\n",
    "XpR=['r1_xp','r2_xp','r3_xp','r4_xp','r5_xp']\n",
    "XpD=['d1_xp','d2_xp','d3_xp','d4_xp','d5_xp']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Reduced4train['cumKillsR']=Reduced4train[mostImportantColumnsR].sum(axis=1)**2\n",
    "#Reduced4train['cumKillsD']=Reduced4train[mostImportantColumnsD].sum(axis=1)**2\n",
    "Reduced4train['cumKillsRD_D2']=(Reduced4train[mostImportantColumnsR].sum(axis=1)-Reduced4train[mostImportantColumnsD].sum(axis=1))**2\n",
    "\n",
    "#Reduced4train['cumKillsR_L']=Reduced4train[mostImportantColumnsR].sum(axis=1)\n",
    "#Reduced4train['cumKillsD_L']=Reduced4train[mostImportantColumnsD].sum(axis=1)\n",
    "Reduced4train['cumKillsRD_LD']=Reduced4train[mostImportantColumnsR].sum(axis=1)-Reduced4train[mostImportantColumnsD].sum(axis=1)\n",
    "\n",
    "#Reduced4train['herR']=Reduced4train[mostImportantHerosR].sum(axis=1)**2\n",
    "#Reduced4train['herD']=Reduced4train[mostImportantHerosD].sum(axis=1)**2\n",
    "\n",
    "#Reduced4train['GoldR']=Reduced4train[XpR].sum(axis=1)**2\n",
    "#Reduced4train['GoldD']=Reduced4train[XpD].sum(axis=1)**2\n",
    "Reduced4train['GoldRD_D2']=(Reduced4train[XpR].sum(axis=1)-Reduced4train[XpD].sum(axis=1))**2\n",
    "\n",
    "#Reduced4train['GoldR_L']=Reduced4train[XpR].sum(axis=1)\n",
    "#Reduced4train['GoldD_L']=Reduced4train[XpD].sum(axis=1)\n",
    "Reduced4train['GoldRD_D']=Reduced4train[XpR].sum(axis=1)-Reduced4train[XpD].sum(axis=1)\n",
    "\n",
    "#Reduced4train['XpR']=Reduced4train[GoldR].sum(axis=1)**2\n",
    "#Reduced4train['XpD']=Reduced4train[GoldD].sum(axis=1)**2\n",
    "Reduced4train['XpRD_D2']=(Reduced4train[GoldR].sum(axis=1)-Reduced4train[GoldD].sum(axis=1))**2\n",
    "\n",
    "#Reduced4train['XpR_L']=Reduced4train[GoldR].sum(axis=1)\n",
    "#Reduced4train['XpD_L']=Reduced4train[GoldD].sum(axis=1)\n",
    "Reduced4train['XpRD_LD']=Reduced4train[GoldR].sum(axis=1)-Reduced4train[GoldD].sum(axis=1)\n",
    "\n",
    "#Reduced4test['cumKillsR']=Reduced4test[mostImportantColumnsR].sum(axis=1)**2\n",
    "#Reduced4test['cumKillsD']=Reduced4test[mostImportantColumnsD].sum(axis=1)**2\n",
    "Reduced4test['cumKillsRD_D2']=(Reduced4test[mostImportantColumnsR].sum(axis=1)-Reduced4test[mostImportantColumnsD].sum(axis=1))**2\n",
    "\n",
    "\n",
    "#Reduced4test['cumKillsR_L']=Reduced4test[mostImportantColumnsR].sum(axis=1)\n",
    "#Reduced4test['cumKillsD_L']=Reduced4test[mostImportantColumnsD].sum(axis=1)\n",
    "Reduced4test['cumKillsRD_D']=Reduced4test[mostImportantColumnsR].sum(axis=1)-Reduced4test[mostImportantColumnsD].sum(axis=1)\n",
    "\n",
    "#Reduced4test['herR']=Reduced4test[mostImportantHerosR].sum(axis=1)**2\n",
    "#Reduced4test['herD']=Reduced4test[mostImportantHerosD].sum(axis=1)**2\n",
    "\n",
    "#Reduced4test['GoldR']=Reduced4test[XpR].sum(axis=1)**2\n",
    "#Reduced4test['GoldD']=Reduced4test[XpD].sum(axis=1)**2\n",
    "Reduced4test['GoldRD_D2']=(Reduced4test[XpR].sum(axis=1)-Reduced4test[XpD].sum(axis=1))**2\n",
    "\n",
    "#Reduced4test['GoldR_L']=Reduced4test[XpR].sum(axis=1)\n",
    "#Reduced4test['GoldD_L']=Reduced4test[XpD].sum(axis=1)\n",
    "Reduced4test['GoldRD_D']=Reduced4test[XpR].sum(axis=1)-Reduced4test[XpD].sum(axis=1)\n",
    "\n",
    "#Reduced4test['XpR']=Reduced4test[GoldR].sum(axis=1)**2\n",
    "#Reduced4test['XpD']=Reduced4test[GoldD].sum(axis=1)**2\n",
    "Reduced4test['XpRD_D2']=(Reduced4test[GoldR].sum(axis=1)-Reduced4test[GoldD].sum(axis=1))**2\n",
    "\n",
    "#Reduced4test['XpR_L']=Reduced4test[GoldR].sum(axis=1)\n",
    "#Reduced4test['XpD_L']=Reduced4test[GoldD].sum(axis=1)\n",
    "Reduced4test['XpRD_LD']=Reduced4test[GoldR].sum(axis=1)-Reduced4test[GoldD].sum(axis=1)\n",
    "\n",
    "Reduced4testCols = list(Reduced4test.columns.values)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "standard_scaler4 = StandardScaler()\n",
    "Reduced4train = standard_scaler4.fit_transform(Reduced4train)\n",
    "Reduced4test = standard_scaler4.transform(Reduced4test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "i 0.001000  score 0.752131, time elapsed:0:00:12.263855\n",
      "i 0.002000  score 0.752186, time elapsed:0:00:13.750017\n",
      "i 0.003000  score 0.752185, time elapsed:0:00:14.661408\n",
      "i 0.004000  score 0.752175, time elapsed:0:00:14.751901\n"
     ]
    }
   ],
   "source": [
    "[roc_auc5,Coeff5,timesElapsed5] = optimize_reg(Reduced4train, Y, np.arange(0.001,0.005,0.001))\n",
    "#best i 0.005000  score 0.752409, time elapsed:0:00:17.513508"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.75218377032045303"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "scores2 = cross_val_score(LogisticRegression2(C=0.002,penalty='l2',random_state=1, dual=False, \n",
    "                                              tol=0.0001,fit_intercept=True, class_weight='balanced'),\n",
    "                            Reduced4train, Y, cv=KFold(len(Reduced4train),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')\n",
    "scores2.mean()\n",
    "#0.75240576702810924"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.linear_model import SGDClassifier\n",
    "class SGDClassifier2(SGDClassifier):\n",
    "    def predict(X):\n",
    "        return SGDClassifier.predict_proba(self, X)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "scores2 = cross_val_score(SGDClassifier2(loss='log',penalty='l2'),\n",
    "                            Reduced4train, Y, cv=KFold(len(Reduced4train),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')\n",
    "scores2.mean()\n",
    "#res 0.74561891051697349 <- with dif params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC\n",
    "class SVC2(SVC):\n",
    "    def predict(X):\n",
    "        return SVC.predict_proba(self, X)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "scores2 = cross_val_score(SVC2(probability=True,verbose=True),\n",
    "                            Reduced4train, Y, cv=KFold(len(Reduced4train),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')\n",
    "scores2.mean()\n",
    "#cant solve very long time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from  sklearn.ensemble import RandomForestClassifier\n",
    "class RandomForestClassifier2(RandomForestClassifier):\n",
    "    def predict(X):\n",
    "        return RandomForestClassifier.predict_proba(self, X)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores2 = cross_val_score(RandomForestClassifier2(n_estimators=1000,verbose=True),\n",
    "                            Reduced4train, Y, cv=KFold(len(Reduced4train),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')\n",
    "scores2.mean()\n",
    "# 0.72049034821612956 for 500 estimators, 0.72151192088147087 - 1000"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores2 = cross_val_score(GradientBoostingClassifier2(n_estimators=500,verbose=True),\n",
    "                            Reduced4train, Y, cv=KFold(len(Reduced4train),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')\n",
    "scores2.mean()\n",
    "# 500 estimators - 0.74885936740953007 , 0.74883431278308077"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PCA(copy=True, n_components=None, whiten=False)"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#http://scikit-learn.org/stable/auto_examples/plot_digits_pipe.html#example-plot-digits-pipe-py\n",
    "from sklearn import decomposition, datasets\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "\n",
    "pca = decomposition.PCA()\n",
    "pipe = Pipeline(steps=[('pca', pca), ('logistic', LogisticRegression2)])\n",
    "\n",
    "pca.fit(Reduced4train)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7fd88facaed0>"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPIAAADFCAYAAABuF6+xAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGOFJREFUeJzt3XeYVdW5x/HvjyYwOIKKgCBYiHo1NhRFTcIYUwzGchOx\na8oTy1UfjXpje/SCucYUzc1DrjERTQzWmFixxHrFGix0FbAjjFJURHqb9/6x9nHODKftPafNmffz\nPPs5++yzy8sw76y111lrbZkZzrn2rVOlA3DOtZ0nsnM1wBPZuRrgiexcDfBEdq4GeCI7VwPKksiS\nLpX0uqSZkm6X1K0c13Wuoyh5IksaApwG7GNmewJdgONLfV3nOpIuZbjG58A6oE5SE9AT+DB9B0ne\nK8W5ApiZMm0veYlsZkuB3wIfAI3AZ2b2ZOv9rrjCMGt/y5gxYyoeg8dd/Usx4s6lHFXrHYHzgSHA\ntkAvSSe23m/dulJH4lztKkfVej/gBTP7FEDSvcBBwB3pOz3zzFjGjg3rDQ0NNDQ0lCE056rXpEmT\nmDRpUmE7l7pKAewFzAK6AwL+Cpzdah875xxrl55++ulKh5CIx11exYg7pGvmPJPlqXsXg6SfAT8E\nNgLTgJ+Y2fq0z+30040bbih5KM61W5KwLI1d5ahaY2bXANfk2mf9+lyfOudyqZqeXd7Y5VxynsjO\n1YCqSWSvWjuXXNUkspfIziXniexcDaiaRPaqtXPJVU0ie4nsXHKeyM7VgKpJZK9aO5dc1SSyl8jO\nJVeOYYw7S5omaWr0ukzSua3380R2LrmS97U2szeBfQAkdQIWAPe13s+r1s4lV+6q9TeAd8xsfusP\nvER2LrlyJ/JxwJ2ZPvBEdi65soxHBpDUlTDp3m5mtqTVZ9arl7F8eVlCca5dqvh45Mh3gCmtkzhl\n1Sqf6se5dHGm+ilniXwn8KiZTcjwmYHR1ATK+PfGOZerRC7XVD89gXnAjma2SQU6lcjr10OXctYR\nnGtHKl61NrNVQN98+61b54nsXBJFa7WWtHtbz+Et184lU8yvn25t6wm8U4hzyRQzkdvcTOUlsnPJ\nFDOR29xq5onsXDJVM/oJvGrtXFLFTOQ2l6deIjuXTMGJrOBkSf8VvR8saf/U52Y2oq3BeCI7l0yc\nEvl64EDghOj9cuAPxQzGq9bOJROn+8UBZjZM0jQIDzCX1K2YwXiJ7FwycUrk9ZI6E7VOS+oLNBVy\noKQtJP1D0mxJr0s6INN+nsjOJROnRP49YWaPbST9AjgGuLzAY8cBj5jZaEldgJ6ZdvKqtXPJFJzI\nZna7pCnAoYTOH0eb2ex8x0mqB75qZj+MzrMB+DzTvl4iO5dMnFbrEUCjmf3BzK4DGrNVkVvZAfhY\n0s3RBHzjJfXItKMnsnPJxLlH/iOwIu39imhbPl2AYcAfzGwYsAq4JNOOXrV2Lpk498iytMHLZtYU\n3e/mswCYb2avRu/vBi7edLex3HUXzJ3rM4Q4ByWaIUTSvcAkmkvhs4BDzOzoAo59BjjNzN6UNAbo\naWYXp31uYIwfD6edVlA4znU4uSYWiFO1PhM4CGgklLIHAKcXeOy5wO2SpgN7AVdn2mnt2hjROOe+\nEKfVejFwfJKLmNkMYHi+/dasSXJ251zBiRx1ADkN2D79ODP7cbGC8UR2Lpk4jV0PAM8BTwIbSxHM\n6tWlOKtztS9OIrdooCoFL5GdSyZOY9dDkkaVLBK8RHYuqTiJfB4hmVdL+lzSckkZu1om5SWyc8nE\nabXevJSBgCeyc0nFmg5eUh/gS0D31DYze7ZYwXjV2rlk4nz99BNC9XoQMB0YAfwL+HqxgvES2blk\n4t4jDwfmmdkhwD7AZ8UMxhPZuWTiJPIaM1sDIGkzM5sD7FLMYLxq7Vwyce6RF0jqDdwPPCFpKeEJ\ni3lJeh9YRpgaaL2Z7Z9pPy+RnUsm0WNVJY0EtiA87zjvdACS3gX2NbOlWT43MHbZBebMiR2Ocx1C\nmx6rKqnezD6XtGXa5lnRay/g00JioIBqvFetnUumkKr1HcB3gSmEGTTV6nXHAs5hhOr4RmC8md2Y\naSevWjuXTEFVa0kCtjOzDxJdRBpgZh9FI6ieAM4xs+fTPjcw6uth2bIkV3Cu9rWpag1gZibpYWCP\nJAGY2UfR6xJJ9wH7A8+33GssK1bA2LE+1Y9zULqpfiYA15nZK3GCkdQT6GRmKyTVAY8DV5rZ42n7\nmGSYwYYN0LlznCs41zG0uUSOHACcJGkesJLoHtnM9sxzXD/gvlB9pgtwe3oSp3TvHhq71q6Fnhmn\nr3fOZRMnkb+d5AJm9h6wd779Uom8erUnsnNxxRn9NA9A0jakDZoolu7RGb3l2rn44jxp4khJbwHv\nAc8A7wP/LFYgPaJnT3giOxdfnL7W/00Y8fSmme1AeAbU5GIFkiqRvVOIc/HFeqyqmX0CdJLUycye\nBvYrViBeIjuXXJzGrs8k9QKeJUw2v5jQel0Ufo/sXHJxSuSjCA9gOx94FHgHOKJYgXjV2rnk4pTI\nZwB3mVkjMKHYgXjV2rnk4pTImwOPS3pO0jmS+hUzkLq68Pp5UefldK5jKDiRzexKM9sdOBsYADwj\n6cliBTJwYHhtbCzWGZ3rOOKUyCmLgYXAJ8A2hR4kqZOkqZImZvp80KDwumBBgoic6+DidAg5S9Ik\n4ClgK8LzjvP1s053HvBGtg89kZ1LLk5j13bAT81seqYPJfXJMZXPIGAU8Avggkz7eCI7l1yce+RL\nsyVx5Kkcn/0O+BlhppCMPJGdSy7JPXI2GcdJSjocWBT9EVC2/QYMgE6dYOFCWL++iFE51wHEemRM\nHtlK24OBI6MnOfYANpd0i5mdmr7TVVeNpa4Oli+He+9t4LjjGooYmnPtT0lmCMl7ImmqmQ3Ls89I\n4EIzO7LVdjMzDjwQJk+G00+HceOae3s553LPEFLyqnUcF1wAXbrA+PEwfDhMz3VH7pz7Qt4SudV8\n1psws09T+6XWYwcRlcgAU6bAiSfCm2+Gz/beG449FkaPhqFDk5zdudqQq0QuJJHfo3ke68HA0mi9\nN/BBNDa5rQFaehwrV8Jll8Ff/9qyy+bPfw5XXNHWqznXPrWpam1mO5jZjsCTwBFmtrWZbUWYtH6T\nSfSKoa4u3CMvXgwTJ8KR0R31k0XrEOpcbYkzHe4sM9sj37ZEQbQqkVubOhX23TdUs6dNa+vVnGuf\nijUd7oeSLgdui96fBHzY1uAKsfnm4XX58nJczbn2J06r9QlAX+A+4N5o/YRSBNVafX149SGOzmUW\n+3tkSXVmVrQpfqJz5qxar1oV7ptTc1871xEV5XtkSQdJegOYHb3fS9L1RYoxpx49wmNk1qzx7pvO\nZRKnav07wtMmPgEwsxnA10oRVGuS3yc7l0usnl1mNr/Vpo1FjCUnv092Lrs4rdbzJR0EmKSuhIkC\nZpcmrE15iexcdnFK5DMJ83UNBBoJD2Y7O99BkjaT9JKkaZJel3R1kkC9RHYuuzgPcfuY8N1xLGa2\nVtIhZrZKUmfgBUkHm9kLcc6TKpE9kZ3bVMGJLKkvcBqwffpxZvbjfMea2apodTNCLSDjlEC5pEpk\nr1o7t6k498gPAM8R+lzHauSS1AmYAuwE/MnMsk7Cl42XyM5lFyeRe5rZxUkuYmZNwD6S6gmT3I80\ns2finMNLZOeyi5PID0kaZWaPJL2YmX0u6WHCUxxbJPLYsWO/WG9oaKChoaHFsV4iu46mJFP9SFoO\n1AFrgfWEMclmZvV5jtua8EjWZZJ6AI8BV5rZU2n75OyiCXDNNXDRRXDhhXDttQWF7FxNKcroJzPb\nPOH1BwATJInQ0HVrehIXyktk57LLm8iSdjWzOZIyTqxnZlNzHW9ms4Cck/IVInWPvGxZW8/kXO0p\npES+ADgd+G2Gzwz4elEjymLLaOawTxPNCuZcbcubyGZ2evR6SOnDyW6rrcLrJ59UMgrnqlOsCeol\nfRnYDfhixmkzu6XYQWXiiexcdnF6do0BGgiJ/AjwHeB5wBPZuQqLM2jiGOBQYKGZ/QjYC9iiJFFl\nUF8fJq9fuRLWri3XVZ1rH+Ik8uqoh9aGqIfWYsKjVstCam7w8lLZuZbiJPKrknoDNxL6TU8F/lWS\nqLLw6rVzmcXpEHJWtPonSY8C9WY2szRhZeaJ7FxmhXQIydqZQ9KwfB1CiskT2bnMCimRM3UESSlb\nhxDwRHYum0I6hLSpI4ikQYSvqPoBTcCNZvb7JOfyRHYuszjfI3cHzgK+QiiJnyNMErAmz6EbgAvM\nbLqkXsAUSY+b2Zy4wXoiO5dZnFbrW4Ddgf8FrovWb813kJktNLPp0foKwsybA+OHCv37h9ePPkpy\ntHO1K04XzS+b2W5p75+OnjxRMEnbE2bffCnOcSlDhoTXDz5IcrRztStOIk+VNMLMJgNIOgB4tdCD\no2r13cB5UcncQr4ZQgAGDw6v8+bFiNq5dqpUM4TMBnYBUuXhYGAu4R7YzGzPHMd2AR4C/mlm4zJ8\nnneGEAhdM3v0CL281q4NXTad6yhyzRASJ5GH5PrczLKWk5JuAT42swuyfF5QIgNsu224R543r7mE\ndq4jKMrTGIEvmdm89AVoSFvPdvGDCRPbfz162sRUSYfF+yc08/tk5zYVp3L6X5K+D/wn0Au4iTAR\n34RcB0VPlOicOMJWBg+GyZM9kZ1LF6dEHgm8A0wnjEO+w8yOKUlUOaSq088+W+4rO1e94iRyH2B/\nQjKvBYZEM2OW1ejRoZHrhhvgwQfLfXXnqlOcRJ4MPGpmhwHDgW2BWA9iK4b994df/jKsX3YZNDWV\nOwLnqk+cVuvBhOr1Dmb28+j99mbW5kpunFZrCF89DR0KCxbAhAlw6qltjcC56lesVutLgRHACdH7\n5eQeGVUym20GV10V1s8/HxobKxGFc9UjTiIfYGZnA2sAzGwp0K0kURXg1FPhm98M81yPGgWLF1cq\nEucqL04ir48eVG7wxfOSK3aHKsEdd8DOO8PMmeH13HN9Yj7XMcW5Rz4JOI7w+JcJhFk1Lzezf7Q5\niJj3yOkWLoRjjoEXoma3ffeF730PdtkF9tsvfF1V/rZ154qvKF00oxPtSpgSV8BTZja7SAEmTmQA\nM3juOTj2WFi0qOVnW28Nw4bBwIHQt29437dvGBK5006w/fbQtWvb4neuHIqWyAkv/mfgu8CibAMr\n2prIKcuWwd13w5w58Npr8Mor+Sch6Nw5lNpDh8Luu4f1vn1hjz3C0inOzYdzJVTpRP4KsAK4pdSJ\n3JpZGFwxc2ZoDFuyBD7+OLwuWADvvAPz54f9MunfH77xDRg0KJTk/frBdtuFZeDA0HruXLlUNJGj\nAIYAD5Y7kQuxdi289x689VYoxT/6KCwvvRSSPJdttgnTD9XXwxZbhNf6eujTJyxbbtm83rt382uv\nXs3DMZ0rlCdyAmYwa1YYoLFkSVgWLgzJPX8+fPghbNzYtmv07Al1dS2XXNvivPbo4bcFtcYTuQQ2\nbAgNa0uXwuefNy/LloVtS5eG77iXLoXPPgtLan3lSli9uvQx9uiR/A9Br16hRpFa+vQJtQ3/41A5\nuRK5aubYKGSqn2rSpUu4Tx6YaBrB0Ed81aqQ1CtXtlxPX1Lb87223rZ6dfNSLJ06tbxlaJ3o2d73\n6QPdKtZ1qP0qyVQ/bRFNuvegme2R5fN2VyJXu6amkMS5kj3XH4Tly5trFamaxfLlyeOpq2uZ2P36\nwW67hW8G9twTdtjBS/t8Kt1qfQfhucpbAYuAMWZ2c6t9PJHbgfXrW94ypCd5aj3b+3ztCXV1cNBB\ncOihoVPP8OGhAdE1q/g9cj6eyLXNLJTm6Une2BgaE1PLhx9uelz//nD00XDggc2t/b17h3nbttoq\n3N50pJZ/T2RX9ZYsCRNFzJgBzz8Ps2fnv7/v2xdGjAjLqFGw1161ndieyK7daWoKJfU998Dbb4cS\nfcWK5tJ86dJNJ5UYOBBOOSUMnhkwoDJxl5Insqs5ZvDuu+F7/mefhfvuC6U6hBby3/wGzjkndMGt\nFZ7IruY1NYXeeNdeC/feG7Ztuy1cfz0cdVRlYysWT2TXodx1V5jP7d13w/vRo2GffUIf+UGDmpfu\n3SsbZ1yeyK7DaWqCcePgkktg3brM+3TvHlq/L7wQzjwz9ISrZp7IrsOaMwceeyyMdluwIPSTX7Ag\nNJht2NC8X9eucMYZcMUVYTBMNfJEdq4Vs/D11v33h4kcZ0dTZEhhyuXbbw8TT1QTT2Tn8pg1Cy69\nFB5+OLzv3BlOPhluuql6nvpZrOlwnatZe+wBDz0UZpQ54ojQpXTCBLjyyuwTT1STsiSypMMkzZH0\npqSLy3HNcil0dEq18bgz23JLmDgxJDWEavfRR7d97vRSx13yRJbUCbgO+DawO3BCNIlfTfCEKK9y\nxX344XDbbWEM9sSJYZLGiy4KDWWffBJGia1bV3hpXeq4y1H73x94K/UMZUl/A44C5pTh2s4ldtJJ\n8LWvha+n7rkHrrkmLK117hxavbt2DYnfvz/suGMYrpna/vLLYfRY167hO+xTTinunG/lSOSBQPrs\nVwsIye1c1dtuO/j730M30PPPD/O5rVkTlvXrw1dYGzeGZc2a0Ce8sRGmTNn0XC++2Lx+9dUwcmTx\nGtLKMR75+8C3zez06P3JwP5mdm7aPu2gOcG5yqvkVD+NwOC094OibV/IFpxzrjDlaLV+BRgqaYik\nbsDxwMQyXNe5DqPkJbKZbZR0DvA44Q/Hn4v1qBnnXFAVPbucc21T8Z5d7amziKT3Jc2QNE3Sy9G2\nPpIelzRX0mOSqmLKOEl/lrRI0sy0bVljlXSppLckzZb0rcpEnTXuMZIWSJoaLYelfVYtcQ+S9H+S\nXpc0S9K50fby/MzNrGIL4Q/J28AQoCswHdi1kjHlifddoE+rbb8GLorWLwZ+Vek4o1i+AuwNzMwX\nK7AbMI1wq7V99H+iKop7DHBBhn3/rYri7g/sHa33AuYCu5brZ17pEvmLziJmth5IdRapVmLTWsxR\nhOdFE70eXdaIsjCz54GlrTZni/VI4G9mtsHM3gfeokLf9WeJG8LPvrWjqJ64F5rZ9Gh9BTCb8A1N\nWX7mlU7kTJ1FEj67oSwMeELSK5J+Em3rZ2aLIPxnAlU6mhWAbbLE2vr/oZHq+384R9J0STelVU+r\nMu7ogQx7A5PJ/vtR1NgrncjtzcFmNgwYBZwt6auE5E7XnloP20us1wM7mtnewELgtxWOJytJvYC7\ngfOikrksvx+VTuS8nUWqiZl9FL0uAe4nVIUWSeoHIKk/sLhyEeaVLdZGYLu0/arq/8HMllh0Ywnc\nSHMVtKriltSFkMS3mtkD0eay/MwrncjtprOIpJ7RX1sk1QHfAmYR4v1htNsPgAcynqAyRMt7y2yx\nTgSOl9RN0g7AUODlcgWZQYu4owRI+R7wWrRebXH/BXjDzMalbSvPz7wSLXytWvsOI7TwvQVcUul4\ncsS5A6FVfRohgS+Jtm8JPBn9Gx4Helc61iiuO4APgbXAB8CPgD7ZYgUuJbSczga+VWVx3wLMjH7+\n9xPuO6st7oOBjWm/I1Oj3+2svx/FjN07hDhXAypdtXbOFYEnsnM1wBPZuRrgiexcDfBEdq4GeCK7\nqiBppKQDKx1He+WJ7KpFA3BQpYNorzyRq0jUw+0NSeMlvSbpUUkZJ02VtJOkJ6KBBK9GvYOQdE00\nHnaGpGOjbSMlTZJ0v6S3Jf1K0smSXo72Sx17s6Q/RoNC5kg6PNq+maS/SJopaYqkhmj7DyTdI+mf\n0XjbX6fF901JL0ax3SWpZ7T9PUljo/PMkLSzpCHAmcBPo/HGB0s6Jvp3TJM0qXQ/9RpR6R5IvrTo\nHTQEWAfsEb2/Czgxy76TgSOj9W5Ad0L3xceibdsA84B+wEjg02hbN0Kf3rHRfucC/xOt3ww8Eq0P\nJYzO6QZcANwUbd8lOm83QpfDtwnjbzcD3ieM4NkKeAboER1zEXB5tP4ecFa0/h/A+Gi9xZhjQk+u\nAdF6faX/b6p98RK5+rxnZrOi9SmEQectRH2+tzWziQBmts7M1hAG5d8ZbVsMTAKGR4e9YmaLzWwd\nIfkei7bPanWNv0fHvw28Qxi8/xXgtmj7XELC7hzt/5SZrTCztcDrhD9GIwgD51+QNA04lZaDY+7L\n9e+LPA9MiIaLVslj1KqX/4Cqz9q09Y2Ekjap9AET6edtSnvfRMvfg/Q+u4o+j3PeLtHnj5vZSVni\nSh2zkSy/g2Z2lqThwHeBKZKGmVmmCQccfo9cjfLO8W1hnOsCSUcBRCNoegDPAcdJ6iSpL/BV4o+o\nGa1gJ8JAkbnReU+KrrUzYfjd3BznmAwcHJ0jNXLsS3muuxyoT72RtKOZvWJmYwhD/7bLeqTzRK5C\nhY5iOQU4V9IM4AXCiKD7CFXlGYQRNz+LqthxrvEBIfkfBs6IquLXA50VJsS7E/iBhamZMp7XzD4m\nDN27M4rvRcK9da5rPwj8e6qxC7gmalybCbxgZjOzHOfw6XBdGkk3Aw+a2b2VjsXF4yWyS+d/1dsp\nL5GrnKTrCIPWjXD/bMA4M5uQ80DXoXgiO1cDvGrtXA3wRHauBngiO1cDPJGdqwGeyM7VAE9k52rA\n/wPptm8FJMnVsAAAAABJRU5ErkJggg==\n",
      "text/html": [
       "\n",
       "\n",
       "<style>\n",
       "\n",
       "</style>\n",
       "\n",
       "<div id=\"fig_el26961405681004334888153751092\"></div>\n",
       "<script>\n",
       "function mpld3_load_lib(url, callback){\n",
       "  var s = document.createElement('script');\n",
       "  s.src = url;\n",
       "  s.async = true;\n",
       "  s.onreadystatechange = s.onload = callback;\n",
       "  s.onerror = function(){console.warn(\"failed to load library \" + url);};\n",
       "  document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "}\n",
       "\n",
       "if(typeof(mpld3) !== \"undefined\" && mpld3._mpld3IsLoaded){\n",
       "   // already loaded: just create the figure\n",
       "   !function(mpld3){\n",
       "       \n",
       "       mpld3.draw_figure(\"fig_el26961405681004334888153751092\", {\"axes\": [{\"xlim\": [0.0, 218.0], \"yscale\": \"linear\", \"axesbg\": \"#FFFFFF\", \"texts\": [{\"v_baseline\": \"hanging\", \"h_anchor\": \"middle\", \"color\": \"#000000\", \"text\": \"n_components\", \"coordinates\": \"axes\", \"zorder\": 3, \"alpha\": 1, \"fontsize\": 10.0, \"position\": [0.5, -0.1190476190476191], \"rotation\": -0.0, \"id\": \"el2696140568100384720\"}, {\"v_baseline\": \"auto\", \"h_anchor\": \"middle\", \"color\": \"#000000\", \"text\": \"explained_variance_\", \"coordinates\": \"axes\", \"zorder\": 3, \"alpha\": 1, \"fontsize\": 10.0, \"position\": [-0.076450892857142849, 0.5], \"rotation\": -90.0, \"id\": \"el2696140568100122320\"}], \"zoomable\": true, \"images\": [], \"xdomain\": [0.0, 218.0], \"ylim\": [9.1426740091010302e-33, 8.149185057152966], \"paths\": [], \"sharey\": [], \"sharex\": [], \"axesbgalpha\": null, \"axes\": [{\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"bottom\", \"nticks\": 6, \"tickvalues\": null}, {\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"left\", \"nticks\": 10, \"tickvalues\": null}], \"lines\": [{\"color\": \"#0000FF\", \"yindex\": 1, \"coordinates\": \"data\", \"dasharray\": \"10,0\", \"zorder\": 2, \"alpha\": 1, \"xindex\": 0, \"linewidth\": 2.0, \"data\": \"data01\", \"id\": \"el2696140568100989968\"}], \"markers\": [], \"id\": \"el2696140568100381584\", \"ydomain\": [9.1426740091010302e-33, 8.149185057152966], \"collections\": [], \"xscale\": \"linear\", \"bbox\": [0.20000000000000001, 0.20000000000000001, 0.69999999999999996, 0.69999999999999996]}], \"height\": 240.0, \"width\": 320.0, \"plugins\": [{\"type\": \"reset\"}, {\"enabled\": false, \"button\": true, \"type\": \"zoom\"}, {\"enabled\": false, \"button\": true, \"type\": \"boxzoom\"}], \"data\": {\"data01\": [[0.0, 8.149185057152966], [1.0, 5.894113669493637], [2.0, 4.3679534705122425], [3.0, 3.914207757353311], [4.0, 3.8764300151201656], [5.0, 3.8439194918007646], [6.0, 3.8072253085930314], [7.0, 3.789555426491603], [8.0, 3.767983145890411], [9.0, 3.7539886324730705], [10.0, 3.7403743905598277], [11.0, 2.795263751939266], [12.0, 2.1244867823454805], [13.0, 1.701988738608703], [14.0, 1.6409844557648756], [15.0, 1.5419368585742637], [16.0, 1.4648909851419936], [17.0, 1.408477709618382], [18.0, 1.3668702967799287], [19.0, 1.3543513840928547], [20.0, 1.307744409126777], [21.0, 1.2884119471571278], [22.0, 1.2845358415874302], [23.0, 1.2738326037288876], [24.0, 1.26815096060797], [25.0, 1.2581641213827455], [26.0, 1.2439978183243536], [27.0, 1.2410010736957646], [28.0, 1.2334655550483342], [29.0, 1.2312738059684305], [30.0, 1.2291250833906915], [31.0, 1.224157919265399], [32.0, 1.217762892002753], [33.0, 1.2145525174379224], [34.0, 1.2043618296988794], [35.0, 1.1966109045560154], [36.0, 1.1896213935916065], [37.0, 1.179705910726117], [38.0, 1.1769677723199907], [39.0, 1.1720702728189947], [40.0, 1.162379088965484], [41.0, 1.146252903913197], [42.0, 1.132366614490787], [43.0, 1.1299809552183604], [44.0, 1.1292243715044978], [45.0, 1.1204033491124965], [46.0, 1.114073067255821], [47.0, 1.108515380236819], [48.0, 1.1041373007973807], [49.0, 1.0975938249926336], [50.0, 1.092288955846302], [51.0, 1.0907504306109563], [52.0, 1.084908754817757], [53.0, 1.0840706447761561], [54.0, 1.0832255200332639], [55.0, 1.078681059031188], [56.0, 1.07631368506946], [57.0, 1.0713617555604429], [58.0, 1.0679409574139471], [59.0, 1.0663719056287817], [60.0, 1.0618573093143244], [61.0, 1.057323083394364], [62.0, 1.0557394847015162], [63.0, 1.0533765314701795], [64.0, 1.0491888538310508], [65.0, 1.0475757835110528], [66.0, 1.0444795838599643], [67.0, 1.0424434417885815], [68.0, 1.039404443333045], [69.0, 1.037123498394614], [70.0, 1.0337821669516547], [71.0, 1.0328363682887758], [72.0, 1.0308338068998082], [73.0, 1.0298385655832758], [74.0, 1.0283773033696257], [75.0, 1.027802742501999], [76.0, 1.02391771075658], [77.0, 1.0220381223752348], [78.0, 1.0186526403183196], [79.0, 1.0182202669823972], [80.0, 1.01566859223344], [81.0, 1.01355517466424], [82.0, 1.012717626875316], [83.0, 1.0116080614898264], [84.0, 1.009624627572526], [85.0, 1.0080715613076057], [86.0, 1.0038838776787335], [87.0, 1.002550422530945], [88.0, 1.0011155718983245], [89.0, 1.0004901311647723], [90.0, 0.9975713180543684], [91.0, 0.9969912067707613], [92.0, 0.9917229568515653], [93.0, 0.9899386395740184], [94.0, 0.9894505812972252], [95.0, 0.9883458844764521], [96.0, 0.9847979902702476], [97.0, 0.9828553710648109], [98.0, 0.98219125373114], [99.0, 0.9808463272577089], [100.0, 0.9771293323517563], [101.0, 0.9768179714240128], [102.0, 0.9725052673634231], [103.0, 0.9691840091150631], [104.0, 0.966042414082371], [105.0, 0.9649211554738639], [106.0, 0.9626427981374229], [107.0, 0.9598893349099412], [108.0, 0.9588165003295667], [109.0, 0.956208607086908], [110.0, 0.9528557208329452], [111.0, 0.9518124358189353], [112.0, 0.9485822541685258], [113.0, 0.944590961383397], [114.0, 0.9406353428390931], [115.0, 0.9370149661057773], [116.0, 0.9328491571581834], [117.0, 0.930103369336051], [118.0, 0.9231107284531164], [119.0, 0.9187935757768372], [120.0, 0.9141798254192028], [121.0, 0.9123736277701688], [122.0, 0.9119443330510347], [123.0, 0.9085722862222256], [124.0, 0.9070694534234754], [125.0, 0.9054875274143135], [126.0, 0.9030800276194679], [127.0, 0.8981033014832931], [128.0, 0.8943205387823496], [129.0, 0.8928731932935295], [130.0, 0.8886258123571492], [131.0, 0.8847597994010928], [132.0, 0.8792620266794916], [133.0, 0.873885163063518], [134.0, 0.8699147532683519], [135.0, 0.8663296055117897], [136.0, 0.8578193401741194], [137.0, 0.8487112868184915], [138.0, 0.8435716482001728], [139.0, 0.8329862020396979], [140.0, 0.8222621073296892], [141.0, 0.8042526290545423], [142.0, 0.7983228229125783], [143.0, 0.784463634892824], [144.0, 0.7783698521264554], [145.0, 0.7694484981638684], [146.0, 0.7486829215487233], [147.0, 0.7407345727141238], [148.0, 0.7400989315122968], [149.0, 0.7368279137244057], [150.0, 0.733726955236806], [151.0, 0.7269819779643645], [152.0, 0.7233508283973208], [153.0, 0.722040085923073], [154.0, 0.711730914967166], [155.0, 0.7051424082429957], [156.0, 0.6893882218149942], [157.0, 0.6711625856886201], [158.0, 0.6594335689185311], [159.0, 0.635648616242941], [160.0, 0.6135574192124026], [161.0, 0.5727250108997626], [162.0, 0.5568431863456136], [163.0, 0.5217742627599448], [164.0, 0.48534614900193], [165.0, 0.4794110274891809], [166.0, 0.4756506470889936], [167.0, 0.47224748038863573], [168.0, 0.4600351833150165], [169.0, 0.45655851334874986], [170.0, 0.4522480023458522], [171.0, 0.45135536517056807], [172.0, 0.4475243066463468], [173.0, 0.4071924205788809], [174.0, 0.4003630541264404], [175.0, 0.3701892072246591], [176.0, 0.34603027965830796], [177.0, 0.3394443305560339], [178.0, 0.29361349426579475], [179.0, 0.2822746375999936], [180.0, 0.21200327790381562], [181.0, 0.1759809343127573], [182.0, 0.14430724444516976], [183.0, 0.12386939420733072], [184.0, 0.08233732882629448], [185.0, 0.07874397698335996], [186.0, 0.07765284829026635], [187.0, 0.07723522468308773], [188.0, 0.07688009445523704], [189.0, 0.07645630397636892], [190.0, 0.07627932048736379], [191.0, 0.0755187295910804], [192.0, 0.07507586323700682], [193.0, 0.07155842142714873], [194.0, 0.06820614574465898], [195.0, 0.06204105797322983], [196.0, 0.05387350690939758], [197.0, 0.05289279474603526], [198.0, 0.051634500390671884], [199.0, 0.050967611825650704], [200.0, 0.05030279027500172], [201.0, 0.049362245511248785], [202.0, 0.04882259412117724], [203.0, 0.04736553963167591], [204.0, 0.016451200096332837], [205.0, 1.6403234603144918e-30], [206.0, 4.710607728861684e-31], [207.0, 3.3140199817776736e-31], [208.0, 1.9878473996854754e-31], [209.0, 1.6162710160562987e-31], [210.0, 1.4804591904391972e-31], [211.0, 9.46062641971616e-32], [212.0, 7.810065887840307e-32], [213.0, 7.497412280933759e-32], [214.0, 3.1818983680684223e-32], [215.0, 3.1818983680684223e-32], [216.0, 1.288270983491311e-32], [217.0, 1.2275273127911376e-32], [218.0, 9.14267400910103e-33]]}, \"id\": \"el2696140568100433488\"});\n",
       "   }(mpld3);\n",
       "}else if(typeof define === \"function\" && define.amd){\n",
       "   // require.js is available: use it to load d3/mpld3\n",
       "   require.config({paths: {d3: \"https://mpld3.github.io/js/d3.v3.min\"}});\n",
       "   require([\"d3\"], function(d3){\n",
       "      window.d3 = d3;\n",
       "      mpld3_load_lib(\"https://mpld3.github.io/js/mpld3.v0.2.js\", function(){\n",
       "         \n",
       "         mpld3.draw_figure(\"fig_el26961405681004334888153751092\", {\"axes\": [{\"xlim\": [0.0, 218.0], \"yscale\": \"linear\", \"axesbg\": \"#FFFFFF\", \"texts\": [{\"v_baseline\": \"hanging\", \"h_anchor\": \"middle\", \"color\": \"#000000\", \"text\": \"n_components\", \"coordinates\": \"axes\", \"zorder\": 3, \"alpha\": 1, \"fontsize\": 10.0, \"position\": [0.5, -0.1190476190476191], \"rotation\": -0.0, \"id\": \"el2696140568100384720\"}, {\"v_baseline\": \"auto\", \"h_anchor\": \"middle\", \"color\": \"#000000\", \"text\": \"explained_variance_\", \"coordinates\": \"axes\", \"zorder\": 3, \"alpha\": 1, \"fontsize\": 10.0, \"position\": [-0.076450892857142849, 0.5], \"rotation\": -90.0, \"id\": \"el2696140568100122320\"}], \"zoomable\": true, \"images\": [], \"xdomain\": [0.0, 218.0], \"ylim\": [9.1426740091010302e-33, 8.149185057152966], \"paths\": [], \"sharey\": [], \"sharex\": [], \"axesbgalpha\": null, \"axes\": [{\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"bottom\", \"nticks\": 6, \"tickvalues\": null}, {\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"left\", \"nticks\": 10, \"tickvalues\": null}], \"lines\": [{\"color\": \"#0000FF\", \"yindex\": 1, \"coordinates\": \"data\", \"dasharray\": \"10,0\", \"zorder\": 2, \"alpha\": 1, \"xindex\": 0, \"linewidth\": 2.0, \"data\": \"data01\", \"id\": \"el2696140568100989968\"}], \"markers\": [], \"id\": \"el2696140568100381584\", \"ydomain\": [9.1426740091010302e-33, 8.149185057152966], \"collections\": [], \"xscale\": \"linear\", \"bbox\": [0.20000000000000001, 0.20000000000000001, 0.69999999999999996, 0.69999999999999996]}], \"height\": 240.0, \"width\": 320.0, \"plugins\": [{\"type\": \"reset\"}, {\"enabled\": false, \"button\": true, \"type\": \"zoom\"}, {\"enabled\": false, \"button\": true, \"type\": \"boxzoom\"}], \"data\": {\"data01\": [[0.0, 8.149185057152966], [1.0, 5.894113669493637], [2.0, 4.3679534705122425], [3.0, 3.914207757353311], [4.0, 3.8764300151201656], [5.0, 3.8439194918007646], [6.0, 3.8072253085930314], [7.0, 3.789555426491603], [8.0, 3.767983145890411], [9.0, 3.7539886324730705], [10.0, 3.7403743905598277], [11.0, 2.795263751939266], [12.0, 2.1244867823454805], [13.0, 1.701988738608703], [14.0, 1.6409844557648756], [15.0, 1.5419368585742637], [16.0, 1.4648909851419936], [17.0, 1.408477709618382], [18.0, 1.3668702967799287], [19.0, 1.3543513840928547], [20.0, 1.307744409126777], [21.0, 1.2884119471571278], [22.0, 1.2845358415874302], [23.0, 1.2738326037288876], [24.0, 1.26815096060797], [25.0, 1.2581641213827455], [26.0, 1.2439978183243536], [27.0, 1.2410010736957646], [28.0, 1.2334655550483342], [29.0, 1.2312738059684305], [30.0, 1.2291250833906915], [31.0, 1.224157919265399], [32.0, 1.217762892002753], [33.0, 1.2145525174379224], [34.0, 1.2043618296988794], [35.0, 1.1966109045560154], [36.0, 1.1896213935916065], [37.0, 1.179705910726117], [38.0, 1.1769677723199907], [39.0, 1.1720702728189947], [40.0, 1.162379088965484], [41.0, 1.146252903913197], [42.0, 1.132366614490787], [43.0, 1.1299809552183604], [44.0, 1.1292243715044978], [45.0, 1.1204033491124965], [46.0, 1.114073067255821], [47.0, 1.108515380236819], [48.0, 1.1041373007973807], [49.0, 1.0975938249926336], [50.0, 1.092288955846302], [51.0, 1.0907504306109563], [52.0, 1.084908754817757], [53.0, 1.0840706447761561], [54.0, 1.0832255200332639], [55.0, 1.078681059031188], [56.0, 1.07631368506946], [57.0, 1.0713617555604429], [58.0, 1.0679409574139471], [59.0, 1.0663719056287817], [60.0, 1.0618573093143244], [61.0, 1.057323083394364], [62.0, 1.0557394847015162], [63.0, 1.0533765314701795], [64.0, 1.0491888538310508], [65.0, 1.0475757835110528], [66.0, 1.0444795838599643], [67.0, 1.0424434417885815], [68.0, 1.039404443333045], [69.0, 1.037123498394614], [70.0, 1.0337821669516547], [71.0, 1.0328363682887758], [72.0, 1.0308338068998082], [73.0, 1.0298385655832758], [74.0, 1.0283773033696257], [75.0, 1.027802742501999], [76.0, 1.02391771075658], [77.0, 1.0220381223752348], [78.0, 1.0186526403183196], [79.0, 1.0182202669823972], [80.0, 1.01566859223344], [81.0, 1.01355517466424], [82.0, 1.012717626875316], [83.0, 1.0116080614898264], [84.0, 1.009624627572526], [85.0, 1.0080715613076057], [86.0, 1.0038838776787335], [87.0, 1.002550422530945], [88.0, 1.0011155718983245], [89.0, 1.0004901311647723], [90.0, 0.9975713180543684], [91.0, 0.9969912067707613], [92.0, 0.9917229568515653], [93.0, 0.9899386395740184], [94.0, 0.9894505812972252], [95.0, 0.9883458844764521], [96.0, 0.9847979902702476], [97.0, 0.9828553710648109], [98.0, 0.98219125373114], [99.0, 0.9808463272577089], [100.0, 0.9771293323517563], [101.0, 0.9768179714240128], [102.0, 0.9725052673634231], [103.0, 0.9691840091150631], [104.0, 0.966042414082371], [105.0, 0.9649211554738639], [106.0, 0.9626427981374229], [107.0, 0.9598893349099412], [108.0, 0.9588165003295667], [109.0, 0.956208607086908], [110.0, 0.9528557208329452], [111.0, 0.9518124358189353], [112.0, 0.9485822541685258], [113.0, 0.944590961383397], [114.0, 0.9406353428390931], [115.0, 0.9370149661057773], [116.0, 0.9328491571581834], [117.0, 0.930103369336051], [118.0, 0.9231107284531164], [119.0, 0.9187935757768372], [120.0, 0.9141798254192028], [121.0, 0.9123736277701688], [122.0, 0.9119443330510347], [123.0, 0.9085722862222256], [124.0, 0.9070694534234754], [125.0, 0.9054875274143135], [126.0, 0.9030800276194679], [127.0, 0.8981033014832931], [128.0, 0.8943205387823496], [129.0, 0.8928731932935295], [130.0, 0.8886258123571492], [131.0, 0.8847597994010928], [132.0, 0.8792620266794916], [133.0, 0.873885163063518], [134.0, 0.8699147532683519], [135.0, 0.8663296055117897], [136.0, 0.8578193401741194], [137.0, 0.8487112868184915], [138.0, 0.8435716482001728], [139.0, 0.8329862020396979], [140.0, 0.8222621073296892], [141.0, 0.8042526290545423], [142.0, 0.7983228229125783], [143.0, 0.784463634892824], [144.0, 0.7783698521264554], [145.0, 0.7694484981638684], [146.0, 0.7486829215487233], [147.0, 0.7407345727141238], [148.0, 0.7400989315122968], [149.0, 0.7368279137244057], [150.0, 0.733726955236806], [151.0, 0.7269819779643645], [152.0, 0.7233508283973208], [153.0, 0.722040085923073], [154.0, 0.711730914967166], [155.0, 0.7051424082429957], [156.0, 0.6893882218149942], [157.0, 0.6711625856886201], [158.0, 0.6594335689185311], [159.0, 0.635648616242941], [160.0, 0.6135574192124026], [161.0, 0.5727250108997626], [162.0, 0.5568431863456136], [163.0, 0.5217742627599448], [164.0, 0.48534614900193], [165.0, 0.4794110274891809], [166.0, 0.4756506470889936], [167.0, 0.47224748038863573], [168.0, 0.4600351833150165], [169.0, 0.45655851334874986], [170.0, 0.4522480023458522], [171.0, 0.45135536517056807], [172.0, 0.4475243066463468], [173.0, 0.4071924205788809], [174.0, 0.4003630541264404], [175.0, 0.3701892072246591], [176.0, 0.34603027965830796], [177.0, 0.3394443305560339], [178.0, 0.29361349426579475], [179.0, 0.2822746375999936], [180.0, 0.21200327790381562], [181.0, 0.1759809343127573], [182.0, 0.14430724444516976], [183.0, 0.12386939420733072], [184.0, 0.08233732882629448], [185.0, 0.07874397698335996], [186.0, 0.07765284829026635], [187.0, 0.07723522468308773], [188.0, 0.07688009445523704], [189.0, 0.07645630397636892], [190.0, 0.07627932048736379], [191.0, 0.0755187295910804], [192.0, 0.07507586323700682], [193.0, 0.07155842142714873], [194.0, 0.06820614574465898], [195.0, 0.06204105797322983], [196.0, 0.05387350690939758], [197.0, 0.05289279474603526], [198.0, 0.051634500390671884], [199.0, 0.050967611825650704], [200.0, 0.05030279027500172], [201.0, 0.049362245511248785], [202.0, 0.04882259412117724], [203.0, 0.04736553963167591], [204.0, 0.016451200096332837], [205.0, 1.6403234603144918e-30], [206.0, 4.710607728861684e-31], [207.0, 3.3140199817776736e-31], [208.0, 1.9878473996854754e-31], [209.0, 1.6162710160562987e-31], [210.0, 1.4804591904391972e-31], [211.0, 9.46062641971616e-32], [212.0, 7.810065887840307e-32], [213.0, 7.497412280933759e-32], [214.0, 3.1818983680684223e-32], [215.0, 3.1818983680684223e-32], [216.0, 1.288270983491311e-32], [217.0, 1.2275273127911376e-32], [218.0, 9.14267400910103e-33]]}, \"id\": \"el2696140568100433488\"});\n",
       "      });\n",
       "    });\n",
       "}else{\n",
       "    // require.js not available: dynamically load d3 & mpld3\n",
       "    mpld3_load_lib(\"https://mpld3.github.io/js/d3.v3.min.js\", function(){\n",
       "         mpld3_load_lib(\"https://mpld3.github.io/js/mpld3.v0.2.js\", function(){\n",
       "                 \n",
       "                 mpld3.draw_figure(\"fig_el26961405681004334888153751092\", {\"axes\": [{\"xlim\": [0.0, 218.0], \"yscale\": \"linear\", \"axesbg\": \"#FFFFFF\", \"texts\": [{\"v_baseline\": \"hanging\", \"h_anchor\": \"middle\", \"color\": \"#000000\", \"text\": \"n_components\", \"coordinates\": \"axes\", \"zorder\": 3, \"alpha\": 1, \"fontsize\": 10.0, \"position\": [0.5, -0.1190476190476191], \"rotation\": -0.0, \"id\": \"el2696140568100384720\"}, {\"v_baseline\": \"auto\", \"h_anchor\": \"middle\", \"color\": \"#000000\", \"text\": \"explained_variance_\", \"coordinates\": \"axes\", \"zorder\": 3, \"alpha\": 1, \"fontsize\": 10.0, \"position\": [-0.076450892857142849, 0.5], \"rotation\": -90.0, \"id\": \"el2696140568100122320\"}], \"zoomable\": true, \"images\": [], \"xdomain\": [0.0, 218.0], \"ylim\": [9.1426740091010302e-33, 8.149185057152966], \"paths\": [], \"sharey\": [], \"sharex\": [], \"axesbgalpha\": null, \"axes\": [{\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"bottom\", \"nticks\": 6, \"tickvalues\": null}, {\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"left\", \"nticks\": 10, \"tickvalues\": null}], \"lines\": [{\"color\": \"#0000FF\", \"yindex\": 1, \"coordinates\": \"data\", \"dasharray\": \"10,0\", \"zorder\": 2, \"alpha\": 1, \"xindex\": 0, \"linewidth\": 2.0, \"data\": \"data01\", \"id\": \"el2696140568100989968\"}], \"markers\": [], \"id\": \"el2696140568100381584\", \"ydomain\": [9.1426740091010302e-33, 8.149185057152966], \"collections\": [], \"xscale\": \"linear\", \"bbox\": [0.20000000000000001, 0.20000000000000001, 0.69999999999999996, 0.69999999999999996]}], \"height\": 240.0, \"width\": 320.0, \"plugins\": [{\"type\": \"reset\"}, {\"enabled\": false, \"button\": true, \"type\": \"zoom\"}, {\"enabled\": false, \"button\": true, \"type\": \"boxzoom\"}], \"data\": {\"data01\": [[0.0, 8.149185057152966], [1.0, 5.894113669493637], [2.0, 4.3679534705122425], [3.0, 3.914207757353311], [4.0, 3.8764300151201656], [5.0, 3.8439194918007646], [6.0, 3.8072253085930314], [7.0, 3.789555426491603], [8.0, 3.767983145890411], [9.0, 3.7539886324730705], [10.0, 3.7403743905598277], [11.0, 2.795263751939266], [12.0, 2.1244867823454805], [13.0, 1.701988738608703], [14.0, 1.6409844557648756], [15.0, 1.5419368585742637], [16.0, 1.4648909851419936], [17.0, 1.408477709618382], [18.0, 1.3668702967799287], [19.0, 1.3543513840928547], [20.0, 1.307744409126777], [21.0, 1.2884119471571278], [22.0, 1.2845358415874302], [23.0, 1.2738326037288876], [24.0, 1.26815096060797], [25.0, 1.2581641213827455], [26.0, 1.2439978183243536], [27.0, 1.2410010736957646], [28.0, 1.2334655550483342], [29.0, 1.2312738059684305], [30.0, 1.2291250833906915], [31.0, 1.224157919265399], [32.0, 1.217762892002753], [33.0, 1.2145525174379224], [34.0, 1.2043618296988794], [35.0, 1.1966109045560154], [36.0, 1.1896213935916065], [37.0, 1.179705910726117], [38.0, 1.1769677723199907], [39.0, 1.1720702728189947], [40.0, 1.162379088965484], [41.0, 1.146252903913197], [42.0, 1.132366614490787], [43.0, 1.1299809552183604], [44.0, 1.1292243715044978], [45.0, 1.1204033491124965], [46.0, 1.114073067255821], [47.0, 1.108515380236819], [48.0, 1.1041373007973807], [49.0, 1.0975938249926336], [50.0, 1.092288955846302], [51.0, 1.0907504306109563], [52.0, 1.084908754817757], [53.0, 1.0840706447761561], [54.0, 1.0832255200332639], [55.0, 1.078681059031188], [56.0, 1.07631368506946], [57.0, 1.0713617555604429], [58.0, 1.0679409574139471], [59.0, 1.0663719056287817], [60.0, 1.0618573093143244], [61.0, 1.057323083394364], [62.0, 1.0557394847015162], [63.0, 1.0533765314701795], [64.0, 1.0491888538310508], [65.0, 1.0475757835110528], [66.0, 1.0444795838599643], [67.0, 1.0424434417885815], [68.0, 1.039404443333045], [69.0, 1.037123498394614], [70.0, 1.0337821669516547], [71.0, 1.0328363682887758], [72.0, 1.0308338068998082], [73.0, 1.0298385655832758], [74.0, 1.0283773033696257], [75.0, 1.027802742501999], [76.0, 1.02391771075658], [77.0, 1.0220381223752348], [78.0, 1.0186526403183196], [79.0, 1.0182202669823972], [80.0, 1.01566859223344], [81.0, 1.01355517466424], [82.0, 1.012717626875316], [83.0, 1.0116080614898264], [84.0, 1.009624627572526], [85.0, 1.0080715613076057], [86.0, 1.0038838776787335], [87.0, 1.002550422530945], [88.0, 1.0011155718983245], [89.0, 1.0004901311647723], [90.0, 0.9975713180543684], [91.0, 0.9969912067707613], [92.0, 0.9917229568515653], [93.0, 0.9899386395740184], [94.0, 0.9894505812972252], [95.0, 0.9883458844764521], [96.0, 0.9847979902702476], [97.0, 0.9828553710648109], [98.0, 0.98219125373114], [99.0, 0.9808463272577089], [100.0, 0.9771293323517563], [101.0, 0.9768179714240128], [102.0, 0.9725052673634231], [103.0, 0.9691840091150631], [104.0, 0.966042414082371], [105.0, 0.9649211554738639], [106.0, 0.9626427981374229], [107.0, 0.9598893349099412], [108.0, 0.9588165003295667], [109.0, 0.956208607086908], [110.0, 0.9528557208329452], [111.0, 0.9518124358189353], [112.0, 0.9485822541685258], [113.0, 0.944590961383397], [114.0, 0.9406353428390931], [115.0, 0.9370149661057773], [116.0, 0.9328491571581834], [117.0, 0.930103369336051], [118.0, 0.9231107284531164], [119.0, 0.9187935757768372], [120.0, 0.9141798254192028], [121.0, 0.9123736277701688], [122.0, 0.9119443330510347], [123.0, 0.9085722862222256], [124.0, 0.9070694534234754], [125.0, 0.9054875274143135], [126.0, 0.9030800276194679], [127.0, 0.8981033014832931], [128.0, 0.8943205387823496], [129.0, 0.8928731932935295], [130.0, 0.8886258123571492], [131.0, 0.8847597994010928], [132.0, 0.8792620266794916], [133.0, 0.873885163063518], [134.0, 0.8699147532683519], [135.0, 0.8663296055117897], [136.0, 0.8578193401741194], [137.0, 0.8487112868184915], [138.0, 0.8435716482001728], [139.0, 0.8329862020396979], [140.0, 0.8222621073296892], [141.0, 0.8042526290545423], [142.0, 0.7983228229125783], [143.0, 0.784463634892824], [144.0, 0.7783698521264554], [145.0, 0.7694484981638684], [146.0, 0.7486829215487233], [147.0, 0.7407345727141238], [148.0, 0.7400989315122968], [149.0, 0.7368279137244057], [150.0, 0.733726955236806], [151.0, 0.7269819779643645], [152.0, 0.7233508283973208], [153.0, 0.722040085923073], [154.0, 0.711730914967166], [155.0, 0.7051424082429957], [156.0, 0.6893882218149942], [157.0, 0.6711625856886201], [158.0, 0.6594335689185311], [159.0, 0.635648616242941], [160.0, 0.6135574192124026], [161.0, 0.5727250108997626], [162.0, 0.5568431863456136], [163.0, 0.5217742627599448], [164.0, 0.48534614900193], [165.0, 0.4794110274891809], [166.0, 0.4756506470889936], [167.0, 0.47224748038863573], [168.0, 0.4600351833150165], [169.0, 0.45655851334874986], [170.0, 0.4522480023458522], [171.0, 0.45135536517056807], [172.0, 0.4475243066463468], [173.0, 0.4071924205788809], [174.0, 0.4003630541264404], [175.0, 0.3701892072246591], [176.0, 0.34603027965830796], [177.0, 0.3394443305560339], [178.0, 0.29361349426579475], [179.0, 0.2822746375999936], [180.0, 0.21200327790381562], [181.0, 0.1759809343127573], [182.0, 0.14430724444516976], [183.0, 0.12386939420733072], [184.0, 0.08233732882629448], [185.0, 0.07874397698335996], [186.0, 0.07765284829026635], [187.0, 0.07723522468308773], [188.0, 0.07688009445523704], [189.0, 0.07645630397636892], [190.0, 0.07627932048736379], [191.0, 0.0755187295910804], [192.0, 0.07507586323700682], [193.0, 0.07155842142714873], [194.0, 0.06820614574465898], [195.0, 0.06204105797322983], [196.0, 0.05387350690939758], [197.0, 0.05289279474603526], [198.0, 0.051634500390671884], [199.0, 0.050967611825650704], [200.0, 0.05030279027500172], [201.0, 0.049362245511248785], [202.0, 0.04882259412117724], [203.0, 0.04736553963167591], [204.0, 0.016451200096332837], [205.0, 1.6403234603144918e-30], [206.0, 4.710607728861684e-31], [207.0, 3.3140199817776736e-31], [208.0, 1.9878473996854754e-31], [209.0, 1.6162710160562987e-31], [210.0, 1.4804591904391972e-31], [211.0, 9.46062641971616e-32], [212.0, 7.810065887840307e-32], [213.0, 7.497412280933759e-32], [214.0, 3.1818983680684223e-32], [215.0, 3.1818983680684223e-32], [216.0, 1.288270983491311e-32], [217.0, 1.2275273127911376e-32], [218.0, 9.14267400910103e-33]]}, \"id\": \"el2696140568100433488\"});\n",
       "            })\n",
       "         });\n",
       "}\n",
       "</script>"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fd88fb16e50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(1, figsize=(4, 3))\n",
    "plt.clf()\n",
    "plt.axes([.2, .2, .7, .7])\n",
    "plt.plot(pca.explained_variance_, linewidth=2)\n",
    "plt.axis('tight')\n",
    "plt.xlabel('n_components')\n",
    "plt.ylabel('explained_variance_')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=0.0025, class_weight=None, dual=False,\n",
       "          fit_intercept=True, intercept_scaling=1, max_iter=100,\n",
       "          multi_class='ovr', n_jobs=-1, penalty='l2', random_state=1,\n",
       "          solver='liblinear', tol=0.0001, verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf4=LogisticRegression(C=0.0025,penalty='l2',random_state=1, n_jobs=-1)\n",
    "clf4.fit(Reduced4train,Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "Y_pred4 = clf4.predict_proba(Reduced4test)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(17177,)"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Y_pred4.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test['radiant_win']=Y_pred4"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "test['radiant_win'].to_csv('prediction_result_6.csv')"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "list(Reduced4testCols)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "\n",
       "<style>\n",
       "\n",
       "</style>\n",
       "\n",
       "<div id=\"fig_el26961405679426583208738058525\"></div>\n",
       "<script>\n",
       "function mpld3_load_lib(url, callback){\n",
       "  var s = document.createElement('script');\n",
       "  s.src = url;\n",
       "  s.async = true;\n",
       "  s.onreadystatechange = s.onload = callback;\n",
       "  s.onerror = function(){console.warn(\"failed to load library \" + url);};\n",
       "  document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "}\n",
       "\n",
       "if(typeof(mpld3) !== \"undefined\" && mpld3._mpld3IsLoaded){\n",
       "   // already loaded: just create the figure\n",
       "   !function(mpld3){\n",
       "       \n",
       "       mpld3.draw_figure(\"fig_el26961405679426583208738058525\", {\"axes\": [{\"xlim\": [0.0, 250.0], \"yscale\": \"linear\", \"axesbg\": \"#EEEEEE\", \"texts\": [], \"zoomable\": true, \"images\": [], \"xdomain\": [0.0, 250.0], \"ylim\": [-0.30000000000000004, 0.40000000000000002], \"paths\": [], \"sharey\": [], \"sharex\": [], \"axesbgalpha\": null, \"axes\": [{\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"bottom\", \"nticks\": 6, \"tickvalues\": null}, {\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"left\", \"nticks\": 10, \"tickvalues\": null}], \"lines\": [{\"color\": \"#FF0000\", \"yindex\": 1, \"coordinates\": \"data\", \"dasharray\": \"10,0\", \"zorder\": 2, \"alpha\": 1, \"xindex\": 0, \"linewidth\": 1.0, \"data\": \"data01\", \"id\": \"el2696140567943080336\"}], \"markers\": [{\"edgecolor\": \"#000000\", \"facecolor\": \"#FF0000\", \"edgewidth\": 0.5, \"yindex\": 1, \"coordinates\": \"data\", \"zorder\": 2, \"markerpath\": [[[0.0, 3.0], [0.7956093000000001, 3.0], [1.5587396123545605, 2.683901074764725], [2.121320343559643, 2.121320343559643], [2.683901074764725, 1.5587396123545605], [3.0, 0.7956093000000001], [3.0, 0.0], [3.0, -0.7956093000000001], [2.683901074764725, -1.5587396123545605], [2.121320343559643, -2.121320343559643], [1.5587396123545605, -2.683901074764725], [0.7956093000000001, -3.0], [0.0, -3.0], [-0.7956093000000001, -3.0], [-1.5587396123545605, -2.683901074764725], [-2.121320343559643, -2.121320343559643], [-2.683901074764725, -1.5587396123545605], [-3.0, -0.7956093000000001], [-3.0, 0.0], [-3.0, 0.7956093000000001], [-2.683901074764725, 1.5587396123545605], [-2.121320343559643, 2.121320343559643], [-1.5587396123545605, 2.683901074764725], [-0.7956093000000001, 3.0], [0.0, 3.0]], [\"M\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"Z\"]], \"alpha\": 1, \"xindex\": 0, \"data\": \"data01\", \"id\": \"el2696140567943080336pts\"}], \"id\": \"el2696140567942659728\", \"ydomain\": [-0.30000000000000004, 0.40000000000000002], \"collections\": [], \"xscale\": \"linear\", \"bbox\": [0.125, 0.125, 0.77500000000000002, 0.77500000000000002]}], \"height\": 320.0, \"width\": 480.0, \"plugins\": [{\"type\": \"reset\"}, {\"enabled\": false, \"button\": true, \"type\": \"zoom\"}, {\"enabled\": false, \"button\": true, \"type\": \"boxzoom\"}, {\"voffset\": 10, \"labels\": [\"start_time\", \"r1_level\", \"r1_xp\", \"r1_gold\", \"r1_lh\", \"r1_kills\", \"r1_deaths\", \"r1_items\", \"r2_level\", \"r2_xp\", \"r2_gold\", \"r2_lh\", \"r2_kills\", \"r2_deaths\", \"r2_items\", \"r3_level\", \"r3_xp\", \"r3_gold\", \"r3_lh\", \"r3_kills\", \"r3_deaths\", \"r3_items\", \"r4_level\", \"r4_xp\", \"r4_gold\", \"r4_lh\", \"r4_kills\", \"r4_deaths\", \"r4_items\", \"r5_level\", \"r5_xp\", \"r5_gold\", \"r5_lh\", \"r5_kills\", \"r5_deaths\", \"r5_items\", \"d1_level\", \"d1_xp\", \"d1_gold\", \"d1_lh\", \"d1_kills\", \"d1_deaths\", \"d1_items\", \"d2_level\", \"d2_xp\", \"d2_gold\", \"d2_lh\", \"d2_kills\", \"d2_deaths\", \"d2_items\", \"d3_level\", \"d3_xp\", \"d3_gold\", \"d3_lh\", \"d3_kills\", \"d3_deaths\", \"d3_items\", \"d4_level\", \"d4_xp\", \"d4_gold\", \"d4_lh\", \"d4_kills\", \"d4_deaths\", \"d4_items\", \"d5_level\", \"d5_xp\", \"d5_gold\", \"d5_lh\", \"d5_kills\", \"d5_deaths\", \"d5_items\", \"first_blood_time\", \"first_blood_team\", \"first_blood_player1\", \"first_blood_player2\", \"radiant_bottle_time\", \"radiant_courier_time\", \"radiant_flying_courier_time\", \"radiant_tpscroll_count\", \"radiant_boots_count\", \"radiant_ward_observer_count\", \"radiant_ward_sentry_count\", \"radiant_first_ward_time\", \"dire_bottle_time\", \"dire_courier_time\", \"dire_flying_courier_time\", \"dire_tpscroll_count\", \"dire_boots_count\", \"dire_ward_observer_count\", \"dire_ward_sentry_count\", \"dire_first_ward_time\", \"r1_hero\", \"r2_hero\", \"r3_hero\", \"r4_hero\", \"r5_hero\", \"r6_hero\", \"r7_hero\", \"r8_hero\", \"r9_hero\", \"r10_hero\", \"r11_hero\", \"r12_hero\", \"r13_hero\", \"r14_hero\", \"r15_hero\", \"r16_hero\", \"r17_hero\", \"r18_hero\", \"r19_hero\", \"r20_hero\", \"r21_hero\", \"r22_hero\", \"r23_hero\", \"r24_hero\", \"r25_hero\", \"r26_hero\", \"r27_hero\", \"r28_hero\", \"r29_hero\", \"r30_hero\", \"r31_hero\", \"r32_hero\", \"r33_hero\", \"r34_hero\", \"r35_hero\", \"r36_hero\", \"r37_hero\", \"r38_hero\", \"r39_hero\", \"r40_hero\", \"r41_hero\", \"r42_hero\", \"r43_hero\", \"r44_hero\", \"r45_hero\", \"r46_hero\", \"r47_hero\", \"r48_hero\", \"r49_hero\", \"r50_hero\", \"r51_hero\", \"r52_hero\", \"r53_hero\", \"r54_hero\", \"r55_hero\", \"r56_hero\", \"r57_hero\", \"r58_hero\", \"r59_hero\", \"r60_hero\", \"r61_hero\", \"r62_hero\", \"r63_hero\", \"r64_hero\", \"r65_hero\", \"r66_hero\", \"r67_hero\", \"r68_hero\", \"r69_hero\", \"r70_hero\", \"r71_hero\", \"r72_hero\", \"r73_hero\", \"r74_hero\", \"r75_hero\", \"r76_hero\", \"r77_hero\", \"r78_hero\", \"r79_hero\", \"r80_hero\", \"r81_hero\", \"r82_hero\", \"r83_hero\", \"r84_hero\", \"r85_hero\", \"r86_hero\", \"r87_hero\", \"r88_hero\", \"r89_hero\", \"r90_hero\", \"r91_hero\", \"r92_hero\", \"r93_hero\", \"r94_hero\", \"r95_hero\", \"r96_hero\", \"r97_hero\", \"r98_hero\", \"r99_hero\", \"r100_hero\", \"r101_hero\", \"r102_hero\", \"r103_hero\", \"r104_hero\", \"r105_hero\", \"r106_hero\", \"r107_hero\", \"r108_hero\", \"r109_hero\", \"r110_hero\", \"r111_hero\", \"r112_hero\", \"r113_hero\", \"lobby_type_0\", \"lobby_type_1\", \"lobby_type_7\", \"cumKillsRD_D2\", \"cumKillsRD_D\", \"GoldRD_D2\", \"GoldRD_D\", \"XpRD_D2\", \"XpRD_LD\"], \"hoffset\": 0, \"location\": \"mouse\", \"type\": \"tooltip\", \"id\": \"el2696140567943080336pts\"}], \"data\": {\"data01\": [[0.0, 0.04343866501342595], [1.0, -0.001705142111257298], [2.0, 0.02084503409787244], [3.0, 0.08466054124526572], [4.0, 0.05079288063156598], [5.0, -0.00509112417224215], [6.0, -0.026397898571421347], [7.0, 0.03786239524077758], [8.0, 0.01918302492327302], [9.0, 0.006202300914325171], [10.0, 0.08185628707607753], [11.0, 0.05687549841535872], [12.0, -0.010282447379282003], [13.0, -0.029249477204991152], [14.0, 0.02328291574307087], [15.0, 0.006577227863646518], [16.0, 0.01737180869265604], [17.0, 0.05432699759003162], [18.0, 0.05800243628325038], [19.0, -0.007988424232712885], [20.0, -0.031279345539883326], [21.0, 0.04468620633414526], [22.0, 0.012368212807059159], [23.0, 0.032872350552853535], [24.0, 0.055162377494871434], [25.0, 0.057934639114027145], [26.0, -0.011307217344837204], [27.0, -0.037716417595541535], [28.0, 0.03398720286802103], [29.0, 0.035772313601415466], [30.0, 0.013329412275467526], [31.0, 0.06101564324380741], [32.0, 0.043677883789217234], [33.0, -0.012094868945103995], [34.0, -0.03634134542467395], [35.0, 0.04852170028705332], [36.0, -0.013168142117334584], [37.0, -0.012388860539733336], [38.0, -0.08150778524271493], [39.0, -0.05325112452364768], [40.0, 0.012434403364277938], [41.0, 0.03357293795500361], [42.0, -0.030589057175367817], [43.0, -0.006983283344704031], [44.0, -0.024355307166050316], [45.0, -0.09191816101268903], [46.0, -0.056185907242755816], [47.0, 0.014673807805073437], [48.0, 0.027549859930011474], [49.0, -0.041074135350320895], [50.0, -0.0010703429739588799], [51.0, -0.044070146214212236], [52.0, -0.03679420300789202], [53.0, -0.06698254579256653], [54.0, -0.0015906587822146232], [55.0, 0.022845883309598157], [56.0, -0.03974269347807829], [57.0, -0.01944998662406794], [58.0, -0.004503243133027334], [59.0, -0.07610839811798041], [60.0, -0.05645346967908379], [61.0, 0.010204710919884875], [62.0, 0.028438358994540152], [63.0, -0.04229466277683008], [64.0, -0.0007257586917263519], [65.0, -0.04182655703745765], [66.0, -0.05661083396585451], [67.0, -0.06243958861870004], [68.0, 0.00025018381715973984], [69.0, 0.0413637731684287], [70.0, -0.04430441006329099], [71.0, 0.004546798803041796], [72.0, 0.0033841188248591096], [73.0, -0.002329321035437579], [74.0, -0.037907018936319876], [75.0, -0.003998686927639658], [76.0, -0.030975112773046886], [77.0, 0.03665170165622784], [78.0, -0.041654370165183405], [79.0, 0.02622663109434148], [80.0, -0.0076993738950903185], [81.0, -0.000823744233946271], [82.0, -0.040990739044905716], [83.0, 0.005958778220305869], [84.0, 0.019422505471510424], [85.0, -0.010421586682746018], [86.0, 0.0410812665255505], [87.0, -0.0174554432807253], [88.0, 0.007226499303826192], [89.0, -0.006525361618573388], [90.0, 0.03249148563683502], [91.0, 0.0], [92.0, 0.036633668758320824], [93.0, -0.13539816057492163], [94.0, -0.06477779260323838], [95.0, -0.021397585508479557], [96.0, 0.02662737883192611], [97.0, 0.029112667702740683], [98.0, 0.13856985260892923], [99.0, 0.05507776193121345], [100.0, -0.007614285713023931], [101.0, -0.008658463073237685], [102.0, 0.047681787935845904], [103.0, -0.021620010224688582], [104.0, -0.09319853259409584], [105.0, 0.008891852639581877], [106.0, -0.034500641584801185], [107.0, -0.0018185020282853747], [108.0, -0.026793355397504024], [109.0, 0.015565626301343258], [110.0, 0.02249737336897395], [111.0, 0.008953065884176471], [112.0, -0.11248998050740967], [113.0, 0.07601237130926362], [114.0, -0.001209149534553118], [115.0, 0.0], [116.0, -0.15283920438631568], [117.0, -0.07834216802802536], [118.0, -0.0098188803231899], [119.0, 0.08872090297557292], [120.0, -0.006939068896118171], [121.0, 0.019002486586566798], [122.0, 0.026647554186052202], [123.0, 0.045883753935802574], [124.0, -0.07263095430602783], [125.0, -0.02935724314764153], [126.0, -0.018881192475443797], [127.0, 0.02200516414352026], [128.0, 0.011156860877373666], [129.0, 0.03044592713969998], [130.0, -0.21150511767179528], [131.0, 0.03717171354800524], [132.0, -0.04451931505565348], [133.0, 0.10329772966718742], [134.0, -0.024501406587228658], [135.0, 0.0010703826734452355], [136.0, -0.001314384328804943], [137.0, -0.016777934697237703], [138.0, -0.11587665166921923], [139.0, 0.01694167730727831], [140.0, 0.03095874160639729], [141.0, -0.01426333608799834], [142.0, 0.019816119055028888], [143.0, 0.049169423533978705], [144.0, -0.03581691068829402], [145.0, -0.0276640773159885], [146.0, -0.07824065661237388], [147.0, -0.011058269638459533], [148.0, 0.12410782832855408], [149.0, -0.04020983920321127], [150.0, 0.01189667397444962], [151.0, 0.01045991699616193], [152.0, -0.05301784895393844], [153.0, 0.04848964808373938], [154.0, -0.02190240651714622], [155.0, -0.017205322721245763], [156.0, -0.013070626474708243], [157.0, -0.020298434143483743], [158.0, 0.16360142276400755], [159.0, -0.010857349496940248], [160.0, 0.08201298779034745], [161.0, 0.04892727002029711], [162.0, 0.12190551663318878], [163.0, 0.009935433918045137], [164.0, 0.0530287573812335], [165.0, -0.007948950509785993], [166.0, -0.02325367280742661], [167.0, -0.06255482726339454], [168.0, 0.033069149942552764], [169.0, -0.0017261469318314477], [170.0, -0.03260185228497803], [171.0, -0.012533182574527717], [172.0, 0.003181277537282552], [173.0, 0.03398722230298177], [174.0, 0.015914392389184247], [175.0, -0.008439243767245566], [176.0, 0.04589054257887874], [177.0, -0.06471035363964749], [178.0, -0.03491803295446905], [179.0, 0.005638733748987111], [180.0, -0.011929174694570076], [181.0, -0.009691393829634649], [182.0, 0.035677201688073305], [183.0, 0.02195100255344405], [184.0, 0.026816053943929035], [185.0, 0.037065458696058956], [186.0, -0.019639458333663838], [187.0, 0.01090799927261999], [188.0, -0.03427482365566722], [189.0, -0.02260391885314092], [190.0, -0.06644060716958644], [191.0, 0.05738753385560041], [192.0, -0.07929972982113749], [193.0, 0.06676794492338915], [194.0, 0.01875417562269879], [195.0, -0.04875431758692161], [196.0, -0.0023737428859068543], [197.0, 0.01272551940603005], [198.0, 0.0], [199.0, 0.0], [200.0, -0.0063082813412410335], [201.0, 0.04393786207153275], [202.0, 0.0], [203.0, -0.003213849525892959], [204.0, -0.03497502471790447], [205.0, 0.027464167932828878], [206.0, -0.003911300354477009], [207.0, -0.006648233324910291], [208.0, 0.3025258300365734], [209.0, 0.016521358500612513], [210.0, 0.12388925004710116], [211.0, 0.019371416462265527], [212.0, 0.2629359504866758]]}, \"id\": \"el2696140567942658320\"});\n",
       "   }(mpld3);\n",
       "}else if(typeof define === \"function\" && define.amd){\n",
       "   // require.js is available: use it to load d3/mpld3\n",
       "   require.config({paths: {d3: \"https://mpld3.github.io/js/d3.v3.min\"}});\n",
       "   require([\"d3\"], function(d3){\n",
       "      window.d3 = d3;\n",
       "      mpld3_load_lib(\"https://mpld3.github.io/js/mpld3.v0.2.js\", function(){\n",
       "         \n",
       "         mpld3.draw_figure(\"fig_el26961405679426583208738058525\", {\"axes\": [{\"xlim\": [0.0, 250.0], \"yscale\": \"linear\", \"axesbg\": \"#EEEEEE\", \"texts\": [], \"zoomable\": true, \"images\": [], \"xdomain\": [0.0, 250.0], \"ylim\": [-0.30000000000000004, 0.40000000000000002], \"paths\": [], \"sharey\": [], \"sharex\": [], \"axesbgalpha\": null, \"axes\": [{\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"bottom\", \"nticks\": 6, \"tickvalues\": null}, {\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"left\", \"nticks\": 10, \"tickvalues\": null}], \"lines\": [{\"color\": \"#FF0000\", \"yindex\": 1, \"coordinates\": \"data\", \"dasharray\": \"10,0\", \"zorder\": 2, \"alpha\": 1, \"xindex\": 0, \"linewidth\": 1.0, \"data\": \"data01\", \"id\": \"el2696140567943080336\"}], \"markers\": [{\"edgecolor\": \"#000000\", \"facecolor\": \"#FF0000\", \"edgewidth\": 0.5, \"yindex\": 1, \"coordinates\": \"data\", \"zorder\": 2, \"markerpath\": [[[0.0, 3.0], [0.7956093000000001, 3.0], [1.5587396123545605, 2.683901074764725], [2.121320343559643, 2.121320343559643], [2.683901074764725, 1.5587396123545605], [3.0, 0.7956093000000001], [3.0, 0.0], [3.0, -0.7956093000000001], [2.683901074764725, -1.5587396123545605], [2.121320343559643, -2.121320343559643], [1.5587396123545605, -2.683901074764725], [0.7956093000000001, -3.0], [0.0, -3.0], [-0.7956093000000001, -3.0], [-1.5587396123545605, -2.683901074764725], [-2.121320343559643, -2.121320343559643], [-2.683901074764725, -1.5587396123545605], [-3.0, -0.7956093000000001], [-3.0, 0.0], [-3.0, 0.7956093000000001], [-2.683901074764725, 1.5587396123545605], [-2.121320343559643, 2.121320343559643], [-1.5587396123545605, 2.683901074764725], [-0.7956093000000001, 3.0], [0.0, 3.0]], [\"M\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"Z\"]], \"alpha\": 1, \"xindex\": 0, \"data\": \"data01\", \"id\": \"el2696140567943080336pts\"}], \"id\": \"el2696140567942659728\", \"ydomain\": [-0.30000000000000004, 0.40000000000000002], \"collections\": [], \"xscale\": \"linear\", \"bbox\": [0.125, 0.125, 0.77500000000000002, 0.77500000000000002]}], \"height\": 320.0, \"width\": 480.0, \"plugins\": [{\"type\": \"reset\"}, {\"enabled\": false, \"button\": true, \"type\": \"zoom\"}, {\"enabled\": false, \"button\": true, \"type\": \"boxzoom\"}, {\"voffset\": 10, \"labels\": [\"start_time\", \"r1_level\", \"r1_xp\", \"r1_gold\", \"r1_lh\", \"r1_kills\", \"r1_deaths\", \"r1_items\", \"r2_level\", \"r2_xp\", \"r2_gold\", \"r2_lh\", \"r2_kills\", \"r2_deaths\", \"r2_items\", \"r3_level\", \"r3_xp\", \"r3_gold\", \"r3_lh\", \"r3_kills\", \"r3_deaths\", \"r3_items\", \"r4_level\", \"r4_xp\", \"r4_gold\", \"r4_lh\", \"r4_kills\", \"r4_deaths\", \"r4_items\", \"r5_level\", \"r5_xp\", \"r5_gold\", \"r5_lh\", \"r5_kills\", \"r5_deaths\", \"r5_items\", \"d1_level\", \"d1_xp\", \"d1_gold\", \"d1_lh\", \"d1_kills\", \"d1_deaths\", \"d1_items\", \"d2_level\", \"d2_xp\", \"d2_gold\", \"d2_lh\", \"d2_kills\", \"d2_deaths\", \"d2_items\", \"d3_level\", \"d3_xp\", \"d3_gold\", \"d3_lh\", \"d3_kills\", \"d3_deaths\", \"d3_items\", \"d4_level\", \"d4_xp\", \"d4_gold\", \"d4_lh\", \"d4_kills\", \"d4_deaths\", \"d4_items\", \"d5_level\", \"d5_xp\", \"d5_gold\", \"d5_lh\", \"d5_kills\", \"d5_deaths\", \"d5_items\", \"first_blood_time\", \"first_blood_team\", \"first_blood_player1\", \"first_blood_player2\", \"radiant_bottle_time\", \"radiant_courier_time\", \"radiant_flying_courier_time\", \"radiant_tpscroll_count\", \"radiant_boots_count\", \"radiant_ward_observer_count\", \"radiant_ward_sentry_count\", \"radiant_first_ward_time\", \"dire_bottle_time\", \"dire_courier_time\", \"dire_flying_courier_time\", \"dire_tpscroll_count\", \"dire_boots_count\", \"dire_ward_observer_count\", \"dire_ward_sentry_count\", \"dire_first_ward_time\", \"r1_hero\", \"r2_hero\", \"r3_hero\", \"r4_hero\", \"r5_hero\", \"r6_hero\", \"r7_hero\", \"r8_hero\", \"r9_hero\", \"r10_hero\", \"r11_hero\", \"r12_hero\", \"r13_hero\", \"r14_hero\", \"r15_hero\", \"r16_hero\", \"r17_hero\", \"r18_hero\", \"r19_hero\", \"r20_hero\", \"r21_hero\", \"r22_hero\", \"r23_hero\", \"r24_hero\", \"r25_hero\", \"r26_hero\", \"r27_hero\", \"r28_hero\", \"r29_hero\", \"r30_hero\", \"r31_hero\", \"r32_hero\", \"r33_hero\", \"r34_hero\", \"r35_hero\", \"r36_hero\", \"r37_hero\", \"r38_hero\", \"r39_hero\", \"r40_hero\", \"r41_hero\", \"r42_hero\", \"r43_hero\", \"r44_hero\", \"r45_hero\", \"r46_hero\", \"r47_hero\", \"r48_hero\", \"r49_hero\", \"r50_hero\", \"r51_hero\", \"r52_hero\", \"r53_hero\", \"r54_hero\", \"r55_hero\", \"r56_hero\", \"r57_hero\", \"r58_hero\", \"r59_hero\", \"r60_hero\", \"r61_hero\", \"r62_hero\", \"r63_hero\", \"r64_hero\", \"r65_hero\", \"r66_hero\", \"r67_hero\", \"r68_hero\", \"r69_hero\", \"r70_hero\", \"r71_hero\", \"r72_hero\", \"r73_hero\", \"r74_hero\", \"r75_hero\", \"r76_hero\", \"r77_hero\", \"r78_hero\", \"r79_hero\", \"r80_hero\", \"r81_hero\", \"r82_hero\", \"r83_hero\", \"r84_hero\", \"r85_hero\", \"r86_hero\", \"r87_hero\", \"r88_hero\", \"r89_hero\", \"r90_hero\", \"r91_hero\", \"r92_hero\", \"r93_hero\", \"r94_hero\", \"r95_hero\", \"r96_hero\", \"r97_hero\", \"r98_hero\", \"r99_hero\", \"r100_hero\", \"r101_hero\", \"r102_hero\", \"r103_hero\", \"r104_hero\", \"r105_hero\", \"r106_hero\", \"r107_hero\", \"r108_hero\", \"r109_hero\", \"r110_hero\", \"r111_hero\", \"r112_hero\", \"r113_hero\", \"lobby_type_0\", \"lobby_type_1\", \"lobby_type_7\", \"cumKillsRD_D2\", \"cumKillsRD_D\", \"GoldRD_D2\", \"GoldRD_D\", \"XpRD_D2\", \"XpRD_LD\"], \"hoffset\": 0, \"location\": \"mouse\", \"type\": \"tooltip\", \"id\": \"el2696140567943080336pts\"}], \"data\": {\"data01\": [[0.0, 0.04343866501342595], [1.0, -0.001705142111257298], [2.0, 0.02084503409787244], [3.0, 0.08466054124526572], [4.0, 0.05079288063156598], [5.0, -0.00509112417224215], [6.0, -0.026397898571421347], [7.0, 0.03786239524077758], [8.0, 0.01918302492327302], [9.0, 0.006202300914325171], [10.0, 0.08185628707607753], [11.0, 0.05687549841535872], [12.0, -0.010282447379282003], [13.0, -0.029249477204991152], [14.0, 0.02328291574307087], [15.0, 0.006577227863646518], [16.0, 0.01737180869265604], [17.0, 0.05432699759003162], [18.0, 0.05800243628325038], [19.0, -0.007988424232712885], [20.0, -0.031279345539883326], [21.0, 0.04468620633414526], [22.0, 0.012368212807059159], [23.0, 0.032872350552853535], [24.0, 0.055162377494871434], [25.0, 0.057934639114027145], [26.0, -0.011307217344837204], [27.0, -0.037716417595541535], [28.0, 0.03398720286802103], [29.0, 0.035772313601415466], [30.0, 0.013329412275467526], [31.0, 0.06101564324380741], [32.0, 0.043677883789217234], [33.0, -0.012094868945103995], [34.0, -0.03634134542467395], [35.0, 0.04852170028705332], [36.0, -0.013168142117334584], [37.0, -0.012388860539733336], [38.0, -0.08150778524271493], [39.0, -0.05325112452364768], [40.0, 0.012434403364277938], [41.0, 0.03357293795500361], [42.0, -0.030589057175367817], [43.0, -0.006983283344704031], [44.0, -0.024355307166050316], [45.0, -0.09191816101268903], [46.0, -0.056185907242755816], [47.0, 0.014673807805073437], [48.0, 0.027549859930011474], [49.0, -0.041074135350320895], [50.0, -0.0010703429739588799], [51.0, -0.044070146214212236], [52.0, -0.03679420300789202], [53.0, -0.06698254579256653], [54.0, -0.0015906587822146232], [55.0, 0.022845883309598157], [56.0, -0.03974269347807829], [57.0, -0.01944998662406794], [58.0, -0.004503243133027334], [59.0, -0.07610839811798041], [60.0, -0.05645346967908379], [61.0, 0.010204710919884875], [62.0, 0.028438358994540152], [63.0, -0.04229466277683008], [64.0, -0.0007257586917263519], [65.0, -0.04182655703745765], [66.0, -0.05661083396585451], [67.0, -0.06243958861870004], [68.0, 0.00025018381715973984], [69.0, 0.0413637731684287], [70.0, -0.04430441006329099], [71.0, 0.004546798803041796], [72.0, 0.0033841188248591096], [73.0, -0.002329321035437579], [74.0, -0.037907018936319876], [75.0, -0.003998686927639658], [76.0, -0.030975112773046886], [77.0, 0.03665170165622784], [78.0, -0.041654370165183405], [79.0, 0.02622663109434148], [80.0, -0.0076993738950903185], [81.0, -0.000823744233946271], [82.0, -0.040990739044905716], [83.0, 0.005958778220305869], [84.0, 0.019422505471510424], [85.0, -0.010421586682746018], [86.0, 0.0410812665255505], [87.0, -0.0174554432807253], [88.0, 0.007226499303826192], [89.0, -0.006525361618573388], [90.0, 0.03249148563683502], [91.0, 0.0], [92.0, 0.036633668758320824], [93.0, -0.13539816057492163], [94.0, -0.06477779260323838], [95.0, -0.021397585508479557], [96.0, 0.02662737883192611], [97.0, 0.029112667702740683], [98.0, 0.13856985260892923], [99.0, 0.05507776193121345], [100.0, -0.007614285713023931], [101.0, -0.008658463073237685], [102.0, 0.047681787935845904], [103.0, -0.021620010224688582], [104.0, -0.09319853259409584], [105.0, 0.008891852639581877], [106.0, -0.034500641584801185], [107.0, -0.0018185020282853747], [108.0, -0.026793355397504024], [109.0, 0.015565626301343258], [110.0, 0.02249737336897395], [111.0, 0.008953065884176471], [112.0, -0.11248998050740967], [113.0, 0.07601237130926362], [114.0, -0.001209149534553118], [115.0, 0.0], [116.0, -0.15283920438631568], [117.0, -0.07834216802802536], [118.0, -0.0098188803231899], [119.0, 0.08872090297557292], [120.0, -0.006939068896118171], [121.0, 0.019002486586566798], [122.0, 0.026647554186052202], [123.0, 0.045883753935802574], [124.0, -0.07263095430602783], [125.0, -0.02935724314764153], [126.0, -0.018881192475443797], [127.0, 0.02200516414352026], [128.0, 0.011156860877373666], [129.0, 0.03044592713969998], [130.0, -0.21150511767179528], [131.0, 0.03717171354800524], [132.0, -0.04451931505565348], [133.0, 0.10329772966718742], [134.0, -0.024501406587228658], [135.0, 0.0010703826734452355], [136.0, -0.001314384328804943], [137.0, -0.016777934697237703], [138.0, -0.11587665166921923], [139.0, 0.01694167730727831], [140.0, 0.03095874160639729], [141.0, -0.01426333608799834], [142.0, 0.019816119055028888], [143.0, 0.049169423533978705], [144.0, -0.03581691068829402], [145.0, -0.0276640773159885], [146.0, -0.07824065661237388], [147.0, -0.011058269638459533], [148.0, 0.12410782832855408], [149.0, -0.04020983920321127], [150.0, 0.01189667397444962], [151.0, 0.01045991699616193], [152.0, -0.05301784895393844], [153.0, 0.04848964808373938], [154.0, -0.02190240651714622], [155.0, -0.017205322721245763], [156.0, -0.013070626474708243], [157.0, -0.020298434143483743], [158.0, 0.16360142276400755], [159.0, -0.010857349496940248], [160.0, 0.08201298779034745], [161.0, 0.04892727002029711], [162.0, 0.12190551663318878], [163.0, 0.009935433918045137], [164.0, 0.0530287573812335], [165.0, -0.007948950509785993], [166.0, -0.02325367280742661], [167.0, -0.06255482726339454], [168.0, 0.033069149942552764], [169.0, -0.0017261469318314477], [170.0, -0.03260185228497803], [171.0, -0.012533182574527717], [172.0, 0.003181277537282552], [173.0, 0.03398722230298177], [174.0, 0.015914392389184247], [175.0, -0.008439243767245566], [176.0, 0.04589054257887874], [177.0, -0.06471035363964749], [178.0, -0.03491803295446905], [179.0, 0.005638733748987111], [180.0, -0.011929174694570076], [181.0, -0.009691393829634649], [182.0, 0.035677201688073305], [183.0, 0.02195100255344405], [184.0, 0.026816053943929035], [185.0, 0.037065458696058956], [186.0, -0.019639458333663838], [187.0, 0.01090799927261999], [188.0, -0.03427482365566722], [189.0, -0.02260391885314092], [190.0, -0.06644060716958644], [191.0, 0.05738753385560041], [192.0, -0.07929972982113749], [193.0, 0.06676794492338915], [194.0, 0.01875417562269879], [195.0, -0.04875431758692161], [196.0, -0.0023737428859068543], [197.0, 0.01272551940603005], [198.0, 0.0], [199.0, 0.0], [200.0, -0.0063082813412410335], [201.0, 0.04393786207153275], [202.0, 0.0], [203.0, -0.003213849525892959], [204.0, -0.03497502471790447], [205.0, 0.027464167932828878], [206.0, -0.003911300354477009], [207.0, -0.006648233324910291], [208.0, 0.3025258300365734], [209.0, 0.016521358500612513], [210.0, 0.12388925004710116], [211.0, 0.019371416462265527], [212.0, 0.2629359504866758]]}, \"id\": \"el2696140567942658320\"});\n",
       "      });\n",
       "    });\n",
       "}else{\n",
       "    // require.js not available: dynamically load d3 & mpld3\n",
       "    mpld3_load_lib(\"https://mpld3.github.io/js/d3.v3.min.js\", function(){\n",
       "         mpld3_load_lib(\"https://mpld3.github.io/js/mpld3.v0.2.js\", function(){\n",
       "                 \n",
       "                 mpld3.draw_figure(\"fig_el26961405679426583208738058525\", {\"axes\": [{\"xlim\": [0.0, 250.0], \"yscale\": \"linear\", \"axesbg\": \"#EEEEEE\", \"texts\": [], \"zoomable\": true, \"images\": [], \"xdomain\": [0.0, 250.0], \"ylim\": [-0.30000000000000004, 0.40000000000000002], \"paths\": [], \"sharey\": [], \"sharex\": [], \"axesbgalpha\": null, \"axes\": [{\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"bottom\", \"nticks\": 6, \"tickvalues\": null}, {\"scale\": \"linear\", \"tickformat\": null, \"grid\": {\"gridOn\": false}, \"fontsize\": 10.0, \"position\": \"left\", \"nticks\": 10, \"tickvalues\": null}], \"lines\": [{\"color\": \"#FF0000\", \"yindex\": 1, \"coordinates\": \"data\", \"dasharray\": \"10,0\", \"zorder\": 2, \"alpha\": 1, \"xindex\": 0, \"linewidth\": 1.0, \"data\": \"data01\", \"id\": \"el2696140567943080336\"}], \"markers\": [{\"edgecolor\": \"#000000\", \"facecolor\": \"#FF0000\", \"edgewidth\": 0.5, \"yindex\": 1, \"coordinates\": \"data\", \"zorder\": 2, \"markerpath\": [[[0.0, 3.0], [0.7956093000000001, 3.0], [1.5587396123545605, 2.683901074764725], [2.121320343559643, 2.121320343559643], [2.683901074764725, 1.5587396123545605], [3.0, 0.7956093000000001], [3.0, 0.0], [3.0, -0.7956093000000001], [2.683901074764725, -1.5587396123545605], [2.121320343559643, -2.121320343559643], [1.5587396123545605, -2.683901074764725], [0.7956093000000001, -3.0], [0.0, -3.0], [-0.7956093000000001, -3.0], [-1.5587396123545605, -2.683901074764725], [-2.121320343559643, -2.121320343559643], [-2.683901074764725, -1.5587396123545605], [-3.0, -0.7956093000000001], [-3.0, 0.0], [-3.0, 0.7956093000000001], [-2.683901074764725, 1.5587396123545605], [-2.121320343559643, 2.121320343559643], [-1.5587396123545605, 2.683901074764725], [-0.7956093000000001, 3.0], [0.0, 3.0]], [\"M\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"C\", \"Z\"]], \"alpha\": 1, \"xindex\": 0, \"data\": \"data01\", \"id\": \"el2696140567943080336pts\"}], \"id\": \"el2696140567942659728\", \"ydomain\": [-0.30000000000000004, 0.40000000000000002], \"collections\": [], \"xscale\": \"linear\", \"bbox\": [0.125, 0.125, 0.77500000000000002, 0.77500000000000002]}], \"height\": 320.0, \"width\": 480.0, \"plugins\": [{\"type\": \"reset\"}, {\"enabled\": false, \"button\": true, \"type\": \"zoom\"}, {\"enabled\": false, \"button\": true, \"type\": \"boxzoom\"}, {\"voffset\": 10, \"labels\": [\"start_time\", \"r1_level\", \"r1_xp\", \"r1_gold\", \"r1_lh\", \"r1_kills\", \"r1_deaths\", \"r1_items\", \"r2_level\", \"r2_xp\", \"r2_gold\", \"r2_lh\", \"r2_kills\", \"r2_deaths\", \"r2_items\", \"r3_level\", \"r3_xp\", \"r3_gold\", \"r3_lh\", \"r3_kills\", \"r3_deaths\", \"r3_items\", \"r4_level\", \"r4_xp\", \"r4_gold\", \"r4_lh\", \"r4_kills\", \"r4_deaths\", \"r4_items\", \"r5_level\", \"r5_xp\", \"r5_gold\", \"r5_lh\", \"r5_kills\", \"r5_deaths\", \"r5_items\", \"d1_level\", \"d1_xp\", \"d1_gold\", \"d1_lh\", \"d1_kills\", \"d1_deaths\", \"d1_items\", \"d2_level\", \"d2_xp\", \"d2_gold\", \"d2_lh\", \"d2_kills\", \"d2_deaths\", \"d2_items\", \"d3_level\", \"d3_xp\", \"d3_gold\", \"d3_lh\", \"d3_kills\", \"d3_deaths\", \"d3_items\", \"d4_level\", \"d4_xp\", \"d4_gold\", \"d4_lh\", \"d4_kills\", \"d4_deaths\", \"d4_items\", \"d5_level\", \"d5_xp\", \"d5_gold\", \"d5_lh\", \"d5_kills\", \"d5_deaths\", \"d5_items\", \"first_blood_time\", \"first_blood_team\", \"first_blood_player1\", \"first_blood_player2\", \"radiant_bottle_time\", \"radiant_courier_time\", \"radiant_flying_courier_time\", \"radiant_tpscroll_count\", \"radiant_boots_count\", \"radiant_ward_observer_count\", \"radiant_ward_sentry_count\", \"radiant_first_ward_time\", \"dire_bottle_time\", \"dire_courier_time\", \"dire_flying_courier_time\", \"dire_tpscroll_count\", \"dire_boots_count\", \"dire_ward_observer_count\", \"dire_ward_sentry_count\", \"dire_first_ward_time\", \"r1_hero\", \"r2_hero\", \"r3_hero\", \"r4_hero\", \"r5_hero\", \"r6_hero\", \"r7_hero\", \"r8_hero\", \"r9_hero\", \"r10_hero\", \"r11_hero\", \"r12_hero\", \"r13_hero\", \"r14_hero\", \"r15_hero\", \"r16_hero\", \"r17_hero\", \"r18_hero\", \"r19_hero\", \"r20_hero\", \"r21_hero\", \"r22_hero\", \"r23_hero\", \"r24_hero\", \"r25_hero\", \"r26_hero\", \"r27_hero\", \"r28_hero\", \"r29_hero\", \"r30_hero\", \"r31_hero\", \"r32_hero\", \"r33_hero\", \"r34_hero\", \"r35_hero\", \"r36_hero\", \"r37_hero\", \"r38_hero\", \"r39_hero\", \"r40_hero\", \"r41_hero\", \"r42_hero\", \"r43_hero\", \"r44_hero\", \"r45_hero\", \"r46_hero\", \"r47_hero\", \"r48_hero\", \"r49_hero\", \"r50_hero\", \"r51_hero\", \"r52_hero\", \"r53_hero\", \"r54_hero\", \"r55_hero\", \"r56_hero\", \"r57_hero\", \"r58_hero\", \"r59_hero\", \"r60_hero\", \"r61_hero\", \"r62_hero\", \"r63_hero\", \"r64_hero\", \"r65_hero\", \"r66_hero\", \"r67_hero\", \"r68_hero\", \"r69_hero\", \"r70_hero\", \"r71_hero\", \"r72_hero\", \"r73_hero\", \"r74_hero\", \"r75_hero\", \"r76_hero\", \"r77_hero\", \"r78_hero\", \"r79_hero\", \"r80_hero\", \"r81_hero\", \"r82_hero\", \"r83_hero\", \"r84_hero\", \"r85_hero\", \"r86_hero\", \"r87_hero\", \"r88_hero\", \"r89_hero\", \"r90_hero\", \"r91_hero\", \"r92_hero\", \"r93_hero\", \"r94_hero\", \"r95_hero\", \"r96_hero\", \"r97_hero\", \"r98_hero\", \"r99_hero\", \"r100_hero\", \"r101_hero\", \"r102_hero\", \"r103_hero\", \"r104_hero\", \"r105_hero\", \"r106_hero\", \"r107_hero\", \"r108_hero\", \"r109_hero\", \"r110_hero\", \"r111_hero\", \"r112_hero\", \"r113_hero\", \"lobby_type_0\", \"lobby_type_1\", \"lobby_type_7\", \"cumKillsRD_D2\", \"cumKillsRD_D\", \"GoldRD_D2\", \"GoldRD_D\", \"XpRD_D2\", \"XpRD_LD\"], \"hoffset\": 0, \"location\": \"mouse\", \"type\": \"tooltip\", \"id\": \"el2696140567943080336pts\"}], \"data\": {\"data01\": [[0.0, 0.04343866501342595], [1.0, -0.001705142111257298], [2.0, 0.02084503409787244], [3.0, 0.08466054124526572], [4.0, 0.05079288063156598], [5.0, -0.00509112417224215], [6.0, -0.026397898571421347], [7.0, 0.03786239524077758], [8.0, 0.01918302492327302], [9.0, 0.006202300914325171], [10.0, 0.08185628707607753], [11.0, 0.05687549841535872], [12.0, -0.010282447379282003], [13.0, -0.029249477204991152], [14.0, 0.02328291574307087], [15.0, 0.006577227863646518], [16.0, 0.01737180869265604], [17.0, 0.05432699759003162], [18.0, 0.05800243628325038], [19.0, -0.007988424232712885], [20.0, -0.031279345539883326], [21.0, 0.04468620633414526], [22.0, 0.012368212807059159], [23.0, 0.032872350552853535], [24.0, 0.055162377494871434], [25.0, 0.057934639114027145], [26.0, -0.011307217344837204], [27.0, -0.037716417595541535], [28.0, 0.03398720286802103], [29.0, 0.035772313601415466], [30.0, 0.013329412275467526], [31.0, 0.06101564324380741], [32.0, 0.043677883789217234], [33.0, -0.012094868945103995], [34.0, -0.03634134542467395], [35.0, 0.04852170028705332], [36.0, -0.013168142117334584], [37.0, -0.012388860539733336], [38.0, -0.08150778524271493], [39.0, -0.05325112452364768], [40.0, 0.012434403364277938], [41.0, 0.03357293795500361], [42.0, -0.030589057175367817], [43.0, -0.006983283344704031], [44.0, -0.024355307166050316], [45.0, -0.09191816101268903], [46.0, -0.056185907242755816], [47.0, 0.014673807805073437], [48.0, 0.027549859930011474], [49.0, -0.041074135350320895], [50.0, -0.0010703429739588799], [51.0, -0.044070146214212236], [52.0, -0.03679420300789202], [53.0, -0.06698254579256653], [54.0, -0.0015906587822146232], [55.0, 0.022845883309598157], [56.0, -0.03974269347807829], [57.0, -0.01944998662406794], [58.0, -0.004503243133027334], [59.0, -0.07610839811798041], [60.0, -0.05645346967908379], [61.0, 0.010204710919884875], [62.0, 0.028438358994540152], [63.0, -0.04229466277683008], [64.0, -0.0007257586917263519], [65.0, -0.04182655703745765], [66.0, -0.05661083396585451], [67.0, -0.06243958861870004], [68.0, 0.00025018381715973984], [69.0, 0.0413637731684287], [70.0, -0.04430441006329099], [71.0, 0.004546798803041796], [72.0, 0.0033841188248591096], [73.0, -0.002329321035437579], [74.0, -0.037907018936319876], [75.0, -0.003998686927639658], [76.0, -0.030975112773046886], [77.0, 0.03665170165622784], [78.0, -0.041654370165183405], [79.0, 0.02622663109434148], [80.0, -0.0076993738950903185], [81.0, -0.000823744233946271], [82.0, -0.040990739044905716], [83.0, 0.005958778220305869], [84.0, 0.019422505471510424], [85.0, -0.010421586682746018], [86.0, 0.0410812665255505], [87.0, -0.0174554432807253], [88.0, 0.007226499303826192], [89.0, -0.006525361618573388], [90.0, 0.03249148563683502], [91.0, 0.0], [92.0, 0.036633668758320824], [93.0, -0.13539816057492163], [94.0, -0.06477779260323838], [95.0, -0.021397585508479557], [96.0, 0.02662737883192611], [97.0, 0.029112667702740683], [98.0, 0.13856985260892923], [99.0, 0.05507776193121345], [100.0, -0.007614285713023931], [101.0, -0.008658463073237685], [102.0, 0.047681787935845904], [103.0, -0.021620010224688582], [104.0, -0.09319853259409584], [105.0, 0.008891852639581877], [106.0, -0.034500641584801185], [107.0, -0.0018185020282853747], [108.0, -0.026793355397504024], [109.0, 0.015565626301343258], [110.0, 0.02249737336897395], [111.0, 0.008953065884176471], [112.0, -0.11248998050740967], [113.0, 0.07601237130926362], [114.0, -0.001209149534553118], [115.0, 0.0], [116.0, -0.15283920438631568], [117.0, -0.07834216802802536], [118.0, -0.0098188803231899], [119.0, 0.08872090297557292], [120.0, -0.006939068896118171], [121.0, 0.019002486586566798], [122.0, 0.026647554186052202], [123.0, 0.045883753935802574], [124.0, -0.07263095430602783], [125.0, -0.02935724314764153], [126.0, -0.018881192475443797], [127.0, 0.02200516414352026], [128.0, 0.011156860877373666], [129.0, 0.03044592713969998], [130.0, -0.21150511767179528], [131.0, 0.03717171354800524], [132.0, -0.04451931505565348], [133.0, 0.10329772966718742], [134.0, -0.024501406587228658], [135.0, 0.0010703826734452355], [136.0, -0.001314384328804943], [137.0, -0.016777934697237703], [138.0, -0.11587665166921923], [139.0, 0.01694167730727831], [140.0, 0.03095874160639729], [141.0, -0.01426333608799834], [142.0, 0.019816119055028888], [143.0, 0.049169423533978705], [144.0, -0.03581691068829402], [145.0, -0.0276640773159885], [146.0, -0.07824065661237388], [147.0, -0.011058269638459533], [148.0, 0.12410782832855408], [149.0, -0.04020983920321127], [150.0, 0.01189667397444962], [151.0, 0.01045991699616193], [152.0, -0.05301784895393844], [153.0, 0.04848964808373938], [154.0, -0.02190240651714622], [155.0, -0.017205322721245763], [156.0, -0.013070626474708243], [157.0, -0.020298434143483743], [158.0, 0.16360142276400755], [159.0, -0.010857349496940248], [160.0, 0.08201298779034745], [161.0, 0.04892727002029711], [162.0, 0.12190551663318878], [163.0, 0.009935433918045137], [164.0, 0.0530287573812335], [165.0, -0.007948950509785993], [166.0, -0.02325367280742661], [167.0, -0.06255482726339454], [168.0, 0.033069149942552764], [169.0, -0.0017261469318314477], [170.0, -0.03260185228497803], [171.0, -0.012533182574527717], [172.0, 0.003181277537282552], [173.0, 0.03398722230298177], [174.0, 0.015914392389184247], [175.0, -0.008439243767245566], [176.0, 0.04589054257887874], [177.0, -0.06471035363964749], [178.0, -0.03491803295446905], [179.0, 0.005638733748987111], [180.0, -0.011929174694570076], [181.0, -0.009691393829634649], [182.0, 0.035677201688073305], [183.0, 0.02195100255344405], [184.0, 0.026816053943929035], [185.0, 0.037065458696058956], [186.0, -0.019639458333663838], [187.0, 0.01090799927261999], [188.0, -0.03427482365566722], [189.0, -0.02260391885314092], [190.0, -0.06644060716958644], [191.0, 0.05738753385560041], [192.0, -0.07929972982113749], [193.0, 0.06676794492338915], [194.0, 0.01875417562269879], [195.0, -0.04875431758692161], [196.0, -0.0023737428859068543], [197.0, 0.01272551940603005], [198.0, 0.0], [199.0, 0.0], [200.0, -0.0063082813412410335], [201.0, 0.04393786207153275], [202.0, 0.0], [203.0, -0.003213849525892959], [204.0, -0.03497502471790447], [205.0, 0.027464167932828878], [206.0, -0.003911300354477009], [207.0, -0.006648233324910291], [208.0, 0.3025258300365734], [209.0, 0.016521358500612513], [210.0, 0.12388925004710116], [211.0, 0.019371416462265527], [212.0, 0.2629359504866758]]}, \"id\": \"el2696140567942658320\"});\n",
       "            })\n",
       "         });\n",
       "}\n",
       "</script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import mpld3\n",
    "\n",
    "mpld3.enable_notebook()\n",
    "fig, ax = plt.subplots(subplot_kw=dict(axisbg='#EEEEEE'))\n",
    "\n",
    "somefig=ax.plot(clf4.coef_[0],'ro-')\n",
    "#x=range(len(allColumns))\n",
    "#plt.xticks(clf2.coef_[0],allColumns, rotation='vertical')\n",
    "\n",
    "#labels = ['point {0}'.format(i + 1) for i in range(N)]\n",
    "tooltip = mpld3.plugins.PointLabelTooltip(somefig[0], labels=Reduced4testCols)\n",
    "mpld3.plugins.connect(fig, tooltip)\n",
    "mpld3.display()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "    scores4 = cross_val_score(GradientBoostingClassifier2(n_estimators=40,random_state=1),\n",
    "                            X_concat2, Y, cv=KFold(len(X_concat2),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores4.mean()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "from sklearn.svm import SVC"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "from sklearn.preprocessing import MinMaxScaler"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "MinMaxScaler1 = MinMaxScaler()\n",
    "X_concat_sc = MinMaxScaler1.fit_transform(X_concat)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores5 = cross_val_score(SVC(random_state=1,cache_size=9000),\n",
    "                            X_concat_sc, Y, cv=KFold(len(X_concat2),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_concat2_sc_df=pandas.DataFrame(X_concat2_sc)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_concat2_sc_df.head()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_concat2_sc_df.shape"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Y.shape"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "X_concat2_sc_df_sample=X_concat2_sc_df.sample(10000)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Y_sample = Y.sample(10000)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "scores5 = cross_val_score(SVC(random_state=1,cache_size=9000),\n",
    "                            X_concat2_sc_df_sample, Y_sample, cv=KFold(len(X_concat2_sc_df_sample),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores5.mean()"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "from sklearn.ensemble import RandomForestClassifier"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores6 = cross_val_score(RandomForestClassifier(random_state=1,n_estimators=400),\n",
    "                            X_concat2_sc, Y, cv=KFold(len(X_concat2_sc),\n",
    "                            n_folds=5, shuffle=True, random_state=241),scoring='roc_auc')"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "scores6.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Добавление полиномиальных характеристик"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "poly = PolynomialFeatures(degree=2,interaction_only=True)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_concat.shape"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Xtr_s2_p=poly.fit_transform(Xtr_s2)\n",
    "#Xts_s2_p=poly.fit_transform(Xts_s2)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "#reset_selective X_concat3"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_tr_s2 = np.hstack((X_pick,Xtr_s2_p))\n",
    "#X_concat3_test = np.hstack((X_pick_test,Xts_s2_p))"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "X_tr_s2 = np.hstack((X_tr_s2,X_pick3))\n",
    "#X_concat3_test = np.hstack((X_concat3_test,X_pick3_test))"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "clf5=LogisticRegression(C=0.05,penalty='l2',random_state=1, n_jobs=-1)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "clf5.fit(X_tr_s2,Y)"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "trainReduced2.columns"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "PolyFeaturesolumns = ['r1_level','r1_xp','r1_gold','r1_lh',\n",
    "                      'r1_kills','r1_deaths','r1_items',\n",
    "                      'radiant_tpscroll_count','radiant_boots_count',\n",
    "                     'radiant_ward_observer_count','radiant_ward_sentry_count']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}