AI4H_practical3a.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-vXVsjNO_NZW"
      },
      "source": [
        "# Scikit-learn model training and evaluation example\n",
        "\n",
        "This notebook contains the code for two models predicting a binary outcome. This notebook is just meant as a first introduction towards the steps you go through when training models using scikit-learn. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2yHZPGOMxeri"
      },
      "source": [
        "We start with importing the relevant modules. Next to importing numpy and pandas, which you have worked with by now, we need different funcionalities of scikit-learn (sklearn), related to preprocessing the data, the models we use, model selection and metrics used to evaluate model performance."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "id": "2qB--SMu81rW"
      },
      "outputs": [],
      "source": [
        "#import relevant modules\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "\n",
        "from sklearn.model_selection import StratifiedShuffleSplit\n",
        "from sklearn.model_selection import RepeatedKFold\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "\n",
        "from sklearn.metrics import confusion_matrix\n",
        "from sklearn.metrics import ConfusionMatrixDisplay\n",
        "from sklearn.metrics import accuracy_score, precision_score"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z-XUSYKcynSN"
      },
      "source": [
        "### Importing and describing the data\n",
        "\n",
        "We work with a toy dataset, containing sonar data which can be used to classify objects as being either a Mine or a Rock. More information on the dataset can be found at: https://datahub.io/machine-learning/sonar."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "EtEXZweE9NPD"
      },
      "outputs": [],
      "source": [
        "df = pd.read_csv('https://datahub.io/machine-learning/sonar/r/sonar.csv')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "     attribute_1  attribute_2  attribute_3  attribute_4  attribute_5  \\\n",
            "0         0.0200       0.0371       0.0428       0.0207       0.0954   \n",
            "1         0.0453       0.0523       0.0843       0.0689       0.1183   \n",
            "2         0.0262       0.0582       0.1099       0.1083       0.0974   \n",
            "3         0.0100       0.0171       0.0623       0.0205       0.0205   \n",
            "4         0.0762       0.0666       0.0481       0.0394       0.0590   \n",
            "..           ...          ...          ...          ...          ...   \n",
            "203       0.0187       0.0346       0.0168       0.0177       0.0393   \n",
            "204       0.0323       0.0101       0.0298       0.0564       0.0760   \n",
            "205       0.0522       0.0437       0.0180       0.0292       0.0351   \n",
            "206       0.0303       0.0353       0.0490       0.0608       0.0167   \n",
            "207       0.0260       0.0363       0.0136       0.0272       0.0214   \n",
            "\n",
            "     attribute_6  attribute_7  attribute_8  attribute_9  attribute_10  ...  \\\n",
            "0         0.0986       0.1539       0.1601       0.3109        0.2111  ...   \n",
            "1         0.2583       0.2156       0.3481       0.3337        0.2872  ...   \n",
            "2         0.2280       0.2431       0.3771       0.5598        0.6194  ...   \n",
            "3         0.0368       0.1098       0.1276       0.0598        0.1264  ...   \n",
            "4         0.0649       0.1209       0.2467       0.3564        0.4459  ...   \n",
            "..           ...          ...          ...          ...           ...  ...   \n",
            "203       0.1630       0.2028       0.1694       0.2328        0.2684  ...   \n",
            "204       0.0958       0.0990       0.1018       0.1030        0.2154  ...   \n",
            "205       0.1171       0.1257       0.1178       0.1258        0.2529  ...   \n",
            "206       0.1354       0.1465       0.1123       0.1945        0.2354  ...   \n",
            "207       0.0338       0.0655       0.1400       0.1843        0.2354  ...   \n",
            "\n",
            "     attribute_52  attribute_53  attribute_54  attribute_55  attribute_56  \\\n",
            "0          0.0027        0.0065        0.0159        0.0072        0.0167   \n",
            "1          0.0084        0.0089        0.0048        0.0094        0.0191   \n",
            "2          0.0232        0.0166        0.0095        0.0180        0.0244   \n",
            "3          0.0121        0.0036        0.0150        0.0085        0.0073   \n",
            "4          0.0031        0.0054        0.0105        0.0110        0.0015   \n",
            "..            ...           ...           ...           ...           ...   \n",
            "203        0.0116        0.0098        0.0199        0.0033        0.0101   \n",
            "204        0.0061        0.0093        0.0135        0.0063        0.0063   \n",
            "205        0.0160        0.0029        0.0051        0.0062        0.0089   \n",
            "206        0.0086        0.0046        0.0126        0.0036        0.0035   \n",
            "207        0.0146        0.0129        0.0047        0.0039        0.0061   \n",
            "\n",
            "     attribute_57  attribute_58  attribute_59  attribute_60  Class  \n",
            "0          0.0180        0.0084        0.0090        0.0032   Rock  \n",
            "1          0.0140        0.0049        0.0052        0.0044   Rock  \n",
            "2          0.0316        0.0164        0.0095        0.0078   Rock  \n",
            "3          0.0050        0.0044        0.0040        0.0117   Rock  \n",
            "4          0.0072        0.0048        0.0107        0.0094   Rock  \n",
            "..            ...           ...           ...           ...    ...  \n",
            "203        0.0065        0.0115        0.0193        0.0157   Mine  \n",
            "204        0.0034        0.0032        0.0062        0.0067   Mine  \n",
            "205        0.0140        0.0138        0.0077        0.0031   Mine  \n",
            "206        0.0034        0.0079        0.0036        0.0048   Mine  \n",
            "207        0.0040        0.0036        0.0061        0.0115   Mine  \n",
            "\n",
            "[208 rows x 61 columns]\n"
          ]
        }
      ],
      "source": [
        "print(df)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 270
        },
        "id": "Dk9vp_Ezzd0u",
        "outputId": "d40c3b51-829a-45da-c012-5a66a27c83fe"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>attribute_1</th>\n",
              "      <th>attribute_2</th>\n",
              "      <th>attribute_3</th>\n",
              "      <th>attribute_4</th>\n",
              "      <th>attribute_5</th>\n",
              "      <th>attribute_6</th>\n",
              "      <th>attribute_7</th>\n",
              "      <th>attribute_8</th>\n",
              "      <th>attribute_9</th>\n",
              "      <th>attribute_10</th>\n",
              "      <th>...</th>\n",
              "      <th>attribute_52</th>\n",
              "      <th>attribute_53</th>\n",
              "      <th>attribute_54</th>\n",
              "      <th>attribute_55</th>\n",
              "      <th>attribute_56</th>\n",
              "      <th>attribute_57</th>\n",
              "      <th>attribute_58</th>\n",
              "      <th>attribute_59</th>\n",
              "      <th>attribute_60</th>\n",
              "      <th>Class</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.0200</td>\n",
              "      <td>0.0371</td>\n",
              "      <td>0.0428</td>\n",
              "      <td>0.0207</td>\n",
              "      <td>0.0954</td>\n",
              "      <td>0.0986</td>\n",
              "      <td>0.1539</td>\n",
              "      <td>0.1601</td>\n",
              "      <td>0.3109</td>\n",
              "      <td>0.2111</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0027</td>\n",
              "      <td>0.0065</td>\n",
              "      <td>0.0159</td>\n",
              "      <td>0.0072</td>\n",
              "      <td>0.0167</td>\n",
              "      <td>0.0180</td>\n",
              "      <td>0.0084</td>\n",
              "      <td>0.0090</td>\n",
              "      <td>0.0032</td>\n",
              "      <td>Rock</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.0453</td>\n",
              "      <td>0.0523</td>\n",
              "      <td>0.0843</td>\n",
              "      <td>0.0689</td>\n",
              "      <td>0.1183</td>\n",
              "      <td>0.2583</td>\n",
              "      <td>0.2156</td>\n",
              "      <td>0.3481</td>\n",
              "      <td>0.3337</td>\n",
              "      <td>0.2872</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0084</td>\n",
              "      <td>0.0089</td>\n",
              "      <td>0.0048</td>\n",
              "      <td>0.0094</td>\n",
              "      <td>0.0191</td>\n",
              "      <td>0.0140</td>\n",
              "      <td>0.0049</td>\n",
              "      <td>0.0052</td>\n",
              "      <td>0.0044</td>\n",
              "      <td>Rock</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.0262</td>\n",
              "      <td>0.0582</td>\n",
              "      <td>0.1099</td>\n",
              "      <td>0.1083</td>\n",
              "      <td>0.0974</td>\n",
              "      <td>0.2280</td>\n",
              "      <td>0.2431</td>\n",
              "      <td>0.3771</td>\n",
              "      <td>0.5598</td>\n",
              "      <td>0.6194</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0232</td>\n",
              "      <td>0.0166</td>\n",
              "      <td>0.0095</td>\n",
              "      <td>0.0180</td>\n",
              "      <td>0.0244</td>\n",
              "      <td>0.0316</td>\n",
              "      <td>0.0164</td>\n",
              "      <td>0.0095</td>\n",
              "      <td>0.0078</td>\n",
              "      <td>Rock</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.0100</td>\n",
              "      <td>0.0171</td>\n",
              "      <td>0.0623</td>\n",
              "      <td>0.0205</td>\n",
              "      <td>0.0205</td>\n",
              "      <td>0.0368</td>\n",
              "      <td>0.1098</td>\n",
              "      <td>0.1276</td>\n",
              "      <td>0.0598</td>\n",
              "      <td>0.1264</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0121</td>\n",
              "      <td>0.0036</td>\n",
              "      <td>0.0150</td>\n",
              "      <td>0.0085</td>\n",
              "      <td>0.0073</td>\n",
              "      <td>0.0050</td>\n",
              "      <td>0.0044</td>\n",
              "      <td>0.0040</td>\n",
              "      <td>0.0117</td>\n",
              "      <td>Rock</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.0762</td>\n",
              "      <td>0.0666</td>\n",
              "      <td>0.0481</td>\n",
              "      <td>0.0394</td>\n",
              "      <td>0.0590</td>\n",
              "      <td>0.0649</td>\n",
              "      <td>0.1209</td>\n",
              "      <td>0.2467</td>\n",
              "      <td>0.3564</td>\n",
              "      <td>0.4459</td>\n",
              "      <td>...</td>\n",
              "      <td>0.0031</td>\n",
              "      <td>0.0054</td>\n",
              "      <td>0.0105</td>\n",
              "      <td>0.0110</td>\n",
              "      <td>0.0015</td>\n",
              "      <td>0.0072</td>\n",
              "      <td>0.0048</td>\n",
              "      <td>0.0107</td>\n",
              "      <td>0.0094</td>\n",
              "      <td>Rock</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 61 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "   attribute_1  attribute_2  attribute_3  attribute_4  attribute_5  \\\n",
              "0       0.0200       0.0371       0.0428       0.0207       0.0954   \n",
              "1       0.0453       0.0523       0.0843       0.0689       0.1183   \n",
              "2       0.0262       0.0582       0.1099       0.1083       0.0974   \n",
              "3       0.0100       0.0171       0.0623       0.0205       0.0205   \n",
              "4       0.0762       0.0666       0.0481       0.0394       0.0590   \n",
              "\n",
              "   attribute_6  attribute_7  attribute_8  attribute_9  attribute_10  ...  \\\n",
              "0       0.0986       0.1539       0.1601       0.3109        0.2111  ...   \n",
              "1       0.2583       0.2156       0.3481       0.3337        0.2872  ...   \n",
              "2       0.2280       0.2431       0.3771       0.5598        0.6194  ...   \n",
              "3       0.0368       0.1098       0.1276       0.0598        0.1264  ...   \n",
              "4       0.0649       0.1209       0.2467       0.3564        0.4459  ...   \n",
              "\n",
              "   attribute_52  attribute_53  attribute_54  attribute_55  attribute_56  \\\n",
              "0        0.0027        0.0065        0.0159        0.0072        0.0167   \n",
              "1        0.0084        0.0089        0.0048        0.0094        0.0191   \n",
              "2        0.0232        0.0166        0.0095        0.0180        0.0244   \n",
              "3        0.0121        0.0036        0.0150        0.0085        0.0073   \n",
              "4        0.0031        0.0054        0.0105        0.0110        0.0015   \n",
              "\n",
              "   attribute_57  attribute_58  attribute_59  attribute_60  Class  \n",
              "0        0.0180        0.0084        0.0090        0.0032   Rock  \n",
              "1        0.0140        0.0049        0.0052        0.0044   Rock  \n",
              "2        0.0316        0.0164        0.0095        0.0078   Rock  \n",
              "3        0.0050        0.0044        0.0040        0.0117   Rock  \n",
              "4        0.0072        0.0048        0.0107        0.0094   Rock  \n",
              "\n",
              "[5 rows x 61 columns]"
            ]
          },
          "execution_count": 3,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# describing the dataframe - 1\n",
        "df.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "kkL61FFAzN39",
        "outputId": "e3329e44-9c11-482e-fb75-6fb989402e07"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "['attribute_1',\n",
              " 'attribute_2',\n",
              " 'attribute_3',\n",
              " 'attribute_4',\n",
              " 'attribute_5',\n",
              " 'attribute_6',\n",
              " 'attribute_7',\n",
              " 'attribute_8',\n",
              " 'attribute_9',\n",
              " 'attribute_10',\n",
              " 'attribute_11',\n",
              " 'attribute_12',\n",
              " 'attribute_13',\n",
              " 'attribute_14',\n",
              " 'attribute_15',\n",
              " 'attribute_16',\n",
              " 'attribute_17',\n",
              " 'attribute_18',\n",
              " 'attribute_19',\n",
              " 'attribute_20',\n",
              " 'attribute_21',\n",
              " 'attribute_22',\n",
              " 'attribute_23',\n",
              " 'attribute_24',\n",
              " 'attribute_25',\n",
              " 'attribute_26',\n",
              " 'attribute_27',\n",
              " 'attribute_28',\n",
              " 'attribute_29',\n",
              " 'attribute_30',\n",
              " 'attribute_31',\n",
              " 'attribute_32',\n",
              " 'attribute_33',\n",
              " 'attribute_34',\n",
              " 'attribute_35',\n",
              " 'attribute_36',\n",
              " 'attribute_37',\n",
              " 'attribute_38',\n",
              " 'attribute_39',\n",
              " 'attribute_40',\n",
              " 'attribute_41',\n",
              " 'attribute_42',\n",
              " 'attribute_43',\n",
              " 'attribute_44',\n",
              " 'attribute_45',\n",
              " 'attribute_46',\n",
              " 'attribute_47',\n",
              " 'attribute_48',\n",
              " 'attribute_49',\n",
              " 'attribute_50',\n",
              " 'attribute_51',\n",
              " 'attribute_52',\n",
              " 'attribute_53',\n",
              " 'attribute_54',\n",
              " 'attribute_55',\n",
              " 'attribute_56',\n",
              " 'attribute_57',\n",
              " 'attribute_58',\n",
              " 'attribute_59',\n",
              " 'attribute_60',\n",
              " 'Class']"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# describing the dataframe - 2\n",
        "list(df.columns)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 364
        },
        "id": "zta_Am-o-0PJ",
        "outputId": "caede051-ed25-4817-8352-20f722c532d6"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>attribute_1</th>\n",
              "      <th>attribute_2</th>\n",
              "      <th>attribute_3</th>\n",
              "      <th>attribute_4</th>\n",
              "      <th>attribute_5</th>\n",
              "      <th>attribute_6</th>\n",
              "      <th>attribute_7</th>\n",
              "      <th>attribute_8</th>\n",
              "      <th>attribute_9</th>\n",
              "      <th>attribute_10</th>\n",
              "      <th>...</th>\n",
              "      <th>attribute_51</th>\n",
              "      <th>attribute_52</th>\n",
              "      <th>attribute_53</th>\n",
              "      <th>attribute_54</th>\n",
              "      <th>attribute_55</th>\n",
              "      <th>attribute_56</th>\n",
              "      <th>attribute_57</th>\n",
              "      <th>attribute_58</th>\n",
              "      <th>attribute_59</th>\n",
              "      <th>attribute_60</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "      <td>208.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.029164</td>\n",
              "      <td>0.038437</td>\n",
              "      <td>0.043832</td>\n",
              "      <td>0.053892</td>\n",
              "      <td>0.075202</td>\n",
              "      <td>0.104570</td>\n",
              "      <td>0.121747</td>\n",
              "      <td>0.134799</td>\n",
              "      <td>0.178003</td>\n",
              "      <td>0.208259</td>\n",
              "      <td>...</td>\n",
              "      <td>0.016069</td>\n",
              "      <td>0.013420</td>\n",
              "      <td>0.010709</td>\n",
              "      <td>0.010941</td>\n",
              "      <td>0.009290</td>\n",
              "      <td>0.008222</td>\n",
              "      <td>0.007820</td>\n",
              "      <td>0.007949</td>\n",
              "      <td>0.007941</td>\n",
              "      <td>0.006507</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.022991</td>\n",
              "      <td>0.032960</td>\n",
              "      <td>0.038428</td>\n",
              "      <td>0.046528</td>\n",
              "      <td>0.055552</td>\n",
              "      <td>0.059105</td>\n",
              "      <td>0.061788</td>\n",
              "      <td>0.085152</td>\n",
              "      <td>0.118387</td>\n",
              "      <td>0.134416</td>\n",
              "      <td>...</td>\n",
              "      <td>0.012008</td>\n",
              "      <td>0.009634</td>\n",
              "      <td>0.007060</td>\n",
              "      <td>0.007301</td>\n",
              "      <td>0.007088</td>\n",
              "      <td>0.005736</td>\n",
              "      <td>0.005785</td>\n",
              "      <td>0.006470</td>\n",
              "      <td>0.006181</td>\n",
              "      <td>0.005031</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.001500</td>\n",
              "      <td>0.000600</td>\n",
              "      <td>0.001500</td>\n",
              "      <td>0.005800</td>\n",
              "      <td>0.006700</td>\n",
              "      <td>0.010200</td>\n",
              "      <td>0.003300</td>\n",
              "      <td>0.005500</td>\n",
              "      <td>0.007500</td>\n",
              "      <td>0.011300</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000800</td>\n",
              "      <td>0.000500</td>\n",
              "      <td>0.001000</td>\n",
              "      <td>0.000600</td>\n",
              "      <td>0.000400</td>\n",
              "      <td>0.000300</td>\n",
              "      <td>0.000300</td>\n",
              "      <td>0.000100</td>\n",
              "      <td>0.000600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.013350</td>\n",
              "      <td>0.016450</td>\n",
              "      <td>0.018950</td>\n",
              "      <td>0.024375</td>\n",
              "      <td>0.038050</td>\n",
              "      <td>0.067025</td>\n",
              "      <td>0.080900</td>\n",
              "      <td>0.080425</td>\n",
              "      <td>0.097025</td>\n",
              "      <td>0.111275</td>\n",
              "      <td>...</td>\n",
              "      <td>0.008425</td>\n",
              "      <td>0.007275</td>\n",
              "      <td>0.005075</td>\n",
              "      <td>0.005375</td>\n",
              "      <td>0.004150</td>\n",
              "      <td>0.004400</td>\n",
              "      <td>0.003700</td>\n",
              "      <td>0.003600</td>\n",
              "      <td>0.003675</td>\n",
              "      <td>0.003100</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.022800</td>\n",
              "      <td>0.030800</td>\n",
              "      <td>0.034300</td>\n",
              "      <td>0.044050</td>\n",
              "      <td>0.062500</td>\n",
              "      <td>0.092150</td>\n",
              "      <td>0.106950</td>\n",
              "      <td>0.112100</td>\n",
              "      <td>0.152250</td>\n",
              "      <td>0.182400</td>\n",
              "      <td>...</td>\n",
              "      <td>0.013900</td>\n",
              "      <td>0.011400</td>\n",
              "      <td>0.009550</td>\n",
              "      <td>0.009300</td>\n",
              "      <td>0.007500</td>\n",
              "      <td>0.006850</td>\n",
              "      <td>0.005950</td>\n",
              "      <td>0.005800</td>\n",
              "      <td>0.006400</td>\n",
              "      <td>0.005300</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.035550</td>\n",
              "      <td>0.047950</td>\n",
              "      <td>0.057950</td>\n",
              "      <td>0.064500</td>\n",
              "      <td>0.100275</td>\n",
              "      <td>0.134125</td>\n",
              "      <td>0.154000</td>\n",
              "      <td>0.169600</td>\n",
              "      <td>0.233425</td>\n",
              "      <td>0.268700</td>\n",
              "      <td>...</td>\n",
              "      <td>0.020825</td>\n",
              "      <td>0.016725</td>\n",
              "      <td>0.014900</td>\n",
              "      <td>0.014500</td>\n",
              "      <td>0.012100</td>\n",
              "      <td>0.010575</td>\n",
              "      <td>0.010425</td>\n",
              "      <td>0.010350</td>\n",
              "      <td>0.010325</td>\n",
              "      <td>0.008525</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>0.137100</td>\n",
              "      <td>0.233900</td>\n",
              "      <td>0.305900</td>\n",
              "      <td>0.426400</td>\n",
              "      <td>0.401000</td>\n",
              "      <td>0.382300</td>\n",
              "      <td>0.372900</td>\n",
              "      <td>0.459000</td>\n",
              "      <td>0.682800</td>\n",
              "      <td>0.710600</td>\n",
              "      <td>...</td>\n",
              "      <td>0.100400</td>\n",
              "      <td>0.070900</td>\n",
              "      <td>0.039000</td>\n",
              "      <td>0.035200</td>\n",
              "      <td>0.044700</td>\n",
              "      <td>0.039400</td>\n",
              "      <td>0.035500</td>\n",
              "      <td>0.044000</td>\n",
              "      <td>0.036400</td>\n",
              "      <td>0.043900</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>8 rows × 60 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "       attribute_1  attribute_2  attribute_3  attribute_4  attribute_5  \\\n",
              "count   208.000000   208.000000   208.000000   208.000000   208.000000   \n",
              "mean      0.029164     0.038437     0.043832     0.053892     0.075202   \n",
              "std       0.022991     0.032960     0.038428     0.046528     0.055552   \n",
              "min       0.001500     0.000600     0.001500     0.005800     0.006700   \n",
              "25%       0.013350     0.016450     0.018950     0.024375     0.038050   \n",
              "50%       0.022800     0.030800     0.034300     0.044050     0.062500   \n",
              "75%       0.035550     0.047950     0.057950     0.064500     0.100275   \n",
              "max       0.137100     0.233900     0.305900     0.426400     0.401000   \n",
              "\n",
              "       attribute_6  attribute_7  attribute_8  attribute_9  attribute_10  ...  \\\n",
              "count   208.000000   208.000000   208.000000   208.000000    208.000000  ...   \n",
              "mean      0.104570     0.121747     0.134799     0.178003      0.208259  ...   \n",
              "std       0.059105     0.061788     0.085152     0.118387      0.134416  ...   \n",
              "min       0.010200     0.003300     0.005500     0.007500      0.011300  ...   \n",
              "25%       0.067025     0.080900     0.080425     0.097025      0.111275  ...   \n",
              "50%       0.092150     0.106950     0.112100     0.152250      0.182400  ...   \n",
              "75%       0.134125     0.154000     0.169600     0.233425      0.268700  ...   \n",
              "max       0.382300     0.372900     0.459000     0.682800      0.710600  ...   \n",
              "\n",
              "       attribute_51  attribute_52  attribute_53  attribute_54  attribute_55  \\\n",
              "count    208.000000    208.000000    208.000000    208.000000    208.000000   \n",
              "mean       0.016069      0.013420      0.010709      0.010941      0.009290   \n",
              "std        0.012008      0.009634      0.007060      0.007301      0.007088   \n",
              "min        0.000000      0.000800      0.000500      0.001000      0.000600   \n",
              "25%        0.008425      0.007275      0.005075      0.005375      0.004150   \n",
              "50%        0.013900      0.011400      0.009550      0.009300      0.007500   \n",
              "75%        0.020825      0.016725      0.014900      0.014500      0.012100   \n",
              "max        0.100400      0.070900      0.039000      0.035200      0.044700   \n",
              "\n",
              "       attribute_56  attribute_57  attribute_58  attribute_59  attribute_60  \n",
              "count    208.000000    208.000000    208.000000    208.000000    208.000000  \n",
              "mean       0.008222      0.007820      0.007949      0.007941      0.006507  \n",
              "std        0.005736      0.005785      0.006470      0.006181      0.005031  \n",
              "min        0.000400      0.000300      0.000300      0.000100      0.000600  \n",
              "25%        0.004400      0.003700      0.003600      0.003675      0.003100  \n",
              "50%        0.006850      0.005950      0.005800      0.006400      0.005300  \n",
              "75%        0.010575      0.010425      0.010350      0.010325      0.008525  \n",
              "max        0.039400      0.035500      0.044000      0.036400      0.043900  \n",
              "\n",
              "[8 rows x 60 columns]"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# describing the dataframe - 3\n",
        "df.describe()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "erU0PeC1-7Px",
        "outputId": "e1b9509d-36c7-41b0-a583-bea025460bd9"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(208, 61)"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# describing the dataframe - 4\n",
        "df.shape"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vw3o5M_t-hsO",
        "outputId": "54754270-ad7a-43ad-9896-6bb706be9845"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "Class\n",
              "Mine    111\n",
              "Rock     97\n",
              "Name: count, dtype: int64"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# exploring outcome data\n",
        "df['Class'].value_counts()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P3iDyjSh_Eme"
      },
      "source": [
        "We split the data in a train and test set, while ensuring an equal distribution of the outcome variable in both sets."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "iDRA7xvD_HUT"
      },
      "outputs": [],
      "source": [
        "split = StratifiedShuffleSplit(n_splits=1, test_size=0.3, random_state=42)\n",
        "for train_index, test_index in split.split(df, df['Class']):\n",
        "  strat_train_set = df.loc[train_index]\n",
        "  strat_test_set = df.loc[test_index]"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2W6ShGz-0NvL"
      },
      "source": [
        "### Question 1\n",
        "- Check whether splitting the data indeed resulted in 70% of the data being contained in the train set and 30% in the test set. \n",
        "- Check whether both datasets contain an equal distribution of mines vs. rocks.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "0.6971153846153846\n"
          ]
        }
      ],
      "source": [
        "print(strat_train_set.shape[0]/(strat_test_set.shape[0] + strat_train_set.shape[0]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Class\n",
            "Mine    77\n",
            "Rock    68\n",
            "Name: count, dtype: int64 Class\n",
            "Mine    34\n",
            "Rock    29\n",
            "Name: count, dtype: int64\n"
          ]
        }
      ],
      "source": [
        "print(strat_train_set['Class'].value_counts(),\n",
        "strat_test_set['Class'].value_counts())"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FRKJccRY1wzP"
      },
      "source": [
        "We now split the features from the outcome data for both the train and the test set, to align with the input needed when using sklearn to train models."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "id": "SSqWxbi24D5v"
      },
      "outputs": [],
      "source": [
        "# split features and outcome data\n",
        "X_train = strat_train_set.loc[:,strat_train_set.columns != 'Class']\n",
        "y_train = strat_train_set['Class']\n",
        "X_test = strat_test_set.loc[:,strat_test_set.columns != 'Class']\n",
        "y_test = strat_test_set['Class']\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BT7luOepCXHP"
      },
      "source": [
        "## Model 1: Logistic Regression"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RqxP39W92oWT"
      },
      "source": [
        "We start with running a basic logistic regression, which only requires two lines of code: "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yCYN0Cqmeq2q",
        "outputId": "bdcd74ae-a8e0-4626-eda1-e9e55992ace1"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/Users/junda_huang/anaconda3/envs/ai4h_dl/lib/python3.11/site-packages/sklearn/linear_model/_logistic.py:1173: FutureWarning: `penalty='none'`has been deprecated in 1.2 and will be removed in 1.4. To keep the past behaviour, set `penalty=None`.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LogisticRegression(max_iter=1000, penalty=&#x27;none&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression(max_iter=1000, penalty=&#x27;none&#x27;)</pre></div></div></div></div></div>"
            ],
            "text/plain": [
              "LogisticRegression(max_iter=1000, penalty='none')"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "log_clf = LogisticRegression(penalty='none', max_iter=1000)\n",
        "log_clf.fit(X_train, y_train)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ae7hZzBJ3vkA"
      },
      "source": [
        "### Question 2\n",
        "- look into the documentation for sklearn LogisticRegression to find out what the penalty and max_iter arguments stand for\n",
        "- see what happens when you set max_iter to its default value"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/html": [
              "<style>#sk-container-id-2 {color: black;background-color: white;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LogisticRegression()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" checked><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression()</pre></div></div></div></div></div>"
            ],
            "text/plain": [
              "LogisticRegression()"
            ]
          },
          "execution_count": 21,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "log_clf = LogisticRegression()\n",
        "log_clf.fit(X_train, y_train)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "arFlB5Zn5vsP"
      },
      "source": [
        "### Model evaluation \n",
        "We can now apply our model to the test set and compare predictions to actual outcomes."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 301
        },
        "id": "l1p3u0k-DwzS",
        "outputId": "cf834e9c-7bb8-4153-d549-4a97e45b9c0f"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<sklearn.metrics._plot.confusion_matrix.ConfusionMatrixDisplay at 0x13d885010>"
            ]
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhgAAAGwCAYAAADrIxwOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABDp0lEQVR4nO3deXxU5dn/8e+EZRJCMhggG4SwCoLIDgH8sWgFggu4AUJZhMojm1AUKFUEaiVSBRFUal0S2rK1AhI3BMoSUaCyBFwQQYMJQoQqEBLIOuf3B2X0SAgZ5g5Z+nk/r/N6nHPOfc81aV7Mleu6zzkOy7IsAQAAGORX2gEAAICKhwQDAAAYR4IBAACMI8EAAADGkWAAAADjSDAAAIBxJBgAAMC4yqUdQFnndrt17NgxBQUFyeFwlHY4AAAvWJals2fPKjIyUn5+Jfc3dXZ2tnJzc43MVbVqVfn7+xuZqzSRYFzBsWPHFBUVVdphAAB8kJaWprp165bI3NnZ2WoQXV3pJwqMzBceHq6UlJRyn2SQYFxBUFCQJOnbPfUVXJ2OEiqmu69vWdohACUiX3napvc8/5aXhNzcXKWfKNC3u+srOMi374mMs25Ftzui3NxcEoyK7mJbJLi6n8+/OEBZVdlRpbRDAErGfx+GcS1a3NWDHKoe5Nv7uFVxWvEkGAAAGFBguVXg49O9Ciy3mWDKABIMAAAMcMuSW75lGL6OL0uo+QMAUA7FxcWpQ4cOCgoKUmhoqPr376+DBw/aznE4HIVuzz777GXnTUhIKHRMdna2V/GRYAAAYIDb0P8V19atWzVu3Djt2LFDGzZsUH5+vnr16qWsrCzPOcePH7dtb7zxhhwOh+69994i5w4ODr5krLeLTmmRAABgQIFlqcDyrcXhzfh169bZXsfHxys0NFS7d+9Wt27dJF245PXn1q5dq549e6phw4ZFzu1wOC4Z6y0qGAAAlDEZGRm2LScn54pjzpw5I0kKCQkp9Pj333+vd999V6NGjbriXJmZmYqOjlbdunV1xx13aO/evd59AJFgAABgxMVFnr5ukhQVFSWXy+XZ4uLiinxvy7I0efJk3XzzzbrxxhsLPWfJkiUKCgrSPffcU+RczZo1U0JCghITE7V8+XL5+/ura9euOnTokFc/D1okAAAY4JalAkNXkaSlpSk4ONiz3+l0Fjlu/Pjx2r9/v7Zt23bZc9544w0NGTLkimspYmJiFBMT43ndtWtXtW3bVosWLdLChQuL8zEkkWAAAFDmBAcH2xKMokyYMEGJiYlKSkq67O3QP/zwQx08eFArV670OhY/Pz916NDB6woGLRIAAAww2SIpDsuyNH78eK1evVqbNm1SgwYNLnvu66+/rnbt2qlVq1Zefy7LspScnKyIiAivxlHBAADAgGt9Fcm4ceO0bNkyrV27VkFBQUpPT5ckuVwuBQQEeM7LyMjQP//5T82bN6/QeYYNG6Y6dep41nnMnj1bMTExatKkiTIyMrRw4UIlJyfrpZde8uqzkGAAAFAOLV68WJLUo0cP2/74+HiNGDHC83rFihWyLEsPPPBAofOkpqbaHmV/+vRpjR49Wunp6XK5XGrTpo2SkpLUsWNHr+JzWJaP6VYFl5GRIZfLpVNfNeRhZ6iweke2Lu0QgBKRb+Vpi9bqzJkzxV7T4K2L3xNfHghTkI/fE2fPutXshu9LNN5rhQoGAAAGFBi4isTX8WUJCQYAAAYUWDLwNFUzsZQF1PwBAIBxVDAAADDA/d/N1zkqChIMAAAMcMuhAjl8nqOioEUCAACMo4IBAIABbuvC5uscFQUJBgAABhQYaJH4Or4soUUCAACMo4IBAIABVDDsSDAAADDAbTnktny8isTH8WUJLRIAAGAcFQwAAAygRWJHggEAgAEF8lOBj42BAkOxlAUkGAAAGGAZWINhsQYDAADg8qhgAABgAGsw7EgwAAAwoMDyU4Hl4xqMCnSrcFokAADAOCoYAAAY4JZDbh//bner4pQwSDAAADCANRh2tEgAAIBxVDAAADDAzCJPWiQAAOBnLqzB8PFhZ7RIAAAALo8KBgAABrgNPIuEq0gAAIANazDsSDAAADDALT/ug/EzrMEAAADGUcEAAMCAAsuhAh8ft+7r+LKEBAMAAAMKDCzyLKBFAgAASlNcXJw6dOigoKAghYaGqn///jp48KDtnBEjRsjhcNi2mJiYK869atUqNW/eXE6nU82bN9eaNWu8jo8EAwAAA9yWn5GtuLZu3apx48Zpx44d2rBhg/Lz89WrVy9lZWXZzuvTp4+OHz/u2d57770i592+fbsGDhyooUOHat++fRo6dKgGDBignTt3evXzoEUCAIAB17pFsm7dOtvr+Ph4hYaGavfu3erWrZtnv9PpVHh4eLHnXbBggW677TZNnz5dkjR9+nRt3bpVCxYs0PLly4s9DxUMAADKmIyMDNuWk5NzxTFnzpyRJIWEhNj2b9myRaGhobr++uv10EMP6cSJE0XOs337dvXq1cu2r3fv3vr444+9+gwkGAAAGODWT1eSXO3m/u9cUVFRcrlcni0uLq7I97YsS5MnT9bNN9+sG2+80bM/NjZWS5cu1aZNmzRv3jx98sknuuWWW4pMWNLT0xUWFmbbFxYWpvT0dK9+HrRIAAAwwMyNti6MT0tLU3BwsGe/0+ksctz48eO1f/9+bdu2zbZ/4MCBnv++8cYb1b59e0VHR+vdd9/VPffcc9n5HA775bKWZV2y70pIMAAAKGOCg4NtCUZRJkyYoMTERCUlJalu3bpFnhsREaHo6GgdOnTosueEh4dfUq04ceLEJVWNK6FFAgCAARefReLrVlyWZWn8+PFavXq1Nm3apAYNGlxxzA8//KC0tDRFRERc9pzOnTtrw4YNtn3r169Xly5dih2bRAUDAAAj3HLILd/uxOnN+HHjxmnZsmVau3atgoKCPFUHl8ulgIAAZWZmatasWbr33nsVERGhI0eO6Pe//71q1aqlu+++2zPPsGHDVKdOHc86j4kTJ6pbt26aO3eu+vXrp7Vr12rjxo2XtF+uhAQDAAADzDxNtfjjFy9eLEnq0aOHbX98fLxGjBihSpUq6dNPP9Vf//pXnT59WhEREerZs6dWrlypoKAgz/mpqany8/vpfbt06aIVK1boiSee0IwZM9SoUSOtXLlSnTp18uqzkGAAAFAOWVd4tHtAQIA++OCDK86zZcuWS/bdd999uu+++642NEkkGAAAGGHmRlsVZ2kkCQYAAAa4LYfcPj4N1dfxZUnFSZUAAECZQQUDAAAD3AZaJL7eqKssIcEAAMAAb5+Gerk5KoqK80kAAECZQQUDAAADCuRQgY832vJ1fFlCggEAgAG0SOwqzicBAABlBhUMAAAMKJDvLY4CM6GUCSQYAAAYQIvEjgQDAAADrvXDzsq6ivNJAABAmUEFAwAAAyw55PZxDYbFZaoAAODnaJHYVZxPAgAAygwqGAAAGMDj2u1IMAAAMKDAwNNUfR1fllScTwIAAMoMKhgAABhAi8SOBAMAAAPc8pPbx8aAr+PLkorzSQAAQJlBBQMAAAMKLIcKfGxx+Dq+LCHBAADAANZg2JFgAABggGXgaaoWd/IEAAC4PCoYAAAYUCCHCnx8WJmv48sSEgwAAAxwW76voXBbhoIpA2iRAAAA46hg4JpbsShUH71XQ2mHnarq71bz9uc06vFjimqc4znnfJafXn86Qts/cCnjVGWF1c1Vv1EndefwH0oxcuDq1QzP06jHj6lDz7OqGuDWd984NX9ylA5/Wq20Q4MhbgOLPH0dX5aUmwSjR48eat26tRYsWFDaocBH+7dX150j/qPrW59TQb6UMDdCv3+gkV7d+qX8q7klSX+eWUf7Pq6uqYtSFRaVqz1bg7Roel3VDMtTlz4ZpfwJAO9Ud+Vr/tpD2v9xdT3x64Y6/Z/Kiqifo6yMSqUdGgxyyyG3j2sofB1flpRqqjRixAg5HA49/PDDlxwbO3asHA6HRowYIUlavXq1nnrqqWscIUrCnGXfqNfAH1W/abYatcjWo8+n6sR3VXVof4DnnAO7q+m2+39Uqy6ZCo/KVd9f/6CGzc/r0H7+2kP5M2DcCf3nWFXN+209HUyupu+PVlXytiAd/9ZZ2qGhHIuLi1OHDh0UFBSk0NBQ9e/fXwcPHvQcz8vL07Rp09SyZUsFBgYqMjJSw4YN07Fjx4qcNyEhQQ6H45ItOzvbq/hKvRYTFRWlFStW6Pz585592dnZWr58uerVq+fZFxISoqCgoNIIESXs4l9xQTUKPPtadMzSjvUu/ed4FVmWlPxRdX33jVPtup8trTCBqxbTK0Nf7QvQ468c0cr9n+ul9QcVO5h2X0Vz8U6evm7FtXXrVo0bN047duzQhg0blJ+fr169eikrK0uSdO7cOe3Zs0czZszQnj17tHr1an311Ve66667rjh3cHCwjh8/btv8/f29+nmUeoukbdu2+uabb7R69WoNGTJE0oVqRVRUlBo2bOg575ctkvr162v06NE6fPiw/vnPf+q6667TE088odGjR3vGfPfdd5o8ebLWr18vPz8/3XzzzXrhhRdUv379a/kRUQTLkv4yq45adMxU/WY/Zcdjn/pOC6ZEaUi7FqpU2ZKfn6VJz6Xpxk5ZpRgtcHUi6uXqjmE/aPVfamvFolA1bX1eY576Tnm5Dm18M6S0w4Mh13oNxrp162yv4+PjFRoaqt27d6tbt25yuVzasGGD7ZxFixapY8eOSk1Ntf0R/0sOh0Ph4eHeBf8LpV7BkKQHH3xQ8fHxntdvvPGGRo4cecVx8+bNU/v27bV3716NHTtWY8aM0ZdffinpQubWs2dPVa9eXUlJSdq2bZuqV6+uPn36KDc397Jz5uTkKCMjw7ah5Lz0+zpKORCg6S9/a9v/1uu19OXuapqd8I1eXHdQDz15TC9Or6s9SdVLKVLg6jn8pMOfBSj+mQh9/Vk1vff3mnp/WU3dPowqBgr3y++hnJycK445c+aMpAsV/6LOcTgcqlGjRpFzZWZmKjo6WnXr1tUdd9yhvXv3ehW/VEYSjKFDh2rbtm06cuSIvv32W3300Uf69a9/fcVxffv21dixY9W4cWNNmzZNtWrV0pYtWyRJK1askJ+fn1577TW1bNlSN9xwg+Lj45Wamuo5pzBxcXFyuVyeLSoqytCnxC+99HgdbV/v0p/ePKzakXme/TnnHUp4JkKjZx1TTK8MNWyerX4j/6Pud53Wm38OLcWIgavz44nK+vYre3k57ZBToXUu/8cOyh+3HJ7nkVz19t9FnlFRUbbvori4uCLf27IsTZ48WTfffLNuvPHGQs/Jzs7W7373Ow0ePFjBwcGXnatZs2ZKSEhQYmKili9fLn9/f3Xt2lWHDh3y6udR6i0SSapVq5Zuv/12LVmyRJZl6fbbb1etWrWuOO6mm27y/PfFcs6JEyckSbt379bhw4cvWbeRnZ2tr7/++rJzTp8+XZMnT/a8zsjIIMkwzLIuJBcfr3Pp2TcPK7ye/R/Z/HyH8vP85Odnv+OMXyVLlvtaRgqY8cUngYpqZP8LtE7DHJ34rmopRYSSYBm4isT67/i0tDRbEuB0Fr0gePz48dq/f7+2bdtW6PG8vDwNGjRIbrdbL7/8cpFzxcTEKCYmxvO6a9euatu2rRYtWqSFCxcW96OUjQRDkkaOHKnx48dLkl566aVijalSpYrttcPhkNt94RvI7XarXbt2Wrp06SXjateufdk5nU7nFf+HhG9e/H1dbV5znWbFf6OA6m79eOLCr2FgUIGcAZYCg9y6qXOmXn0qUlX9v1NY3Vzt315dG98M0eiZ35Vy9ID3Vv+ltp5PPKRBE75X0ts11LTNOfX99Y9aMKVuaYcGg0w+TTU4OLjIKsPPTZgwQYmJiUpKSlLdupf+TuXl5WnAgAFKSUnRpk2bij3vRX5+furQoUP5rGBIsq2N6N27t8/ztW3bVitXrlRoaKjXP0yUrHeWXKhOTbm3iW3/o8+nqtfAHyVJ0xcf0RtzIjR3fD2dPV1ZoXVyNWLacd1Bzxrl0Ff7qukPoxrowenHNeS33ys9rar+/GSkNq+5rrRDQzlmWZYmTJigNWvWaMuWLWrQoMEl51xMLg4dOqTNmzerZs2aV/U+ycnJatmypVfjykyCUalSJR04cMDz374aMmSInn32WfXr109/+MMfVLduXaWmpmr16tWaMmVKoVkero0PjiVf8ZyQ0Hw9tiCt5IMBrpGdG4O1cyN/7FRk1/oqknHjxmnZsmVau3atgoKClJ6eLklyuVwKCAhQfn6+7rvvPu3Zs0fvvPOOCgoKPOeEhISoatULLbphw4apTp06nnUes2fPVkxMjJo0aaKMjAwtXLhQycnJxe4uXFRmEgxJRisN1apVU1JSkqZNm6Z77rlHZ8+eVZ06dXTrrbdS0QAAGGeyRVIcixcvlnThNg4/Fx8frxEjRujo0aNKTEyUJLVu3dp2zubNmz3jUlNT5ef3U2Jz+vRpjR49Wunp6XK5XGrTpo2SkpLUsWNHrz6Lw7KsCvTsNvMyMjLkcrl06quGCg4qExfdAMb1jmxd2iEAJSLfytMWrdWZM2dK7I/Li98T/daPVJVA3xbu5mXlam2vN0o03mulTFUwAAAor3gWiR0JBgAABlzrFklZR80fAAAYRwUDAAADqGDYkWAAAGAACYYdLRIAAGAcFQwAAAyggmFHggEAgAGWfL/MtCLdmIoEAwAAA6hg2LEGAwAAGEcFAwAAA6hg2JFgAABgAAmGHS0SAABgHBUMAAAMoIJhR4IBAIABluWQ5WOC4Ov4soQWCQAAMI4KBgAABrjl8PlGW76OL0tIMAAAMIA1GHa0SAAAgHFUMAAAMIBFnnYkGAAAGECLxI4EAwAAA6hg2LEGAwAAGEcFAwAAAywDLZKKVMEgwQAAwABLkmX5PkdFQYsEAAAYRwUDAAAD3HLIwZ08PUgwAAAwgKtI7GiRAAAA46hgAABggNtyyMGNtjxIMAAAMMCyDFxFUoEuI6FFAgBAORQXF6cOHTooKChIoaGh6t+/vw4ePGg7x7IszZo1S5GRkQoICFCPHj30+eefX3HuVatWqXnz5nI6nWrevLnWrFnjdXwkGAAAGHBxkaevW3Ft3bpV48aN044dO7Rhwwbl5+erV69eysrK8pzzpz/9SfPnz9eLL76oTz75ROHh4brtttt09uzZy867fft2DRw4UEOHDtW+ffs0dOhQDRgwQDt37vTq5+GwrIpUkDEvIyNDLpdLp75qqOAg8jFUTL0jW5d2CECJyLfytEVrdebMGQUHB5fIe1z8nrhh+TRVqub0aa6Cczk68MDcq4r35MmTCg0N1datW9WtWzdZlqXIyEhNmjRJ06ZNkyTl5OQoLCxMc+fO1f/93/8VOs/AgQOVkZGh999/37OvT58+uu6667R8+fJix8M3JgAABlx8mqqvm3Qhafn5lpOTc8X3P3PmjCQpJCREkpSSkqL09HT16tXLc47T6VT37t318ccfX3ae7du328ZIUu/evYscUxgSDAAAypioqCi5XC7PFhcXV+T5lmVp8uTJuvnmm3XjjTdKktLT0yVJYWFhtnPDwsI8xwqTnp7u9ZjCcBUJAAAGmLyKJC0tzdYicTqLbr2MHz9e+/fv17Zt2y455nDY13VYlnXJPhNjfokEAwAAAy4kGL7eyfPC/w8ODi72GowJEyYoMTFRSUlJqlu3rmd/eHi4pAsViYiICM/+EydOXFKh+Lnw8PBLqhVXGlMYWiQAAJRDlmVp/PjxWr16tTZt2qQGDRrYjjdo0EDh4eHasGGDZ19ubq62bt2qLl26XHbezp0728ZI0vr164scUxgqGAAAGHCtn0Uybtw4LVu2TGvXrlVQUJCn6uByuRQQECCHw6FJkyZpzpw5atKkiZo0aaI5c+aoWrVqGjx4sGeeYcOGqU6dOp51HhMnTlS3bt00d+5c9evXT2vXrtXGjRsLbb8UhQQDAAADrP9uvs5RXIsXL5Yk9ejRw7Y/Pj5eI0aMkCRNnTpV58+f19ixY3Xq1Cl16tRJ69evV1BQkOf81NRU+fn91NDo0qWLVqxYoSeeeEIzZsxQo0aNtHLlSnXq1Mmrz8J9MK6A+2DgfwH3wUBFdS3vg9Hob9NVqZq/T3MVnMvW10PjSjTea4UKBgAABvC4djsSDAAATLjWPZIyjgQDAAATDFQwVIEqGCwqAAAAxlHBAADAAJN38qwISDAAADCARZ52tEgAAIBxVDAAADDBcvi+SLMCVTBIMAAAMIA1GHa0SAAAgHFUMAAAMIEbbdmQYAAAYABXkdgVK8FYuHBhsSd85JFHrjoYAABQMRQrwXj++eeLNZnD4SDBAAD876pALQ5fFSvBSElJKek4AAAo12iR2F31VSS5ubk6ePCg8vPzTcYDAED5ZBnaKgivE4xz585p1KhRqlatmlq0aKHU1FRJF9ZePPPMM8YDBAAA5Y/XCcb06dO1b98+bdmyRf7+/p79v/rVr7Ry5UqjwQEAUH44DG0Vg9eXqb711ltauXKlYmJi5HD89INo3ry5vv76a6PBAQBQbnAfDBuvKxgnT55UaGjoJfuzsrJsCQcAAPjf5XWC0aFDB7377rue1xeTildffVWdO3c2FxkAAOUJizxtvG6RxMXFqU+fPvriiy+Un5+vF154QZ9//rm2b9+urVu3lkSMAACUfTxN1cbrCkaXLl300Ucf6dy5c2rUqJHWr1+vsLAwbd++Xe3atSuJGAEAQDlzVc8iadmypZYsWWI6FgAAyi0e1253VQlGQUGB1qxZowMHDsjhcOiGG25Qv379VLkyz04DAPyP4ioSG68zgs8++0z9+vVTenq6mjZtKkn66quvVLt2bSUmJqply5bGgwQAAOWL12swfvOb36hFixY6evSo9uzZoz179igtLU033XSTRo8eXRIxAgBQ9l1c5OnrVkF4XcHYt2+fdu3apeuuu86z77rrrtPTTz+tDh06GA0OAIDywmFd2Hydo6LwuoLRtGlTff/995fsP3HihBo3bmwkKAAAyh3ug2FTrAQjIyPDs82ZM0ePPPKI3nzzTR09elRHjx7Vm2++qUmTJmnu3LklHS8AACgHitUiqVGjhu024JZlacCAAZ591n+vq7nzzjtVUFBQAmECAFDGcaMtm2IlGJs3by7pOAAAKN9K4TLVpKQkPfvss9q9e7eOHz+uNWvWqH///p7jl3tG2J/+9CdNmTKl0GMJCQl68MEHL9l//vx521PUr6RYCUb37t2LPSEAALg2srKy1KpVKz344IO69957Lzl+/Phx2+v3339fo0aNKvTcnwsODtbBgwdt+7xJLqSrvNGWJJ07d06pqanKzc217b/pppuudkoAAMqvUqhgxMbGKjY29rLHw8PDba/Xrl2rnj17qmHDhkXO63A4LhnrLa8TjJMnT+rBBx/U+++/X+hx1mAAAP4nGUwwMjIybLudTqecTqdPU3///fd69913i/Woj8zMTEVHR6ugoECtW7fWU089pTZt2nj1fl5fpjpp0iSdOnVKO3bsUEBAgNatW6clS5aoSZMmSkxM9HY6AADwC1FRUXK5XJ4tLi7O5zmXLFmioKAg3XPPPUWe16xZMyUkJCgxMVHLly+Xv7+/unbtqkOHDnn1fl5XMDZt2qS1a9eqQ4cO8vPzU3R0tG677TYFBwcrLi5Ot99+u7dTAgBQ/hm8iiQtLU3BwcGe3b5WLyTpjTfe0JAhQ664liImJkYxMTGe1127dlXbtm21aNEiLVy4sNjv53WCkZWVpdDQUElSSEiITp48qeuvv14tW7bUnj17vJ0OAIAKweSdPIODg20Jhq8+/PBDHTx4UCtXrvR6rJ+fnzp06OB1BeOq7uR5cWVp69at9corr+i7777Tn//8Z0VERHg7HQAAKGGvv/662rVrp1atWnk91rIsJScne/0d73UFY9KkSZ7LXmbOnKnevXtr6dKlqlq1qhISErydDgCAiqEUriLJzMzU4cOHPa9TUlKUnJyskJAQ1atXT9KFBaP//Oc/NW/evELnGDZsmOrUqeNZ5zF79mzFxMSoSZMmysjI0MKFC5WcnKyXXnrJq9i8TjCGDBni+e82bdroyJEj+vLLL1WvXj3VqlXL2+kAAMBV2rVrl3r27Ol5PXnyZEnS8OHDPX/0r1ixQpZl6YEHHih0jtTUVPn5/dTQOH36tEaPHq309HS5XC61adNGSUlJ6tixo1exOayL9/lGoTIyMuRyuXTqq4YKDvK6owSUC70jW5d2CECJyLfytEVrdebMGaNrGn7u4vdE9Nw/ys/Lm1H9kjs7W99Oe6JE471WilXBuJgRFcf8+fOvOhgAAFAxFCvB2Lt3b7Emu9w9zyuCe++5V5Ur+X6ZEFAWHVtDdQ4VU8G5HGnw2mvzZjzszIaHnQEAYEIpLPIsy/izBQAAGHfVDzsDAAA/QwXDhgQDAAADTN7JsyKgRQIAAIyjggEAgAm0SGyuqoLxt7/9TV27dlVkZKS+/fZbSdKCBQu0du01uhQIAICyxjK0VRBeJxiLFy/W5MmT1bdvX50+fVoFBQWSpBo1amjBggWm4wMAAOWQ1wnGokWL9Oqrr+rxxx9XpUqVPPvbt2+vTz/91GhwAACUFxcXefq6VRRer8FISUlRmzZtLtnvdDqVlZVlJCgAAMod7uRp43UFo0GDBkpOTr5k//vvv6/mzZubiAkAgPKHNRg2XlcwpkyZonHjxik7O1uWZenf//63li9frri4OL322mslESMAAChnvE4wHnzwQeXn52vq1Kk6d+6cBg8erDp16uiFF17QoEGDSiJGAADKPG60ZXdV98F46KGH9NBDD+k///mP3G63QkNDTccFAED5wn0wbHy60VatWrVMxQEAACoQrxOMBg0ayOG4/CrXb775xqeAAAAol0xcZvq/XMGYNGmS7XVeXp727t2rdevWacqUKabiAgCgfKFFYuN1gjFx4sRC97/00kvatWuXzwEBAIDyz9jTVGNjY7Vq1SpT0wEAUL5wHwwbY09TffPNNxUSEmJqOgAAyhUuU7XzOsFo06aNbZGnZVlKT0/XyZMn9fLLLxsNDgAAlE9eJxj9+/e3vfbz81Pt2rXVo0cPNWvWzFRcAACgHPMqwcjPz1f9+vXVu3dvhYeHl1RMAACUP1xFYuPVIs/KlStrzJgxysnJKal4AAAol3hcu53XV5F06tRJe/fuLYlYAABABeH1GoyxY8fq0Ucf1dGjR9WuXTsFBgbajt90003GggMAoFypQBUIXxU7wRg5cqQWLFiggQMHSpIeeeQRzzGHwyHLsuRwOFRQUGA+SgAAyjrWYNgUO8FYsmSJnnnmGaWkpJRkPAAAoAIodoJhWRfSqujo6BILBgCA8oobbdl5tcizqKeoAgDwP60UbhWelJSkO++8U5GRkXI4HHrrrbdsx0eMGCGHw2HbYmJirjjvqlWr1Lx5czmdTjVv3lxr1qzxLjB5mWBcf/31CgkJKXIDAADXRlZWllq1aqUXX3zxsuf06dNHx48f92zvvfdekXNu375dAwcO1NChQ7Vv3z4NHTpUAwYM0M6dO72KzaurSGbPni2Xy+XVGwAA8L+gNFoksbGxio2NLfIcp9Pp1c0xFyxYoNtuu03Tp0+XJE2fPl1bt27VggULtHz58mLP41WCMWjQIIWGhnozBACA/w0GryLJyMiw7XY6nXI6nVc15ZYtWxQaGqoaNWqoe/fuevrpp4v8Lt++fbt++9vf2vb17t1bCxYs8Op9i90iYf0FAADXRlRUlFwul2eLi4u7qnliY2O1dOlSbdq0SfPmzdMnn3yiW265pcg7cqenpyssLMy2LywsTOnp6V69t9dXkQAAgEIYrGCkpaUpODjYs/tqqxcX710lSTfeeKPat2+v6Ohovfvuu7rnnnsuO+6XRYWL97ryRrETDLfb7dXEAAD8LzG5BiM4ONiWYJgSERGh6OhoHTp06LLnhIeHX1KtOHHixCVVjSvx+lkkAACgEKVwmaq3fvjhB6WlpSkiIuKy53Tu3FkbNmyw7Vu/fr26dOni1Xt5/SwSAABQNmRmZurw4cOe1ykpKUpOTvbcOmLWrFm69957FRERoSNHjuj3v/+9atWqpbvvvtszZtiwYapTp45nncfEiRPVrVs3zZ07V/369dPatWu1ceNGbdu2zavYSDAAADChFJ5FsmvXLvXs2dPzevLkyZKk4cOHa/Hixfr000/117/+VadPn1ZERIR69uyplStXKigoyDMmNTVVfn4/NTS6dOmiFStW6IknntCMGTPUqFEjrVy5Up06dfIqNhIMAAAMKI37YPTo0aPIizA++OCDK86xZcuWS/bdd999uu+++7wL5hdYgwEAAIyjggEAgAk8rt2GBAMAAAN4mqodLRIAAGAcFQwAAEygRWJDggEAgAkkGDa0SAAAgHFUMAAAMMDx383XOSoKEgwAAEygRWJDggEAgAFcpmrHGgwAAGAcFQwAAEygRWJDggEAgCkVKEHwFS0SAABgHBUMAAAMYJGnHQkGAAAmsAbDhhYJAAAwjgoGAAAG0CKxI8EAAMAEWiQ2tEgAAIBxVDAAADCAFokdCQYAACbQIrEhwQAAwAQSDBvWYAAAAOOoYAAAYABrMOxIMAAAMIEWiQ0tEgAAYBwVDAAADHBYlhyWbyUIX8eXJSQYAACYQIvEhhYJAAAwjgoGAAAGcBWJHRUMAABMsAxtXkhKStKdd96pyMhIORwOvfXWW55jeXl5mjZtmlq2bKnAwEBFRkZq2LBhOnbsWJFzJiQkyOFwXLJlZ2d7FRsJBgAA5VRWVpZatWqlF1988ZJj586d0549ezRjxgzt2bNHq1ev1ldffaW77rrrivMGBwfr+PHjts3f39+r2GiRAABgQGm0SGJjYxUbG1voMZfLpQ0bNtj2LVq0SB07dlRqaqrq1at3+TgcDoWHh3sXzC9QwQAAwASDLZKMjAzblpOTYyTEM2fOyOFwqEaNGkWel5mZqejoaNWtW1d33HGH9u7d6/V7kWAAAGDAxQqGr5skRUVFyeVyeba4uDif48vOztbvfvc7DR48WMHBwZc9r1mzZkpISFBiYqKWL18uf39/de3aVYcOHfLq/WiRAABQxqSlpdmSAKfT6dN8eXl5GjRokNxut15++eUiz42JiVFMTIznddeuXdW2bVstWrRICxcuLPZ7kmAAAGCCwRttBQcHF1ll8EZeXp4GDBiglJQUbdq0yet5/fz81KFDB68rGLRIAAAwxER7xKSLycWhQ4e0ceNG1axZ0+s5LMtScnKyIiIivBpHBQMAgHIqMzNThw8f9rxOSUlRcnKyQkJCFBkZqfvuu0979uzRO++8o4KCAqWnp0uSQkJCVLVqVUnSsGHDVKdOHc86j9mzZysmJkZNmjRRRkaGFi5cqOTkZL300ktexUaCAQCACZZ1YfN1Di/s2rVLPXv29LyePHmyJGn48OGaNWuWEhMTJUmtW7e2jdu8ebN69OghSUpNTZWf308NjdOnT2v06NFKT0+Xy+VSmzZtlJSUpI4dO3oVGwkGAAAGlMZ9MHr06CGriKSkqGMXbdmyxfb6+eef1/PPP+9dIIVgDQYAADCOCgYAACbwuHYbEgwAAAxwuC9svs5RUdAiAQAAxlHBQJkQEJCnYcM+Vecu36lGjRx9/XUNvfLnNvrqK++v2QZKW/VV/5H/jgxVPporq6pDuc2qKWNYqArq/OxujJaloJUnVW39afllFSi3SYDOjA5Xfj3vnliJMoQWiU2FrmDMmjXrkktzUDZNnPSJ2rT9Xs8920ljHu6tPXvCNSduq2rWPFfaoQFeq/p5lrJiQ/SfufX1w6xoOQos1ZydKkf2T/Xv6mt+UGDijzrzULhO/qmB3NdVVs1ZqXKcLyjFyOELk88iqQhKNcEYMWKEHA6HHA6HKleurHr16mnMmDE6depUaYaFa6xq1XzdfPNRvf56K332WaiOHw/S0r/fqPT0QN1+x9elHR7gtR+fjNb5W2oov56/8hv46/SESFU+macqX5+/cIJlKfCdH5V5Xy1ldw5WfrS/Tj0SKUeOWwFJGaUbPK7exftg+LpVEKVewejTp4+OHz+uI0eO6LXXXtPbb7+tsWPHlnZYuIYqVbJUqZKlvNxKtv25uZXUosXJUooKMMdx7kLlwl39wu94pe/zVOlUvrJbB/50UhU/5bSopqpfUrVDxVDqCYbT6VR4eLjq1q2rXr16aeDAgVq/fr0kye126w9/+IPq1q0rp9Op1q1ba926dbbxR48e1aBBgxQSEqLAwEC1b99eO3fuLPS9UlJS1LhxY40ZM0Zud+FLdXNycpSRkWHbULLOn6+iL76oqQcGf66QkPPy83Or5y1H1LTpDwoJyS7t8ADfWJZc8enKuSFA+dEX1lf4nc6XJLlr2JfBuWtU9hxD+UOLxK7UE4yf++abb7Ru3TpVqVJFkvTCCy9o3rx5eu6557R//3717t1bd911l+eJbpmZmerevbuOHTumxMRE7du3T1OnTi00efjss8/UtWtX3X///Vq8eLHttqg/FxcXJ5fL5dmioqJK7gPD47lnY+SQtHRZohLfflP9+h3Sli3Rchc4Sjs0wCeuv6Sr8pEcnZpc98onW5L4lS+/LENbBVHqV5G88847ql69ugoKCpSdfeGv1fnz50uSnnvuOU2bNk2DBg2SJM2dO1ebN2/WggUL9NJLL2nZsmU6efKkPvnkE4WEhEiSGjdufMl7bN++XXfccYemT5+uxx57rMh4pk+f7rmXuyRlZGSQZFwDx49X19Spt8jpzFe1wDyd+jFAv5v+sdK/D7zyYKCMCn71uPw/Oav/PF1f7lpVPPsvVi78TufLHfLTfr8z+XK7Sv2fZcCIUv9N7tmzpxYvXqxz587ptdde01dffaUJEyYoIyNDx44dU9euXW3nd+3aVfv27ZMkJScnq02bNp7kojCpqan61a9+pT/+8Y/67W9/e8V4nE6nnE7nFc9DycjJqaycnMqqXj1X7dql643XW5V2SID3LEuuV9Plv/Os/vNUtArCqtoOF4RVUcF1leW/L0uZDQMu7Myz5Pz8nDKGhZVCwDChNJ5FUpaVeoskMDBQjRs31k033aSFCxcqJydHs2fP9hx3OOz1QsuyPPsCAgKuOH/t2rXVsWNHrVixgvUUZVjbdsfVrt1xhYVlqk2bdD0zd7OOHg3S+vUNSjs0wGuuv6QrYOsZnfptHVkBleR3Kl9+p/KlnP+2bx0OZd0Roupv/vd+Gd9mq8ai72Q5/XS+W3DpBo+rx1UkNqVewfilmTNnKjY2VmPGjFFkZKS2bdumbt26eY5//PHHnkfG3nTTTXrttdf0448/XraKERAQoHfeeUd9+/ZV7969tX79egUFBV2Tz4LiC6yWpwcf3K9atc7rbGZVbdtWV0sSWqqgoNRzYMBrgesuXGpfa8a3tv2nJkTq/C01JEmZd9eUI9ct11/S5Zd54UZbP8ysJyug0i+nA8qlMpdg9OjRQy1atNCcOXM0ZcoUzZw5U40aNVLr1q0VHx+v5ORkLV26VJL0wAMPaM6cOerfv7/i4uIUERGhvXv3KjIyUp07d/bMGRgYqHfffVexsbGKjY3VunXrVL169dL6iCjEhx/W04cf1ivtMAAjjq1pfuWTHA6dHRSqs4NCSz4gXBO0SOzK5J+HkydP1quvvqq7775bjz76qB599FG1bNlS69atU2Jiopo0aSJJqlq1qtavX6/Q0FD17dtXLVu21DPPPKNKlS79C6B69ep6//33ZVmW+vbtq6ysrGv9sQAAFRlXkdg4LKsCNXxKQEZGhlwul265cYoqV2LxJyqmo7PL5N8agM8KzuXo4OC5OnPmjIKDS2Z9y8Xvic59/qDKVXx7lkx+Xra2r3uyROO9VspciwQAgPKIFokdCQYAACa4rQubr3NUECQYAACYwOPabWi8AgAA46hgAABggEMG1mAYiaRsIMEAAMAEE3firEAXdtIiAQAAxlHBAADAAC5TtSPBAADABK4isaFFAgAAjKOCAQCAAQ7LksPHRZq+ji9LSDAAADDB/d/N1zkqCFokAADAOCoYAAAYQIvEjgoGAAAmWIY2LyQlJenOO+9UZGSkHA6H3nrrLXtIlqVZs2YpMjJSAQEB6tGjhz7//PMrzrtq1So1b95cTqdTzZs315o1a7wLTCQYAACYcfFOnr5uXsjKylKrVq304osvFnr8T3/6k+bPn68XX3xRn3zyicLDw3Xbbbfp7Nmzl51z+/btGjhwoIYOHap9+/Zp6NChGjBggHbu3OlVbLRIAAAop2JjYxUbG1voMcuytGDBAj3++OO65557JElLlixRWFiYli1bpv/7v/8rdNyCBQt02223afr06ZKk6dOna+vWrVqwYIGWL19e7NioYAAAYMDFO3n6uklSRkaGbcvJyfE6npSUFKWnp6tXr16efU6nU927d9fHH3982XHbt2+3jZGk3r17FzmmMCQYAACYYLBFEhUVJZfL5dni4uK8Dic9PV2SFBYWZtsfFhbmOXa5cd6OKQwtEgAAypi0tDQFBwd7Xjudzquey+GwPwTesqxL9pkY80skGAAAGOBwX9h8nUOSgoODbQnG1QgPD5d0oSIRERHh2X/ixIlLKhS/HPfLasWVxhSGFgkAACaUwlUkRWnQoIHCw8O1YcMGz77c3Fxt3bpVXbp0uey4zp0728ZI0vr164scUxgqGAAAlFOZmZk6fPiw53VKSoqSk5MVEhKievXqadKkSZozZ46aNGmiJk2aaM6cOapWrZoGDx7sGTNs2DDVqVPHs85j4sSJ6tatm+bOnat+/fpp7dq12rhxo7Zt2+ZVbCQYAACYUAqPa9+1a5d69uzpeT158mRJ0vDhw5WQkKCpU6fq/PnzGjt2rE6dOqVOnTpp/fr1CgoK8oxJTU2Vn99PDY0uXbpoxYoVeuKJJzRjxgw1atRIK1euVKdOnbyKzWFZFei+pCUgIyNDLpdLt9w4RZUrXf0iG6AsOzqbbikqpoJzOTo4eK7OnDnj85qGy7n4PdGz/e9VubK/T3Pl52dr8645JRrvtcK/KgAAwDhaJAAAmGBikWYFaiqQYAAAYIIlycfLVH1ew1GGkGAAAGAAj2u3Yw0GAAAwjgoGAAAmWDKwBsNIJGUCCQYAACawyNOGFgkAADCOCgYAACa4JXn3wNHC56ggSDAAADCAq0jsaJEAAADjqGAAAGACizxtSDAAADCBBMOGFgkAADCOCgYAACZQwbAhwQAAwAQuU7UhwQAAwAAuU7VjDQYAADCOCgYAACawBsOGBAMAABPcluTwMUFwV5wEgxYJAAAwjgoGAAAm0CKxIcEAAMAIAwmGKk6CQYsEAAAYRwUDAAATaJHYkGAAAGCC25LPLQ6uIgEAALg8KhgAAJhguS9svs5RQZBgAABgAmswbEgwAAAwgTUYNqzBAAAAxpFgAABgwsUWia9bMdWvX18Oh+OSbdy4cYWev2XLlkLP//LLL039BGxokQAAYIIlA2swin/qJ598ooKCAs/rzz77TLfddpvuv//+IscdPHhQwcHBnte1a9f2OsziIMEAAKAc+mVi8Mwzz6hRo0bq3r17keNCQ0NVo0aNEozsAlokAACYYLBFkpGRYdtycnKKfOvc3Fz9/e9/18iRI+VwOIo8t02bNoqIiNCtt96qzZs3G/v4v0SCAQCACW63mU1SVFSUXC6XZ4uLiyvyrd966y2dPn1aI0aMuOw5ERER+stf/qJVq1Zp9erVatq0qW699VYlJSWZ/Cl40CIBAKCMSUtLs62TcDqdRZ7/+uuvKzY2VpGRkZc9p2nTpmratKnndefOnZWWlqbnnntO3bp18z3oXyDBAADABIM32goODrYlGEX59ttvtXHjRq1evdrrt4uJidHf//53r8cVBwkGAAAmlNKdPOPj4xUaGqrbb7/d67F79+5VRESE1+OKgwQDAIByyu12Kz4+XsOHD1flyvav9OnTp+u7777TX//6V0nSggULVL9+fbVo0cKzKHTVqlVatWpVicRGggEAgAmlcKvwjRs3KjU1VSNHjrzk2PHjx5Wamup5nZubq8cee0zfffedAgIC1KJFC7377rvq27evbzFfBgkGAAAGWJZblo9PQ/V2fK9evWRdpq2SkJBgez116lRNnTr1akPzGgkGAAAmWJbvDyurQE9T5T4YAADAOCoYAACYYBlYg1GBKhgkGAAAmOB2Sw7f1mDIxzUcZQktEgAAYBwVDAAATKBFYkOCAQCAAZbbLcvHFomvl7mWJbRIAACAcVQwAAAwgRaJDQkGAAAmuC3JQYJxES0SAABgHBUMAABMsCxJvt4Ho+JUMEgwAAAwwHJbsnxskVzuwWXlEQkGAAAmWG75XsHgMlUAAIDLooIBAIABtEjsSDAAADCBFokNCcYVXMwm8wtySjkSoOQUnKNbioqp4NyFf7uvRWUgX3k+32crX3lmgikDHFZFqseUgKNHjyoqKqq0wwAA+CAtLU1169Ytkbmzs7PVoEEDpaenG5kvPDxcKSkp8vf3NzJfaSHBuAK3261jx44pKChIDoejtMOp8DIyMhQVFaW0tDQFBweXdjiAcfyOX1uWZens2bOKjIyUn1/JVeqys7OVm5trZK6qVauW++RCokVyRX5+fiWW9eLygoOD+ccXFRq/49eOy+Uq8ffw9/evEEmBSTReAQCAcSQYAADAOBIMlClOp1MzZ86U0+ks7VCAEsHvOP5XsMgTAAAYRwUDAAAYR4IBAACMI8EAAADGkWCg1PXo0UOTJk0q7TCAMmXWrFlq3bp1aYcBXDUSDJSIESNGyOFw6OGHH77k2NixY+VwODRixAhJ0urVq/XUU09d4wgB31z8HXc4HKpcubLq1aunMWPG6NSpU6UdGlAmkGCgxERFRWnFihU6f/68Z192draWL1+uevXqefaFhIQoKCioNEIEfNKnTx8dP35cR44c0Wuvvaa3335bY8eOLe2wgDKBBAMlpm3btqpXr55Wr17t2bd69WpFRUWpTZs2nn2/bJHUr19fc+bM0ciRIxUUFKR69erpL3/5i23u7777TgMHDtR1112nmjVrql+/fjpy5EhJfyTAxul0Kjw8XHXr1lWvXr00cOBArV+/XtKF5xj94Q9/UN26deV0OtW6dWutW7fONv7o0aMaNGiQQkJCFBgYqPbt22vnzp2FvldKSooaN26sMWPGyO2uOI/0RsVFgoES9eCDDyo+Pt7z+o033tDIkSOvOG7evHlq37699u7dq7Fjx2rMmDH68ssvJUnnzp1Tz549Vb16dSUlJWnbtm2qXr26+vTpY+xhQ4C3vvnmG61bt05VqlSRJL3wwguaN2+ennvuOe3fv1+9e/fWXXfdpUOHDkmSMjMz1b17dx07dkyJiYnat2+fpk6dWmjy8Nlnn6lr1666//77tXjx4hJ9aBdgjAWUgOHDh1v9+vWzTp48aTmdTislJcU6cuSI5e/vb508edLq16+fNXz4cMuyLKt79+7WxIkTPWOjo6OtX//6157XbrfbCg0NtRYvXmxZlmW9/vrrVtOmTS232+05JycnxwoICLA++OCDa/L5gOHDh1uVKlWyAgMDLX9/f0uSJcmaP3++ZVmWFRkZaT399NO2MR06dLDGjh1rWZZlvfLKK1ZQUJD1ww8/FDr/zJkzrVatWlkff/yxFRISYj377LMl+4EAw3iaKkpUrVq1dPvtt2vJkiWyLEu33367atWqdcVxN910k+e/HQ6HwsPDdeLECUnS7t27dfjw4UvWbWRnZ+vrr782+wGAIvTs2VOLFy/WuXPn9Nprr+mrr77ShAkTlJGRoWPHjqlr166287t27ap9+/ZJkpKTk9WmTRuFhIRcdv7U1FT96le/0h//+Ef99re/LdHPAphGgoESN3LkSI0fP16S9NJLLxVrzMUy80UOh8NTOna73WrXrp2WLl16ybjatWv7GC1QfIGBgWrcuLEkaeHCherZs6dmz56tKVOmSLrwe/tzlmV59gUEBFxx/tq1aysyMlIrVqzQqFGjeLw7yhUaeShxF9dG5Obmqnfv3j7P17ZtWx06dEihoaFq3LixbXO5XAYiBq7OzJkz9dxzzykzM1ORkZHatm2b7fjHH3+sG264QdKFKl1ycrJ+/PHHy84XEBCgd955R/7+/urdu7fOnj1bovEDJpFgoMRVqlRJBw4c0IEDB1SpUiWf5xsyZIhq1aqlfv366cMPP1RKSoq2bt2qiRMn6ujRowYiBq5Ojx491KJFC82ZM0dTpkzR3LlztXLlSh08eFC/+93vlJycrIkTJ0qSHnjgAYWHh6t///766KOP9M0332jVqlXavn27bc7AwEC9++67qly5smJjY5WZmVkaHw3wGgkGrong4GBj5d1q1aopKSlJ9erV0z333KMbbrhBI0eO1Pnz5ykho9RNnjxZr776qu6++249+uijevTRR9WyZUutW7dOiYmJatKkiSSpatWqWr9+vUJDQ9W3b1+1bNlSzzzzTKFJePXq1fX+++/Lsiz17dtXWVlZ1/pjAV7jce0AAMA4KhgAAMA4EgwAAGAcCQYAADCOBAMAABhHggEAAIwjwQAAAMaRYAAAAONIMAAAgHEkGEA5MGvWLLVu3drzesSIEerfv/81j+PIkSNyOBxKTk6+7Dn169fXggULij1nQkKCatSo4XNsDodDb731ls/zADCDBAO4SiNGjJDD4ZDD4VCVKlXUsGFDPfbYY9fkNs4vvPCCEhISinVucZICADCNx7UDPujTp4/i4+OVl5enDz/8UL/5zW+UlZWlxYsXX3JuXl7eJY+hv1o8NRZAWUcFA/CB0+lUeHi4oqKiNHjwYA0ZMsRTpr/Y1njjjTfUsGFDOZ1OWZalM2fOaPTo0QoNDVVwcLBuueUW7du3zzbvM888o7CwMAUFBWnUqFHKzs62Hf9li8Ttdmvu3Llq3LixnE6n6tWrp6efflqS1KBBA0lSmzZt5HA41KNHD8+4+Ph43XDDDfL391ezZs308ssv297n3//+t9q0aSN/f3+1b99ee/fu9fpnNH/+fLVs2VKBgYGKiorS2LFjC30i6FtvvaXrr79e/v7+uu2225SWlmY7/vbbb6tdu3by9/dXw4YNNXv2bOXn53sdD4BrgwQDMCggIEB5eXme14cPH9Y//vEPrVq1ytOiuP3225Wenq733ntPu3fvVtu2bXXrrbfqxx9/lCT94x//0MyZM/X0009r165dioiIuOSL/5emT5+uuXPnasaMGfriiy+0bNkyhYWFSbqQJEjSxo0bdfz4ca1evVqS9Oqrr+rxxx/X008/rQMHDmjOnDmaMWOGlixZIknKysrSHXfcoaZNm2r37t2aNWuWHnvsMa9/Jn5+flq4cKE+++wzLVmyRJs2bdLUqVNt55w7d05PP/20lixZoo8++kgZGRkaNGiQ5/gHH3ygX//613rkkUf0xRdf6JVXXlFCQoIniQJQBlkArsrw4cOtfv36eV7v3LnTqlmzpjVgwADLsixr5syZVpUqVawTJ054zvnXv/5lBQcHW9nZ2ba5GjVqZL3yyiuWZVlW586drYcffth2vFOnTlarVq0Kfe+MjAzL6XRar776aqFxpqSkWJKsvXv32vZHRUVZy5Yts+176qmnrM6dO1uWZVmvvPKKFRISYmVlZXmOL168uNC5fi46Otp6/vnnL3v8H//4h1WzZk3P6/j4eEuStWPHDs++AwcOWJKsnTt3WpZlWf/v//0/a86cObZ5/va3v1kRERGe15KsNWvWXPZ9AVxbrMEAfPDOO++oevXqys/PV15envr166dFixZ5jkdHR6t27dqe17t371ZmZqZq1qxpm+f8+fP6+uuvJUkHDhzQww8/bDveuXNnbd68udAYDhw4oJycHN16663FjvvkyZNKS0vTqFGj9NBDD3n25+fne9Z3HDhwQK1atVK1atVscXhr8+bNmjNnjr744gtlZGQoPz9f2dnZysrKUmBgoCSpcuXKat++vWdMs2bNVKNGDR04cEAdO3bU7t279cknn9gqFgUFBcrOzta5c+dsMQIoG0gwAB/07NlTixcvVpUqVRQZGXnJIs6LX6AXud1uRUREaMuWLZfMdbWXagYEBHg9xu12S7rQJunUqZPtWKVKlSRJlmVdVTw/9+2336pv3756+OGH9dRTTykkJETbtm3TqFGjbK0k6cJlpr90cZ/b7dbs2bN1zz33XHKOv7+/z3ECMI8EA/BBYGCgGjduXOzz27Ztq/T0dFWuXFn169cv9JwbbrhBO3bs0LBhwzz7duzYcdk5mzRpooCAAP3rX//Sb37zm0uOV61aVdKFv/gvCgsLU506dfTNN99oyJAhhc7bvHlz/e1vf9P58+c9SUxRcRRm165dys/P17x58+Tnd2HJ1z/+8Y9LzsvPz9euXbvUsWNHSdLBgwd1+vRpNWvWTNKFn9vBgwe9+lkDKF0kGMA19Ktf/UqdO3dW//79NXfuXDVt2lTHjh3Te++9p/79+6t9+/aaOHGihg8frvbt2+vmm2/W0qVL9fnnn6thw4aFzunv769p06Zp6tSpqlq1qrp27aqTJ0/q888/16hRoxQaGqqAgACtW7dOdevWlb+/v1wul2bNmqVHHnlEwcHBio2NVU5Ojnbt2qVTp05p8uTJGjx4sB5//HGNGjVKTzzxhI4cOaLnnnvOq8/bqFEj5efna9GiRbrzzjv10Ucf6c9//vMl51WpUkUTJkzQwoULVaVKFY0fP14xMTGehOPJJ5/UHXfcoaioKN1///3y8/PT/v379emnn+qPf/yj9/9DAChxXEUCXEMOh0PvvfeeunXrppEjR+r666/XoEGDdOTIEc9VHwMHDtSTTz6padOmqV27dvr22281ZsyYIuedMWOGHn30UT355JO64YYbNHDgQJ04cULShfUNCxcu1CuvvKLIyEj169dPkvSb3/xGr732mhISEtSyZUt1795dCQkJnstaq1evrrfffltffPGF2rRpo8cff1xz58716vO2bt1a8+fP19y5c3XjjTdq6dKliouLu+S8atWqadq0aRo8eLA6d+6sgIAArVixwnO8d+/eeuedd7RhwwZ16NBBMTExmj9/vqKjo72KB8C147BMNFoBAAB+hgoGAAAwjgQDAAAYR4IBAACMI8EAAADGkWAAAADjSDAAAIBxJBgAAMA4EgwAAGAcCQYAADCOBAMAABhHggEAAIz7//YgsjAs7FqCAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 640x480 with 2 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "cm = confusion_matrix(y_test, log_clf.predict(X_test), labels=log_clf.classes_)\n",
        "disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=log_clf.classes_)\n",
        "disp.plot()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZvmBh4E9fIdd",
        "outputId": "1ef2d7e1-f78a-48fc-ae35-ce94fd1b0b80"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.7619047619047619"
            ]
          },
          "execution_count": 23,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "accuracy_score(log_clf.predict(X_test), y_test)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "14yEz9MHfbQL",
        "outputId": "96047029-9287-489c-cf8d-469170c0f6d7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "accuracy equals:  0.6825396825396826\n"
          ]
        }
      ],
      "source": [
        "print('accuracy equals: ', str((26+17)/len(y_test)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P9o0DnEn7Ly5"
      },
      "source": [
        "### Question 3\n",
        "use sklearn to determine precision and check the answer as was done above with accuracy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 32,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.6896551724137931"
            ]
          },
          "execution_count": 32,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "precision_score(log_clf.predict(X_test), y_test, pos_label='Rock')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ejCIIR167c94"
      },
      "source": [
        "### Question 4\n",
        "- why were there no hyperparameters tuned in this model?\n",
        "- how is the assessment of model performance in this example different from how you would normally do it at this stage of the process?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d0nUMIaJf0MV"
      },
      "source": [
        "## Model 2: LASSO"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LNef_BInwr3D"
      },
      "source": [
        "Let's now train a LASSO model, which includes a penalty hyperparameter. We know from the inner workings of LASSO models that these models depend on the scale in which features are measured. We therefore start off with standardizing all features."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 33,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 270
        },
        "id": "sblWEKGklJZv",
        "outputId": "8f8baee4-8dca-4ada-ae7c-f6177172bceb"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>attribute_1</th>\n",
              "      <th>attribute_2</th>\n",
              "      <th>attribute_3</th>\n",
              "      <th>attribute_4</th>\n",
              "      <th>attribute_5</th>\n",
              "      <th>attribute_6</th>\n",
              "      <th>attribute_7</th>\n",
              "      <th>attribute_8</th>\n",
              "      <th>attribute_9</th>\n",
              "      <th>attribute_10</th>\n",
              "      <th>...</th>\n",
              "      <th>attribute_51</th>\n",
              "      <th>attribute_52</th>\n",
              "      <th>attribute_53</th>\n",
              "      <th>attribute_54</th>\n",
              "      <th>attribute_55</th>\n",
              "      <th>attribute_56</th>\n",
              "      <th>attribute_57</th>\n",
              "      <th>attribute_58</th>\n",
              "      <th>attribute_59</th>\n",
              "      <th>attribute_60</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.034283</td>\n",
              "      <td>0.067601</td>\n",
              "      <td>-0.540083</td>\n",
              "      <td>-1.216550</td>\n",
              "      <td>-1.170987</td>\n",
              "      <td>-0.701419</td>\n",
              "      <td>-0.516137</td>\n",
              "      <td>-0.268162</td>\n",
              "      <td>-0.688366</td>\n",
              "      <td>-0.999476</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.652782</td>\n",
              "      <td>-0.663675</td>\n",
              "      <td>-0.450920</td>\n",
              "      <td>-0.007605</td>\n",
              "      <td>-0.567176</td>\n",
              "      <td>0.293379</td>\n",
              "      <td>-0.654754</td>\n",
              "      <td>-0.351269</td>\n",
              "      <td>-0.475797</td>\n",
              "      <td>0.409389</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1.858457</td>\n",
              "      <td>0.744916</td>\n",
              "      <td>1.373396</td>\n",
              "      <td>-0.358238</td>\n",
              "      <td>-0.523457</td>\n",
              "      <td>-0.436807</td>\n",
              "      <td>-0.715491</td>\n",
              "      <td>-0.240968</td>\n",
              "      <td>0.468459</td>\n",
              "      <td>-0.183085</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.202379</td>\n",
              "      <td>0.072554</td>\n",
              "      <td>1.193419</td>\n",
              "      <td>-0.903566</td>\n",
              "      <td>-0.777689</td>\n",
              "      <td>-0.962169</td>\n",
              "      <td>-0.960487</td>\n",
              "      <td>1.190439</td>\n",
              "      <td>0.430963</td>\n",
              "      <td>0.258869</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>-0.413156</td>\n",
              "      <td>-0.874452</td>\n",
              "      <td>-0.317661</td>\n",
              "      <td>-0.726481</td>\n",
              "      <td>0.165531</td>\n",
              "      <td>0.824675</td>\n",
              "      <td>-0.099744</td>\n",
              "      <td>-0.858148</td>\n",
              "      <td>-0.399998</td>\n",
              "      <td>-0.534576</td>\n",
              "      <td>...</td>\n",
              "      <td>0.202221</td>\n",
              "      <td>-0.063067</td>\n",
              "      <td>-0.325504</td>\n",
              "      <td>-0.572750</td>\n",
              "      <td>-0.679450</td>\n",
              "      <td>-0.317875</td>\n",
              "      <td>-0.510879</td>\n",
              "      <td>-0.365677</td>\n",
              "      <td>-0.122315</td>\n",
              "      <td>-0.211507</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.181790</td>\n",
              "      <td>-0.884766</td>\n",
              "      <td>-0.405976</td>\n",
              "      <td>0.173362</td>\n",
              "      <td>0.072745</td>\n",
              "      <td>-0.125597</td>\n",
              "      <td>-0.363406</td>\n",
              "      <td>-0.373390</td>\n",
              "      <td>-0.640584</td>\n",
              "      <td>0.054246</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.843631</td>\n",
              "      <td>-0.702424</td>\n",
              "      <td>-0.158283</td>\n",
              "      <td>0.364563</td>\n",
              "      <td>-0.398765</td>\n",
              "      <td>-0.350915</td>\n",
              "      <td>-0.780644</td>\n",
              "      <td>-0.682664</td>\n",
              "      <td>-0.291371</td>\n",
              "      <td>0.033088</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.403051</td>\n",
              "      <td>0.153555</td>\n",
              "      <td>-0.343828</td>\n",
              "      <td>-0.576969</td>\n",
              "      <td>0.191196</td>\n",
              "      <td>0.513465</td>\n",
              "      <td>-0.088490</td>\n",
              "      <td>-0.793119</td>\n",
              "      <td>-1.374078</td>\n",
              "      <td>-1.183483</td>\n",
              "      <td>...</td>\n",
              "      <td>-0.790193</td>\n",
              "      <td>-0.411807</td>\n",
              "      <td>0.775367</td>\n",
              "      <td>0.709164</td>\n",
              "      <td>-0.146149</td>\n",
              "      <td>-0.235273</td>\n",
              "      <td>1.035773</td>\n",
              "      <td>-0.178367</td>\n",
              "      <td>-0.045470</td>\n",
              "      <td>0.051904</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>5 rows × 60 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "   attribute_1  attribute_2  attribute_3  attribute_4  attribute_5  \\\n",
              "0     0.034283     0.067601    -0.540083    -1.216550    -1.170987   \n",
              "1     1.858457     0.744916     1.373396    -0.358238    -0.523457   \n",
              "2    -0.413156    -0.874452    -0.317661    -0.726481     0.165531   \n",
              "3     0.181790    -0.884766    -0.405976     0.173362     0.072745   \n",
              "4     0.403051     0.153555    -0.343828    -0.576969     0.191196   \n",
              "\n",
              "   attribute_6  attribute_7  attribute_8  attribute_9  attribute_10  ...  \\\n",
              "0    -0.701419    -0.516137    -0.268162    -0.688366     -0.999476  ...   \n",
              "1    -0.436807    -0.715491    -0.240968     0.468459     -0.183085  ...   \n",
              "2     0.824675    -0.099744    -0.858148    -0.399998     -0.534576  ...   \n",
              "3    -0.125597    -0.363406    -0.373390    -0.640584      0.054246  ...   \n",
              "4     0.513465    -0.088490    -0.793119    -1.374078     -1.183483  ...   \n",
              "\n",
              "   attribute_51  attribute_52  attribute_53  attribute_54  attribute_55  \\\n",
              "0     -0.652782     -0.663675     -0.450920     -0.007605     -0.567176   \n",
              "1     -0.202379      0.072554      1.193419     -0.903566     -0.777689   \n",
              "2      0.202221     -0.063067     -0.325504     -0.572750     -0.679450   \n",
              "3     -0.843631     -0.702424     -0.158283      0.364563     -0.398765   \n",
              "4     -0.790193     -0.411807      0.775367      0.709164     -0.146149   \n",
              "\n",
              "   attribute_56  attribute_57  attribute_58  attribute_59  attribute_60  \n",
              "0      0.293379     -0.654754     -0.351269     -0.475797      0.409389  \n",
              "1     -0.962169     -0.960487      1.190439      0.430963      0.258869  \n",
              "2     -0.317875     -0.510879     -0.365677     -0.122315     -0.211507  \n",
              "3     -0.350915     -0.780644     -0.682664     -0.291371      0.033088  \n",
              "4     -0.235273      1.035773     -0.178367     -0.045470      0.051904  \n",
              "\n",
              "[5 rows x 60 columns]"
            ]
          },
          "execution_count": 33,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sc = StandardScaler()\n",
        "X_train_prep = sc.fit_transform(X_train)\n",
        "X_train_prep = pd.DataFrame(data = X_train_prep, columns = X_train.columns)\n",
        "X_train_prep.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wrTm0umZxk9I"
      },
      "source": [
        "And we can now train a LASSO model, simply by changing the penalty argument in the LogisticRegression. Moreover, we now have a hyperparameter, which means that we define a grid with a large range of possible hyperparameter values. As model results need to be compared for different settings of the hyperparameter, we now need to introduce cross-validation as well, to generate feedback from our training data while not using the test data.\n",
        "\n",
        "Please note how the hyperparameter values are stored in a dictionary under the name 'C', which is because 'C' is the argument containing the penalty (inverse strenght of regularization) in LogisticRegression (always make sure to refer to the documentation when working with sklearn models: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 34,
      "metadata": {
        "id": "OzVH2GuFfr3g"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n",
            "Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.\n"
          ]
        }
      ],
      "source": [
        "logL1_clf = LogisticRegression(penalty='l1', solver='liblinear')\n",
        "cv = RepeatedKFold(n_splits=5, n_repeats=3, random_state=42)\n",
        "lasso_alphas = np.linspace(0.1, 20, 200)\n",
        "grid = dict()\n",
        "grid['C'] = lasso_alphas\n",
        "gscv = GridSearchCV(logL1_clf, grid, scoring='accuracy', cv = cv, n_jobs=-1)\n",
        "results = gscv.fit(X_train_prep, y_train)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qaUJe0UNzyYR"
      },
      "source": [
        "We now have 200 model outcomes. To find out what the settings are of the best model, we simply use best_estimator_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 35,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iEUYrCtooEQV",
        "outputId": "30706d26-3384-4c20-9ce9-60c508027a0d"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<style>#sk-container-id-3 {color: black;background-color: white;}#sk-container-id-3 pre{padding: 0;}#sk-container-id-3 div.sk-toggleable {background-color: white;}#sk-container-id-3 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-3 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-3 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-3 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-3 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-3 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-3 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-3 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-3 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-3 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-3 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-3 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-3 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-3 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-3 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-3 div.sk-item {position: relative;z-index: 1;}#sk-container-id-3 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-3 div.sk-item::before, #sk-container-id-3 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-3 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-3 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-3 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-3 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-3 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-3 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-3 div.sk-label-container {text-align: center;}#sk-container-id-3 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-3 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-3\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LogisticRegression(C=1.7, penalty=&#x27;l1&#x27;, solver=&#x27;liblinear&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" checked><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression(C=1.7, penalty=&#x27;l1&#x27;, solver=&#x27;liblinear&#x27;)</pre></div></div></div></div></div>"
            ],
            "text/plain": [
              "LogisticRegression(C=1.7, penalty='l1', solver='liblinear')"
            ]
          },
          "execution_count": 35,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "results.best_estimator_"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rmA1gHEi0sr0"
      },
      "source": [
        "We can plot model results for the entire range of hyperparameter values considered."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 36,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        },
        "id": "irMTtZSJrqZw",
        "outputId": "36e27172-cc36-448f-86cb-3296ca84d99b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x140b01d50>]"
            ]
          },
          "execution_count": 36,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABaQklEQVR4nO3deXhU5dk/8O+ZmcxM9m3IRhZCwr4oBEQS0GoRxar1tRXUvrhU+6vVtiJdxLpU7YK1ra2thaoFqa1V3xa3Ki5R2YOgEATZQwIJZCP7Msms5/fHzDmZycxklsyW8P1c11yamXPOPCcnw7nnee7nfgRRFEUQERERRTFFpBtARERE5A0DFiIiIop6DFiIiIgo6jFgISIioqjHgIWIiIiiHgMWIiIiinoMWIiIiCjqMWAhIiKiqKeKdAOCxWq1or6+HomJiRAEIdLNISIiIh+Iooju7m7k5ORAofDcjzJqApb6+nrk5eVFuhlEREQUgLq6OuTm5np8fdQELImJiQBsJ5yUlBTh1hAREZEvurq6kJeXJ9/HPRk1AYs0DJSUlMSAhYiIaITxls7BpFsiIiKKegxYiIiIKOoxYCEiIqKox4CFiIiIoh4DFiIiIop6DFiIiIgo6jFgISIioqjHgIWIiIiiHgMWIiIiinoMWIiIiCjqMWAhIiKiqMeAhYiIiKLeqFn88HzR0mPAiztroDdaoBAEXHtBDi7MS4l0s3y27fg5bD7W7PH1r0zKwKUTx4SxRURENBIwYBlh1m45iXU7auSfd1a14P0Vl0SwRb4zmC343j/3otdo8bjNa5/VofLRK6BRKcPYMiIiinYMWEaY7SfOAQAWTcnER0eaUN3SC6tVhEIx9LLc0aCytgO9RgtS4mLwrXn5Lq+/vLsWHXoT9p3uwPyi9Ai0kIiIohUDlhGkubsfx5t6IAjA6htmYPPqZhjNVpzrMSAzSRvp5nlVUdUCALhkwhj85MrJLq/XtfXh7S/qUXGyhQELERE5YdLtCLLrZCsAYGp2EsYkapCTYgtSatv0kWyWz3ba219W7D4YWVCss21nD2yIiIgkDFhGkB0nbDdy6caenxYHAKhtjf6ApbvfhP11HQCA0iKd221K7YHMF2c60d1vClfTiIhoBGDAMkKIoogKew9FqT1gyUu1BSx17dEfsOypaYPFKqIgPQ559kBrsNzUOBSkx8FiFbG7ui3MLSQiomjGgGWEON2qx9mOPsQoBcwdlwoA8o1/JAwJ7ayyB1seelckZdKw0EkOCxER0QAGLCOEdAOfnZ+KOLUtV1oKWOpGRMBia7+n/BVJWRHzWIiIyBVnCY0QAzf8gR6KfDlg6QtbO/788Qk8v60aVlF0+/qF+Sn4+x0XQaVUYF9tO773z73o6TfLtVfmjx86YJFmBx1v6sG0R9/3q20KQcDdXynCvZcV+7UfERFFP/awjABWqyjPEHLsoZAClsaufvSbPBdjCxaLVcQL26vRbbAFIO4eO6ta8cWZDgDAq3tq0dRlkIOVhRN0SE/QDPkeafFqudKtp/fw9Og2mLFx35mQ/g6IiCgy2MMyAhxu6EK73oR4tRIzc1Pk51PjYhCvVqLXaMGZ9j4UZySEtB1fnu1EV78ZiVoV3v7+AgyuVffEfw/j46PN2FnVitn5qXLeytNLL8CcgjSMTY316X3W3z4XZ9v7IMJ9L447p1v1uHX9HrT2GH3eh4iIRg4GLCOANBx08fh0xCgHOsUEQUBeWhyONnajrl0f8oBFyqO5eHw6CnXxLq9fNjkDHx9txo6qFlx3QY6cJHzV9Cw578YXSoWA/HT3M4k8SdLGAAA6+0wwmq1Qq9h5SEQ0mvBf9RFg56DpzI7yw5h4W2HvMSnzUIVWyq+prG3HR0eaAACzHJKEQyk5NgZKe5dPWy97WYiIRhsGLFHOYLbgsxpbTRJ3M2zywlQ8rt9kwWenpHa4n5o8Lj0OY1NiYbKIeG5btW1bL9OYg0WhEJAerwZgW9GaiIhGFwYsUa6ytgN9Jgt0CWpMykx0eV3uYQlx8bh9p9thMFuRkajxOPQkCAJK7b0v57ptQcOCCeFbE0hK6GXAQkQ0+jBgiXLSgoGlRToIguuKzHlptkTW2hBPbd7hMK3aXTskCyYM9KgMThIONV2C1MPCISEiotGGSbch1tTVj3a9EZOzkgLa39uCgY45LKIoQhAEiKKIfbUdmJaTBG2MMrCGe2hHqZdVlB1XWZ43KEk41HT2HpbWKO1hMVms2FHVAr3B/yno+WlxmJGbHIJWERGNDAxYQuy29XtworkHux68HBmJWr/27TdZvC4YmGtfT6jHYMa5HgMyErX49+dn8NONB3DngkI8cs3UYbUfAHoNZhy011bxlL8iyUjUYmJmAo439XgNboJtoIclOgOWf+w6jSfeORzQvgoB+PhHX3E7O4uI6HzAgCXEqlt6YbGKqO/o9ztgqW3Tw2IVkahVeVwwUBujxOSsRBxt7Mbu6jZce0EO3j/UCAD44FBjUAKW2jY9rCKQEheDnBTvtVQevWYa3tp/Fsvm5g37vf2RLvewROeQkDRzakJGAtLsCcK+OHmuBy09Rmw51oxCXWGomkdEFNUYsIRQv8kCo9kKAOjpN/u9vzTzJ99DsCIpK9bhaGM3Kk624KrpWdhdbRu+OdPeh9pWvd81TQaTpkx7a4dkwQSdUy5LuEhDQi1ROK2532TB56fbAQB/XV6CojG+18xZu+UkfvP+UeysasUdZQxYiOj8xKTbEOrsM8n/32PwP2CRZv54D1hsQy87q1px4EyHXAofGEiWHQ5pNWhPvTzRIl0aEuqOviGhz0+1w2i2IitJi/F+DutI13d3dSvMFmsomkdEFPUYsITQcAMWXwOFiwrToVIIqG3T45U9dQAAaSKPVJ12OKQelrzU6A5YdPH2IaHe6AtYpOtQWpw+5Cwrd6blJCM5NgbdBjMOnO0MRfOIiKIeA5YQcgxYegPpYfExYEnQqHBhXgoA4HX74n//c+FYAMCuk62wWn1fk8dtO9ptU6Z9HRKKFF2irYeltcc47HMONml6+gIvScvuKBWCvMp1RRB6zIiIRiIGLCHUqR/mkFCb74GCVLZfuk/fc1kR4tRKtPUacaSxy+/3dlTrZw5LpEiJrGariK5+k5etw6dTb5J7RrzNsvJEGhYKxhAfEdFIxIAlhBxvmv4GLKIoDgwJ+bDKseP6PmNTYlE0JgHzCtMADKwBFAhRFB16enxbbTlSNColkrS2PPJomtq8q7oVoggUjYlHZpJ/M8UkUqCz73QH+oz+13EhIhrpOEsohJxyWPycJdTSY0SfyQJBAMb6ELDMyk9FbIwSfSYLyux5EmXFOmw+dg47T7bgO5eM93qM/35Rj//sPQMRQIJGiQeXTIFGpYDBbIVCgE9TmiNNl6BBV78ZLT1GFGc4v2axinjqg6O4IDcFV8/IBgDsqWnDc1tPwuRhCKkwPQ6PXDMVqmEUwKs4GfhwkNwOXTyyk7Vo6OzH56fbsHDCmCG37zdZ8OR7R3H55AxcMtG27fYT5/DR4SY8ePWUYRUU/PJsJ/6x6zQeWDLZ4/Tso41dePrD4+g3W6EUgFvnj8NlkzPcbuvOZ6fa8GblWTx49RQkaNz/M/Xl2U78a08tfnrlJKTE+T5NnIhGJgYsITScHBZphlB2khYalfebi1qlwOWTM/DuwQZcMTULAHCxPe9BKj7nza/ePYLGrn7554xELa6Zabux56TEhrVqbaB0CRpUt/S67WHZcqwZz22tRqJGhcVTM6FSKvDU+0fl6cbubANw2eQMfGWS7zfbwaRhHHerbftKEARcPD4db1Sexf7aDq8By9v767Gh4hQ+OdqMbT+9DADw6FuHUNPSi8nZSbj5ovyA2/Kb949i+4kWpCeo8dOrJrvd5s8fV+HDw03yz6da9X4FLI//9xC+PNuFcenxHoPtP39yAh8cakJ6vBo/WjzJv5MgohEn+u9AI5hjwNLtb8BiH4bJ9SNv5Nc3zMC/756PK6ZmAoBcFbVDb/Ka09FvssjBym3zCwAAO6taHIalojt/RSJNbXZXPE4KHLoNZhw824keg1kO5p74+jT8/sYLnB4L7bVkKk4GPqTW2NmP6nO9UAgDAWSgpOspXZOhSOda26ZHXZseZzv6UNPSC8B2XQPVb7Jgj3318J0efi9Wqyj3Kq1YNAEKAahp6UV9h2/rXbX3GnGovsv+Hp7betpep2g450NEIwd7WEJoOD0svhaNc5QcG4O549Lkn+M1KqTHq9Haa0Rdmx7TcjyvRXPG3qOToFHh/ism4qVPT+NEcw/22nsfoj3hVqIbYsVmx1yeipOtaNcbYbaKyE+Lw63zx7lsr1IK2H6iZVg3RGnfGbkpSI6NCfg4wMA18BawiOJAwCC1QaEYmEotzRxzfM5X+2ptq3YDwMEzHejsM7mc1+GGLrTrTYhXK3HvZcXYevwcKms7sLOqBTfO8V79WMr5AWxDdkazFWqV83crxxyvL850orvfhETt8H6/RBTd2MMSQl19A0GKv0m3vhaN8yZPXhxx6G+30ut5aXFIiVNjuj24eXt/va0dw6yWGy7pHlZsbu7ux7GmbvnnnVUt2Fk19MKS0vpNh+q70BZg9VwpYCkLwrpK0rU80z70tTzW1O10/jtPtjpNh27tNeJoY7e7Xb1yDN6sIvBptWsvixQsSYtfltl/j74Gfo4zofRGC76wr2PlqK3XCL09+dhiFbG7us3ncyCikYkBSwh1DaNwXG2QZubkOazm7Mv75dvfr9R+E5eGsnJ9SPyNBp5WbN5lH75ItyeJfn66HZuPNgPwvLDkmEQNJmUmOu3vD1EU5SGNQKczO5L+Fuo7++QlH9yRAjHpXCuqWuThG/m5AAsKuju2p22kxS+lv6WdJ1shit7r40jHlN7DXaAzuJcpGAUSiSi6MWAJoeHMEvKnBstQpADE2zDC4FyVskE38ZEzJOR+xWbppveNklxkJWlhNFtRbc/pGGpV6YGbrf83xJPnetHUZYBGpUBJQarf+w82JkEDbYwCoogh80GkG/63FxQiNkaJ1l4jznUboI1R4NsLbGsRBTLM1dVvwgF7b8e9lxXbjjMokDOarXKOi7Se1Oz8VGhjFDjXbUBVc8+Q73GmXY9TrXooFQLuvrTIY1ulYobSqNZwpu4T0cjAgCWEAs1hMZqtaOgcGKIZDinQkIaYPJEXOLQP/cwdlwa1w6ygkRKwyCs2OwzhiKLo9K2/1GEIaEp2kryPO/4OZziS9pkzLnVY04glgiB4zWMxWazyMM2lE8fgosKBnKa549JwmX220257bog/Pj3ZCqtoS/69YfZYCAJQ1dyDxs6BmWWVte3oM1mgS1DLvVPaGKWcW+Wt8J0UeMzMTcaV07Lsx+xw+fxIf6/S+Rxr6kZzdz+IaPRi0m0IOQUsRouc6NhjMLv8A5wcGyPf1Oo7+mAVAW2MAmOGuJn6Quox8bmHxX5DjFUrMbsgBZ9WtyFOrfRYbyPayEm33QY02Wc91Xf04WxHH2KUAi4qTENbrxGv7zsLwHtuybzxaVAqBJxu1eNMux65g2ZLiaKIc90GuBvo2Hr8HADPQ06ByEuNw/GmHvl6SUMs0vpE0uKXqXExmJqdhLLidLkdZcU6TM5KlBOxtx0/hxm5nhOxB9t8rNl+nHSkxKkxY2wyDpzpxIeHG+Xg4mP7MNv8Ip3TmkmlRTpsP9GCrcfPyTVw3Nly3Lb/gmId8tPjkJcWi7q2Pnx0pAkXj0/HmAQNFApBTkqfmZuChs5+HG7owoeHmuQZcoDzZwqwfWmQhmbT49VOtXX6TRaPQeXgBOVAE5aJaHgYsISI0WxFn8m5Immv0Yyall58c+0uGAetupuoVeHjlZciI0nrNDzj70J5g8mJmm19Hv+hdapm63BDLivS4dPqNuSnDb8d4SIl3fYaLZj364+dXpuVn4o4tcopn6RswtDBRKI2BhfkJmNfbQcqqlqxdK5zwLLy/77AG5VnhzzGcArGDZbn0GPWZ7RgyTPbkJ6gwX/ung9BELDjhNSTpINCITid64Ji23Pzi9LxzoEG3PXS5wG1Qep1Ki3S4cCZTjz61iE8+tYhp20WDEpkXlCsw28AbDl2zuW6uCMFeWVFOrzaVof7Xt0PwNZr9PdvXzSQlJ4eiwUTdDjc0IWH3/wSD7/5pXyMRK0K5fdfiqxkLY42duG6Z3fKvUqTsxKx6YcLoVAIeOdAPe57dT9+d+NM/M+sXKd2PF1+HC9sq8br95RiSnYSWnoMuPqZ7bioMA3P3jLb918aEQ0bh4RCRKp7Igi2xesAW+LtZ6fa5WBFpRCgsr/W3W/GvtoOAMGbIQQA2claqBQCjBYrmrvdl6tv15vQa59x4Zhce0NJLiZkJGCpD1NRo0WiRoUrpmbKv1vpEa9W4lvzbMXSMpO0+GZJLi4qTJMXFRyKdNMfPJzRZ7Tg3QMNAGzXePB7qhQC5hWmYfpY33sxvMl3SKL+tKYVp1r12Hu6Xc4NcVwVGgCmZCVh0ZQMfHVyBqZmJwEAbr4oH0laldv2entMykzEQnvl3Btmj4UuQeOyzbj0OLl4oWRqThJKi9J9eo9Z+Slyzs+Nc/KQGhcjf062Hj+Hlh6D0/pWtnaonY4B2D5THx2xFa9790ADjGarvIr50cZu+fOw7fg5WKwith13Ha768FAj+kwWvLnfFpR+cqQZzd0GlB9u8imBmIiChz0sISINByVqVBAEAZ19JvQazHIy6O2l4/DYddMAAN//1z68c6BB7uUYPDwzHCqlAjkpsaht06O2TY+sZNe1bKT3y0zSOHWLj02JRfnKS4fdhnASBAEv3DrH63a/u/ECn49ZVqzDnz+pQoV9lovU2/TZqTYYLVZkJ2tRserysPRC5TnksDjO0NlR1YKxqbGorLXVzZF6QRQKAX+7ba7L+Rx47Mpht2ViZiI+f3iRT9sqFQL+9Z2L/X6PkoJUVD66GABw1R+34WhjN7afOCcnHeelxiEjSYvPH77Cab8/f3wCvy8/jp1VLfjfiwvkYPM3N8zEs5urnD4PtYM+dxLHnkcpH0k6jsFsxbluAzICXBuKiPzHHpYQkQKWpNgYeS2U7n6zPN1Wms0CuCbG1gUxYHE8vqc8lpGyGnOkzMpPgTZGgZYeA443DcxycZyyHK4hs3yHujo7HGbG7KxqxZ6aNpgsIsamxKJghNTN8Yc0tPbvz8/AKgIalQJjEt3neEnLIOyqbkVnnwkHzthXy56gk6eHS58zaUbe4Kn/jj2Ph+q70N5rdKp67C2RnYiCiwFLiEgBS7JDwNJrsMgl43UOybR5gwKKYE1pHnx8T7VYgh0gjTYalftZLhVeCs+FgnSz7ewz4UhDl/z87upWbD8hBVDpIybnyB/S0JwUNAyVW3VBbjISNCp06E3YsPMULFYR49LjMDYl1imAd5yR19xtcFoJ2zHAF0XgpV2nnabL+7JEAhEFDwOWEOlyCFjiNbZhlh6HISHHqbT5gwKKYBWNkwz+RjmYu4RbcibfLO0BS4feiC/rbd/agzkLyJs4tcqpd25iZgKSY2PQbTDjtc/qnNo62lxUmCbnpwBDB9gqpQIXj7cFmS9srwYw0OvimLgszciTnHHoNRkckEjHkV9v9W1tJCIKDgYsIeLUw2Jf48QWsEg9LO6GhPrQoTfK+wYrgOCQ0PBJwxG7a9pgslix66RtvZsJGQnIDHMeg+ONeuGEMXLisDRlN5wBVDjFa1SYlZ8i/+zt71X6PUi/F+kaSp+rOnseiyPHn6VAXqq46zglevC2RBR6DFhCpMtpSMjew9JvQmuvlMMy0MOSnayFUiHAaLbKiw3qEtSI1wQnJ9pb8biBKaIMWDyZmp2ElLgY9BjMOHCmI6gl9/3leKMuK053GpKalJnoMa9jNHAMxrwNYTpeG0GAHNg5BvCDPxN1bgKW62eNhUPHDr5RYpv6zBwWovBiwBIi7pJum7sN6DfZpjSnO/Sw2Gby2L6lSxVZBxcoGw7pG2VTlwH9g2rDmCxW1Hf0O21HrhQKQb7hPftJFT46LK1DFL78FYl0nVQKARcVpstDHQCcqviORo5BSJ6X9a0mZibIXwymZich1d4zIgUsTV0GnHBIogaAWodFQqWAZGp2EmbmpgAAUuJi5CJ5Q63P9cGhRvzp4xP408cn8Nb+gTo9FquIl3efll9zXNOpz2jB/31e5/e6Y+ejUy29+PfndbBaObX8fMJpzSHiOCRkstddOW2vzhmnViJO7fyrz0+LQ11bn/wPWDCHZ1LiYpCoVaG734yT53owLWegLsjp1l5YrCI0KgUyRvE382BYMEGH975sxOZjtsqxKoWAeT7UcQm2oox4ALY1ehI0KsTr4jE2JRZnO/qw0EshvJHuwrwUJGhU6DGYMX5MwpDbCoKAhRN0eKPyrLyuEWD7PEjHkJYxKNTFo6al12mYRx4qTY/DJRN02F/XgbIinTwDq7GrHwazBRqVc4Xc6nM9+O4/9jo9l5cWh9n5qXj3YAMeemOguJ1GpcDeR65AgkaFP358HM9trcbZ9j7cf8XEAH47549H3voS20+0QJeokZdnoNGPAUuIOPawGOy9GqdabYvtOfauSGzfmltxtLEbQHADFkEQMDs/FVuPn8Ouk61OAYu0CvHs/FSWG/fiG7Nz0dDRL69TtKBYh+TYmLC345qZOWjqMmDRFFsZekEQ8KebZ+HgmY5R/4+3WqXAc8tLUN/Rh+KMoQMWAFi1ZDIK0uPkRR8B2+8rLy0ORxq65M9baVE6alp65aRbs0PPY35aHP7fpUVQKhS4cU4u0uPViFMroTdacLa9zyVwkmZr5afFQaNS4ERzD7YdP2f7DNqD3QvzUlDXpkdrrxF7alpx+eRMbDlqe+2YvU3kmdS7VdXUM+r/5mkAh4RCpNNplpAtLpR6WHRu1gcaPB4frBlCEinPoWLQ6ro7IzA1d6TSxijx4ysnYfUNM7D6hhn42kzPa+KEUoxSgbsvLXK6YZcUpOL2ssJROZ15sLJiHW70sfpyZpIWKxZNRJLWObAcPJwkJeTWtukhiiIaOvthsYpQq2zreSVoVLhv0QTkpMR6XYRSmvq+bG6e0+rYtkU4ba/9aPFELLYPLe040Yrm7n4ca+r2eExyJpWH4O/q/MKAJUQ6+2zj0LZZQraAZWCWgWvAMrhHJdg1UaSx/93VrfIQlcUqykNQo3UqLJE7gz9vF49PhyAAeqMFrb1Gh/W8Yt32PHqqbWR2WC27rFgnVxyurO3AofouNHb1Q61UYE5BmsOXiBa5p1M6Jsv+e9ZvsqDb/m8pE5/PLwxYQqTLTeE4ic7dkNCgf0CDPcV4SlYS0uLV6DVa8EVdBwDgUH0nuvrNSNSoMCOI690QRTvHGXHp8WqkxquRZZ+e7jjd2dPnUJ4a3e5ci+XL+i5095uRqLV9pvLT45CbGguzVcQfyo8DsPWGxaqV8oyno43deGt/vXyMboPZaaV3ciYNyQLsYTnfMGAJESlgSdKq3AQsQ/ewqBQCspODOyTkOMtlx6B1UeaNT4dKyT8FOn84zoiTvixIz9W26b1Wf863D9nWtjrfMKUhn/nj0+VFT6Xhpo+P2maWST0rafFqeUHKT+yvSXgj9qzFYRFXaRV6Oj8EdJdas2YNCgsLodVqUVJSgu3bt3vc9vbbb4cgCC6PadOmud3+1VdfhSAIuP766wNpWlSwWEW5y9Ixh0XiLuk2NW6gJ2Zsaqz8j10wDVRrbXX67wLmr9B5xjEQkb4sOA7zeOthkXpoBgcWUsDiOCupdNBwq+Pwq+N2MUoBU+wBTF0bq+h6ItWyAgCjxYqm7v4ItobCye+A5bXXXsOKFSvw0EMPobKyEgsXLsSSJUtQW1vrdvtnnnkGDQ0N8qOurg5paWm48cYbXbY9ffo0fvzjH2PhwoX+n0kU6XLozk1yOyTk2sMizVwAQldxVvpmV1nXjvZeIz471WZ/nvkrdH7JdUi6zR/0uatr6/Paw+JYLVfKN+k3WfC5vfCjY4E7x1o9g4dfHV+bnZ+KyVmJANjDMpSWbqPTzwzuzh9+T2t++umnceedd+Kuu+4CAPzxj3/EBx98gLVr12L16tUu2ycnJyM5eeAD+uabb6K9vR133HGH03YWiwXf+ta38Pjjj2P79u3o6Ojwt2lRQxp/jlcrEaNUuAQs7npYAFuC35GGrqAWjXOUnxYn1+tY9PRWGMxWZCRqfJoeSjSaaGOUyEzSoKnLIM/Ik/771hdnYbLYghBPxRSlz2i3wYwFv9kMQQDMFhFGsxWZSRoUjYmXt9UlaDA5KxFHG7tdhl8vKkxDjFKAySKirFgHsz0hXgpYHnz9ILafOOf03hqVAo9cMxVfCcJ0XotVxH2vVmK/Pa8tJzkWL9w6B8lx4Z+uL3l+20n849PTEEVbzaonvzETs/NT5ddbHHpYANvv6qLCtHA3M6g+OtyENVuq8LsbL/BaX+h85lcPi9FoxN69e7F48WKn5xcvXoyKigqfjrFu3TosWrQIBQUFTs8/8cQTGDNmDO68806fjmMwGNDV1eX0iBbd/bbhoET7VEpplpBkjJseFgCYM872oZxTkOr29eESBAFXz7BNpZQS166annVeTIUlGmzOuDQIAjDLfjOclZ8KhQD0m6ywWEWkxatRqIt3u2+sWonpY23DN2c7+nCmvQ+NXbahiSXTs10+U1+/cCwA4PpZOU7Px6lVWDw1C2qVAldNz5J7dM6061Hbqscre2pxpr3P6XHyXC/+tr0mKL+Dg2c78c6BBvnYe0614f1DDUE5diDMFiv+/HEV6tps7Tne1IOXKk45bSNNaZaMht6of+2pxb7aDnxwqCnSTYlqfvWwtLS0wGKxIDMz0+n5zMxMNDY2et2/oaEB7733Hv71r385Pb9z506sW7cO+/fv97ktq1evxuOPP+7z9uHUbbD1sEiBimsPi/uA5TsLx+NrM3OQkxy6xfRWLZmC62eNhckiQqUQ5C5oovPNH5ddiEevmSovXlmoi0fFqq/Kgce49DjEqpUe9//3d0vl2ikSlWIgD8XRdy8Zj/+ZNRZZbj7bv196AXoMZugSNGjvHagvIq1XdUFuMh7/+nQAQGNnH+7+5z58dqoN/SYLtDGe2+cLOeemWIdxujj889Na7KxqxbK5+cM6bqAOnO1Et8GM5NgY/OzqyXhg40HsPNkKURTlIFBa8T4tXo22XiPOjIKARQq6pHMj9wJKuh387cHxj2koGzZsQEpKilNCbXd3N/73f/8XL7zwAnQ633MpHnzwQXR2dsqPuro6n/cNtV6DrbKtFKhoVAqo7Em0SoWAFA/VUQVBwFh7YapQUSoETMtJxoV5KZg+Npmzg+i8FaNUuKy0nZWsxYV5KbgwLwUpce6HbiWxaqW8rfSYPjbZbcK8QiG4DVYA2/CUlNcm9bCcbe/DtuO2oaDLJmfIx79yWhYyEjUwmK3YV9vu9zkPJtVhumJqJq6dmSM/F6k6MBUOs6yunzUW2hgFznUbcKJ5YM0nqYdlVl4KgJHfwyKKopwz1cqAZUh+3a10Oh2USqVLb0pzc7NLr8tgoihi/fr1WL58OdTqgX8ITp48iVOnTuHaa6+FSqWCSqXCSy+9hLfffhsqlQonT550ezyNRoOkpCSnR7Totc8QkgIWQRDkmUJp8WqWwCcitzKTtFArFTBbRXx8RJoG7bjqtCAn6kq9I4HqN1nw2al2+3ukY1Z+KmJjlGjpMbr0HIWLVGqhrDgdGpUSc8fZclMcz1XqhZiVnwJg5Acs57oNMJhtuUstg4a7yJlfAYtarUZJSQnKy8udni8vL0dpaemQ+27duhVVVVUuOSqTJ0/GwYMHsX//fvlx3XXX4bLLLsP+/fuRl+dbCe5o0j0oYHH8//T4ob+1EdH5S6kQMNY+g8losSLe3ovjSApgpGU1ArX3dLtDknAC1CqFnLw63GMHos9owb7THQAGztHduUo3dSn3qLnbdRX6kcQx4OKQ0ND8niW0cuVKLF++HHPmzMH8+fPx/PPPo7a2FnfffTcA21DN2bNn8dJLLzntt27dOsybNw/Tp093el6r1bo8l5KSAgAuz48UUg9LvJuAZQxXRCaiIeSlxaGmxbZQqm0WkfP3SukmfuBMB7r6TS7rJPlK6rUoK9LJw9BlxenYevwcdla14E6HBSPD4fPTbTBarMhO1srJztLSBrurW2G2WCEIAtrss4SKMxLkVbfPtOtRnDEy8/EcAxbHKr7kyu8EhmXLluGPf/wjnnjiCVx44YXYtm0bNm3aJM/6aWhocKnJ0tnZiY0bN/o8A2ik65FnCTkELFr2sBCRd/kOC5+6q5GUkxKLQl08rCLw6cnAe0J22vd1LGxX6hAgSGuOhYs0HFTqEEBNzUlCcmwMug1mHDjbiQ69EVJh27R4tZzzM5KHhRzryLT1Glm5dwh+97AAwD333IN77rnH7WsbNmxweS45ORl6ve9/UO6OMZL0yD0sAxn8Um+LpxlCRESAc+0XT0Udy4rTUdPSi4qTrfKqzwazBV/Udcq1XIZitFhx8EyHfCzJ1OwkpMbFoF1vwhd1HZhjzyE5123ACXteS3qCBpMcZhfWtPSiocO5eFtyXAymZie5TCDo7DPh0NlOALYvcTPGJsvbyJW3Jwy0R6mw5ey892Ujdp5oQcJ027mmxsUgRqlAfpqtdtX2Ey3Qqgb+vS3QxWNsSnCXNwFs083T4zUus8dEUURVcw/Gj0nwu0q5Y7BlsYro6DMhzc8vtv0mCxo7+zHOwzT8wW093apHXlqcX23dfuIcErUxmOEhsTwcAgpYaGg9cg7LQFet1NvirsotEZFEqribHq/GpEz3wxxlRTr7FOSBZNQn/nsYL+92X3Hck/Fj4p3WLVMoBJQW6fDuwQbsrGrFnHFpMJgtuPpP23HOYQ2f//vufFxUmIbqcz244g/bYHHTK/C3W+dg0VTnyRi3vPApDtUP1Mx6eukFuGF2Ljr0RnxZbwtkHKsEA7YeoPe+bMTOky0osdeokr74Sb+rF3eewos7T8n7xKuV2P7A5X7f+IfyRV0Hblhbga9fkIOnl13o9Nore+rwszcO4sElk/HdS4v8Ou7gFb9begx+t/u+VyvxwaEmbPrhQkzNGXoCygeHmnD3P/fivq9OwP1XTPT5PX7xzmEcb+rB2m/NxpIZ2X61L1g4pzUEBmYJDUThN8/Nx8IJOnwtQheaiEaGyyZn4Gszs/Gzq6d4nFE4vygdggCcaO5BU1c/RFHEh4dtRcfGpcdhYmaC18eU7CSsWOR6wyq197hIdWAqaztwrtsAtUohD2l/cMg2U/TjI82wWEUkaVXycaU8PWkbyZl2PQ7Vd0EhAJlJztvsOtkKUbTlpQyeal5mnxW173QHzthXx5ZWvP9GSS5m56c4nVecWoleo2XYs6gG2/RlAyxWEbtr2lxeO9JgC8KkisH+qGt3DVj8Jb3vF2e8v7+07X+/qB96QwfNXf043tQDQQAuHh+5tefYwxICcg+LQw7Lggk6p4XOiIjc0cYo8ZdbZg+5TUqcGtNzknHwbCcqTrZgWk4yznUboFEp8P6KS4ZVUE5aXbqyth16o1mujXLVtCwsmpqJH75SKQcDUlDzw69OwF0LxwMAthxrxu0vfoadVS1ONbqkIZ8L8lLw6DVT8T9rKrDrZCssVlE+zgI3Q2CFunjkJGtR39mP9+0BjtTDMjkrCa/fU+a0/RP/PYz1O2tQcbIF116Q43K8QEntb+jsg9FshVo18H1fWpDR31yafpNFLlRYnJGAquYev6c295ssaOry/f2lWi/VLb2o7+hDjg9DZxX2fKdpOUlIjWAeJntYQkDOYVEzHiSi0JB7Qqpa5QDiosK0YVe/ldYcM1lE7Klpk5NzFxTr5BowRxu70djZjz01rguoSusj1Xf241TrwA10h0NV3Rljk5GoVaGr34xD9Z1yMOC4GKREEAQ5MVgqpudpeRNgIAcmmFOzHYesrCJQPyhnR1qQ0d+A5WxHH0TRNoQ1MdO2hpC/xePOOPTQ+PL+jj04vvZCOc4oiyQGLCEgzRIaXJKfiChYpJvHzqoW+YYyOP8jEIIgyIm4HxxqkocQSovT5YUcAWDNlirojRaXXJs4tUqukSK1SxRF+Vt6aZEOKqVCHlr49+dnUN3SC4UAzPMw3CC1x2zPlRlqtuVFhelQKQTUtuld8kMCJQ1ZSVyGcew9LN39ZnTqTT4f13FVcCm/cfBaSd6PMRA8+bJMgePU6QofZpmJojjw9+UhCTxcGLCEQK+bISEiomCaOy4NaqUCDZ392GrveXA3pBIIqcfkP3vrYLGKGJceJ69QLb3HK3tsCb7zi9Jdcm2kbaTS/8ebetDSY4A2RoHZBSm297D3prz6me04M3NTkOxh2ZLB3+x1Q9SzStCo5GJ7wcpjkYasJIN7MhyDDH96WdwFLP7msDi+n29DQgNtlYbthnKqVY/6zn6olQrMHReahXl9xYAlBLrdFI4jIgqmWLVSvvmbLCKSY2O8zhDxldRTY7LYbmaO36ylYEZ6zd3Ua6lHpOJkK6xWUR4OmjsuDRr79GPX43hO5sxI0mJCRoL8s7d6VlJ7dw6jTo0jaXgpz14jxzEwMJqt6Owb6FXxJ2CRts1Pi0O6PZHY3xwWx16kdr0J3f2ee3hEUcQ5h4CouduAKod1mtyRgr5Z+SmIi3CaAwOWIBNFUe5hSWTAQkQh5NjzMH98etDqY4xJ1DgN8zi+z0WFafJiroD7Xp2ZuSmIVyvRoTfhcEOXnLjrGNwUZyQgw6GnxFt+hOO+3upZSb03u4KwkOPZjj7U2IesvjE7FwBwZlCxN0eDh4uGIg3n5Aeph8XxmO70GMww2tctkqaIe+uFclzRO9J4Rw2Cg2c68c7Bevzw8gkQBMiVGNnDQkShVDZBh9+XH5f/P6jHLtbhWFM3BME27COJ16gwKz8Fn51qR15arFxt1lGMPUfl46PNeOStL3G0wVZ0bsGghRzLinV4o/IsNCoFZhcMPdxQVqzDhopTAIZOugXgtJDj9/65D9qYge/mV03PwlXTnctL9BrM+P2Hx+Wy/5dOGoP/mWULTqQb9szcFEzNtvVgDbX+TyA9LHlpsfJwmDTjSFJR1YIv6zvxnYXjXQrxuXu/2jY9JmYm4Pflx+WCfhfmpeD2skK59yZercRXp2Rg7+l2vFhxCvvrOjAhMxH3fKXI6T2sVhG7ql0rIkcK76hB8MzHx/HRkWZMzEjEwom2iyoIQJx6eNn6RERDmTk2GboENdr1JlwS5IDl8skZWL+zBrPyUlwKmV02OQOfnWrH5ZMyPO7/lUlj8PHRZlTWdgCwFc2UbviOx3mj8iwWFOu8zm6aNz4NcWolFILgdU02tUqBsmIdPjrSJE+FlpQfbsLlkzOdpiW/e6AB63fWyD+/c6ABi6ZkIlEbI/cOLSjWIT/ddSmAwQGLr4m+RrMV1S224ZiC9Hi510qacSR58I2DON2qx4TMRFw26PctiqJcm2ZqdhION3Shrk2Pj440Y+2Wk/J2b+6vx5IZ2fIMpPQEDS6dOAZPvX8Mp1v1OG2fzVValC4nTANATWsvOvQmxMYocUFusk/nFUoMWIKgw54V3tTdPzBDSK1yGw0TEQWLSqnAv75zMTr7TChI916W3R8LJujw4u1zMTHLtdrudxaOR15qHC6b7DlgWTY3H5oYJbrs+R1lxTqX5NxrZ2ZDo1JgVn6K1/YkaWPwf9+dD1GES2l8d371P9OxcILOaU2kZzdXoUNvwv66DnllasB2YwaAi8en4XSrHg2d/dhd3YavTslwWHMpXV42obPPhM4+E5JjY+Rei9gYJfpMFp97WCpr29FvskKXoMZ4XTx6jbYVp/tMFuiNZsSpVTCarXIAtONEi0vA0q43yWU05hel2wKWdj1Ot9nOZ0GxDkcbu9HSY8Cpll60621t1SWoMS0nGX+7dQ5Otfbiv1/U44szndhZ1eIUsEjnMk4XD5Uy8hkkDFiCQG//Q2vpNqLXYPt/zhAionCY6KF8fzB4CkhilAqvRdnUKgWWzskbchtBEHClfS0kX0wf6/u3/MwkLW4rHef0XGVdB9490ICdVS1OAYt0Y140JRPVLb341+5a7DzZgvz0OLkg3+z8VGhjlEiPV6O114i6Nj2SxybLvRYzc5Oxu6YNZ9v7YLGKXvOJdg5a7DFerYQ2RoF+kxUt3Ubkp6tQ39Enpxi4yzWR2p2ZpJGTkmvb9Ki195jcOr8A//j0NLafMKC2TQ+DPX9FygGSlk5QqxT2gKUV3798gnx8eRZTavDXZQpE5EOmUaDPZAtSWnsN6DbYvk0wf4WIKLoMnm4tqXOYrSMl/1Z4KMgn5exI+0hDQtNykhGjFGC2imjo9Jz4KpF6bqTZUYIgID3ennjrpnKu1FPiqd1Su/bXdTjVtXFsrzSlefCadlJC897T7eizfwEHIAc++W7ylCKBAUsQ6I22LrmWHsNADwsDFiKiqCIFI5W1HfJsTsC5Hoq0TtOxpm68td+23o5jQT7p5i3NBpKCgIwkjVyrxtuwUHe/aaAgn8Oxpfoy0jEHzzgaXOit1qHdUrukFAWpro00jFXX3icHPNJaTJLxunhkJWlhtFjx+emBtZKk93eXWB0JDFiCQG8PUlp7jOix97AwYCEiii756XHITY2F2SrKywp09ZvQbr/J56XFIS1eLScHS0GF4+ymwbVYWnoHei2kG/uZIaYWA8CemjZYrCIK0uOcggFdvFSLxbmHRUqHrKhy3zOUlxqH7GSt0zCU1HMjBTK1bXp5BtLgOja25Q9clzSodZh2HQ0YsAyTKIrQ24eEWnoM6GEPCxFR1JKCD2m4R7rpp8Wr5X+3HWu+DC7INxAA2G7mLd3SzBu1nOvhrYdlh4elFAbK8xuc2ia1ecfggKV9YMhGpVQgJ2VgpWupN8kxwJJmILmrFDx4uEwURaeep2jAgGWYjBYrLPasqLZeo5wRzxwWIqLoUzro5i8VWnO8KTsuwji4IN9AL4p9SMjea6GL18jBzKnWXhjMFo8PabHHwdV9B1e7ldr2zZJcqBQCzrT3oaq5Rz6OXCnXPt1aen/HujbSc+e6DfJCiVKujCMpSDt4thMdeiM6HGYg5UZJ0i3vqsPkmKBkFQdWzkzkLCEioqjjuOJ0S4/BKXFVIq04bbKILgX5pJyQM+19MFusA4msiWo5mHnnQAPeOdDgQ1vc97CcGzQkNCkrUS7Wt+jprS7Hkdpk+28r5o4bSBJOjo1BolaF7n4z6jv7AQBjEl2XNshM0qJoTDxOnuvFp9WtyE6OtT+vGfYK4MHCHpZh6nUIWADgVIvtDyxeEx0XmIiIBjiuOL3rZOtAYqlDL0KcWoVvluQhO1mLK6dlOu2fkxKLRI0KRosVFSdb5RWk0+LVmDMu1es6R5Irp2W6FOQbp7MFHgfPdMq1Xmxti8ONc/Lgbqb0hXkp8hIHV03PQrxaif+9OF9+XRAEOaCRuOthARyHyxx/L9ExHASwh2XY+oxmp59P2wsQcUiIiCg6ldkLqu2sakFjl63XYXBi6eobZrjdV6kQMG98Oj460iTPIkrSqqBRKZGRqMSnP/uqXOpiKO7Wmps7Lg1KhYDaNj122WcE6RLUiNeosHROHq6dmQOT1epyHKlI6VcmZeDQE1e5HDc/LQ6HG7oAACqF4HFV7NJiHf6+6zR2VrUg254PEy0JtwB7WIZNP6iHpcH+x8+FD4mIopOUO7LzZIvTisn+7v+Bvey/Y12TGKUCSdoYrw93ldATtTFyCfzXPqsFAHmqNGCr8OvLcQaTclwAW0/Q4IrDkovHp0MhANUtvfIsqmhJuAUYsAzb4IBF5MKHRERR7aLCdKgUAurabCsxA/7dmKWhEykpdXAhtuGQjr3l+DkAwenhcBzuGqqtybExmJGbAgDYan9/BiyjSJ/RfdcfpzUTEUWnBI0KF+alALB9yVQqBGQna4feyUFxRoKcNwIMzO4JBmkWk/TlNygBi8MxvLW1zJ6UHMz3DxYGLMPUOyiHRcKAhYgoepU61FrJSdH6tbifIAhOU5+D2cMyKz8F2piBtkh1VIbDMWAZ46WtjjVoAAYso8rgISEJFz8kIopejtVrA7kpO97Yg9nDolEpcVHhQDAUjCGZsSmxcrVcb20tKUiFRmULDdQqhVNPUqQxYBkmaUhIrXL+VTKHhYgoel2Yl4JYe32R4QYswexhAQaGZYDg9HBoY5TISrINeXlrqzZGiTnjbEXnclNjPSboRgIDlmGSelgGL7/NWUJERNFLrVJg3vg0AMC49Hi/989JicV4nW0/KRgIFikY0qgUcgG34SqwzxTK9KGt0vsXBvB7CSXeVYdJWqk5Py0OJ8/1ys+zh4WIKLo9/LWpGK+rxc3z8r1v7MZvvjkTW4414yuTxgS1XdPHJuPRa6ZiTKLGaVmA4fjJlZPwzoEGLJqa6XXb2+aPQ4fehOsvHBuU9w4W3lWHSe5hcei2EwQgTs1Kt0RE0aw4IwGPXjs14P3njkvD3HFpQWzRgG8vKAzq8UoK0lBS4Ftb4zUq/OzqKUF9/2DgkNAwSQFLerxGHg9NUKt8KuZDREREvmHAMkxSaf54jRI6+4JSnCFEREQUXAxYhknqYYlVK+UFpZi/QkREFFwMWIZJClji1Ero7PPbWTSOiIgouBiwDJM0Syg2RiXPb2fAQkREFFwMWIZJ6mGJ1yjlCoLxGs4QIiIiCiYGLMPUZxoYEpqclQQAKNQlRLJJREREow7HLoap12BPuo1R4ZqZ2Rg/Jh4TMxMj3CoiIqLRhQHLMEnTmuPUSgiCgGk5yRFuERER0ejDIaFhEEUReochISIiIgoNBizDYDBbIYq2/4/jzCAiIqKQYcAyDNIMIQByWX4iIiIKPgYsw9BrsOWvaFSKoK2oSURERK4YsAxDH/NXiIiIwoIByzAMlOVn/goREVEoMWAZBr3DlGYiIiIKHQYsw6A3cEiIiIgoHBiwDINUgyWWAQsREVFIMWAZhoEqt8xhISIiCiUGLMMwkHTLHhYiIqJQYsAyDAxYiIiIwoMByzDoOSREREQUFgxYhkHqYWHSLRERUWgxYBmGPmlIiOsIERERhRQDlmGQc1i4UjMREVFIMWAZBla6JSIiCg8GLMPAWUJEREThwYBlGOSkW+awEBERhRQDlmHo42rNREREYcGAZRj0JnsOi4Y9LERERKHEgGUYuFozERFReDBgGQY56TaGQ0JEREShxIAlQFariD4TK90SERGFAwOWAPWbLfL/xzOHhYiIKKQYsASo1zAQsGhVDFiIiIhCKaCAZc2aNSgsLIRWq0VJSQm2b9/ucdvbb78dgiC4PKZNmyZv88ILL2DhwoVITU1FamoqFi1ahD179gTStLDpc6jBolAIEW4NERHR6OZ3wPLaa69hxYoVeOihh1BZWYmFCxdiyZIlqK2tdbv9M888g4aGBvlRV1eHtLQ03HjjjfI2W7Zswc0334zNmzdj165dyM/Px+LFi3H27NnAzyzE5CnNzF8hIiIKOUEURdGfHebNm4fZs2dj7dq18nNTpkzB9ddfj9WrV3vd/80338QNN9yAmpoaFBQUuN3GYrEgNTUVzz77LG699Vaf2tXV1YXk5GR0dnYiKSnJt5MZhn217bhhTQVyU2Ox44HLQ/5+REREo5Gv92+/eliMRiP27t2LxYsXOz2/ePFiVFRU+HSMdevWYdGiRR6DFQDQ6/UwmUxIS0vzuI3BYEBXV5fTI5ykIaF4VrklIiIKOb8ClpaWFlgsFmRmZjo9n5mZicbGRq/7NzQ04L333sNdd9015HarVq3C2LFjsWjRIo/brF69GsnJyfIjLy/Pt5MIkl6DbUiIU5qJiIhCL6CkW0FwTjIVRdHlOXc2bNiAlJQUXH/99R63eeqpp/DKK6/g9ddfh1ar9bjdgw8+iM7OTvlRV1fnc/uDQarBwhwWIiKi0PNrPEOn00GpVLr0pjQ3N7v0ugwmiiLWr1+P5cuXQ61Wu93md7/7HX7961/jo48+wsyZM4c8nkajgUaj8af5QSVXuWXAQkREFHJ+9bCo1WqUlJSgvLzc6fny8nKUlpYOue/WrVtRVVWFO++80+3rv/3tb/GLX/wC77//PubMmeNPsyJCz5WaiYiIwsbvu+3KlSuxfPlyzJkzB/Pnz8fzzz+P2tpa3H333QBsQzVnz57FSy+95LTfunXrMG/ePEyfPt3lmE899RQeeeQR/Otf/8K4cePkHpyEhAQkJCQEcl4hpzdwWjMREVG4+B2wLFu2DK2trXjiiSfQ0NCA6dOnY9OmTfKsn4aGBpeaLJ2dndi4cSOeeeYZt8dcs2YNjEYjvvnNbzo9//Of/xyPPfaYv00MCz3XESIiIgqbgMYz7rnnHtxzzz1uX9uwYYPLc8nJydDr9R6Pd+rUqUCaEVF9zGEhIiIKG64lFCC9URoSYg4LERFRqDFgCRBnCREREYUPA5YAMWAhIiIKHwYsAZKGhGI5JERERBRyDFgCJCfdxrCHhYiIKNQYsASIQ0JEREThw4AlQHLAouGQEBERUagxYAnQwLRm9rAQERGFGgOWAEk9LLHMYSEiIgo5BiwBsFhFGMxWAOxhISIiCgcGLAHos68jBADxzGEhIiIKOQYsAZBWahYEQKPir5CIiCjUeLcNgN6hBosgCBFuDRER0ejHgCUAcsItq9wSERGFBQOWAPSZOKWZiIgonBiwBKDXwCq3RERE4cSAJQAsy09ERBReDFgCMDAkxBwWIiKicGDAEoCBpFv2sBAREYUDA5YA9HFIiIiIKKwYsARgIOmWQ0JEREThwIAlAHpOayYiIgorBiwB4JAQERFReDFgCQCTbomIiMKLAUsA9EbbkFA8c1iIiIjCggFLANjDQkREFF4MWALASrdEREThxYAlAEy6JSIiCi8GLAGQclhiY5jDQkREFA4MWAIgDQnFa9jDQkREFA4MWALAHBYiIqLwYsASgD55lhCHhIiIiMKBAYufTBYrjBYrACAuhj0sRERE4cCAxU/ScBAAxDGHhYiIKCwYsPhJGg5SKgSolfz1ERERhQPvuH6SpjTHxSghCEKEW0NERHR+YMDiJ5blJyIiCj8GLH7q7rf3sDBgISIiChsGLH6qrGsHABRnJEa4JUREROcPBix+2lnVAgBYUJwe4ZYQERGdPxiw+KHfZMHnp2w9LGXFugi3hoiI6PzBgMUP+063w2C2IiNRg+KMhEg3h4iI6LzBgMUPO0/ahoPKinWc0kxERBRGDFj8sKOqFQBQWsT8FSIionBiwOKjzj4TDp7pAMD8FSIionBjwOKj3dWtsIrAeF08clJiI90cIiKi8woDFh8dqu8CAMwZlxrhlhAREZ1/GLD4yGixAgASNDERbgkREdH5hwGLjyxWEQCgUnJ2EBERUbgxYPGRFLAoOJ2ZiIgo7Biw+EjuYVEwYCEiIgo3Biw+MlttOSxKBixERERhx4DFR/acW/awEBERRQADFh9Z7D0sCgYsREREYceAxUdm5rAQERFFDAMWH0lJt8xhISIiCj8GLD7iLCEiIqLIYcDiI/awEBERRQ4DFh+Z5YCFvzIiIqJw493XRxwSIiIiihwGLD7ikBAREVHkMGDxEQMWIiKiyGHA4iOW5iciIoocBiw+Yg4LERFR5AQUsKxZswaFhYXQarUoKSnB9u3bPW57++23QxAEl8e0adOcttu4cSOmTp0KjUaDqVOn4o033gikaSHDISEiIqLI8Ttgee2117BixQo89NBDqKysxMKFC7FkyRLU1ta63f6ZZ55BQ0OD/Kirq0NaWhpuvPFGeZtdu3Zh2bJlWL58Ob744gssX74cS5cuxe7duwM/syBjwEJERBQ5giiKoj87zJs3D7Nnz8batWvl56ZMmYLrr78eq1ev9rr/m2++iRtuuAE1NTUoKCgAACxbtgxdXV1477335O2uuuoqpKam4pVXXvGpXV1dXUhOTkZnZyeSkpL8OSWffO1P23Govgsb7piLr0zKCPrxiYiIzke+3r/96mExGo3Yu3cvFi9e7PT84sWLUVFR4dMx1q1bh0WLFsnBCmDrYRl8zCuvvHLIYxoMBnR1dTk9Qmkgh4VpP0REROHm1923paUFFosFmZmZTs9nZmaisbHR6/4NDQ147733cNdddzk939jY6PcxV69ejeTkZPmRl5fnx5n4j0NCREREkRNQd4EgON+0RVF0ec6dDRs2ICUlBddff/2wj/nggw+is7NTftTV1fnW+AAxYCEiIooclT8b63Q6KJVKl56P5uZmlx6SwURRxPr167F8+XKo1Wqn17Kysvw+pkajgUaj8af5w2JmwEJERBQxfvWwqNVqlJSUoLy83On58vJylJaWDrnv1q1bUVVVhTvvvNPltfnz57sc88MPP/R6zHBiHRYiIqLI8auHBQBWrlyJ5cuXY86cOZg/fz6ef/551NbW4u677wZgG6o5e/YsXnrpJaf91q1bh3nz5mH69Okux7zvvvtwySWX4De/+Q2+/vWv46233sJHH32EHTt2BHhawcchISIiosjxO2BZtmwZWltb8cQTT6ChoQHTp0/Hpk2b5Fk/DQ0NLjVZOjs7sXHjRjzzzDNuj1laWopXX30VDz/8MB555BEUFRXhtddew7x58wI4pdDgkBAREVHk+F2HJVqFug7LrCc+RLvehPL7L8GEzMSgH5+IiOh8FJI6LOcz9rAQERFFDgMWH1lZOI6IiChiePf1kdTDwniFiIgo/Hj79RFL8xMREUUO774+sojMYSEiIooUBiw+sFpFSHOpGLAQERGFHwMWH0j5KwADFiIiokhgwOIDi0PAwtL8RERE4ceAxQcWkT0sREREkcSAxQcWCwMWIiKiSGLA4gOz1Sr/v1JgwEJERBRuDFh8IOWwKARAwR4WIiKisGPA4gMph4VF44iIiCKDd2AfmC0sy09ERBRJvAX7gGX5iYiIIot3YB9IheM4Q4iIiCgyGLD4wCrnsDBgISIiigQGLD4YyGFhwEJERBQJDFh8MJDDwoCFiIgoEhiw+EAqHMccFiIioshgwOID5rAQERFFFgMWHzCHhYiIKLIYsPiAOSxERESRxYDFBwN1WPjrIiIiigTegX1gYQ4LERFRRDFg8YGFOSxEREQRxYDFB2bmsBAREUUUAxYfWLiWEBERUUQxYPEBc1iIiIgiiwGLDyysdEtERBRRDFh8IBWOY8BCREQUGQxYfMDCcURERJHFgMUHUg4Le1iIiIgigwGLDzhLiIiIKLIYsPhgIIeFvy4iIqJI4B3YB1ZOayYiIoooBiw+MHNIiIiIKKIYsPhAzmERGLAQERFFAgMWH8g5LEoGLERERJHAgMUHLM1PREQUWQxYfMDS/ERERJHFgMUHZuawEBERRRQDFh9YmMNCREQUUQxYfMAcFiIioshiwOKDgdL8/HURERFFAu/APmAOCxERUWQxYPGBlMOiYg4LERFRRDBg8YGUw8JpzURERJHBgMUHLM1PREQUWQxYfMDFD4mIiCKLAYsPpEq3zGEhIiKKDAYsPrCwh4WIiCiiGLD4gDksREREkcWAxQfMYSEiIoosBiw+kHpYmMNCREQUGQxYfMDS/ERERJHFO7APWJqfiIgoshiw+ICzhIiIiCKLAYsPpB4WFQMWIiKiiGDA4gOr1MPCpFsiIqKIYMDiA+awEBERRRYDFh/Ipfk5JERERBQRDFh8wKRbIiKiyGLA4gMWjiMiIoqsgAKWNWvWoLCwEFqtFiUlJdi+ffuQ2xsMBjz00EMoKCiARqNBUVER1q9f77TNH//4R0yaNAmxsbHIy8vD/fffj/7+/kCaF3RSDouCOSxEREQRofJ3h9deew0rVqzAmjVrUFZWhueeew5LlizB4cOHkZ+f73afpUuXoqmpCevWrUNxcTGam5thNpvl119++WWsWrUK69evR2lpKY4fP47bb78dAPCHP/whsDMLIrmHhZVuiYiIIsLvgOXpp5/GnXfeibvuuguArWfkgw8+wNq1a7F69WqX7d9//31s3boV1dXVSEtLAwCMGzfOaZtdu3ahrKwMt9xyi/z6zTffjD179vjbvJBgDgsREVFk+dVlYDQasXfvXixevNjp+cWLF6OiosLtPm+//TbmzJmDp556CmPHjsXEiRPx4x//GH19ffI2CxYswN69e+UApbq6Gps2bcLXvvY1j20xGAzo6upyeoQKc1iIiIgiy68elpaWFlgsFmRmZjo9n5mZicbGRrf7VFdXY8eOHdBqtXjjjTfQ0tKCe+65B21tbXIey0033YRz585hwYIFEEURZrMZ3/ve97Bq1SqPbVm9ejUef/xxf5ofMOawEBERRVZASRnCoBu3KIouz0msVisEQcDLL7+Miy66CFdffTWefvppbNiwQe5l2bJlC371q19hzZo12LdvH15//XW88847+MUvfuGxDQ8++CA6OzvlR11dXSCn4hMLS/MTERFFlF89LDqdDkql0qU3pbm52aXXRZKdnY2xY8ciOTlZfm7KlCkQRRFnzpzBhAkT8Mgjj2D58uVyXsyMGTPQ29uL//f//h8eeughKNwku2o0Gmg0Gn+aHzDmsBAREUWWXz0sarUaJSUlKC8vd3q+vLwcpaWlbvcpKytDfX09enp65OeOHz8OhUKB3NxcAIBer3cJSpRKJURRhCiK/jQxJJjDQkREFFl+DwmtXLkSf/vb37B+/XocOXIE999/P2pra3H33XcDsA3V3HrrrfL2t9xyC9LT03HHHXfg8OHD2LZtG37yk5/g29/+NmJjYwEA1157LdauXYtXX30VNTU1KC8vxyOPPILrrrsOSqUySKcaOLO9ND/XEiIiIooMv6c1L1u2DK2trXjiiSfQ0NCA6dOnY9OmTSgoKAAANDQ0oLa2Vt4+ISEB5eXl+MEPfoA5c+YgPT0dS5cuxS9/+Ut5m4cffhiCIODhhx/G2bNnMWbMGFx77bX41a9+FYRTHB5RFGHvYOGQEBERUYQIYjSMuQRBV1cXkpOT0dnZiaSkpKAd12yxovih9wAAXzy6GMlxMUE7NhER0fnO1/s3S7d6IU1pBgAlc1iIiIgiggGLFxbHgIU5LERERBHBgMULpx4W5rAQERFFBAMWL6wOAQsLxxEREUUGAxYvpB4WQQAUDFiIiIgiggGLF3KVW+avEBERRQwDFi/konHsXSEiIooYBixe2OMV5q8QERFFEAMWL9jDQkREFHkMWLzgSs1ERESRx4DFC7McsPBXRUREFCm8C3sh9bAwh4WIiChyGLB4wSEhIiKiyGPA4oWZAQsREVHEMWDxwipySIiIiCjSGLB4Ybawh4WIiCjSGLB4wRwWIiKiyGPA4gULxxEREUUeAxYvmMNCREQUeQxYvGAOCxERUeQxYPGCOSxERESRx4DFC9ZhISIiijwGLF4M5LDwV0VERBQpvAt7wRwWIiKiyGPA4gVzWIiIiCKPAYsXzGEhIiKKPAYsXlhYh4WIiCjiGLB4YbGw0i0REVGkMWDxgkNCREREkceAxQsm3RIREUUeAxYvmMNCREQUeQxYvLDIdVj4qyIiIooU3oW9GMhhiXBDiIiIzmO8DXsh5bCwND8REVHk8C7shZTDwqRbIiKiyGHA4sVADwsDFiIiokhhwOKFtPihggELERFRxDBg8cJitVW6ZQ8LERFR5DBg8YI5LERERJHHgMUL5rAQERFFHgMWL5jDQkREFHkMWLxgaX4iIqLIY8DixcDih/xVERERRQrvwl6YmcNCREQUcQxYvLAwh4WIiCjiGLB4wRwWIiKiyGPA4sVADgsDFiIiokhhwOIFc1iIiIgijwGLF1JpfvawEBERRQ4DFi84JERERBR5DFi8YGl+IiKiyGPA4oWUw6IQGLAQERFFiirSDYh23yzJxfzx6Rg/Jj7STSEiIjpvMWDx4lvzCiLdBCIiovMeh4SIiIgo6jFgISIioqjHgIWIiIiiHgMWIiIiinoMWIiIiCjqMWAhIiKiqMeAhYiIiKIeAxYiIiKKegxYiIiIKOoxYCEiIqKoF1DAsmbNGhQWFkKr1aKkpATbt28fcnuDwYCHHnoIBQUF0Gg0KCoqwvr165226ejowL333ovs7GxotVpMmTIFmzZtCqR5RERENMr4vZbQa6+9hhUrVmDNmjUoKyvDc889hyVLluDw4cPIz893u8/SpUvR1NSEdevWobi4GM3NzTCbzfLrRqMRV1xxBTIyMvCf//wHubm5qKurQ2JiYuBnRkRERKOGIIqi6M8O8+bNw+zZs7F27Vr5uSlTpuD666/H6tWrXbZ///33cdNNN6G6uhppaWluj/nXv/4Vv/3tb3H06FHExMT4eQo2XV1dSE5ORmdnJ5KSkgI6BhEREYWXr/dvv3pYjEYj9u7di1WrVjk9v3jxYlRUVLjd5+2338acOXPw1FNP4R//+Afi4+Nx3XXX4Re/+AViY2PlbebPn497770Xb731FsaMGYNbbrkFDzzwAJRKpdvjGgwGGAwG+efOzk75xImIiGhkkO7b3vpP/ApYWlpaYLFYkJmZ6fR8ZmYmGhsb3e5TXV2NHTt2QKvV4o033kBLSwvuuecetLW1yXks1dXV+OSTT/Ctb30LmzZtwokTJ3DvvffCbDbj0UcfdXvc1atX4/HHH3d5Pi8vz59TIiIioijQ3d2N5ORkj6/7ncMCAIIgOP0siqLLcxKr1QpBEPDyyy/LDXn66afxzW9+E3/5y18QGxsLq9WKjIwMPP/881AqlSgpKUF9fT1++9vfegxYHnzwQaxcudLpfdra2pCenu6xLf7o6upCXl4e6urqRu0Q02g/x9F+fgDPcTQY7ecH8BxHg1CenyiK6O7uRk5OzpDb+RWw6HQ6KJVKl96U5uZml14XSXZ2NsaOHesUNU2ZMgWiKOLMmTOYMGECsrOzERMT4zT8M2XKFDQ2NsJoNEKtVrscV6PRQKPROD2XkpLiz+n4JCkpaVT+8Tka7ec42s8P4DmOBqP9/ACe42gQqvMbqmdF4te0ZrVajZKSEpSXlzs9X15ejtLSUrf7lJWVob6+Hj09PfJzx48fh0KhQG5urrxNVVUVrFar0zbZ2dlugxUiIiI6v/hdh2XlypX429/+hvXr1+PIkSO4//77UVtbi7vvvhuAbajm1ltvlbe/5ZZbkJ6ejjvuuAOHDx/Gtm3b8JOf/ATf/va35aTb733ve2htbcV9992H48eP491338Wvf/1r3HvvvUE6TSIiIhrJ/M5hWbZsGVpbW/HEE0+goaEB06dPx6ZNm1BQUAAAaGhoQG1trbx9QkICysvL8YMf/ABz5sxBeno6li5dil/+8pfyNnl5efjwww9x//33Y+bMmRg7dizuu+8+PPDAA0E4xcBoNBr8/Oc/dxl2Gk1G+zmO9vMDeI6jwWg/P4DnOBpEw/n5XYeFiIiIKNy4lhARERFFPQYsREREFPUYsBAREVHUY8BCREREUe+8DljWrFmDwsJCaLValJSUYPv27UNuv3XrVpSUlECr1WL8+PH461//GqaW+m/16tWYO3cuEhMTkZGRgeuvvx7Hjh0bcp8tW7ZAEASXx9GjR8PUat899thjLu3Mysoacp+RdP0AYNy4cW6vh6fp/iPh+m3btg3XXnstcnJyIAgC3nzzTafXRVHEY489hpycHMTGxuIrX/kKDh065PW4GzduxNSpU6HRaDB16lS88cYbITqDoQ11fiaTCQ888ABmzJiB+Ph45OTk4NZbb0V9ff2Qx9ywYYPb69rf3x/is3HP2zW8/fbbXdp68cUXez1utFxDwPs5ursegiDgt7/9rcdjRtN19OX+EI2fxfM2YHnttdewYsUKPPTQQ6isrMTChQuxZMkSpynZjmpqanD11Vdj4cKFqKysxM9+9jP88Ic/xMaNG8Pcct9s3boV9957Lz799FOUl5fDbDZj8eLF6O3t9brvsWPH0NDQID8mTJgQhhb7b9q0aU7tPHjwoMdtR9r1A4DPPvvM6fykgo033njjkPtF8/Xr7e3FBRdcgGeffdbt60899RSefvppPPvss/jss8+QlZWFK664At3d3R6PuWvXLixbtgzLly/HF198geXLl2Pp0qXYvXt3qE7Do6HOT6/XY9++fXjkkUewb98+vP766zh+/Diuu+46r8dNSkpyuqYNDQ3QarWhOAWvvF1DALjqqquc2rpp06YhjxlN1xDwfo6Dr8X69eshCAK+8Y1vDHncaLmOvtwfovKzKJ6nLrroIvHuu+92em7y5MniqlWr3G7/05/+VJw8ebLTc9/97nfFiy++OGRtDKbm5mYRgLh161aP22zevFkEILa3t4evYQH6+c9/Ll5wwQU+bz/Sr58oiuJ9990nFhUViVar1e3rI+n6iaIoAhDfeOMN+Wer1SpmZWWJTz75pPxcf3+/mJycLP71r3/1eJylS5eKV111ldNzV155pXjTTTcFvc3+GHx+7uzZs0cEIJ4+fdrjNi+++KKYnJwc3MYFibtzvO2228Svf/3rfh0nWq+hKPp2Hb/+9a+Ll19++ZDbRPN1HHx/iNbP4nnZw2I0GrF3714sXrzY6fnFixejoqLC7T67du1y2f7KK6/E559/DpPJFLK2BktnZycAIC0tzeu2s2bNQnZ2Nr761a9i8+bNoW5awE6cOIGcnBwUFhbipptuQnV1tcdtR/r1MxqN+Oc//4lvf/vbXhf3HCnXb7Camho0NjY6XSeNRoNLL73U4+cS8Hxth9onWnR2dkIQBK/roPX09KCgoAC5ubm45pprUFlZGZ4GBmjLli3IyMjAxIkT8Z3vfAfNzc1Dbj+Sr2FTUxPeffdd3HnnnV63jdbrOPj+EK2fxfMyYGlpaYHFYnFZsDEzM9NlYUdJY2Oj2+3NZjNaWlpC1tZgEEURK1euxIIFCzB9+nSP22VnZ+P555/Hxo0b8frrr2PSpEn46le/im3btoWxtb6ZN28eXnrpJXzwwQd44YUX0NjYiNLSUrS2trrdfiRfPwB488030dHRgdtvv93jNiPp+rkjffb8+VxK+/m7TzTo7+/HqlWrcMsttwy5mNzkyZOxYcMGvP3223jllVeg1WpRVlaGEydOhLG1vluyZAlefvllfPLJJ/j973+Pzz77DJdffjkMBoPHfUbqNQSAv//970hMTMQNN9ww5HbReh3d3R+i9bPod2n+0WTwN1VRFIf89upue3fPR5vvf//7OHDgAHbs2DHkdpMmTcKkSZPkn+fPn4+6ujr87ne/wyWXXBLqZvplyZIl8v/PmDED8+fPR1FREf7+979j5cqVbvcZqdcPANatW4clS5YMufz6SLp+Q/H3cxnoPpFkMplw0003wWq1Ys2aNUNue/HFFzslrZaVlWH27Nn485//jD/96U+hbqrfli1bJv//9OnTMWfOHBQUFODdd98d8qY+0q6hZP369fjWt77lNRclWq/jUPeHaPssnpc9LDqdDkql0iXqa25udokOJVlZWW63V6lUSE9PD1lbh+sHP/gB3n77bWzevFleHdsfF198ccS/AfgiPj4eM2bM8NjWkXr9AOD06dP46KOPcNddd/m970i5fgDkWV7+fC6l/fzdJ5JMJhOWLl2KmpoalJeXD9m74o5CocDcuXNHzHXNzs5GQUHBkO0daddQsn37dhw7diygz2Y0XEdP94do/SyelwGLWq1GSUmJPOtCUl5ejtLSUrf7zJ8/32X7Dz/8EHPmzEFMTEzI2hooURTx/e9/H6+//jo++eQTFBYWBnScyspKZGdnB7l1wWcwGHDkyBGPbR1p18/Riy++iIyMDHzta1/ze9+Rcv0AoLCwEFlZWU7XyWg0YuvWrR4/l4DnazvUPpEiBSsnTpzARx99FFCwLIoi9u/fP2Kua2trK+rq6oZs70i6ho7WrVuHkpISXHDBBX7vG8nr6O3+ELWfxaCk7o5Ar776qhgTEyOuW7dOPHz4sLhixQoxPj5ePHXqlCiKorhq1Spx+fLl8vbV1dViXFyceP/994uHDx8W161bJ8bExIj/+c9/InUKQ/re974nJicni1u2bBEbGhrkh16vl7cZfI5/+MMfxDfeeEM8fvy4+OWXX4qrVq0SAYgbN26MxCkM6Uc/+pG4ZcsWsbq6Wvz000/Fa665RkxMTBw1109isVjE/Px88YEHHnB5bSRev+7ubrGyslKsrKwUAYhPP/20WFlZKc+SefLJJ8Xk5GTx9ddfFw8ePCjefPPNYnZ2ttjV1SUfY/ny5U6z+Xbu3CkqlUrxySefFI8cOSI++eSTokqlEj/99NOoOj+TySRed911Ym5urrh//36nz6XBYPB4fo899pj4/vvviydPnhQrKyvFO+64Q1SpVOLu3bvDfn6iOPQ5dnd3iz/60Y/EiooKsaamRty8ebM4f/58cezYsSPmGoqi979TURTFzs5OMS4uTly7dq3bY0TzdfTl/hCNn8XzNmARRVH8y1/+IhYUFIhqtVqcPXu205Tf2267Tbz00kudtt+yZYs4a9YsUa1Wi+PGjfP4hxoNALh9vPjii/I2g8/xN7/5jVhUVCRqtVoxNTVVXLBggfjuu++Gv/E+WLZsmZidnS3GxMSIOTk54g033CAeOnRIfn2kXz/JBx98IAIQjx075vLaSLx+0tTrwY/bbrtNFEXbdMqf//znYlZWlqjRaMRLLrlEPHjwoNMxLr30Unl7yb///W9x0qRJYkxMjDh58uSIBWlDnV9NTY3Hz+XmzZvlYww+vxUrVoj5+fmiWq0Wx4wZIy5evFisqKgI/8nZDXWOer1eXLx4sThmzBgxJiZGzM/PF2+77TaxtrbW6RjRfA1F0fvfqSiK4nPPPSfGxsaKHR0dbo8RzdfRl/tDNH4WBXvjiYiIiKLWeZnDQkRERCMLAxYiIiKKegxYiIiIKOoxYCEiIqKox4CFiIiIoh4DFiIiIop6DFiIiIgo6jFgISIioqjHgIWIiIiiHgMWIiIiinoMWIiIiCjqMWAhIiKiqPf/ARvHDVx6+VqzAAAAAElFTkSuQmCC",
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# plot accuracy for different values of C\n",
        "import matplotlib.pyplot as plt\n",
        "plt.plot(lasso_alphas, results.cv_results_ ['mean_test_score'])"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZeFhEy1D03J4"
      },
      "source": [
        "One of the defining features of the LASSO is that it pushes out features that do not contribute enough to the prediction of the outcome Y. We can see which features were pushed out of the model by using the code below."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 37,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8duVyeV1mEUL",
        "outputId": "0f3ba183-e72f-4717-f0e3-b58d085fb36f"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "array([[-0.10956943,  0.        ,  0.36541408, -0.6049318 ,  0.        ,\n",
              "         0.45419673,  0.        ,  1.84794809, -1.5039843 , -0.13483124,\n",
              "        -0.49382215, -1.13648139, -0.02201648,  0.        ,  0.        ,\n",
              "         0.        ,  1.84689614, -0.98488648,  0.        , -0.34572761,\n",
              "         0.        , -0.94607016,  0.        , -0.6197419 ,  0.        ,\n",
              "         0.54516814,  0.        , -0.03862823, -0.5592675 , -0.56872407,\n",
              "         2.02565409, -1.03187124,  0.        ,  0.55933043, -0.26513308,\n",
              "         0.        ,  1.11490617,  0.        , -0.82927453,  0.40732211,\n",
              "         0.        ,  0.0637823 , -0.22552552, -0.19015441, -0.37980055,\n",
              "         0.        ,  0.        , -0.89972939, -1.17803469,  0.59588761,\n",
              "        -0.39860215, -0.5694664 , -0.35693997, -0.72967807,  0.64828519,\n",
              "        -0.48301988,  0.24404671, -0.48547807,  0.        ,  0.0745299 ]])"
            ]
          },
          "execution_count": 37,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# get coefficients\n",
        "results.best_estimator_.coef_"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iiO9FLWv1RUH"
      },
      "source": [
        "### Question 5\n",
        "How many features were pushed out of the model by LASSO's built-in feature selection?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q0PY0C6K1hLX"
      },
      "source": [
        "Finally, we can check the performance of our best LASSO model on the test set. This requires us to prepare the test features exactly like we prepared our training features. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 38,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 356
        },
        "id": "SNok3GWnDQG-",
        "outputId": "f7be04d6-f555-44c2-9c6a-c62da9e3dc9d"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/Users/junda_huang/anaconda3/envs/ai4h_dl/lib/python3.11/site-packages/sklearn/base.py:439: UserWarning: X does not have valid feature names, but LogisticRegression was fitted with feature names\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "<sklearn.metrics._plot.confusion_matrix.ConfusionMatrixDisplay at 0x140b1e490>"
            ]
          },
          "execution_count": 38,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgsAAAG2CAYAAADx48X4AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA6oklEQVR4nO3de1hVZfr/8c9ScIOcFBURBQ9ppuZZKye/iZOp6BiOM5OOZZ5qUjxkOmqOY2AlxEw5HlMnv6nf0qzfpGZWpFMeMrUJFZ3KNBOVNAYbTQREgb1+f5C7duiWzV7IBt+v61rX5XrWWs+6t5H75n6e9SzDNE1TAAAA11CtogMAAADejWQBAAC4RLIAAABcIlkAAAAukSwAAACXSBYAAIBLJAsAAMAlkgUAAOASyQIAAHCJZAEAALhEsgAAQCWUlJSkrl27KigoSGFhYRo4cKAOHz58zfMfe+wxGYahefPmuX0vkgUAACqh7du3a9y4cdqzZ4+2bNmiwsJC9e7dW7m5uSXO3bBhgz755BNFRESU6V4+ngYLAABuvJSUFKf9FStWKCwsTHv37tU999zjaD916pTGjx+v999/X/379y/TvUgWrsNut+v06dMKCgqSYRgVHQ4AwA2maerChQuKiIhQtWrlV0zPz8/X5cuXLenLNM0S3zc2m002m83ldefPn5ckhYaGOtrsdruGDRumqVOnqk2bNmWOiWThOk6fPq3IyMiKDgMA4IGMjAw1atSoXPrOz89X08aByswqsqS/wMBA5eTkOLXFx8crISHhmteYpqnJkyere/fuuv322x3tycnJ8vHx0cSJEz2KiWThOoKCgiRJJ/Y1UXAgUzxQNf02dlBFhwCUi8KiS9p+ZKHj3/LycPnyZWVmFenE3iYKDvLseyL7gl2NOx9XRkaGgoODHe3XqyqMHz9eBw8e1M6dOx1te/fu1fz587Vv3z6PK+MkC9dx5S84OLCaxz8EgLfyqe76HyKgsrsRw8iBQYYCgzy7j10/fOcEBzslC65MmDBBGzdu1I4dO5yqJx999JGysrIUFRXlaCsqKtKUKVM0b948HT9+vNRxkSwAAGCBItOuItPzPkrLNE1NmDBB69ev17Zt29S0aVOn48OGDVOvXr2c2vr06aNhw4Zp5MiRbsVFsgAAgAXsMmWXZ9mCO9ePGzdOa9as0VtvvaWgoCBlZmZKkkJCQuTv7686deqoTp06Ttf4+voqPDxcLVu2dCsu6uoAAFRCS5Ys0fnz5xUdHa0GDRo4ttdff93ye1FZAADAAnbZVfpBhGv3UVqm6X4Vw515Cj9FsgAAgAWKTFNFZfgC/3kf3ohhCAAA4BKVBQAALHCjJzjeSCQLAABYwC5TRVU0WWAYAgAAuERlAQAACzAMAQAAXOJpCAAAcNOisgAAgAXsP2ye9uGNSBYAALBAkQVPQ3h6fXkhWQAAwAJFpix466Q1sViNOQsAAMAlKgsAAFiAOQsAAMAluwwVyfC4D2/EMAQAAHCJygIAABawm8Wbp314I5IFAAAsUGTBMISn15cXhiEAAIBLVBYAALBAVa4skCwAAGABu2nIbnr4NISH15cXhiEAAIBLVBYAALAAwxAAAMClIlVTkYcF+yKLYrEayQIAABYwLZizYDJnAQAAVEZUFgAAsABzFgAAgEtFZjUVmR7OWfDS5Z4ZhgAAAC5RWQAAwAJ2GbJ7+Du4Xd5ZWiBZAADAAlV5zgLDEAAAwCUqCwAAWMCaCY4MQwAAUGUVz1nw8EVSDEMAAIDKiMoCAAAWsFvwbghvfRqCygIAABa4MmfB0620kpKS1LVrVwUFBSksLEwDBw7U4cOHHccLCgo0ffp0tW3bVgEBAYqIiNDDDz+s06dPu/3ZSBYAALCAXdUs2Upr+/btGjdunPbs2aMtW7aosLBQvXv3Vm5uriQpLy9P+/bt06xZs7Rv3z6tW7dOR44c0f333+/2Z2MYAgCASiglJcVpf8WKFQoLC9PevXt1zz33KCQkRFu2bHE6Z+HChbrjjjt08uRJRUVFlfpeJAsAAFigyDRU5OErpq9cn52d7dRus9lks9lcXnv+/HlJUmhoqMtzDMNQrVq13IqLYQgAACxQ9MMER083SYqMjFRISIhjS0pKcnlv0zQ1efJkde/eXbfffvtVz8nPz9eTTz6poUOHKjg42K3PRmUBAAAvk5GR4fSFfr2qwvjx43Xw4EHt3LnzqscLCgo0ZMgQ2e12vfjii27HQ7IAAIAF7GY12T1cwdH+wwqOwcHBpf7tf8KECdq4caN27NihRo0alTheUFCgBx54QOnp6frwww/dripIJAsAAFjip8MIZe+j9OssmKapCRMmaP369dq2bZuaNm1a4pwricJXX32lrVu3qk6dOmWKi2QBAIBKaNy4cVqzZo3eeustBQUFKTMzU5IUEhIif39/FRYW6re//a327dunTZs2qaioyHFOaGioatSoUep7kSwAAGABu+Tx0xB2N85dsmSJJCk6OtqpfcWKFRoxYoS++eYbbdy4UZLUoUMHp3O2bt1a4jpXSBYAALCAu4sqXauP0jKv84bKJk2aXPec0uLRSQAA4BKVBQAALODuux2u1Yc3IlkAAMACdhmyy9M5C55dX15IFgAAsEBVrix4Z1QAAMBrUFkAAMAC1izK5J2/w5MsAABgAbtpyO7pOgseXl9evDOFAQAAXoPKAgAAFrBbMAzh6aJO5YVkAQAAC1jz1knvTBa8MyoAAOA1qCwAAGCBIhkq8nBRJU+vLy8kCwAAWIBhCAAAcNOisgAAgAWK5PkwQpE1oViOZAEAAAtU5WEIkgUAACzAi6QAAMBNi8oCAAAWMGXI7uGcBZNHJwEAqLoYhgAAADctKgsAAFigKr+immQBAAALFFnw1klPry8v3hkVAADwGlQWAACwAMMQAADAJbuqye5hwd7T68uLd0YFAAC8BpUFAAAsUGQaKvJwGMHT68sLyQIAABZgzgIAAHDJtOCtkyYrOAIAgMqIygIAABYokqEiD18E5en15YVkAQAAC9hNz+cc2E2LgrEYwxAAAMClSlNZiI6OVocOHTRv3ryKDgUeWrswTB+/W0sZR22q4WdX6y55Gj3ztCKbX7rq+fOnNdK7r9bVY7NPadCjZ25wtEDZ3N72jH7zuy/V/NZzqlMnX8/E363duxr+5AxTDw77XH37H1NgYIEOfxmqFxd20skTIRUWMzxjt2CCo6fXl5cKjWrEiBEyDENjxowpcSwuLk6GYWjEiBGSpHXr1umZZ565wRGiPBzcHagBI77TvE1fKWnt1yoqkv70+1uUn1fyx3HXeyH6cl+A6oRfroBIgbLz8ytU+rFaWrKo01WP/3bwl/r1b45oyaJOmjS+l86d9dOc5O3y9y+4wZHCKnYZlmyllZSUpK5duyooKEhhYWEaOHCgDh8+7HSOaZpKSEhQRESE/P39FR0drc8//9ztz1bhKUxkZKTWrl2rixcvOtry8/P12muvKSoqytEWGhqqoKCgiggRFktcc0y9B59Vk5b5uqVNvqb87aSyTtXQVwf9nc777ltfLf5zQ01ffEI+laYGBhRL/bSB/m9lW+3a2egqR00N/PVXWvtaK+3a2Ugnjofohb/eIZutSNG/PHnDY0XltH37do0bN0579uzRli1bVFhYqN69eys3N9dxzl/+8hfNnTtXixYt0qeffqrw8HDdd999unDhglv3qvBkoVOnToqKitK6descbevWrVNkZKQ6duzoaIuOjtakSZMc+02aNFFiYqJGjRqloKAgRUVF6e9//7tT36dOndLgwYNVu3Zt1alTR7GxsTp+/Hh5fyS4KTe7uiQpqFaRo81ul/4yMUq/HZulJi3zKyo0oFyEh+cqtE6+9qWGO9oKC6rr3wfrqVXr7yowMnjiygqOnm6llZKSohEjRqhNmzZq3769VqxYoZMnT2rv3r2SiqsK8+bN08yZMzVo0CDdfvvtWrVqlfLy8rRmzRq3PluFJwuSNHLkSK1YscKx//LLL2vUqFHXve6FF15Qly5dtH//fsXFxWns2LH68ssvJUl5eXnq2bOnAgMDtWPHDu3cuVOBgYHq27evLl+mpO0tTFP6e0JDtbkjR01u+zEpeGNxmKpXNzVwNP9wouqpHVr8s/79935O7d+f83McQ+VzZc6Cp1tZnT9/XlJxJV6S0tPTlZmZqd69ezvOsdls6tGjh3bt2uVW316RLAwbNkw7d+7U8ePHdeLECX388cd66KGHrntdv379FBcXp+bNm2v69OmqW7eutm3bJklau3atqlWrpuXLl6tt27Zq1aqVI+u6cs7VXLp0SdnZ2U4bys/iPzVU+iF/zXjxhKPtq4P+2rC8nv4476QM73zkGLCE+bPH5AzDlOmly/3ixvr599ClS1efAH6FaZqaPHmyunfvrttvv12SlJmZKUmqX7++07n169d3HCstrxgJrlu3rvr3769Vq1bJNE31799fdevWve517dq1c/zZMAyFh4crKytLkrR3714dPXq0xDyH/Px8ff3119fsMykpSbNnzy7jJ4E7Fs9sqN2bQ/TC+qOqF/HjpK5/fxKo77/z0UNd2zja7EWGXpodoQ0v1dP//euLiggXsMy5s8UVhdq183Xu7I9zdUJqXdL352wVFRY8ZJcF74b4YYJjZGSkU3t8fLwSEhKued348eN18OBB7dy5s8Qx42e/dZmmWaLterwiWZCkUaNGafz48ZKkxYsXl+oaX19fp33DMGS32yVJdrtdnTt31urVq0tcV69evWv2OWPGDE2ePNmxn52dXeI/GjxjmsWJwq6UEP31H0cVHuU8LNTrN2fV6X+cJ9/8aWgz3fubc+o9+OyNDBUoF5mZATr7Xz916vwfHfu6tiTJx6dIbdud0Yrl7a5zNbyV6ebTDNfqQ5IyMjIUHBzsaLfZrp1ETpgwQRs3btSOHTvUqNGPE2rDw4vnxGRmZqpBgwaO9qysrBLVhuvxmmThp3MJ+vTp43F/nTp10uuvv66wsDCnv/DrsdlsLv+jwHOL/tRIW9fXVsKKY/IPtOtsVvGPYUBQkWz+poJDixQcWuR0jY+PVDus8JprMQDexs+vQBENcxz79cNz1OyWc7qQXUNnzgRow/oWeuD3h3TqVKBOnwrS4N8f0qVL1bXtwygXvcKbWfnWyeDg4Ot+d5mmqQkTJmj9+vXatm2bmjZt6nS8adOmCg8P15YtWxwPDFy+fFnbt29XcnKyW3F5TbJQvXp1HTp0yPFnTz344IP661//qtjYWD399NNq1KiRTp48qXXr1mnq1KlO2RdurE2rioeYpv6mhVP7lL+dpHKAKqPFreeU/MI2x/4fxh6QJG3Z3ER/++sd+sfrt8lWo0jjJuxTYNBlHf6yjv78ZA9dvOh7jR4BZ+PGjdOaNWv01ltvKSgoyDEPISQkRP7+/jIMQ5MmTVJiYqJatGihFi1aKDExUTVr1tTQoUPdupfXJAuS3KoAXE/NmjW1Y8cOTZ8+XYMGDdKFCxfUsGFD3XvvvZbeB+57/3Sa29cwTwGVzb8PhqnffQ+4OMPQ6ldu1+pXbr9hMaF83egVHJcsWSKpeGmBn1qxYoVjQcNp06bp4sWLiouL07lz53TnnXdq8+bNbq9bZJjmz+fj4qeys7MVEhKic0eaKTjIKx4eASzX777BFR0CUC4Kiy7pg0PP6/z58+X2i+KV74nYzaPkG1DDo74Kci/rrd4vl2u8ZcG3HwAAcMmrhiEAAKis3H23w7X68EYkCwAAWMDKpyG8DcMQAADAJSoLAABYoCpXFkgWAACwQFVOFhiGAAAALlFZAADAAlW5skCyAACABUx5/uijt66SSLIAAIAFqnJlgTkLAADAJSoLAABYoCpXFkgWAACwQFVOFhiGAAAALlFZAADAAlW5skCyAACABUzTkOnhl72n15cXhiEAAIBLVBYAALCAXYbHizJ5en15IVkAAMACVXnOAsMQAADAJSoLAABYoCpPcCRZAADAAlV5GIJkAQAAC1TlygJzFgAAgEtUFgAAsIBpwTCEt1YWSBYAALCAKck0Pe/DGzEMAQAAXKKyAACABewyZLCCIwAAuBaehgAAADctKgsAAFjAbhoyWJQJAABci2la8DSElz4OwTAEAABwicoCAAAWqMoTHEkWAACwAMkCAABwqSpPcGTOAgAAldSOHTs0YMAARUREyDAMbdiwwel4Tk6Oxo8fr0aNGsnf31+tWrXSkiVL3L4PyQIAABa48jSEp5s7cnNz1b59ey1atOiqx5944gmlpKTo1Vdf1aFDh/TEE09owoQJeuutt9y6D8MQAABYoPjL3tM5C+6dHxMTo5iYmGse3717t4YPH67o6GhJ0h/+8ActW7ZMqampio2NLfV9qCwAAOBlsrOznbZLly6VqZ/u3btr48aNOnXqlEzT1NatW3XkyBH16dPHrX5IFgAAsMCVpyE83SQpMjJSISEhji0pKalMMS1YsECtW7dWo0aNVKNGDfXt21cvvviiunfv7lY/DEMAAGAB84fN0z4kKSMjQ8HBwY52m81Wpv4WLFigPXv2aOPGjWrcuLF27NihuLg4NWjQQL169Sp1PyQLAAB4meDgYKdkoSwuXryoP/3pT1q/fr369+8vSWrXrp3S0tL0/PPPkywAAHCjeduiTAUFBSooKFC1as4zDqpXry673e5WXyQLAABYwcpxiFLKycnR0aNHHfvp6elKS0tTaGiooqKi1KNHD02dOlX+/v5q3Lixtm/frv/7v//T3Llz3boPyQIAAFawoLIgN69PTU1Vz549HfuTJ0+WJA0fPlwrV67U2rVrNWPGDD344IM6e/asGjdurDlz5mjMmDFu3YdkAQCASio6Olqmi8UZwsPDtWLFCo/vQ7IAAIAFyrIC49X68EYkCwAAWMDbJjhaiUWZAACAS1QWAACwgmm4PUHxqn14IZIFAAAsUJXnLDAMAQAAXKKyAACAFSpgUaYbpVTJwoIFC0rd4cSJE8scDAAAlVVVfhqiVMnC3/72t1J1ZhgGyQIAAFVMqZKF9PT08o4DAIDKz0uHETxV5gmOly9f1uHDh1VYWGhlPAAAVEpXhiE83byR28lCXl6eRo8erZo1a6pNmzY6efKkpOK5Cs8995zlAQIAUCmYFm1eyO1kYcaMGTpw4IC2bdsmPz8/R3uvXr30+uuvWxocAACoeG4/Orlhwwa9/vrruuuuu2QYP5ZLWrdura+//trS4AAAqDyMHzZP+/A+bicLZ86cUVhYWIn23Nxcp+QBAICbShVeZ8HtYYiuXbvqnXfecexfSRBeeukldevWzbrIAACAV3C7spCUlKS+ffvqiy++UGFhoebPn6/PP/9cu3fv1vbt28sjRgAAvB+VhR/94he/0Mcff6y8vDzdcsst2rx5s+rXr6/du3erc+fO5REjAADe78pbJz3dvFCZ3g3Rtm1brVq1yupYAACAFypTslBUVKT169fr0KFDMgxDrVq1UmxsrHx8eC8VAODmVJVfUe32t/tnn32m2NhYZWZmqmXLlpKkI0eOqF69etq4caPatm1reZAAAHg95iz86JFHHlGbNm30zTffaN++fdq3b58yMjLUrl07/eEPfyiPGAEAQAVyu7Jw4MABpaamqnbt2o622rVra86cOerataulwQEAUGlYMUHRSyc4ul1ZaNmypf7zn/+UaM/KylLz5s0tCQoAgMrGMK3ZvFGpKgvZ2dmOPycmJmrixIlKSEjQXXfdJUnas2ePnn76aSUnJ5dPlAAAeLsqPGehVMlCrVq1nJZyNk1TDzzwgKPN/GH65oABA1RUVFQOYQIAgIpSqmRh69at5R0HAACVWxWes1CqZKFHjx7lHQcAAJXbzT4McTV5eXk6efKkLl++7NTerl07j4MCAADeo0yvqB45cqTee++9qx5nzgIA4KZUhSsLbj86OWnSJJ07d0579uyRv7+/UlJStGrVKrVo0UIbN24sjxgBAPB+pkWbF3K7svDhhx/qrbfeUteuXVWtWjU1btxY9913n4KDg5WUlKT+/fuXR5wAAKCCuF1ZyM3NVVhYmCQpNDRUZ86ckVT8Jsp9+/ZZGx0AAJVFFX5FdZlWcDx8+LAkqUOHDlq2bJlOnTqlpUuXqkGDBpYHCABAZXDTr+D4U5MmTdK3334rSYqPj1efPn20evVq1ahRQytXrrQ6PgAAUMHcriw8+OCDGjFihCSpY8eOOn78uD799FNlZGRo8ODBVscHAEDlUAETHHfs2KEBAwYoIiJChmFow4YNJc45dOiQ7r//foWEhCgoKEh33XWXTp486dZ93E4Wfq5mzZrq1KmT6tat62lXAADADbm5uWrfvr0WLVp01eNff/21unfvrttuu03btm3TgQMHNGvWLPn5+bl1n1INQ0yePLnUHc6dO9etAAAAqAoMeT7nwN3pjTExMYqJibnm8ZkzZ6pfv376y1/+4mhr1qyZ23GVKlnYv39/qTr76cumAABA2fz0bc+SZLPZZLPZ3OrDbrfrnXfe0bRp09SnTx/t379fTZs21YwZMzRw4EC3+uJFUqX061vbysfwregwgHLR42BaRYcAlIv8nAJ90O0G3czCF0lFRkY6NcfHxyshIcGtrrKyspSTk6PnnntOzz77rJKTk5WSkqJBgwZp69atbr33qczvhgAAAD9h4XLPGRkZCg4OdjS7W1WQiisLkhQbG6snnnhCUvGSB7t27dLSpUtJFgAAqMyCg4OdkoWyqFu3rnx8fNS6dWun9latWmnnzp1u9UWyAACAFbzsRVI1atRQ165dHQspXnHkyBE1btzYrb5IFgAAsIAVKzC6e31OTo6OHj3q2E9PT1daWppCQ0MVFRWlqVOnavDgwbrnnnvUs2dPpaSk6O2339a2bdvcug/JAgAAlVRqaqp69uzp2L+y1MHw4cO1cuVK/frXv9bSpUuVlJSkiRMnqmXLlnrzzTfVvXt3t+5TpmThlVde0dKlS5Wenq7du3ercePGmjdvnpo2barY2NiydAkAQOVWAcMQ0dHRMk3XF40aNUqjRo3yIKgyrOC4ZMkSTZ48Wf369dP333+voqIiSVKtWrU0b948j4IBAKDSqoDlnm8Ut5OFhQsX6qWXXtLMmTNVvXp1R3uXLl3073//29LgAABAxXN7GCI9PV0dO3Ys0W6z2ZSbm2tJUAAAVDYVMcHxRnG7stC0aVOlpaWVaH/vvfdKPMsJAMBN48oKjp5uXsjtysLUqVM1btw45efnyzRN/etf/9Jrr72mpKQkLV++vDxiBADA+3nZOgtWcjtZGDlypAoLCzVt2jTl5eVp6NChatiwoebPn68hQ4aUR4wAAKAClenRyUcffVSPPvqovvvuO9ntdoWFhVkdFwAAlUpVnrPg0aJMdevWtSoOAAAqN4YhftS0aVMZxrUnYBw7dsyjgAAAgHdxO1mYNGmS035BQYH279+vlJQUTZ061aq4AACoXCwYhqgylYXHH3/8qu2LFy9WamqqxwEBAFApVeFhCLfXWbiWmJgYvfnmm1Z1BwAAvIRlb538xz/+odDQUKu6AwCgcqnClQW3k4WOHTs6TXA0TVOZmZk6c+aMXnzxRUuDAwCgsuDRyZ8YOHCg0361atVUr149RUdH67bbbrMqLgAA4CXcShYKCwvVpEkT9enTR+Hh4eUVEwAA8CJuTXD08fHR2LFjdenSpfKKBwCAysm0aPNCbj8Nceedd2r//v3lEQsAAJXWlTkLnm7eyO05C3FxcZoyZYq++eYbde7cWQEBAU7H27VrZ1lwAACg4pU6WRg1apTmzZunwYMHS5ImTpzoOGYYhkzTlGEYKioqsj5KAAAqAy+tDHiq1MnCqlWr9Nxzzyk9Pb084wEAoHJinYXi9RQkqXHjxuUWDAAA8D5uzVlw9bZJAABuZizK9INbb731ugnD2bNnPQoIAIBKiWGIYrNnz1ZISEh5xQIAALyQW8nCkCFDFBYWVl6xAABQaTEMIeYrAADgUhUehij1Co5XnoYAAAA3l1JXFux2e3nGAQBA5VaFKwtuL/cMAABKYs4CAABwrQpXFtx+6yQAALi5UFkAAMAKVbiyQLIAAIAFqvKcBYYhAACASyQLAABYwbRoc8OOHTs0YMAARUREyDAMbdiw4ZrnPvbYYzIMQ/PmzXPvJiJZAADAEleGITzd3JGbm6v27dtr0aJFLs/bsGGDPvnkE0VERJTpszFnAQCASiomJkYxMTEuzzl16pTGjx+v999/X/379y/TfUgWAACwgoVPQ2RnZzs122w22Ww2t7uz2+0aNmyYpk6dqjZt2pQ5LIYhAACwgoVzFiIjIxUSEuLYkpKSyhRScnKyfHx8NHHixLJ/LlFZAADA62RkZCg4ONixX5aqwt69ezV//nzt27fP4zdHU1kAAMAChkWbJAUHBzttZUkWPvroI2VlZSkqKko+Pj7y8fHRiRMnNGXKFDVp0sStvqgsAABgBS9bwXHYsGHq1auXU1ufPn00bNgwjRw50q2+SBYAALBARazgmJOTo6NHjzr209PTlZaWptDQUEVFRalOnTpO5/v6+io8PFwtW7Z06z4kCwAAVFKpqanq2bOnY3/y5MmSpOHDh2vlypWW3YdkAQAAK1TAMER0dLRMs/QXHT9+3L0b/IBkAQAAq3jpi6A8xdMQAADAJSoLAABYoCq/oppkAQAAK3jZo5NWYhgCAAC4RGUBAAALMAwBAABcYxgCAADcrKgsAABgAYYhAACAa1V4GIJkAQAAK1ThZIE5CwAAwCUqCwAAWIA5CwAAwDWGIQAAwM2KygIAABYwTFOG6VlpwNPrywvJAgAAVmAYAgAA3KyoLAAAYAGehgAAAK4xDAEAAG5WVBYAALAAwxAAAMC1KjwMQbIAAIAFqnJlgTkLAADAJSoLAABYgWEIAABwPd46jOAphiEAAIBLVBYAALCCaRZvnvbhhUgWAACwAE9DAACAmxaVBQAArMDTEAAAwBXDXrx52oc3YhgCAAC4VKUrCwkJCdqwYYPS0tIqOhRcR7XqpoZNydQvB32v2vUKdDbLV1veqK018+rLNI2KDg9wy8nlPvrug+rKSzdUzSYFd7Cr2aQC1Wz6Y435zD+r6dt/+OjCF9VU+L2hzm/kK/A2L61Bo3Sq8DBEhVYWRowYIcMwZBiGfHx8FBUVpbFjx+rcuXMVGRYqwOBxWer/8H+1eGZDPdrjNi1/toF+O/aMYkd9V9GhAW77PrWaIoYUquOrl9Tu75dkFkkHx9RQUd6P59gvGsVJxOMFFRcoLHXlaQhPN3fs2LFDAwYMUEREhAzD0IYNGxzHCgoKNH36dLVt21YBAQGKiIjQww8/rNOnT7v92Sp8GKJv37769ttvdfz4cS1fvlxvv/224uLiKjos3GCtOudq9/sh+tcHwfrPNzW0851a2rc9SC3aX6zo0AC3tVt6WeGxRQpobiqwpamWT1/WpW+r6cIXP/6TW39AkZqMKVTtu7x0kBruu7LOgqebG3Jzc9W+fXstWrSoxLG8vDzt27dPs2bN0r59+7Ru3TodOXJE999/v9sfrcKTBZvNpvDwcDVq1Ei9e/fW4MGDtXnzZkmS3W7X008/rUaNGslms6lDhw5KSUlxuv6bb77RkCFDFBoaqoCAAHXp0kWffPLJVe+Vnp6u5s2ba+zYsbLb+R/Um3z2aYA6dL+ghs0uSZKatb6oNnfk6tMPgyo4MsBzRTnFQ2m+IV5aY0alFRMTo2effVaDBg0qcSwkJERbtmzRAw88oJYtW+quu+7SwoULtXfvXp08edKt+3jVnIVjx44pJSVFvr6+kqT58+frhRde0LJly9SxY0e9/PLLuv/++/X555+rRYsWysnJUY8ePdSwYUNt3LhR4eHh2rdv31UTgc8++0y9e/fW8OHDlZSUdM0YLl26pEuXLjn2s7Ozrf+gKOGNRWEKCLJr+Y4vZS+SqlWXVj4Xrm0bald0aIBHTFP6+q++Cu5YpIAWJAtVmZWLMv38u8dms8lms3nWuaTz58/LMAzVqlXLresqPFnYtGmTAgMDVVRUpPz8fEnS3LlzJUnPP/+8pk+friFDhkiSkpOTtXXrVs2bN0+LFy/WmjVrdObMGX366acKDQ2VJDVv3rzEPXbv3q1f/epXmjFjhv74xz+6jCcpKUmzZ8+28iOiFHrEfq97f3NOz42L0onDfrqlzUWNmX1a//2Pr/75/0IrOjygzI4m+irnK0MdV166/smo3Cyc4BgZGenUHB8fr4SEBI+6zs/P15NPPqmhQ4cqODjYrWsrPFno2bOnlixZory8PC1fvlxHjhzRhAkTlJ2drdOnT+vuu+92Ov/uu+/WgQMHJElpaWnq2LGjI1G4mpMnT6pXr1569tln9cQTT1w3nhkzZmjy5MmO/ezs7BL/0WC9R2d9q9cXhWn7W8WVhONf+iusUYGGTMgiWUCl9VWSr/67rZrar7gsW3hFR4PKJCMjw+kL3dOqQkFBgYYMGSK73a4XX3zR7esrfM5CQECAmjdvrnbt2mnBggW6dOmS02/2huH82Jxpmo42f3//6/Zfr1493XHHHVq7dm2phhRsNpuCg4OdNpQ/m59d5s9Gj+xFkuGtC6UDLpim9FWir777oLraLb8s/0b8HN8MrHwa4uffQ54kCwUFBXrggQeUnp6uLVu2lOl7rcKThZ+Lj4/X888/r5ycHEVERGjnzp1Ox3ft2qVWrVpJktq1a6e0tDSdPXv2mv35+/tr06ZN8vPzU58+fXThwoVyjR9ls2dLsIZMzNId92arfqPL+kXf8xr02BntSgmp6NAAtx2d46v/vFNdrZ67LJ8AU5e/ky5/JxXl/3hOwXkp50tDuceKf/nJO15NOV8auszTwpVXBTwNcT1XEoWvvvpK//znP1WnTp0y9VPhwxA/Fx0drTZt2igxMVFTp05VfHy8brnlFnXo0EErVqxQWlqaVq9eLUn6/e9/r8TERA0cOFBJSUlq0KCB9u/fr4iICHXr1s3RZ0BAgN555x3FxMQoJiZGKSkpCgwMrKiPiKt48c8NNXxapsYnfaNadQr13//46t1X6mj13+pXdGiA206/UfxP64FRzr8Ntnym+JFKSfrvtuo6PKuG49ihacV/bjymQE3iCm9QpKjscnJydPToUcd+enq60tLSFBoaqoiICP32t7/Vvn37tGnTJhUVFSkzM1OSFBoaqho1alyr2xK8LlmQpMmTJ2vkyJE6cuSIsrOzNWXKFGVlZal169bauHGjWrRoIUmqUaOGNm/erClTpqhfv34qLCxU69attXjx4hJ9BgYG6r333lOfPn3Ur18/vffeewoICLjRHw3XcDG3upbGN9TS+IYVHQrgsR4Hr78+SHhskcJjWUekKqmIV1SnpqaqZ8+ejv0rc+6GDx+uhIQEbdy4UZLUoUMHp+u2bt2q6OhoN+IyLa55VDHZ2dkKCQlRtGLlY/hWdDhAuSjNlxtQGeXnFGhOt806f/58uc1Bu/I90a3v0/Lx9fOor8KCfO1Oeapc4y0Lr5uzAAAAvItXDkMAAFDZVMQwxI1CsgAAgBXsZvHmaR9eiGQBAAAr8IpqAABws6KyAACABQxZMGfBkkisR7IAAIAVrFiB0UtXM2AYAgAAuERlAQAAC/DoJAAAcI2nIQAAwM2KygIAABYwTFOGhxMUPb2+vJAsAABgBfsPm6d9eCGGIQAAgEtUFgAAsADDEAAAwLUq/DQEyQIAAFZgBUcAAHCzorIAAIAFWMERAAC4xjAEAAC4WVFZAADAAoa9ePO0D29EsgAAgBUYhgAAADcrKgsAAFiBRZkAAIArVXm5Z4YhAACAS1QWAACwQhWe4EiyAACAFUxJnj766J25AskCAABWYM4CAAC4aVFZAADACqYsmLNgSSSWI1kAAMAKVXiCI8MQAADAJSoLAABYwS7JsKAPL0RlAQAAC1x5GsLTzR07duzQgAEDFBERIcMwtGHDBqfjpmkqISFBERER8vf3V3R0tD7//HO3PxvJAgAAlVRubq7at2+vRYsWXfX4X/7yF82dO1eLFi3Sp59+qvDwcN133326cOGCW/dhGAIAACtUwATHmJgYxcTEXKMrU/PmzdPMmTM1aNAgSdKqVatUv359rVmzRo899lip70NlAQAAK1xJFjzdLJKenq7MzEz17t3b0Waz2dSjRw/t2rXLrb6oLAAA4GWys7Od9m02m2w2m1t9ZGZmSpLq16/v1F6/fn2dOHHCrb6oLAAAYAULKwuRkZEKCQlxbElJSWUOyzCcH9EwTbNE2/VQWQAAwAoWPjqZkZGh4OBgR7O7VQVJCg8Pl1RcYWjQoIGjPSsrq0S14XqoLAAAYAErH50MDg522sqSLDRt2lTh4eHasmWLo+3y5cvavn27fvGLX7jVF5UFAAAqqZycHB09etSxn56errS0NIWGhioqKkqTJk1SYmKiWrRooRYtWigxMVE1a9bU0KFD3boPyQIAAFaogEcnU1NT1bNnT8f+5MmTJUnDhw/XypUrNW3aNF28eFFxcXE6d+6c7rzzTm3evFlBQUFu3YdkAQAAK9hNyfAwWbC7d310dLRMFwmGYRhKSEhQQkKCR2ExZwEAALhEZQEAACtU4VdUkywAAGAJK1Zg9M5kgWEIAADgEpUFAACswDAEAABwyW7K42EEN5+GuFEYhgAAAC5RWQAAwAqmvXjztA8vRLIAAIAVmLMAAABcYs4CAAC4WVFZAADACgxDAAAAl0xZkCxYEonlGIYAAAAuUVkAAMAKDEMAAACX7HZJHq6TYPfOdRYYhgAAAC5RWQAAwAoMQwAAAJeqcLLAMAQAAHCJygIAAFaowss9kywAAGAB07TL9PCtkZ5eX15IFgAAsIJpel4ZYM4CAACojKgsAABgBdOCOQteWlkgWQAAwAp2u2R4OOfAS+csMAwBAABcorIAAIAVGIYAAACumHa7TA+HIbz10UmGIQAAgEtUFgAAsALDEAAAwCW7KRlVM1lgGAIAALhEZQEAACuYpiRP11nwzsoCyQIAABYw7aZMD4chTJIFAACqMNMuzysLPDoJAAAsUlhYqD//+c9q2rSp/P391axZMz399NOy261POKgsAABggRs9DJGcnKylS5dq1apVatOmjVJTUzVy5EiFhITo8ccf9yiOnyNZAADACjd4GGL37t2KjY1V//79JUlNmjTRa6+9ptTUVM9iuAqSheu4kuUVqsDjtTYAb5WfU1DRIQDl4lJuoaQbM3HQiu+JQhX/v5idne3UbrPZZLPZnNq6d++upUuX6siRI7r11lt14MAB7dy5U/PmzfMsiKsx4VJGRsaVJbnY2NjY2CrplpGRUW7fExcvXjTDw8MtizUwMLBEW3x8fIn72u1288knnzQNwzB9fHxMwzDMxMTEcvmMVBauIyIiQhkZGQoKCpJhGBUdTpWXnZ2tyMhIZWRkKDg4uKLDASzHz/iNZZqmLly4oIiIiHK7h5+fn9LT03X58mVL+jNNs8T3zc+rCpL0+uuv69VXX9WaNWvUpk0bpaWladKkSYqIiNDw4cMtieUKwzS99KFO3JSys7MVEhKi8+fP8w8pqiR+xmGVyMhIPfnkkxo3bpyj7dlnn9Wrr76qL7/80tJ78egkAACVUF5enqpVc/4ar169Oo9OAgCAYgMGDNCcOXMUFRWlNm3aaP/+/Zo7d65GjRpl+b1IFuBVbDab4uPjrzo+B1QF/IzDKgsXLtSsWbMUFxenrKwsRURE6LHHHtNTTz1l+b2YswAAAFxizgIAAHCJZAEAALhEsgAAAFwiWUCFi46O1qRJkyo6DMCrJCQkqEOHDhUdBiCJZAHlZMSIETIMQ2PGjClxLC4uToZhaMSIEZKkdevW6ZlnnrnBEQKeufIzbhiGfHx8FBUVpbFjx+rcuXMVHRpgOZIFlJvIyEitXbtWFy9edLTl5+frtddeU1RUlKMtNDRUQUFBFREi4JG+ffvq22+/1fHjx7V8+XK9/fbbiouLq+iwAMuRLKDcdOrUSVFRUVq3bp2jbd26dYqMjFTHjh0dbT8fhmjSpIkSExM1atQoBQUFKSoqSn//+9+d+j516pQGDx6s2rVrq06dOoqNjdXx48fL+yMBTmw2m8LDw9WoUSP17t1bgwcP1ubNmyVJdrtdTz/9tBo1aiSbzaYOHTooJSXF6fpvvvlGQ4YMUWhoqAICAtSlSxd98sknV71Xenq6mjdvrrFjx5bLCn2AKyQLKFcjR47UihUrHPsvv/xyqVYXe+GFF9SlSxft379fcXFxGjt2rGOt87y8PPXs2VOBgYHasWOHdu7cqcDAQPXt29eyF7kA7jp27JhSUlLk6+srSZo/f75eeOEFPf/88zp48KD69Omj+++/X1999ZUkKScnRz169NDp06e1ceNGHThwQNOmTbtqIvDZZ5/p7rvv1u9+9zstWbKkxBK/QLkrl3dZ4qY3fPhwMzY21jxz5oxps9nM9PR08/jx46afn5955swZMzY21hw+fLhpmqbZo0cP8/HHH3dc27hxY/Ohhx5y7NvtdjMsLMxcsmSJaZqm+b//+79my5YtTbvd7jjn0qVLpr+/v/n+++/fkM8HDB8+3KxevboZEBBg+vn5OV4lPHfuXNM0TTMiIsKcM2eO0zVdu3Y14+LiTNM0zWXLlplBQUHmf//736v2Hx8fb7Zv397ctWuXGRoaav71r38t3w8EuMByzyhXdevWVf/+/bVq1SqZpqn+/furbt26172uXbt2jj8bhqHw8HBlZWVJkvbu3aujR4+WmOeQn5+vr7/+2toPALjQs2dPLVmyRHl5eVq+fLmOHDmiCRMmKDs7W6dPn9bdd9/tdP7dd9+tAwcOSJLS0tLUsWNHhYaGXrP/kydPqlevXnr22Wf1xBNPlOtnAVwhWUC5GzVqlMaPHy9JWrx4camuuVLKvcIwDEd51m63q3Pnzlq9enWJ6+rVq+dhtEDpBQQEqHnz5pKkBQsWqGfPnpo9e7amTp0qqfjn9qdM03S0+fv7X7f/evXqKSIiQmvXrtXo0aN5pTUqDANfKHdX5hJcvnxZffr08bi/Tp066auvvlJYWJiaN2/utIWEhFgQMVA28fHxev7555WTk6OIiAjt3LnT6fiuXbvUqlUrScXVs7S0NJ09e/aa/fn7+2vTpk3y8/NTnz59dOHChXKNH7gWkgWUu+rVq+vQoUM6dOiQqlev7nF/Dz74oOrWravY2Fh99NFHSk9P1/bt2/X444/rm2++sSBioGyio6PVpk0bJSYmaurUqUpOTtbrr7+uw4cP68knn1RaWpoef/xxSdLvf/97hYeHa+DAgfr444917Ngxvfnmm9q9e7dTnwEBAXrnnXfk4+OjmJgY5eTkVMRHw02OZAE3RHBwsGUl1Jo1a2rHjh2KiorSoEGD1KpVK40aNUoXL16kTIsKN3nyZL300kv69a9/rSlTpmjKlClq27atUlJStHHjRrVo0UKSVKNGDW3evFlhYWHq16+f2rZtq+eee+6qCXVgYKDee+89maapfv36KTc390Z/LNzkeEU1AABwicoCAABwiWQBAAC4RLIAAABcIlkAAAAukSwAAACXSBYAAIBLJAsAAMAlkgWgEkhISFCHDh0c+yNGjNDAgQNveBzHjx+XYRhKS0u75jlNmjTRvHnzSt3nypUrVatWLY9jMwxDGzZs8LgfACWRLABlNGLECBmGIcMw5Ovrq2bNmumPf/zjDVldb/78+Vq5cmWpzi3NFzwAuMJbJwEP9O3bVytWrFBBQYE++ugjPfLII8rNzdWSJUtKnFtQUFDibZplxQuzANxIVBYAD9hsNoWHhysyMlJDhw7Vgw8+6CiFXxk6ePnll9WsWTPZbDaZpqnz58/rD3/4g8LCwhQcHKxf/vKXOnDggFO/zz33nOrXr6+goCCNHj1a+fn5Tsd/Pgxht9uVnJys5s2by2azKSoqSnPmzJEkNW3aVJLUsWNHGYah6Ohox3UrVqxQq1at5Ofnp9tuu00vvvii033+9a9/qWPHjvLz81OXLl20f/9+t/+O5s6dq7Zt2yogIECRkZGKi4u76suQNmzYoFtvvVV+fn667777lJGR4XT87bffVufOneXn56dmzZpp9uzZKiwsdDseAO4jWQAs5O/vr4KCAsf+0aNH9cYbb+jNN990DAP0799fmZmZevfdd7V371516tRJ9957r+NVxW+88Ybi4+M1Z84cpaamqkGDBiW+xH9uxowZSk5O1qxZs/TFF19ozZo1ql+/vqTiL3xJ+uc//6lvv/1W69atkyS99NJLmjlzpubMmaNDhw4pMTFRs2bN0qpVqyRJubm5+tWvfqWWLVtq7969SkhI0B//+Ee3/06qVaumBQsW6LPPPtOqVav04Ycfatq0aU7n5OXlac6cOVq1apU+/vhjZWdna8iQIY7j77//vh566CFNnDhRX3zxhZYtW6aVK1c6EiIA5cwEUCbDhw83Y2NjHfuffPKJWadOHfOBBx4wTdM04+PjTV9fXzMrK8txzgcffGAGBweb+fn5Tn3dcsst5rJly0zTNM1u3bqZY8aMcTp+5513mu3bt7/qvbOzs02bzWa+9NJLV40zPT3dlGTu37/fqT0yMtJcs2aNU9szzzxjduvWzTRN01y2bJkZGhpq5ubmOo4vWbLkqn39VOPGjc2//e1v1zz+xhtvmHXq1HHsr1ixwpRk7tmzx9F26NAhU5L5ySefmKZpmv/zP/9jJiYmOvXzyiuvmA0aNHDsSzLXr19/zfsCKDvmLAAe2LRpkwIDA1VYWKiCggLFxsZq4cKFjuONGzdWvXr1HPt79+5VTk6O6tSp49TPxYsX9fXXX0uSDh06pDFjxjgd79atm7Zu3XrVGA4dOqRLly7p3nvvLXXcZ86cUUZGhkaPHq1HH33U0V5YWOiYD3Ho0CG1b99eNWvWdIrDXVu3blViYqK++OILZWdnq7CwUPn5+crNzVVAQIAkycfHR126dHFcc9ttt6lWrVo6dOiQ7rjjDu3du1effvqpUyWhqKhI+fn5ysvLc4oRgPVIFgAP9OzZU0uWLJGvr68iIiJKTGC88mV4hd1uV4MGDbRt27YSfZX18UF/f3+3r7Hb7ZKKhyLuvPNOp2PVq1eXJJkWvL3+xIkT6tevn8aMGaNnnnlGoaGh2rlzp0aPHu00XCMVP/r4c1fa7Ha7Zs+erUGDBpU4x8/Pz+M4AbhGsgB4ICAgQM2bNy/1+Z06dVJmZqZ8fHzUpEmTq57TqlUr7dmzRw8//LCjbc+ePdfss0WLFvL399cHH3ygRx55pMTxGjVqSCr+TfyK+vXrq2HDhjp27JgefPDBq/bbunVrvfLKK7p48aIjIXEVx9WkpqaqsLBQL7zwgqpVK54i9cYbb5Q4r7CwUKmpqbrjjjskSYcPH9b333+v2267TVLx39vhw4fd+rsGYB2SBeAG6tWrl7p166aBAwcqOTlZLVu21OnTp/Xuu+9q4MCB6tKlix5//HENHz5cXbp0Uffu3bV69Wp9/vnnatas2VX79PPz0/Tp0zVt2jTVqFFDd999t86cOaPPP/9co0ePVlhYmPz9/ZWSkqJGjRrJz89PISEhSkhI0MSJExUcHKyYmBhdunRJqampOnfunCZPnqyhQ4dq5syZGj16tP785z/r+PHjev755936vLfccosKCwu1cOFCDRgwQB9//LGWLl1a4jxfX19NmDBBCxYskK+vr8aPH6+77rrLkTw89dRT+tWvfqXIyEj97ne/U7Vq1XTw4EH9+9//1rPPPuv+fwgAbuFpCOAGMgxD7777ru655x6NGjVKt956q4YMGaLjx487nl4YPHiwnnrqKU2fPl2dO3fWiRMnNHbsWJf9zpo1S1OmTNFTTz2lVq1aafDgwcrKypJUPB9gwYIFWrZsmSIiIhQbGytJeuSRR7R8+XKtXLlSbdu2VY8ePbRy5UrHo5aBgYF6++239cUXX6hjx46aOXOmkpOT3fq8HTp00Ny5c5WcnKzbb79dq1evVlJSUonzatasqenTp2vo0KHq1q2b/P39tXbtWsfxPn36aNOmTdqyZYu6du2qu+66S3PnzlXjxo3digdA2RimFQOTAACgyqKyAAAAXCJZAAAALpEsAAAAl0gWAACASyQLAADAJZIFAADgEskCAABwiWQBAAC4RLIAAABcIlkAAAAukSwAAACXSBYAAIBL/x+/yl6IXSUS4AAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 640x480 with 2 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "X_test_prep = sc.transform(X_test)\n",
        "cm = confusion_matrix(y_test, results.best_estimator_.predict(X_test_prep), labels=results.classes_)\n",
        "disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=results.classes_)\n",
        "disp.plot()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2vPLVdkb10Hs"
      },
      "source": [
        "### Question 6\n",
        "- Why do we use sc.transform instead of sc.fit_transform to prepare the test data?\n",
        "- What should be added to the code to get rid of the UserWarning that is shown with the confusion matrix above?"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 39,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JjpDwAj_45Ay",
        "outputId": "9def8aaa-5544-4c15-fa82-314b36e72769"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/Users/junda_huang/anaconda3/envs/ai4h_dl/lib/python3.11/site-packages/sklearn/base.py:439: UserWarning: X does not have valid feature names, but LogisticRegression was fitted with feature names\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "data": {
            "text/plain": [
              "0.7142857142857143"
            ]
          },
          "execution_count": 39,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "accuracy_score(results.predict(X_test_prep), y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cfmQRaoY3Pes"
      },
      "source": [
        "### Bonus Question 7\n",
        "Train and evaluate a Random Forest model"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 43,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<sklearn.metrics._plot.confusion_matrix.ConfusionMatrixDisplay at 0x140b05410>"
            ]
          },
          "execution_count": 43,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgsAAAGwCAYAAAAnuiblAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAAA2WElEQVR4nO3deXRUZbb38d8JCZU5GCCTCSGMoswEAfFKsGUIagfpa4NgXxAcIIrSqNhKK3EgERtocKJtUEj7guBVVJwidCvQqNiAgIqRQQIEIYJMgcxJnfcPpK4xoUhRJ6mi+H7WOmtRZ3jOLmWRnb2f8xzDNE1TAAAAZ+Hn6QAAAIB3I1kAAABOkSwAAACnSBYAAIBTJAsAAMApkgUAAOAUyQIAAHDK39MBeDu73a4DBw4oLCxMhmF4OhwAgAtM09TJkycVFxcnP7/6+/24tLRU5eXllozVuHFjBQYGWjKWVUgWzuHAgQNKSEjwdBgAADfk5+crPj6+XsYuLS1VUmKoCg5VWTJeTEyM8vLyvCphIFk4h7CwMEnS3i9bKjyUrg18003tOnk6BKBeVKpC6/SB49/y+lBeXq6CQ1Xau6mlwsPc+zlReNKuxB57VF5eTrJwITnTeggP9XP7LwHgrfyNAE+HANSPn19o0BBt5NAwQ6Fh7t3HLu9sd5MsAABggSrTrio337ZUZdqtCcZiJAsAAFjALlN2uZctuHt9faGuDgAAnKKyAACABeyyy90mgvsj1A+SBQAALFBlmqoy3WsjuHt9faENAQAAnKKyAACABXx5giPJAgAAFrDLVJWPJgu0IQAAgFNUFgAAsABtCAAA4BRPQwAAgIsWlQUAACxg/3lzdwxvRLIAAIAFqix4GsLd6+sLyQIAABaoMmXBWyeticVqzFkAAABOUVkAAMACzFkAAABO2WWoSobbY3gj2hAAAMApKgsAAFjAbp7e3B3DG5EsAABggSoL2hDuXl9faEMAAACnqCwAAGABX64skCwAAGABu2nIbrr5NISb19cX2hAAAMApKgsAAFiANgQAAHCqSn6qcrNgX2VRLFYjWQAAwAKmBXMWTOYsAAAAq8ybN0+dO3dWeHi4wsPD1adPH3344YeO46ZpKiMjQ3FxcQoKClJKSoq2bdt2XvciWQAAwAJn5iy4u9VVfHy8nn76aW3cuFEbN27Utddeq7S0NEdC8Mwzz2j27Nl6/vnntWHDBsXExGjAgAE6efKky9+NZAEAAAtUmX6WbHV14403asiQIWrXrp3atWun6dOnKzQ0VOvXr5dpmpozZ46mTp2qYcOGqWPHjsrOzlZxcbGWLFni8ncjWQAAwMsUFhZW28rKypyeX1VVpaVLl6qoqEh9+vRRXl6eCgoKNHDgQMc5NptN/fr102effeZyPCQLAABYwC5Ddvm5uZ1uQyQkJCgiIsKxZWVl1XrPr7/+WqGhobLZbBo/frzeeustXX755SooKJAkRUdHVzs/OjraccwVPA0BAIAFrFxnIT8/X+Hh4Y79Nput1vPbt2+vLVu26Pjx43rzzTc1evRorVmzxnHcMKrHY5pmjX11QbIAAICXOfOEw7k0btxYbdq0kSQlJydrw4YNmjt3rh566CFJUkFBgWJjYx3nHzp0qEa1oS5oQwAAYIGGnuBYG9M0VVZWpqSkJMXExGjVqlWOY+Xl5VqzZo2uuuoql8elsgAAgAVOz1lw80VSLlz/yCOPKDU1VQkJCTp58qSWLl2q1atXKycnR4ZhaNKkScrMzFTbtm3Vtm1bZWZmKjg4WCNHjnQ5LpIFAAAuQD/++KP+8Ic/6ODBg4qIiFDnzp2Vk5OjAQMGSJKmTJmikpISpaen69ixY+rVq5dWrlypsLAwl+9FsgAAgAXsFrwbwi6zzue+/PLLTo8bhqGMjAxlZGS4FZNEsgAAgCWsmHNQZdY9WWhIJAsAAFjgzFoJ7o3hnckCT0MAAACnqCwAAGCBKtNQlZuvmHb3+vpCsgAAgAWqLJjgWEUbAgAAXIioLAAAYAG76Se7m09D2HkaAgAA30UbAgAAXLSoLAAAYAG73H+awW5NKJYjWQAAwALWLMrknQV/74wKAAB4DSoLAABYwJp3Q3jn7/AkCwAAWMAuQ3a5O2eBFRwBAPBZvlxZ8M6oAACA16CyAACABaxZlMk7f4cnWQAAwAJ205Dd3XUWvPStk96ZwgAAAK9BZQEAAAvYLWhDeOuiTCQLAABYwJq3TnpnsuCdUQEAAK9BZQEAAAtUyVCVm4squXt9fSFZAADAArQhAADARYvKAgAAFqiS+22EKmtCsRzJAgAAFvDlNgTJAgAAFuBFUgAA4KJFZQEAAAuYMmR3c86CyaOTAAD4LtoQAADgokVlAQAAC/jyK6pJFgAAsECVBW+ddPf6+uKdUQEAAK9BZQEAAAvQhgAAAE7Z5Se7mwV7d6+vL94ZFQAA8BpUFgAAsECVaajKzTaCu9fXF5IFAAAswJwFAADglGnBWydNVnAEAAAXIioLAABYoEqGqtx8EZS719cXkgUAACxgN92fc2A3LQrGYrQhAACAU1QW0ODezW6q9//RTD/mN5YkJbYv1ag/FqjntSclSa/OjNHqd5ro8IEABTQ21aZTiW7700Fd1r3Yk2EDbunY65RuTj+stp2K1TSmUhljW+rznAhPhwUL2S2Y4Oju9fXFO6OqRUpKiiZNmuTpMGCB5rEVGvvIAT334Q499+EOdel7Uhm3JWnP9kBJ0qWtSnX39P166ePtmvX2LsUklOvhW1rr+JFGHo4cOH+BwXbt3haoF6Ze6ulQUE/sMizZvJFHk4UxY8bIMAyNHz++xrH09HQZhqExY8ZIkpYvX64nn3yygSNEfeg9sFBX/uak4luXKb51mW77U4ECQ+z6blOwJOnaYcfV/ZpTik0sV8v2pboz4wcVn2ykvG+DPBw5cP42fhKu7Gdi9emHTTwdCnxEVlaWevbsqbCwMEVFRWno0KHavn17tXPO/Jz95da7d2+X7+XxykJCQoKWLl2qkpISx77S0lK99tpratGihWNfZGSkwsLCPBEi6lFVlbT67SYqK/ZTh+SiGscryg198P+aKiS8Sq0uL6llBADwDmdWcHR3q6s1a9bo7rvv1vr167Vq1SpVVlZq4MCBKiqq/m/p4MGDdfDgQcf2wQcfuPzdPD5noXv37tq9e7eWL1+uUaNGSTpdRUhISFCrVq0c56WkpKhr166aM2eOJKlly5a68847tWvXLv3v//6vLrnkEv35z3/WnXfe6bjmhx9+0OTJk7Vy5Ur5+fnp6quv1ty5c9WyZcuG/IqoRV5uoCbd2FblZX4KCrHrsZfzlNiuzHF8/apwZU1IVFmJnyKjK5S1dJcimlZ5MGIAcK6h5yzk5ORU+7xw4UJFRUVp06ZNuuaaaxz7bTabYmJi3IrL45UFSbrtttu0cOFCx+dXXnlFY8eOPed1s2bNUnJysjZv3qz09HRNmDBB3333nSSpuLhY/fv3V2hoqNauXat169YpNDRUgwcPVnl5+VnHLCsrU2FhYbUN1otvXaYXV23X3Pd26Ib/+Ukz70vU3h02x/GufU/pxVXb9dcVO5WcclLT72qp4z95PLcFgAbx659DZWVl57zmxIkTkk5X4n9p9erVioqKUrt27XTHHXfo0KFDLsfjFcnCH/7wB61bt0579uzR3r179emnn+rWW28953VDhgxRenq62rRpo4ceekjNmjXT6tWrJUlLly6Vn5+fFixYoE6dOqlDhw5auHCh9u3b5zinNllZWYqIiHBsCQkJFn1L/FJAY1OXJpWrXZcSjX3koJIuL9HbC5o7jgcG23VpUrk69CjW5Nn5auQv5bwW6WREAPAsuwzH+yHOe/t5gmNCQkK1n0VZWVlO722apiZPnqyrr75aHTt2dOxPTU3V4sWL9fHHH2vWrFnasGGDrr322jolH7/kFb+qNWvWTNdff72ys7Nlmqauv/56NWvW7JzXde7c2fFnwzAUExPjyJg2bdqkXbt21ZjnUFpaqu+///6sYz788MOaPHmy43NhYSEJQwOpKD977mqaUkWZV+S2AFAr04KnGcyfr8/Pz1d4eLhjv81mO9slkqR77rlHX331ldatW1dt//Dhwx1/7tixo5KTk5WYmKj3339fw4YNq3NcXpEsSNLYsWN1zz33SJJeeOGFOl0TEBBQ7bNhGLLb7ZIku92uHj16aPHixTWua968eY19Z9hstnP+T4F7XsmKVc9rC9U8rkIlp/y0+p0m+uqzUD21+HuVFvtpydxo9Rl4QpHRFSo86q/3spvpp4MB+q8bj3s6dOC8BQZXKS7p/1qgMQnlanVFiU4eb6TDPzT2YGSwipVvnQwPD6+WLDgzceJErVixQmvXrlV8fLzTc2NjY5WYmKidO3e6FJfXJAu/nEswaNAgt8fr3r27li1bpqioqDr/B0fDOH7YX3+ZmKijh/wVHFalpA6lemrx9+rR75TKSw3t32XTk//bUoVH/RV2SZXadSnWrLd2qmX7Uk+HDpy3dl1K9Jc3/6+qOf7xA5Kklcsu0aw/tjjbZcBZmaapiRMn6q233tLq1auVlJR0zmuOHDmi/Px8xcbGunQvr0kWGjVqpNzcXMef3TVq1Cj95S9/UVpamp544gnFx8dr3759Wr58uR588MFzZl+oP5Nn55/1WONAU4+9vKfhggEayFefh2pQXBdPh4F61NBPQ9x9991asmSJ3nnnHYWFhamgoECSFBERoaCgIJ06dUoZGRn63e9+p9jYWO3Zs0ePPPKImjVrpptuusmluLyqCexK2eVcgoODtXbtWrVo0ULDhg1Thw4dNHbsWJWUlFBpAABYzu3JjS62MebNm6cTJ04oJSVFsbGxjm3ZsmWSTv/i/fXXXystLU3t2rXT6NGj1a5dO33++ecur1tkmKbppe+48g6FhYWKiIjQsR2tFB7mVbkVYJlBcV09HQJQLyrNCq3WOzpx4kS9/aJ45udE2sqxCghxb/5JRVG53hn4Sr3Gez68pg0BAMCFzIp3O3jruyFIFgAAsICVT0N4G+rqAADAKSoLAABYwJcrCyQLAABYwJeTBdoQAADAKSoLAABYwJcrCyQLAABYwJT7jz5668JHJAsAAFjAlysLzFkAAABOUVkAAMACvlxZIFkAAMACvpws0IYAAABOUVkAAMACvlxZIFkAAMACpmnIdPOHvbvX1xfaEAAAwCkqCwAAWMAuw+1Fmdy9vr6QLAAAYAFfnrNAGwIAADhFZQEAAAv48gRHkgUAACzgy20IkgUAACzgy5UF5iwAAACnqCwAAGAB04I2hLdWFkgWAACwgCnJNN0fwxvRhgAAAE5RWQAAwAJ2GTJYwREAAJwNT0MAAICLFpUFAAAsYDcNGSzKBAAAzsY0LXgawksfh6ANAQAAnKKyAACABXx5giPJAgAAFiBZAAAATvnyBEfmLAAAAKeoLAAAYAFffhqCZAEAAAucThbcnbNgUTAWow0BAACcorIAAIAFeBoCAAA4Zf68uTuGN6INAQAAnKKyAACABWhDAAAA53y4D0GyAACAFSyoLMhLKwvMWQAAAE5RWQAAwAK+vIIjlQUAACxwZoKju1tdZWVlqWfPngoLC1NUVJSGDh2q7du3/yomUxkZGYqLi1NQUJBSUlK0bds2l78byQIAABegNWvW6O6779b69eu1atUqVVZWauDAgSoqKnKc88wzz2j27Nl6/vnntWHDBsXExGjAgAE6efKkS/eiDQEAgBVMw/0Jij9fX1hYWG23zWaTzWarti8nJ6fa54ULFyoqKkqbNm3SNddcI9M0NWfOHE2dOlXDhg2TJGVnZys6OlpLlizRXXfdVeewqCwAAGCBM3MW3N0kKSEhQREREY4tKyvrnPc/ceKEJCkyMlKSlJeXp4KCAg0cONBxjs1mU79+/fTZZ5+59N2oLAAA4GXy8/MVHh7u+PzrqsKvmaapyZMn6+qrr1bHjh0lSQUFBZKk6OjoaudGR0dr7969LsVDsgAAgBUsXJQpPDy8WrJwLvfcc4+++uorrVu3rsYxw6jeGjFNs8a+c6ENAQCABRr6aYgzJk6cqBUrVuiTTz5RfHy8Y39MTIyk/6swnHHo0KEa1YZzqVNl4dlnn63zgPfee69LAQAAANeZpqmJEyfqrbfe0urVq5WUlFTteFJSkmJiYrRq1Sp169ZNklReXq41a9ZoxowZLt2rTsnCX//61zoNZhgGyQIA4OLVgIsq3X333VqyZIneeecdhYWFOSoIERERCgoKkmEYmjRpkjIzM9W2bVu1bdtWmZmZCg4O1siRI126V52Shby8PNe/BQAAF5GGfuvkvHnzJEkpKSnV9i9cuFBjxoyRJE2ZMkUlJSVKT0/XsWPH1KtXL61cuVJhYWEuxXXeExzLy8uVl5en1q1by9+feZIAgItcA7910qzD2tCGYSgjI0MZGRnnH5POY4JjcXGxxo0bp+DgYF1xxRXat2+fpNNzFZ5++mm3ggEAAN7H5WTh4Ycf1tatW7V69WoFBgY69l933XVatmyZpcEBAHDhMCzavI/L/YO3335by5YtU+/evas9p3n55Zfr+++/tzQ4AAAuGA3chmhILlcWDh8+rKioqBr7i4qKXF7kAQAAeD+Xk4WePXvq/fffd3w+kyDMnz9fffr0sS4yAAAuJKZFmxdyuQ2RlZWlwYMH69tvv1VlZaXmzp2rbdu26fPPP9eaNWvqI0YAALyfhW+d9DYuVxauuuoqffrppyouLlbr1q21cuVKRUdH6/PPP1ePHj3qI0YAAOBB57VAQqdOnZSdnW11LAAAXLB++Yppd8bwRueVLFRVVemtt95Sbm6uDMNQhw4dlJaWxuJMAICLlw8/DeHyT/dvvvlGaWlpKigoUPv27SVJO3bsUPPmzbVixQp16tTJ8iABAIDnuDxn4fbbb9cVV1yh/fv368svv9SXX36p/Px8de7cWXfeeWd9xAgAgPc7M8HR3c0LuVxZ2Lp1qzZu3KhLLrnEse+SSy7R9OnT1bNnT0uDAwDgQmGYpzd3x/BGLlcW2rdvrx9//LHG/kOHDqlNmzaWBAUAwAXHh9dZqFOyUFhY6NgyMzN177336o033tD+/fu1f/9+vfHGG5o0aZJmzJhR3/ECAIAGVqc2RJMmTaot5Wyapn7/+9879p15TeaNN96oqqqqeggTAAAv58OLMtUpWfjkk0/qOw4AAC5sF/ujk/369avvOAAAgJc671WUiouLtW/fPpWXl1fb37lzZ7eDAgDggnOxVxZ+6fDhw7rtttv04Ycf1nqcOQsAgIuSDycLLj86OWnSJB07dkzr169XUFCQcnJylJ2drbZt22rFihX1ESMAAPAglysLH3/8sd555x317NlTfn5+SkxM1IABAxQeHq6srCxdf/319REnAADezYefhnC5slBUVKSoqChJUmRkpA4fPizp9Jsov/zyS2ujAwDgAnFmBUd3N290Xis4bt++XZLUtWtXvfTSS/rhhx/0t7/9TbGxsZYHCAAAPMvlNsSkSZN08OBBSdK0adM0aNAgLV68WI0bN9aiRYusjg8AgAuDD09wdDlZGDVqlOPP3bp10549e/Tdd9+pRYsWatasmaXBAQAAzzvvdRbOCA4OVvfu3a2IBQCAC5YhC946aUkk1qtTsjB58uQ6Dzh79uzzDgYAAHifOiULmzdvrtNgv3zZlK+5bupYNQoI9HQYQL0oeKXS0yEA9cJeUiqlv9MwN/PhRyd5kRQAAFbw4QmOLj86CQAALi5uT3AEAADy6coCyQIAABawYgVGn1nBEQAAXFyoLAAAYAUfbkOcV2Xh1VdfVd++fRUXF6e9e/dKkubMmaN33mmgx1MAAPA2pkWbF3I5WZg3b54mT56sIUOG6Pjx46qqqpIkNWnSRHPmzLE6PgAA4GEuJwvPPfec5s+fr6lTp6pRo0aO/cnJyfr6668tDQ4AgAuFL7+i2uU5C3l5eerWrVuN/TabTUVFRZYEBQDABceHV3B0ubKQlJSkLVu21Nj/4Ycf6vLLL7ciJgAALjw+PGfB5crCgw8+qLvvvlulpaUyTVP/+c9/9NprrykrK0sLFiyojxgBAIAHuZws3HbbbaqsrNSUKVNUXFyskSNH6tJLL9XcuXM1YsSI+ogRAACv58uLMp3XOgt33HGH7rjjDv3000+y2+2KioqyOi4AAC4sPrzOgluLMjVr1syqOAAAgJdyOVlISkqSYZx9tubu3bvdCggAgAuSFY8++kplYdKkSdU+V1RUaPPmzcrJydGDDz5oVVwAAFxYaEP8n/vuu6/W/S+88II2btzodkAAAMC7WPbWydTUVL355ptWDQcAwIXFA+ssrF27VjfeeKPi4uJkGIbefvvtasfHjBkjwzCqbb1793b5q1mWLLzxxhuKjIy0ajgAAC4onljuuaioSF26dNHzzz9/1nMGDx6sgwcPOrYPPvjA5e/mchuiW7du1SY4mqapgoICHT58WC+++KLLAQAAgPOTmpqq1NRUp+fYbDbFxMS4dR+Xk4WhQ4dW++zn56fmzZsrJSVFl112mVvBAAAAqbCwsNpnm80mm812XmOtXr1aUVFRatKkifr166fp06e7vD6SS8lCZWWlWrZsqUGDBrmdpQAA4FMsfBoiISGh2u5p06YpIyPD5eFSU1N18803KzExUXl5eXr00Ud17bXXatOmTS4lHy4lC/7+/powYYJyc3NdDhgAAF9m5XLP+fn5Cg8Pd+w/36rC8OHDHX/u2LGjkpOTlZiYqPfff1/Dhg2r8zgutyF69eqlzZs3KzEx0dVLAQBAHYSHh1dLFqwSGxurxMRE7dy506XrXE4W0tPTdf/992v//v3q0aOHQkJCqh3v3Lmzq0MCAOAbvHRRpTOOHDmi/Px8xcbGunRdnZOFsWPHas6cOY6Sxr333us4ZhiGTNOUYRiqqqpyKQAAAHyCB1ZwPHXqlHbt2uX4nJeXpy1btigyMlKRkZHKyMjQ7373O8XGxmrPnj165JFH1KxZM910000u3afOyUJ2draefvpp5eXluXQDAABQPzZu3Kj+/fs7Pk+ePFmSNHr0aM2bN09ff/21/vGPf+j48eOKjY1V//79tWzZMoWFhbl0nzonC6Z5Ot1hrgIAADVZOcGxrlJSUhw/n2vz0UcfuRfQz1yas+DsbZMAAFzUeJHUae3atTtnwnD06FG3AgIAAN7FpWTh8ccfV0RERH3FAgDABcsTbYiG4lKyMGLECJeXiAQA4KLgw22IOr91kvkKAABcnFx+GgIAANTChysLdU4W7HZ7fcYBAMAFjTkLAADAOR+uLNR5zgIAALg4UVkAAMAKPlxZIFkAAMACvjxngTYEAABwisoCAABWoA0BAACcoQ0BAAAuWlQWAACwAm0IAADglA8nC7QhAACAU1QWAACwgPHz5u4Y3ohkAQAAK/hwG4JkAQAAC/DoJAAAuGhRWQAAwAq0IQAAwDl56Q97d9GGAAAATlFZAADAAr48wZFkAQAAK/jwnAXaEAAAwCkqCwAAWIA2BAAAcI42BAAAuFhRWQAAwAK0IQAAgHM+3IYgWQAAwAo+nCwwZwEAADhFZQEAAAswZwEAADhHGwIAAFysqCwAAGABwzRlmO6VBty9vr6QLAAAYAXaEAAA4GJFZQEAAAvwNAQAAHCONgQAALhYUVkAAMACtCEAAIBzPtyGIFkAAMACvlxZYM4CAAAXqLVr1+rGG29UXFycDMPQ22+/Xe24aZrKyMhQXFycgoKClJKSom3btrl8H5IFAACsYFq0uaCoqEhdunTR888/X+vxZ555RrNnz9bzzz+vDRs2KCYmRgMGDNDJkyddug9tCAAALNLQbYTU1FSlpqbWesw0Tc2ZM0dTp07VsGHDJEnZ2dmKjo7WkiVLdNddd9X5PlQWAADwMoWFhdW2srIyl8fIy8tTQUGBBg4c6Nhns9nUr18/ffbZZy6NRbIAAIAVTNOaTVJCQoIiIiIcW1ZWlsvhFBQUSJKio6Or7Y+OjnYcqyvaEAAAWMDKpyHy8/MVHh7u2G+z2c5/TMOo9tk0zRr7zoVkAQAALxMeHl4tWTgfMTExkk5XGGJjYx37Dx06VKPacC60IQAAsIIHnoZwJikpSTExMVq1apVjX3l5udasWaOrrrrKpbGoLAAAYAHDfnpzdwxXnDp1Srt27XJ8zsvL05YtWxQZGakWLVpo0qRJyszMVNu2bdW2bVtlZmYqODhYI0eOdOk+JAsAAFygNm7cqP79+zs+T548WZI0evRoLVq0SFOmTFFJSYnS09N17Ngx9erVSytXrlRYWJhL9yFZgEd0bXVAo1K2qn38T2oeUayHFg7U2m+SfnGGqXEDNymtd67Cg8u0bW+UZi6/Wnk/RnosZsAVQdtP6pKcAgXuKZb/iQr9cE9rFXW/xHG80YkKNXtjv0K+KZRfSZVK2oXq0KgWqogO9GDUcIsH3g2RkpIi0zz7RYZhKCMjQxkZGW6F5dNzFjIyMtS1a1dPh4FaBDau1M4DTTXrrb61Hr+1/1bd0u8rzXqrr8bOGaYjJ4M19673FWwrb+BIgfNjlNlVlhCsQ7e2qHnQNBX3/C4FHC7TD/e20d5pl6uiaWPFz9who6yq4YOFJc48DeHu5o08miyMGTNGhmHIMAz5+/urRYsWmjBhgo4dO+bJsNAA1n/XQn/PuVJrvm5Vy1FTw6/5Wov+2V1rvm6l3QWRevK1/gpsXKmB3XbVcj7gfYo7R+jIsEt1qsclNY4F/FimoO+LdOgPiSpLClFFbKAO/SFRfqVVCvviqAeihSUsXGfB23i8sjB48GAdPHhQe/bs0YIFC/Tuu+8qPT3d02HBg+IiT6pZeLH+syPesa+iqpE2fx+rTi1/9GBkgDWMytOz2MyAXzzr7mfI9PdT0M5THooKODuPJws2m00xMTGKj4/XwIEDNXz4cK1cuVKSZLfb9cQTTyg+Pl42m01du3ZVTk5Otev379+vESNGKDIyUiEhIUpOTtYXX3xR673y8vLUpk0bTZgwQXZ77VNOy8rKaiyziYbVNLxYknT0ZFC1/UdPBiny52PAhaw8JlAVTRur2Rs/yK+oUqq065L3D8r/RIX8j1d4OjycJ9oQDWT37t3KyclRQECAJGnu3LmaNWuWZs6cqa+++kqDBg3Sb3/7W+3cuVPS6UdG+vXrpwMHDmjFihXaunWrpkyZUmsi8M0336hv3766+eabNW/ePPn51f7Vs7Kyqi2xmZCQUH9fGE79uhpnGLL0GWTAY/z9dODu1gr4sVRtJm5R2/FfKnj7SRV1Cveyf5XhEi9bZ8FKHn8a4r333lNoaKiqqqpUWloqSZo9e7YkaebMmXrooYc0YsQISdKMGTP0ySefaM6cOXrhhRe0ZMkSHT58WBs2bFBk5OlZ8m3atKlxj88//1w33HCDHn74YT3wwANO43n44Ycdj55Ip1/mQcLQsI4UBkuSmoaX6MjJEMf+S0JLdPRksKfCAixV1jJE+x6/Qn7FlTIqTVWFByjhyVyVteTvOLyPx5OF/v37a968eSouLtaCBQu0Y8cOTZw4UYWFhTpw4ID69q0+W75v377aunWrJGnLli3q1q2bI1Gozb59+3Tdddfpqaee0h//+MdzxmOz2dxagxvuO3A0TD8VBqtnu/3a8UMzSZJ/oyp1a31QL77Xy8PRAdayB5/+Zzjgx1IF7inSkZviPBwRzpeV74bwNh4veIWEhKhNmzbq3Lmznn32WZWVlenxxx93HHf2AoygoOo97do0b95cV155pZYuXcr8Ay8S1LhCbeN+Utu4nySdntTYNu4nRTc5KcnQsrWdNPo3m9WvY55axRzVoyNWq7TcXys316wcAd7IKK2SbV+xbPtOz7MJ+KlMtn3F8j9y+lXDoRuOKui7QgUcKlPI5mOKn7lDp7o3UXHHCE+GDXf48NMQHq8s/Nq0adOUmpqqCRMmKC4uTuvWrdM111zjOP7ZZ5/pyiuvlCR17txZCxYs0NGjR89aXQgKCtJ7772nIUOGaNCgQee1chWsd1nCYb2Y/q7j831pn0uS3t/QTk8t7a//90kX2QIq9cDv1iksqEzf7ovSpL9fr+Kyxp4KGXBJ4J4iJTyzw/E5aul+SdKJvk3147gk+R+vUPOl+fIvrFRlkwAV9mmqI7+NPdtwgEd5XbKQkpKiK664QpmZmXrwwQc1bdo0tW7dWl27dtXChQu1ZcsWLV68WJJ0yy23KDMzU0OHDlVWVpZiY2O1efNmxcXFqU+fPo4xQ0JC9P777ys1NVWpqanKyclRaGiop74iJG3+Pk597r/LyRmGXl6ZrJdXJjdYTICVSi4L145Xzv739/iAaB0f4Nqb/+DdaEM0sMmTJ2v+/Pm66aabdP/99+v+++9Xp06dlJOToxUrVqht27aSpMaNG2vlypWKiorSkCFD1KlTJz399NNq1KhRjTFDQ0P14YcfyjRNDRkyREVFRQ39tQAAvsyHn4YwTGeLSkOFhYWKiIhQj5ufUqMA1myHbyoYUOnpEIB6YS8p1f70DJ04cULh4eH1co8zPyf6DH5C/m7+nKisKNXnOY/Va7znw+vaEAAAXIh8uQ1BsgAAgBXs5unN3TG8EMkCAABWsGLOgXfmCt45wREAAHgPKgsAAFjAkAVzFiyJxHokCwAAWMGKFRi99AFF2hAAAMApKgsAAFiARycBAIBzPA0BAAAuVlQWAACwgGGaMtycoOju9fWFZAEAACvYf97cHcML0YYAAABOUVkAAMACtCEAAIBzPvw0BMkCAABWYAVHAABwsaKyAACABVjBEQAAOEcbAgAAXKyoLAAAYAHDfnpzdwxvRLIAAIAVaEMAAICLFZUFAACswKJMAADAGV9e7pk2BAAAcIrKAgAAVvDhCY4kCwAAWMGU5O6jj96ZK5AsAABgBeYsAACAixaVBQAArGDKgjkLlkRiOZIFAACs4MMTHGlDAAAAp6gsAABgBbskw4IxvBDJAgAAFuBpCAAA4FUyMjJkGEa1LSYmpl7uRWUBAAAreGCC4xVXXKF//vOfjs+NGjVy7/5nQbIAAIAVPJAs+Pv711s14ZdoQwAA4GUKCwurbWVlZbWet3PnTsXFxSkpKUkjRozQ7t276yUekgUAAKxwprLg7iYpISFBERERji0rK6vG7Xr16qV//OMf+uijjzR//nwVFBToqquu0pEjRyz/arQhAACwgoWPTubn5ys8PNyx22az1Tg1NTXV8edOnTqpT58+at26tbKzszV58mQ3A6mOZAEAAAtY+ehkeHh4tWShLkJCQtSpUyft3LnTrRhqQxsCAAAfUFZWptzcXMXGxlo+NskCAABWsHDOQl088MADWrNmjfLy8vTFF1/ov//7v1VYWKjRo0db/tVoQwAAYAW7KRluPjppr/v1+/fv1y233KKffvpJzZs3V+/evbV+/XolJia6F0MtSBYAALgALV26tMHuRbIAAIAVfPgV1SQLAABYwoJkQd6ZLDDBEQAAOEVlAQAAK9CGAAAATtlNud1GcOFpiIZEGwIAADhFZQEAACuY9tObu2N4IZIFAACswJwFAADgFHMWAADAxYrKAgAAVqANAQAAnDJlQbJgSSSWow0BAACcorIAAIAVaEMAAACn7HZJbq6TYPfOdRZoQwAAAKeoLAAAYAXaEAAAwCkfThZoQwAAAKeoLAAAYAUfXu6ZZAEAAAuYpl2mm2+NdPf6+kKyAACAFUzT/coAcxYAAMCFiMoCAABWMC2Ys+CllQWSBQAArGC3S4abcw68dM4CbQgAAOAUlQUAAKxAGwIAADhj2u0y3WxDeOujk7QhAACAU1QWAACwAm0IAADglN2UDN9MFmhDAAAAp6gsAABgBdOU5O46C95ZWSBZAADAAqbdlOlmG8IkWQAAwIeZdrlfWeDRSQAAcAGisgAAgAVoQwAAAOd8uA1BsnAOZ7K8qopSD0cC1B97SaWnQwDqhb3k9L/dDfEbe6Uq3F6TqVIV1gRjMcP01pqHl9i/f78SEhI8HQYAwA35+fmKj4+vl7FLS0uVlJSkgoICS8aLiYlRXl6eAgMDLRnPCiQL52C323XgwAGFhYXJMAxPh+PzCgsLlZCQoPz8fIWHh3s6HMBy/B1vWKZp6uTJk4qLi5OfX/3N6S8tLVV5ebklYzVu3NirEgWJNsQ5+fn51Vs2irMLDw/nH1L4NP6ON5yIiIh6v0dgYKDX/YC3Eo9OAgAAp0gWAACAUyQL8Co2m03Tpk2TzWbzdChAveDvOC5ETHAEAABOUVkAAABOkSwAAACnSBYAAIBTJAvwuJSUFE2aNMnTYQBeJSMjQ127dvV0GIAkkgXUkzFjxsgwDI0fP77GsfT0dBmGoTFjxkiSli9frieffLKBIwTcc+bvuGEY8vf3V4sWLTRhwgQdO3bM06EBliNZQL1JSEjQ0qVLVVJS4thXWlqq1157TS1atHDsi4yMVFhYmCdCBNwyePBgHTx4UHv27NGCBQv07rvvKj093dNhAZYjWUC96d69u1q0aKHly5c79i1fvlwJCQnq1q2bY9+v2xAtW7ZUZmamxo4dq7CwMLVo0UJ///vfq439ww8/aPjw4brkkkvUtGlTpaWlac+ePfX9lYBqbDabYmJiFB8fr4EDB2r48OFauXKlpNPvlXniiScUHx8vm82mrl27Kicnp9r1+/fv14gRIxQZGamQkBAlJyfriy++qPVeeXl5atOmjSZMmCC73TtfYwzfRbKAenXbbbdp4cKFjs+vvPKKxo4de87rZs2apeTkZG3evFnp6emaMGGCvvvuO0lScXGx+vfvr9DQUK1du1br1q1TaGioBg8ebNmLXABX7d69Wzk5OQoICJAkzZ07V7NmzdLMmTP11VdfadCgQfrtb3+rnTt3SpJOnTqlfv366cCBA1qxYoW2bt2qKVOm1JoIfPPNN+rbt69uvvlmzZs3r15fiATUygTqwejRo820tDTz8OHDps1mM/Py8sw9e/aYgYGB5uHDh820tDRz9OjRpmmaZr9+/cz77rvPcW1iYqJ56623Oj7b7XYzKirKnDdvnmmapvnyyy+b7du3N+12u+OcsrIyMygoyPzoo48a5PsBo0ePNhs1amSGhISYgYGBpiRTkjl79mzTNE0zLi7OnD59erVrevbsaaanp5umaZovvfSSGRYWZh45cqTW8adNm2Z26dLF/Oyzz8zIyEjzL3/5S/1+IcAJ3jqJetWsWTNdf/31ys7Olmmauv7669WsWbNzXte5c2fHnw3DUExMjA4dOiRJ2rRpk3bt2lVjnkNpaam+//57a78A4ET//v01b948FRcXa8GCBdqxY4cmTpyowsJCHThwQH379q12ft++fbV161ZJ0pYtW9StWzdFRkaedfx9+/bpuuuu01NPPaU//vGP9fpdAGdIFlDvxo4dq3vuuUeS9MILL9TpmjOl3DMMw3CUZ+12u3r06KHFixfXuK558+ZuRgvUXUhIiNq0aSNJevbZZ9W/f389/vjjevDBByWd/nv7S6ZpOvYFBQWdc/zmzZsrLi5OS5cu1bhx43ilNTyGxhfq3Zm5BOXl5Ro0aJDb43Xv3l07d+5UVFSU2rRpU21riPfWA2czbdo0zZw5U6dOnVJcXJzWrVtX7fhnn32mDh06SDpdPduyZYuOHj161vGCgoL03nvvKTAwUIMGDdLJkyfrNX7gbEgWUO8aNWqk3Nxc5ebmqlGjRm6PN2rUKDVr1kxpaWn697//rby8PK1Zs0b33Xef9u/fb0HEwPlJSUnRFVdcoczMTD344IOaMWOGli1bpu3bt+tPf/qTtmzZovvuu0+SdMsttygmJkZDhw7Vp59+qt27d+vNN9/U559/Xm3MkJAQvf/++/L391dqaqpOnTrlia+GixzJAhpEeHi4ZSXU4OBgrV27Vi1atNCwYcPUoUMHjR07ViUlJZRp4XGTJ0/W/PnzddNNN+n+++/X/fffr06dOiknJ0crVqxQ27ZtJUmNGzfWypUrFRUVpSFDhqhTp056+umna02oQ0ND9eGHH8o0TQ0ZMkRFRUUN/bVwkeMV1QAAwCkqCwAAwCmSBQAA4BTJAgAAcIpkAQAAOEWyAAAAnCJZAAAATpEsAAAAp0gWAACAUyQLwAUgIyNDXbt2dXweM2aMhg4d2uBx7NmzR4ZhaMuWLWc9p2XLlpozZ06dx1y0aJGaNGnidmyGYejtt992exwANZEsAOdpzJgxMgxDhmEoICBArVq10gMPPNAgS/HOnTtXixYtqtO5dfkBDwDO8IpqwA2DBw/WwoULVVFRoX//+9+6/fbbVVRUpHnz5tU4t6Kiosart88Xb9cE0JCoLABusNlsiomJUUJCgkaOHKlRo0Y5SuFnWgevvPKKWrVqJZvNJtM0deLECd15552KiopSeHi4rr32Wm3durXauE8//bSio6MVFhamcePGqbS0tNrxX7ch7Ha7ZsyYoTZt2shms6lFixaaPn26JCkpKUmS1K1bNxmGoZSUFMd1CxcuVIcOHRQYGKjLLrtML774YrX7/Oc//1G3bt0UGBio5ORkbd682eX/RrNnz1anTp0UEhKihIQEpaen1/rmxLffflvt2rVTYGCgBgwYoPz8/GrH3333XfXo0UOBgYFq1aqVHn/8cVVWVrocDwDXkSwAFgoKClJFRYXj865du/T666/rzTffdLQBrr/+ehUUFOiDDz7Qpk2b1L17d/3mN7/R0aNHJUmvv/66pk2bpunTp2vjxo2KjY2t8UP81x5++GHNmDFDjz76qL799lstWbJE0dHRkk7/wJekf/7znzp48KCWL18uSZo/f76mTp2q6dOnKzc3V5mZmXr00UeVnZ0tSSoqKtINN9yg9u3ba9OmTcrIyNADDzzg8n8TPz8/Pfvss/rmm2+UnZ2tjz/+WFOmTKl2TnFxsaZPn67s7Gx9+umnKiws1IgRIxzHP/roI916662699579e233+qll17SokWLHAkRgHpmAjgvo0ePNtPS0hyfv/jiC7Np06bm73//e9M0TXPatGlmQECAeejQIcc5//rXv8zw8HCztLS02litW7c2X3rpJdM0TbNPnz7m+PHjqx3v1auX2aVLl1rvXVhYaNpsNnP+/Pm1xpmXl2dKMjdv3lxtf0JCgrlkyZJq+5588kmzT58+pmma5ksvvWRGRkaaRUVFjuPz5s2rdaxfSkxMNP/617+e9fjrr79uNm3a1PF54cKFpiRz/fr1jn25ubmmJPOLL74wTdM0/+u//svMzMysNs6rr75qxsbGOj5LMt96662z3hfA+WPOAuCG9957T6GhoaqsrFRFRYXS0tL03HPPOY4nJiaqefPmjs+bNm3SqVOn1LRp02rjlJSU6Pvvv5ck5ebmavz48dWO9+nTR5988kmtMeTm5qqsrEy/+c1v6hz34cOHlZ+fr3HjxumOO+5w7K+srHTMh8jNzVWXLl0UHBxcLQ5XffLJJ8rMzNS3336rwsJCVVZWqrS0VEVFRQoJCZEk+fv7Kzk52XHNZZddpiZNmig3N1dXXnmlNm3apA0bNlSrJFRVVam0tFTFxcXVYgRgPZIFwA39+/fXvHnzFBAQoLi4uBoTGM/8MDzDbrcrNjZWq1evrjHW+T4+GBQU5PI1drtd0ulWRK9evaoda9SokSTJNM3ziueX9u7dqyFDhmj8+PF68sknFRkZqXXr1mncuHHV2jXS6Ucff+3MPrvdrscff1zDhg2rcU5gYKDbcQJwjmQBcENISIjatGlT5/O7d++ugoIC+fv7q2XLlrWe06FDB61fv17/8z//49i3fv36s47Ztm1bBQUF6V//+pduv/32GscbN24s6fRv4mdER0fr0ksv1e7duzVq1Khax7388sv16quvqqSkxJGQOIujNhs3blRlZaVmzZolP7/TU6Ref/31GudVVlZq48aNuvLKKyVJ27dv1/Hjx3XZZZdJOv3fbfv27S79twZgHZIFoAFdd9116tOnj4YOHaoZM2aoffv2OnDggD744AMNHTpUycnJuu+++zR69GglJyfr6quv1uLFi7Vt2za1atWq1jEDAwP10EMPacqUKWrcuLH69u2rw4cPa9u2bRo3bpyioqIUFBSknJwcxcfHKzAwUBEREcrIyNC9996r8PBwpaamqqysTBs3btSxY8c0efJkjRw5UlOnTtW4ceP05z//WXv27NHMmTNd+r6tW7dWZWWlnnvuOd1444369NNP9be//a3GeQEBAZo4caKeffZZBQQE6J577lHv3r0dycNjjz2mG264QQkJCbr55pvl5+enr776Sl9//bWeeuop1/9HAHAJT0MADcgwDH3wwQe65pprNHbsWLVr104jRozQnj17HE8vDB8+XI899pgeeugh9ejRQ3v37tWECROcjvvoo4/q/vvv12OPPaYOHTpo+PDhOnTokKTT8wGeffZZvfTSS4qLi1NaWpok6fbbb9eCBQu0aNEiderUSf369dOiRYscj1qGhobq3Xff1bfffqtu3bpp6tSpmjFjhkvft2vXrpo9e7ZmzJihjh07avHixcrKyqpxXnBwsB566CGNHDlSffr0UVBQkJYuXeo4PmjQIL333ntatWqVevbsqd69e2v27NlKTEx0KR4A58cwrWhMAgAAn0VlAQAAOEWyAAAAnCJZAAAATpEsAAAAp0gWAACAUyQLAADAKZIFAADgFMkCAABwimQBAAA4RbIAAACcIlkAAABO/X+59pW62NB7BgAAAABJRU5ErkJggg==",
            "text/plain": [
              "<Figure size 640x480 with 2 Axes>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "# random forest classifier\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "rf_clf = RandomForestClassifier()\n",
        "rf_clf.fit(X_train, y_train)\n",
        "results = rf_clf.predict(X_test)\n",
        "cm = confusion_matrix(y_test, results, labels=rf_clf.classes_)\n",
        "disp = ConfusionMatrixDisplay(confusion_matrix=cm, display_labels=rf_clf.classes_)\n",
        "disp.plot()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 45,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.8253968253968254"
            ]
          },
          "execution_count": 45,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "accuracy_score(rf_clf.predict(X_test), y_test)"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "Scikit_learn_model_training.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.4"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}