Regularized Multiple Linear Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have already implemented a simple linear regression model by manually implementing gradient descent. For that implementation of linear regression, refer to the notebook titled \"Linear Regression with two variables and manual implementation of gradient descent 22.06.2019\". \n",
    "\n",
    "In this notebook we will solve a case study by using multiple linear regression, and regularised linear regression <span class=\"mark\">[Ridge and Lasso]</span>. We will also look at hyperparameter tuning for regularised regression. Also, we will concentrate on feature engineering and data preperation. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This dataset comes from a loan aggregator who collects loan applications from different people and sends it to various financial institutions. \n",
    "\n",
    "They have collected various information from the loan applicants and have also collected what percentage of interest the bank offered to each of them. Our task now is to predict interest rate for future customers. \n",
    "\n",
    "The name of our output variable is \"Interest.Rate\" in the dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In the class, train_test_split and KFold methods are downloaded from \n",
    "#sklearn.cross_validationlibrary but, python packages have been updated, \n",
    "#so now the correct package is sklearn.model_selection\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import math\n",
    "from sklearn.model_selection import train_test_split \n",
    "from sklearn.linear_model import LinearRegression, Lasso, Ridge\n",
    "from sklearn.model_selection import KFold\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_file=r'C:/Users/HP/Dropbox/Edvancer/CMLEP/Data/Data/loans data.csv'\n",
    "loan_data = pd.read_csv(data_file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Amount.Requested</th>\n",
       "      <th>Amount.Funded.By.Investors</th>\n",
       "      <th>Interest.Rate</th>\n",
       "      <th>Loan.Length</th>\n",
       "      <th>Loan.Purpose</th>\n",
       "      <th>Debt.To.Income.Ratio</th>\n",
       "      <th>State</th>\n",
       "      <th>Home.Ownership</th>\n",
       "      <th>Monthly.Income</th>\n",
       "      <th>FICO.Range</th>\n",
       "      <th>Open.CREDIT.Lines</th>\n",
       "      <th>Revolving.CREDIT.Balance</th>\n",
       "      <th>Inquiries.in.the.Last.6.Months</th>\n",
       "      <th>Employment.Length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>81174.0</td>\n",
       "      <td>20000</td>\n",
       "      <td>20000</td>\n",
       "      <td>8.90%</td>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>14.90%</td>\n",
       "      <td>SC</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>6541.67</td>\n",
       "      <td>735-739</td>\n",
       "      <td>14</td>\n",
       "      <td>14272</td>\n",
       "      <td>2.0</td>\n",
       "      <td>&lt; 1 year</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>99592.0</td>\n",
       "      <td>19200</td>\n",
       "      <td>19200</td>\n",
       "      <td>12.12%</td>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>28.36%</td>\n",
       "      <td>TX</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>4583.33</td>\n",
       "      <td>715-719</td>\n",
       "      <td>12</td>\n",
       "      <td>11140</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>80059.0</td>\n",
       "      <td>35000</td>\n",
       "      <td>35000</td>\n",
       "      <td>21.98%</td>\n",
       "      <td>60 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>23.81%</td>\n",
       "      <td>CA</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>11500.00</td>\n",
       "      <td>690-694</td>\n",
       "      <td>14</td>\n",
       "      <td>21977</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15825.0</td>\n",
       "      <td>10000</td>\n",
       "      <td>9975</td>\n",
       "      <td>9.99%</td>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>14.30%</td>\n",
       "      <td>KS</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>3833.33</td>\n",
       "      <td>695-699</td>\n",
       "      <td>10</td>\n",
       "      <td>9346</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>33182.0</td>\n",
       "      <td>12000</td>\n",
       "      <td>12000</td>\n",
       "      <td>11.71%</td>\n",
       "      <td>36 months</td>\n",
       "      <td>credit_card</td>\n",
       "      <td>18.78%</td>\n",
       "      <td>NJ</td>\n",
       "      <td>RENT</td>\n",
       "      <td>3195.00</td>\n",
       "      <td>695-699</td>\n",
       "      <td>11</td>\n",
       "      <td>14469</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9 years</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        ID Amount.Requested Amount.Funded.By.Investors Interest.Rate  \\\n",
       "0  81174.0            20000                      20000         8.90%   \n",
       "1  99592.0            19200                      19200        12.12%   \n",
       "2  80059.0            35000                      35000        21.98%   \n",
       "3  15825.0            10000                       9975         9.99%   \n",
       "4  33182.0            12000                      12000        11.71%   \n",
       "\n",
       "  Loan.Length        Loan.Purpose Debt.To.Income.Ratio State Home.Ownership  \\\n",
       "0   36 months  debt_consolidation               14.90%    SC       MORTGAGE   \n",
       "1   36 months  debt_consolidation               28.36%    TX       MORTGAGE   \n",
       "2   60 months  debt_consolidation               23.81%    CA       MORTGAGE   \n",
       "3   36 months  debt_consolidation               14.30%    KS       MORTGAGE   \n",
       "4   36 months         credit_card               18.78%    NJ           RENT   \n",
       "\n",
       "   Monthly.Income FICO.Range Open.CREDIT.Lines Revolving.CREDIT.Balance  \\\n",
       "0         6541.67    735-739                14                    14272   \n",
       "1         4583.33    715-719                12                    11140   \n",
       "2        11500.00    690-694                14                    21977   \n",
       "3         3833.33    695-699                10                     9346   \n",
       "4         3195.00    695-699                11                    14469   \n",
       "\n",
       "   Inquiries.in.the.Last.6.Months Employment.Length  \n",
       "0                             2.0          < 1 year  \n",
       "1                             1.0           2 years  \n",
       "2                             1.0           2 years  \n",
       "3                             0.0           5 years  \n",
       "4                             0.0           9 years  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ID                                float64\n",
       "Amount.Requested                   object\n",
       "Amount.Funded.By.Investors         object\n",
       "Interest.Rate                      object\n",
       "Loan.Length                        object\n",
       "Loan.Purpose                       object\n",
       "Debt.To.Income.Ratio               object\n",
       "State                              object\n",
       "Home.Ownership                     object\n",
       "Monthly.Income                    float64\n",
       "FICO.Range                         object\n",
       "Open.CREDIT.Lines                  object\n",
       "Revolving.CREDIT.Balance           object\n",
       "Inquiries.in.the.Last.6.Months    float64\n",
       "Employment.Length                  object\n",
       "dtype: object"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Variables Interest.Rate and Debt.To.Income.Ratio contains \"%\" sign in their values and because of which they are a \"character\" column and not a \"numeric\" column. \n",
    "\n",
    "#### Therefore those \"%\" signs should be removed first. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "col_with_percentage=['Interest.Rate','Debt.To.Income.Ratio']\n",
    "\n",
    "#first we convert the entire column into string so we can easily remove % sign using \n",
    "#string operation then we replace % with '' in each string using list comprehension \n",
    "# and replace the column with this list\n",
    "\n",
    "for col in col_with_percentage:\n",
    "    loan_data[col]= loan_data[col].astype('str')\n",
    "    loan_data[col]= [x.replace('%','') for x in loan_data[col]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ID                                float64\n",
       "Amount.Requested                   object\n",
       "Amount.Funded.By.Investors         object\n",
       "Interest.Rate                      object\n",
       "Loan.Length                        object\n",
       "Loan.Purpose                       object\n",
       "Debt.To.Income.Ratio               object\n",
       "State                              object\n",
       "Home.Ownership                     object\n",
       "Monthly.Income                    float64\n",
       "FICO.Range                         object\n",
       "Open.CREDIT.Lines                  object\n",
       "Revolving.CREDIT.Balance           object\n",
       "Inquiries.in.the.Last.6.Months    float64\n",
       "Employment.Length                  object\n",
       "dtype: object"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Amount.Requested</th>\n",
       "      <th>Amount.Funded.By.Investors</th>\n",
       "      <th>Interest.Rate</th>\n",
       "      <th>Loan.Length</th>\n",
       "      <th>Loan.Purpose</th>\n",
       "      <th>Debt.To.Income.Ratio</th>\n",
       "      <th>State</th>\n",
       "      <th>Home.Ownership</th>\n",
       "      <th>Monthly.Income</th>\n",
       "      <th>FICO.Range</th>\n",
       "      <th>Open.CREDIT.Lines</th>\n",
       "      <th>Revolving.CREDIT.Balance</th>\n",
       "      <th>Inquiries.in.the.Last.6.Months</th>\n",
       "      <th>Employment.Length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>81174.0</td>\n",
       "      <td>20000</td>\n",
       "      <td>20000</td>\n",
       "      <td>8.90</td>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>14.90</td>\n",
       "      <td>SC</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>6541.67</td>\n",
       "      <td>735-739</td>\n",
       "      <td>14</td>\n",
       "      <td>14272</td>\n",
       "      <td>2.0</td>\n",
       "      <td>&lt; 1 year</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>99592.0</td>\n",
       "      <td>19200</td>\n",
       "      <td>19200</td>\n",
       "      <td>12.12</td>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>28.36</td>\n",
       "      <td>TX</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>4583.33</td>\n",
       "      <td>715-719</td>\n",
       "      <td>12</td>\n",
       "      <td>11140</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>80059.0</td>\n",
       "      <td>35000</td>\n",
       "      <td>35000</td>\n",
       "      <td>21.98</td>\n",
       "      <td>60 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>23.81</td>\n",
       "      <td>CA</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>11500.00</td>\n",
       "      <td>690-694</td>\n",
       "      <td>14</td>\n",
       "      <td>21977</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15825.0</td>\n",
       "      <td>10000</td>\n",
       "      <td>9975</td>\n",
       "      <td>9.99</td>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>14.30</td>\n",
       "      <td>KS</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>3833.33</td>\n",
       "      <td>695-699</td>\n",
       "      <td>10</td>\n",
       "      <td>9346</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>33182.0</td>\n",
       "      <td>12000</td>\n",
       "      <td>12000</td>\n",
       "      <td>11.71</td>\n",
       "      <td>36 months</td>\n",
       "      <td>credit_card</td>\n",
       "      <td>18.78</td>\n",
       "      <td>NJ</td>\n",
       "      <td>RENT</td>\n",
       "      <td>3195.00</td>\n",
       "      <td>695-699</td>\n",
       "      <td>11</td>\n",
       "      <td>14469</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9 years</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        ID Amount.Requested Amount.Funded.By.Investors Interest.Rate  \\\n",
       "0  81174.0            20000                      20000          8.90   \n",
       "1  99592.0            19200                      19200         12.12   \n",
       "2  80059.0            35000                      35000         21.98   \n",
       "3  15825.0            10000                       9975          9.99   \n",
       "4  33182.0            12000                      12000         11.71   \n",
       "\n",
       "  Loan.Length        Loan.Purpose Debt.To.Income.Ratio State Home.Ownership  \\\n",
       "0   36 months  debt_consolidation                14.90    SC       MORTGAGE   \n",
       "1   36 months  debt_consolidation                28.36    TX       MORTGAGE   \n",
       "2   60 months  debt_consolidation                23.81    CA       MORTGAGE   \n",
       "3   36 months  debt_consolidation                14.30    KS       MORTGAGE   \n",
       "4   36 months         credit_card                18.78    NJ           RENT   \n",
       "\n",
       "   Monthly.Income FICO.Range Open.CREDIT.Lines Revolving.CREDIT.Balance  \\\n",
       "0         6541.67    735-739                14                    14272   \n",
       "1         4583.33    715-719                12                    11140   \n",
       "2        11500.00    690-694                14                    21977   \n",
       "3         3833.33    695-699                10                     9346   \n",
       "4         3195.00    695-699                11                    14469   \n",
       "\n",
       "   Inquiries.in.the.Last.6.Months Employment.Length  \n",
       "0                             2.0          < 1 year  \n",
       "1                             1.0           2 years  \n",
       "2                             1.0           2 years  \n",
       "3                             0.0           5 years  \n",
       "4                             0.0           9 years  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now the '%' symbol from 'Debt.To.Income.Ratio' and 'Interest.Rate' columns are gone. But still, they are not converted to integers, they are still string values. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ID                                float64\n",
       "Amount.Requested                   object\n",
       "Amount.Funded.By.Investors         object\n",
       "Interest.Rate                      object\n",
       "Loan.Length                        object\n",
       "Loan.Purpose                       object\n",
       "Debt.To.Income.Ratio               object\n",
       "State                              object\n",
       "Home.Ownership                     object\n",
       "Monthly.Income                    float64\n",
       "FICO.Range                         object\n",
       "Open.CREDIT.Lines                  object\n",
       "Revolving.CREDIT.Balance           object\n",
       "Inquiries.in.the.Last.6.Months    float64\n",
       "Employment.Length                  object\n",
       "dtype: object"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Now, lets convert all the columns that has only numbers into numeric data types. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "columns_with_numbers_only = ['Amount.Requested','Amount.Funded.By.Investors',\n",
    "                             'Interest.Rate','Debt.To.Income.Ratio',\n",
    "                             'Monthly.Income','Open.CREDIT.Lines', \n",
    "                             'Revolving.CREDIT.Balance', \n",
    "                             'Inquiries.in.the.Last.6.Months']\n",
    "#Creating a list of column names that should be converted to integers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "for numeric_column_name in columns_with_numbers_only:\n",
    "    loan_data[numeric_column_name]=pd.to_numeric(loan_data[numeric_column_name],\n",
    "                                                 errors='coerce')\n",
    "    \n",
    "#The 'errors=coerce' argument will replace all non-numeric values in the columns\n",
    "# with NaN values If this argument is missing then we will get an error and \n",
    "#execution will stop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ID                                float64\n",
       "Amount.Requested                  float64\n",
       "Amount.Funded.By.Investors        float64\n",
       "Interest.Rate                     float64\n",
       "Loan.Length                        object\n",
       "Loan.Purpose                       object\n",
       "Debt.To.Income.Ratio              float64\n",
       "State                              object\n",
       "Home.Ownership                     object\n",
       "Monthly.Income                    float64\n",
       "FICO.Range                         object\n",
       "Open.CREDIT.Lines                 float64\n",
       "Revolving.CREDIT.Balance          float64\n",
       "Inquiries.in.the.Last.6.Months    float64\n",
       "Employment.Length                  object\n",
       "dtype: object"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.dtypes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Amount.Requested</th>\n",
       "      <th>Amount.Funded.By.Investors</th>\n",
       "      <th>Interest.Rate</th>\n",
       "      <th>Loan.Length</th>\n",
       "      <th>Loan.Purpose</th>\n",
       "      <th>Debt.To.Income.Ratio</th>\n",
       "      <th>State</th>\n",
       "      <th>Home.Ownership</th>\n",
       "      <th>Monthly.Income</th>\n",
       "      <th>FICO.Range</th>\n",
       "      <th>Open.CREDIT.Lines</th>\n",
       "      <th>Revolving.CREDIT.Balance</th>\n",
       "      <th>Inquiries.in.the.Last.6.Months</th>\n",
       "      <th>Employment.Length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>81174.0</td>\n",
       "      <td>20000.0</td>\n",
       "      <td>20000.0</td>\n",
       "      <td>8.90</td>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>14.90</td>\n",
       "      <td>SC</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>6541.67</td>\n",
       "      <td>735-739</td>\n",
       "      <td>14.0</td>\n",
       "      <td>14272.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>&lt; 1 year</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>99592.0</td>\n",
       "      <td>19200.0</td>\n",
       "      <td>19200.0</td>\n",
       "      <td>12.12</td>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>28.36</td>\n",
       "      <td>TX</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>4583.33</td>\n",
       "      <td>715-719</td>\n",
       "      <td>12.0</td>\n",
       "      <td>11140.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>80059.0</td>\n",
       "      <td>35000.0</td>\n",
       "      <td>35000.0</td>\n",
       "      <td>21.98</td>\n",
       "      <td>60 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>23.81</td>\n",
       "      <td>CA</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>11500.00</td>\n",
       "      <td>690-694</td>\n",
       "      <td>14.0</td>\n",
       "      <td>21977.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15825.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>9975.0</td>\n",
       "      <td>9.99</td>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>14.30</td>\n",
       "      <td>KS</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>3833.33</td>\n",
       "      <td>695-699</td>\n",
       "      <td>10.0</td>\n",
       "      <td>9346.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>33182.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>11.71</td>\n",
       "      <td>36 months</td>\n",
       "      <td>credit_card</td>\n",
       "      <td>18.78</td>\n",
       "      <td>NJ</td>\n",
       "      <td>RENT</td>\n",
       "      <td>3195.00</td>\n",
       "      <td>695-699</td>\n",
       "      <td>11.0</td>\n",
       "      <td>14469.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9 years</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        ID  Amount.Requested  Amount.Funded.By.Investors  Interest.Rate  \\\n",
       "0  81174.0           20000.0                     20000.0           8.90   \n",
       "1  99592.0           19200.0                     19200.0          12.12   \n",
       "2  80059.0           35000.0                     35000.0          21.98   \n",
       "3  15825.0           10000.0                      9975.0           9.99   \n",
       "4  33182.0           12000.0                     12000.0          11.71   \n",
       "\n",
       "  Loan.Length        Loan.Purpose  Debt.To.Income.Ratio State Home.Ownership  \\\n",
       "0   36 months  debt_consolidation                 14.90    SC       MORTGAGE   \n",
       "1   36 months  debt_consolidation                 28.36    TX       MORTGAGE   \n",
       "2   60 months  debt_consolidation                 23.81    CA       MORTGAGE   \n",
       "3   36 months  debt_consolidation                 14.30    KS       MORTGAGE   \n",
       "4   36 months         credit_card                 18.78    NJ           RENT   \n",
       "\n",
       "   Monthly.Income FICO.Range  Open.CREDIT.Lines  Revolving.CREDIT.Balance  \\\n",
       "0         6541.67    735-739               14.0                   14272.0   \n",
       "1         4583.33    715-719               12.0                   11140.0   \n",
       "2        11500.00    690-694               14.0                   21977.0   \n",
       "3         3833.33    695-699               10.0                    9346.0   \n",
       "4         3195.00    695-699               11.0                   14469.0   \n",
       "\n",
       "   Inquiries.in.the.Last.6.Months Employment.Length  \n",
       "0                             2.0          < 1 year  \n",
       "1                             1.0           2 years  \n",
       "2                             1.0           2 years  \n",
       "3                             0.0           5 years  \n",
       "4                             0.0           9 years  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(6, 9)"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(loan_data.dtypes=='object').sum(), (loan_data.dtypes=='float64').sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can see that we have only 6 columns as strings/object and the remaining 9 are integers.\n",
    "Let's also see what those object columns are"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Loan.Length</th>\n",
       "      <th>Loan.Purpose</th>\n",
       "      <th>State</th>\n",
       "      <th>Home.Ownership</th>\n",
       "      <th>FICO.Range</th>\n",
       "      <th>Employment.Length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>SC</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>735-739</td>\n",
       "      <td>&lt; 1 year</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>TX</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>715-719</td>\n",
       "      <td>2 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>60 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>CA</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>690-694</td>\n",
       "      <td>2 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>36 months</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>KS</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>695-699</td>\n",
       "      <td>5 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>36 months</td>\n",
       "      <td>credit_card</td>\n",
       "      <td>NJ</td>\n",
       "      <td>RENT</td>\n",
       "      <td>695-699</td>\n",
       "      <td>9 years</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Loan.Length        Loan.Purpose State Home.Ownership FICO.Range  \\\n",
       "0   36 months  debt_consolidation    SC       MORTGAGE    735-739   \n",
       "1   36 months  debt_consolidation    TX       MORTGAGE    715-719   \n",
       "2   60 months  debt_consolidation    CA       MORTGAGE    690-694   \n",
       "3   36 months  debt_consolidation    KS       MORTGAGE    695-699   \n",
       "4   36 months         credit_card    NJ           RENT    695-699   \n",
       "\n",
       "  Employment.Length  \n",
       "0          < 1 year  \n",
       "1           2 years  \n",
       "2           2 years  \n",
       "3           5 years  \n",
       "4           9 years  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "object_column_names = loan_data.columns[loan_data.dtypes=='object']\n",
    "#creating a list of all columns which are of data type 'object'\n",
    "\n",
    "loan_data[object_column_names].head()\n",
    "#slicing the dataframe using those column names only"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### We can see that all the 6 object type columns clearly have strings or some string value in them"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Here the variable 'Loan.Lenght' seems to be clearly categorical, so let's convert that into dummy variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "36 months    1950\n",
       "60 months     548\n",
       ".               1\n",
       "Name: Loan.Length, dtype: int64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data['Loan.Length'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, there are 3 categories in that variable.\n",
    "\n",
    "The function 'get_dummies' from pandas creates dummy variables for all the categorical values we have. This function returns a dataframe. So, we can use that to create dummy variables, then drop the variables that we don't need. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_lenght_dummies = pd.get_dummies(loan_data['Loan.Length'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>.</th>\n",
       "      <th>36 months</th>\n",
       "      <th>60 months</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   .  36 months  60 months\n",
       "0  0          1          0\n",
       "1  0          1          0\n",
       "2  0          0          1\n",
       "3  0          1          0\n",
       "4  0          1          0"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_lenght_dummies.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "since '60_months' and the other value '.' has very less number of values compared to the value '36_months' we can drop them both altogether and add the dummy variable '36 months' to our original dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['Loan_lenght_36']= loan_lenght_dummies['36 months']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we don't need the variable 'loan_length_dummies' we can drop it altogether. \n",
    "Note:dropping variables from notebook environment is a permenant operation. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Once deleted, variables cannot be recovered. Proceed (y/[n])?  y\n"
     ]
    }
   ],
   "source": [
    "%reset_selective loan_lenght_dummies\n",
    "#deleting the variable 'loan_lenght_dummies'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To know what all variables we have in our environment at present we can use the function 'who'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "KFold\t Lasso\t LinearRegression\t Ridge\t col\t col_with_percentage\t columns_with_numbers_only\t data_file\t loan_data\t \n",
      "math\t np\t numeric_column_name\t object_column_names\t pd\t train_test_split\t \n"
     ]
    }
   ],
   "source": [
    "who"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have created dummy variables for the variable Loan.Lenght we can drop that variable from our original dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.drop(labels='Loan.Length', axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Amount.Requested</th>\n",
       "      <th>Amount.Funded.By.Investors</th>\n",
       "      <th>Interest.Rate</th>\n",
       "      <th>Loan.Purpose</th>\n",
       "      <th>Debt.To.Income.Ratio</th>\n",
       "      <th>State</th>\n",
       "      <th>Home.Ownership</th>\n",
       "      <th>Monthly.Income</th>\n",
       "      <th>FICO.Range</th>\n",
       "      <th>Open.CREDIT.Lines</th>\n",
       "      <th>Revolving.CREDIT.Balance</th>\n",
       "      <th>Inquiries.in.the.Last.6.Months</th>\n",
       "      <th>Employment.Length</th>\n",
       "      <th>Loan_lenght_36</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>81174.0</td>\n",
       "      <td>20000.0</td>\n",
       "      <td>20000.0</td>\n",
       "      <td>8.90</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>14.90</td>\n",
       "      <td>SC</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>6541.67</td>\n",
       "      <td>735-739</td>\n",
       "      <td>14.0</td>\n",
       "      <td>14272.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>&lt; 1 year</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>99592.0</td>\n",
       "      <td>19200.0</td>\n",
       "      <td>19200.0</td>\n",
       "      <td>12.12</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>28.36</td>\n",
       "      <td>TX</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>4583.33</td>\n",
       "      <td>715-719</td>\n",
       "      <td>12.0</td>\n",
       "      <td>11140.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>80059.0</td>\n",
       "      <td>35000.0</td>\n",
       "      <td>35000.0</td>\n",
       "      <td>21.98</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>23.81</td>\n",
       "      <td>CA</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>11500.00</td>\n",
       "      <td>690-694</td>\n",
       "      <td>14.0</td>\n",
       "      <td>21977.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15825.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>9975.0</td>\n",
       "      <td>9.99</td>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>14.30</td>\n",
       "      <td>KS</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>3833.33</td>\n",
       "      <td>695-699</td>\n",
       "      <td>10.0</td>\n",
       "      <td>9346.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5 years</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>33182.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>11.71</td>\n",
       "      <td>credit_card</td>\n",
       "      <td>18.78</td>\n",
       "      <td>NJ</td>\n",
       "      <td>RENT</td>\n",
       "      <td>3195.00</td>\n",
       "      <td>695-699</td>\n",
       "      <td>11.0</td>\n",
       "      <td>14469.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9 years</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        ID  Amount.Requested  Amount.Funded.By.Investors  Interest.Rate  \\\n",
       "0  81174.0           20000.0                     20000.0           8.90   \n",
       "1  99592.0           19200.0                     19200.0          12.12   \n",
       "2  80059.0           35000.0                     35000.0          21.98   \n",
       "3  15825.0           10000.0                      9975.0           9.99   \n",
       "4  33182.0           12000.0                     12000.0          11.71   \n",
       "\n",
       "         Loan.Purpose  Debt.To.Income.Ratio State Home.Ownership  \\\n",
       "0  debt_consolidation                 14.90    SC       MORTGAGE   \n",
       "1  debt_consolidation                 28.36    TX       MORTGAGE   \n",
       "2  debt_consolidation                 23.81    CA       MORTGAGE   \n",
       "3  debt_consolidation                 14.30    KS       MORTGAGE   \n",
       "4         credit_card                 18.78    NJ           RENT   \n",
       "\n",
       "   Monthly.Income FICO.Range  Open.CREDIT.Lines  Revolving.CREDIT.Balance  \\\n",
       "0         6541.67    735-739               14.0                   14272.0   \n",
       "1         4583.33    715-719               12.0                   11140.0   \n",
       "2        11500.00    690-694               14.0                   21977.0   \n",
       "3         3833.33    695-699               10.0                    9346.0   \n",
       "4         3195.00    695-699               11.0                   14469.0   \n",
       "\n",
       "   Inquiries.in.the.Last.6.Months Employment.Length  Loan_lenght_36  \n",
       "0                             2.0          < 1 year               1  \n",
       "1                             1.0           2 years               1  \n",
       "2                             1.0           2 years               0  \n",
       "3                             0.0           5 years               1  \n",
       "4                             0.0           9 years               1  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how many string datatype columns we have now"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Loan.Purpose</th>\n",
       "      <th>State</th>\n",
       "      <th>Home.Ownership</th>\n",
       "      <th>FICO.Range</th>\n",
       "      <th>Employment.Length</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>SC</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>735-739</td>\n",
       "      <td>&lt; 1 year</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>TX</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>715-719</td>\n",
       "      <td>2 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>CA</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>690-694</td>\n",
       "      <td>2 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>debt_consolidation</td>\n",
       "      <td>KS</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>695-699</td>\n",
       "      <td>5 years</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>credit_card</td>\n",
       "      <td>NJ</td>\n",
       "      <td>RENT</td>\n",
       "      <td>695-699</td>\n",
       "      <td>9 years</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Loan.Purpose State Home.Ownership FICO.Range Employment.Length\n",
       "0  debt_consolidation    SC       MORTGAGE    735-739          < 1 year\n",
       "1  debt_consolidation    TX       MORTGAGE    715-719           2 years\n",
       "2  debt_consolidation    CA       MORTGAGE    690-694           2 years\n",
       "3  debt_consolidation    KS       MORTGAGE    695-699           5 years\n",
       "4         credit_card    NJ           RENT    695-699           9 years"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "object_column_names = loan_data.columns[loan_data.dtypes=='object']\n",
    "#creating a list of all columns which are of data type 'object'\n",
    "\n",
    "loan_data[object_column_names].head()\n",
    "#slicing the dataframe using those column names only"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we have only 4. Let's start to examine the next variable 'Loan.Purpose'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "debt_consolidation    1307\n",
       "credit_card            444\n",
       "other                  200\n",
       "home_improvement       152\n",
       "major_purchase         101\n",
       "small_business          87\n",
       "car                     50\n",
       "wedding                 39\n",
       "medical                 30\n",
       "moving                  29\n",
       "vacation                21\n",
       "house                   20\n",
       "educational             15\n",
       "renewable_energy         4\n",
       "Name: Loan.Purpose, dtype: int64"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data['Loan.Purpose'].value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "14\n"
     ]
    }
   ],
   "source": [
    "print(len(loan_data['Loan.Purpose'].value_counts()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have 14 categories in the variable 'Loan.Purpose'. \n",
    "\n",
    "We can either make 13 dummy variables or we can group some categories together to reduce the number of effective dummy variables needed. \n",
    "\n",
    "One method we can use to find possibilities to group categories together is by performing group_by operation between this variable and the response variable 'Interest.Rate'. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Loan.Purpose\n",
       "car                   11.0\n",
       "credit_card           13.0\n",
       "debt_consolidation    14.0\n",
       "educational           11.0\n",
       "home_improvement      12.0\n",
       "house                 13.0\n",
       "major_purchase        11.0\n",
       "medical               12.0\n",
       "moving                14.0\n",
       "other                 13.0\n",
       "renewable_energy      10.0\n",
       "small_business        13.0\n",
       "vacation              12.0\n",
       "wedding               12.0\n",
       "Name: Interest.Rate, dtype: float64"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.groupby('Loan.Purpose')['Interest.Rate'].mean().round()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how many effective categories this grouping provides us"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "12.0    4\n",
       "13.0    4\n",
       "11.0    3\n",
       "14.0    2\n",
       "10.0    1\n",
       "Name: Interest.Rate, dtype: int64"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.groupby('Loan.Purpose')['Interest.Rate'].mean().round().value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see there are 4 effective categories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(len(loan_data)):\n",
    "    # grouping all categories with mean value 11(received in line 25) together\n",
    "    if loan_data.loc[i]['Loan.Purpose'] in ['car','educational','major_purchase']:\n",
    "        loan_data.loc[i,'Loan.Purpose']='cem'\n",
    "    # grouping all categories with mean value 12(received in line 25) together\n",
    "    if loan_data.loc[i]['Loan.Purpose'] in  ['home_improvement','medical','vacation',\n",
    "                                             'wedding']:\n",
    "        loan_data.loc[i,'Loan.Purpose']='hmvw'\n",
    "    # grouping all categories with mean value 13(received in line 25) together\n",
    "    if loan_data.loc[i]['Loan.Purpose'] in ['credit_card','house','other',\n",
    "                                            'small_business']:\n",
    "        loan_data.loc[i,'Loan.Purpose'] = 'chos'\n",
    "    # grouping all categories with mean value 14 (received in line 25) together\n",
    "    if loan_data.loc[i]['Loan.Purpose'] in ['debt_consolidation','moving']:\n",
    "        loan_data.loc[i,'Loan.Purpose']='dm'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      dm\n",
       "1      dm\n",
       "2      dm\n",
       "3      dm\n",
       "4    chos\n",
       "Name: Loan.Purpose, dtype: object"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data['Loan.Purpose'].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, we have effectively combined the 13 out of 14 categories into 4 categories. \n",
    "\n",
    "We have not included the category 'renewable_energy' in this grouping because it had only 4 occurences in the entire data frame. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's create dummies for the variable Loan.Purpose"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data_dummies = pd.get_dummies(loan_data['Loan.Purpose'], prefix='lp')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>lp_cem</th>\n",
       "      <th>lp_chos</th>\n",
       "      <th>lp_dm</th>\n",
       "      <th>lp_hmvw</th>\n",
       "      <th>lp_renewable_energy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   lp_cem  lp_chos  lp_dm  lp_hmvw  lp_renewable_energy\n",
       "0       0        0      1        0                    0\n",
       "1       0        0      1        0                    0\n",
       "2       0        0      1        0                    0\n",
       "3       0        0      1        0                    0\n",
       "4       0        1      0        0                    0"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data_dummies.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see we have a dummy variable for 'renewable_energy'. Since we don't need it we can drop this variable and also the original variable 'Loan.Purpose'. But before that let's just concatanate \n",
    "the dummy variables to the original dataframe. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data = loan_data.join(loan_data_dummies)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's drop the variables 'Loan.Purpose' and 'lp_renewable_energy'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.drop(['Loan.Purpose','lp_renewable_energy'], axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's check tha variable 'state'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CA    433\n",
       "NY    255\n",
       "TX    174\n",
       "FL    169\n",
       "IL    101\n",
       "GA     97\n",
       "PA     96\n",
       "NJ     94\n",
       "VA     78\n",
       "MA     73\n",
       "OH     71\n",
       "MD     68\n",
       "NC     64\n",
       "CO     61\n",
       "WA     58\n",
       "CT     50\n",
       "AZ     46\n",
       "MI     45\n",
       "MN     38\n",
       "AL     38\n",
       "MO     33\n",
       "NV     32\n",
       "OR     30\n",
       "SC     28\n",
       "WI     26\n",
       "KY     23\n",
       "LA     22\n",
       "OK     21\n",
       "KS     21\n",
       "UT     16\n",
       "RI     15\n",
       "NH     15\n",
       "WV     14\n",
       "AR     13\n",
       "NM     13\n",
       "HI     12\n",
       "DC     11\n",
       "AK     11\n",
       "DE      8\n",
       "MT      7\n",
       "VT      5\n",
       "SD      4\n",
       "WY      4\n",
       "IN      3\n",
       "IA      1\n",
       ".       1\n",
       "MS      1\n",
       "Name: State, dtype: int64"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data['State'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are too many states. Let's check how many unique values this variable has. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "47"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data['State'].nunique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are 47 unique values in this variable, so for now, let's decide to drop this variable altogether. Although we don't have a reason to drop this variable, we will just to do that to reduce the amount of data preperation needed for now. \n",
    "We can add this variable later if needed. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.drop(labels='State',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Home.Ownership', 'FICO.Range', 'Employment.Length'], dtype='object')"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.columns[loan_data.dtypes=='object']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have effectively reduced 6 'object' variables to 3. Let's go further and work with the other 3 variables."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's process the variable 'Home.Ownership'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MORTGAGE    1147\n",
       "RENT        1146\n",
       "OWN          200\n",
       "OTHER          5\n",
       "NONE           1\n",
       "Name: Home.Ownership, dtype: int64"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data['Home.Ownership'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we can simply ignore the categories 'OTHER' and 'NONE' and create dummy variables for the remaing 3 categories"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['ho_mortgage'] = np.where(loan_data['Home.Ownership']=='MORTGAGE',1,0)\n",
    "loan_data['ho_rent'] = np.where(loan_data['Home.Ownership']== 'RENT',1,0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Amount.Requested</th>\n",
       "      <th>Amount.Funded.By.Investors</th>\n",
       "      <th>Interest.Rate</th>\n",
       "      <th>Debt.To.Income.Ratio</th>\n",
       "      <th>Home.Ownership</th>\n",
       "      <th>Monthly.Income</th>\n",
       "      <th>FICO.Range</th>\n",
       "      <th>Open.CREDIT.Lines</th>\n",
       "      <th>Revolving.CREDIT.Balance</th>\n",
       "      <th>Inquiries.in.the.Last.6.Months</th>\n",
       "      <th>Employment.Length</th>\n",
       "      <th>Loan_lenght_36</th>\n",
       "      <th>lp_cem</th>\n",
       "      <th>lp_chos</th>\n",
       "      <th>lp_dm</th>\n",
       "      <th>lp_hmvw</th>\n",
       "      <th>ho_mortgage</th>\n",
       "      <th>ho_rent</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>81174.0</td>\n",
       "      <td>20000.0</td>\n",
       "      <td>20000.0</td>\n",
       "      <td>8.90</td>\n",
       "      <td>14.90</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>6541.67</td>\n",
       "      <td>735-739</td>\n",
       "      <td>14.0</td>\n",
       "      <td>14272.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>&lt; 1 year</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>99592.0</td>\n",
       "      <td>19200.0</td>\n",
       "      <td>19200.0</td>\n",
       "      <td>12.12</td>\n",
       "      <td>28.36</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>4583.33</td>\n",
       "      <td>715-719</td>\n",
       "      <td>12.0</td>\n",
       "      <td>11140.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>80059.0</td>\n",
       "      <td>35000.0</td>\n",
       "      <td>35000.0</td>\n",
       "      <td>21.98</td>\n",
       "      <td>23.81</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>11500.00</td>\n",
       "      <td>690-694</td>\n",
       "      <td>14.0</td>\n",
       "      <td>21977.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15825.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>9975.0</td>\n",
       "      <td>9.99</td>\n",
       "      <td>14.30</td>\n",
       "      <td>MORTGAGE</td>\n",
       "      <td>3833.33</td>\n",
       "      <td>695-699</td>\n",
       "      <td>10.0</td>\n",
       "      <td>9346.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5 years</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>33182.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>11.71</td>\n",
       "      <td>18.78</td>\n",
       "      <td>RENT</td>\n",
       "      <td>3195.00</td>\n",
       "      <td>695-699</td>\n",
       "      <td>11.0</td>\n",
       "      <td>14469.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9 years</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        ID  Amount.Requested  Amount.Funded.By.Investors  Interest.Rate  \\\n",
       "0  81174.0           20000.0                     20000.0           8.90   \n",
       "1  99592.0           19200.0                     19200.0          12.12   \n",
       "2  80059.0           35000.0                     35000.0          21.98   \n",
       "3  15825.0           10000.0                      9975.0           9.99   \n",
       "4  33182.0           12000.0                     12000.0          11.71   \n",
       "\n",
       "   Debt.To.Income.Ratio Home.Ownership  Monthly.Income FICO.Range  \\\n",
       "0                 14.90       MORTGAGE         6541.67    735-739   \n",
       "1                 28.36       MORTGAGE         4583.33    715-719   \n",
       "2                 23.81       MORTGAGE        11500.00    690-694   \n",
       "3                 14.30       MORTGAGE         3833.33    695-699   \n",
       "4                 18.78           RENT         3195.00    695-699   \n",
       "\n",
       "   Open.CREDIT.Lines  Revolving.CREDIT.Balance  \\\n",
       "0               14.0                   14272.0   \n",
       "1               12.0                   11140.0   \n",
       "2               14.0                   21977.0   \n",
       "3               10.0                    9346.0   \n",
       "4               11.0                   14469.0   \n",
       "\n",
       "   Inquiries.in.the.Last.6.Months Employment.Length  Loan_lenght_36  lp_cem  \\\n",
       "0                             2.0          < 1 year               1       0   \n",
       "1                             1.0           2 years               1       0   \n",
       "2                             1.0           2 years               0       0   \n",
       "3                             0.0           5 years               1       0   \n",
       "4                             0.0           9 years               1       0   \n",
       "\n",
       "   lp_chos  lp_dm  lp_hmvw  ho_mortgage  ho_rent  \n",
       "0        0      1        0            1        0  \n",
       "1        0      1        0            1        0  \n",
       "2        0      1        0            1        0  \n",
       "3        0      1        0            1        0  \n",
       "4        1      0        0            0        1  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have created dummy variables for 'Home.Ownership' we can go ahead and drop that variable. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.drop(labels='Home.Ownership',axis=1, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['FICO.Range', 'Employment.Length'], dtype='object')"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.columns[loan_data.dtypes=='object']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's deal with the variable 'FICO.Range' next"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    735-739\n",
       "1    715-719\n",
       "2    690-694\n",
       "3    695-699\n",
       "4    695-699\n",
       "Name: FICO.Range, dtype: object"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data['FICO.Range'].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since this variable has a range, one easy way is to replace each value with the average of the range.\n",
    "\n",
    "In order to do that, \n",
    "1. first let's split each value in the variable 'FICO.Range' on the hyphen('-') in the middle of the value. \n",
    "2. Then let's put the first part of the split result (which will be the lower limit) in a list called 'lower_limit' and the second part of the split result (which will be the upper limit) in a list called 'upper_limit'. \n",
    "3. Finally let's slice the lower and upper limit values from each list then find the average of these and insert them in a new variable called 'fico' in the original dataframe 'loan_data'\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "lower_limit=[]\n",
    "upper_limit=[]\n",
    "for i in range(len(loan_data)):\n",
    "    lower_limit.append(int(loan_data['FICO.Range'][i].split('-')[0]))\n",
    "    upper_limit.append(int(loan_data['FICO.Range'][i].split('-')[1]))\n",
    "#splitting the column on '-' and appending the results to two columns \n",
    "# called lower_limit and upper_limit \n",
    "\n",
    "for i in range(len(loan_data)):\n",
    "    loan_data.loc[i,'fico']= ((lower_limit[i]+upper_limit[i])/2)\n",
    "    \n",
    "# on a for loop we slice the values from the two lists, find their average and insert \n",
    "# them in a new column called 'fico' in the original dataframe 'loan_data'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have created a new variable for the original variable 'FICO.Range' containing the average value of the range, we can drop the original variable 'FICO.Range' and the temproary lists 'lower_limit' and 'upper_limit'. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Once deleted, variables cannot be recovered. Proceed (y/[n])?  y\n"
     ]
    }
   ],
   "source": [
    "loan_data.drop(labels='FICO.Range', axis=1, inplace=True)\n",
    "%reset_selective lower_limit, upper_limit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID</th>\n",
       "      <th>Amount.Requested</th>\n",
       "      <th>Amount.Funded.By.Investors</th>\n",
       "      <th>Interest.Rate</th>\n",
       "      <th>Debt.To.Income.Ratio</th>\n",
       "      <th>Monthly.Income</th>\n",
       "      <th>Open.CREDIT.Lines</th>\n",
       "      <th>Revolving.CREDIT.Balance</th>\n",
       "      <th>Inquiries.in.the.Last.6.Months</th>\n",
       "      <th>Employment.Length</th>\n",
       "      <th>Loan_lenght_36</th>\n",
       "      <th>lp_cem</th>\n",
       "      <th>lp_chos</th>\n",
       "      <th>lp_dm</th>\n",
       "      <th>lp_hmvw</th>\n",
       "      <th>ho_mortgage</th>\n",
       "      <th>ho_rent</th>\n",
       "      <th>fico</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>81174.0</td>\n",
       "      <td>20000.0</td>\n",
       "      <td>20000.0</td>\n",
       "      <td>8.90</td>\n",
       "      <td>14.90</td>\n",
       "      <td>6541.67</td>\n",
       "      <td>14.0</td>\n",
       "      <td>14272.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>&lt; 1 year</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>737.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>99592.0</td>\n",
       "      <td>19200.0</td>\n",
       "      <td>19200.0</td>\n",
       "      <td>12.12</td>\n",
       "      <td>28.36</td>\n",
       "      <td>4583.33</td>\n",
       "      <td>12.0</td>\n",
       "      <td>11140.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>717.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>80059.0</td>\n",
       "      <td>35000.0</td>\n",
       "      <td>35000.0</td>\n",
       "      <td>21.98</td>\n",
       "      <td>23.81</td>\n",
       "      <td>11500.00</td>\n",
       "      <td>14.0</td>\n",
       "      <td>21977.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2 years</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>692.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15825.0</td>\n",
       "      <td>10000.0</td>\n",
       "      <td>9975.0</td>\n",
       "      <td>9.99</td>\n",
       "      <td>14.30</td>\n",
       "      <td>3833.33</td>\n",
       "      <td>10.0</td>\n",
       "      <td>9346.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5 years</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>697.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>33182.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>11.71</td>\n",
       "      <td>18.78</td>\n",
       "      <td>3195.00</td>\n",
       "      <td>11.0</td>\n",
       "      <td>14469.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9 years</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>697.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        ID  Amount.Requested  Amount.Funded.By.Investors  Interest.Rate  \\\n",
       "0  81174.0           20000.0                     20000.0           8.90   \n",
       "1  99592.0           19200.0                     19200.0          12.12   \n",
       "2  80059.0           35000.0                     35000.0          21.98   \n",
       "3  15825.0           10000.0                      9975.0           9.99   \n",
       "4  33182.0           12000.0                     12000.0          11.71   \n",
       "\n",
       "   Debt.To.Income.Ratio  Monthly.Income  Open.CREDIT.Lines  \\\n",
       "0                 14.90         6541.67               14.0   \n",
       "1                 28.36         4583.33               12.0   \n",
       "2                 23.81        11500.00               14.0   \n",
       "3                 14.30         3833.33               10.0   \n",
       "4                 18.78         3195.00               11.0   \n",
       "\n",
       "   Revolving.CREDIT.Balance  Inquiries.in.the.Last.6.Months Employment.Length  \\\n",
       "0                   14272.0                             2.0          < 1 year   \n",
       "1                   11140.0                             1.0           2 years   \n",
       "2                   21977.0                             1.0           2 years   \n",
       "3                    9346.0                             0.0           5 years   \n",
       "4                   14469.0                             0.0           9 years   \n",
       "\n",
       "   Loan_lenght_36  lp_cem  lp_chos  lp_dm  lp_hmvw  ho_mortgage  ho_rent  \\\n",
       "0               1       0        0      1        0            1        0   \n",
       "1               1       0        0      1        0            1        0   \n",
       "2               0       0        0      1        0            1        0   \n",
       "3               1       0        0      1        0            1        0   \n",
       "4               1       0        1      0        0            0        1   \n",
       "\n",
       "    fico  \n",
       "0  737.0  \n",
       "1  717.0  \n",
       "2  692.0  \n",
       "3  697.0  \n",
       "4  697.0  "
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['Employment.Length'], dtype='object')"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.columns[loan_data.dtypes=='object']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The only remaining 'object' column is 'Employment.Length'. Let's deal with that now. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10+ years    653\n",
       "< 1 year     249\n",
       "2 years      243\n",
       "3 years      235\n",
       "5 years      202\n",
       "4 years      191\n",
       "1 year       177\n",
       "6 years      163\n",
       "7 years      127\n",
       "8 years      108\n",
       "9 years       72\n",
       ".              2\n",
       "Name: Employment.Length, dtype: int64"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data['Employment.Length'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's first fix remove the words 'years' and 'year' from the variable. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['Employment.Length'] = loan_data['Employment.Length'].astype('str')\n",
    "    # first let's convert the variable to string datatype so that removing words and letters will\n",
    "    # be easy\n",
    "\n",
    "loan_data['Employment.Length'] = [x.replace('years','') for x in\n",
    "                                  loan_data['Employment.Length']]\n",
    "loan_data['Employment.Length'] = [x.replace('year', '') for x in \n",
    "                                  loan_data['Employment.Length']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10+     653\n",
       "< 1     249\n",
       "2       243\n",
       "3       235\n",
       "5       202\n",
       "4       191\n",
       "1       177\n",
       "6       163\n",
       "7       127\n",
       "8       108\n",
       "nan      78\n",
       "9        72\n",
       ".         2\n",
       "Name: Employment.Length, dtype: int64"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data['Employment.Length'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's group this variable with the response variable 'Interest.Rate' so we can combine categories togther"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Employment.Length\n",
       ".       11.34\n",
       "1       12.49\n",
       "10+     13.34\n",
       "2       12.87\n",
       "3       12.77\n",
       "4       13.14\n",
       "5       13.40\n",
       "6       13.29\n",
       "7       13.10\n",
       "8       13.01\n",
       "9       13.15\n",
       "< 1     12.86\n",
       "nan     12.78\n",
       "Name: Interest.Rate, dtype: float64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.groupby('Employment.Length')['Interest.Rate'].mean().round(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. As we can see, 'nan' and '<1' are similar to each other in mean. so we can compare them together. \n",
    "2. '<1' means 0, so we can replace that with 0. \n",
    "3. '10+' can be replaced by 10. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data['Employment.Length']=[x.replace('nan','< 1') for x in\n",
    "                                loan_data['Employment.Length']]\n",
    "loan_data['Employment.Length']=[x.replace('10+','10') for x in\n",
    "                                loan_data['Employment.Length']]\n",
    "loan_data['Employment.Length']=[x.replace('< 1','0') for x in\n",
    "                                loan_data['Employment.Length']]\n",
    "loan_data['Employment.Length']=pd.to_numeric(loan_data['Employment.Length'],\n",
    "                                             errors='coerce')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.0\n",
       "1    2.0\n",
       "2    2.0\n",
       "3    5.0\n",
       "4    9.0\n",
       "Name: Employment.Length, dtype: float64"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data['Employment.Length'].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ID                                float64\n",
       "Amount.Requested                  float64\n",
       "Amount.Funded.By.Investors        float64\n",
       "Interest.Rate                     float64\n",
       "Debt.To.Income.Ratio              float64\n",
       "Monthly.Income                    float64\n",
       "Open.CREDIT.Lines                 float64\n",
       "Revolving.CREDIT.Balance          float64\n",
       "Inquiries.in.the.Last.6.Months    float64\n",
       "Employment.Length                 float64\n",
       "Loan_lenght_36                      uint8\n",
       "lp_cem                              uint8\n",
       "lp_chos                             uint8\n",
       "lp_dm                               uint8\n",
       "lp_hmvw                             uint8\n",
       "ho_mortgage                         int32\n",
       "ho_rent                             int32\n",
       "fico                              float64\n",
       "dtype: object"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.dtypes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, all our variables are numeric. \n",
    "We can also drop the variable 'ID' because it's the identity number for every person and doesn't solve any real purpose. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.drop(labels='ID',axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Amount.Requested</th>\n",
       "      <th>Amount.Funded.By.Investors</th>\n",
       "      <th>Interest.Rate</th>\n",
       "      <th>Debt.To.Income.Ratio</th>\n",
       "      <th>Monthly.Income</th>\n",
       "      <th>Open.CREDIT.Lines</th>\n",
       "      <th>Revolving.CREDIT.Balance</th>\n",
       "      <th>Inquiries.in.the.Last.6.Months</th>\n",
       "      <th>Employment.Length</th>\n",
       "      <th>Loan_lenght_36</th>\n",
       "      <th>lp_cem</th>\n",
       "      <th>lp_chos</th>\n",
       "      <th>lp_dm</th>\n",
       "      <th>lp_hmvw</th>\n",
       "      <th>ho_mortgage</th>\n",
       "      <th>ho_rent</th>\n",
       "      <th>fico</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>20000.0</td>\n",
       "      <td>20000.0</td>\n",
       "      <td>8.90</td>\n",
       "      <td>14.90</td>\n",
       "      <td>6541.67</td>\n",
       "      <td>14.0</td>\n",
       "      <td>14272.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>737.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>19200.0</td>\n",
       "      <td>19200.0</td>\n",
       "      <td>12.12</td>\n",
       "      <td>28.36</td>\n",
       "      <td>4583.33</td>\n",
       "      <td>12.0</td>\n",
       "      <td>11140.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>717.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>35000.0</td>\n",
       "      <td>35000.0</td>\n",
       "      <td>21.98</td>\n",
       "      <td>23.81</td>\n",
       "      <td>11500.00</td>\n",
       "      <td>14.0</td>\n",
       "      <td>21977.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>692.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10000.0</td>\n",
       "      <td>9975.0</td>\n",
       "      <td>9.99</td>\n",
       "      <td>14.30</td>\n",
       "      <td>3833.33</td>\n",
       "      <td>10.0</td>\n",
       "      <td>9346.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>697.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>12000.0</td>\n",
       "      <td>12000.0</td>\n",
       "      <td>11.71</td>\n",
       "      <td>18.78</td>\n",
       "      <td>3195.00</td>\n",
       "      <td>11.0</td>\n",
       "      <td>14469.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>697.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Amount.Requested  Amount.Funded.By.Investors  Interest.Rate  \\\n",
       "0           20000.0                     20000.0           8.90   \n",
       "1           19200.0                     19200.0          12.12   \n",
       "2           35000.0                     35000.0          21.98   \n",
       "3           10000.0                      9975.0           9.99   \n",
       "4           12000.0                     12000.0          11.71   \n",
       "\n",
       "   Debt.To.Income.Ratio  Monthly.Income  Open.CREDIT.Lines  \\\n",
       "0                 14.90         6541.67               14.0   \n",
       "1                 28.36         4583.33               12.0   \n",
       "2                 23.81        11500.00               14.0   \n",
       "3                 14.30         3833.33               10.0   \n",
       "4                 18.78         3195.00               11.0   \n",
       "\n",
       "   Revolving.CREDIT.Balance  Inquiries.in.the.Last.6.Months  \\\n",
       "0                   14272.0                             2.0   \n",
       "1                   11140.0                             1.0   \n",
       "2                   21977.0                             1.0   \n",
       "3                    9346.0                             0.0   \n",
       "4                   14469.0                             0.0   \n",
       "\n",
       "   Employment.Length  Loan_lenght_36  lp_cem  lp_chos  lp_dm  lp_hmvw  \\\n",
       "0                0.0               1       0        0      1        0   \n",
       "1                2.0               1       0        0      1        0   \n",
       "2                2.0               0       0        0      1        0   \n",
       "3                5.0               1       0        0      1        0   \n",
       "4                9.0               1       0        1      0        0   \n",
       "\n",
       "   ho_mortgage  ho_rent   fico  \n",
       "0            1        0  737.0  \n",
       "1            1        0  717.0  \n",
       "2            1        0  692.0  \n",
       "3            1        0  697.0  \n",
       "4            0        1  697.0  "
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "33"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.isna().sum().sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2500, 17)"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "loan_data.dropna(axis=0,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2473, 17)"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "loan_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's split the data into train and test data to proceed with modelling. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_loan_data, test_loan_data = train_test_split(loan_data, test_size=0.2, \n",
    "                                                   random_state=2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1978, 17), (495, 17))"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_loan_data.shape, test_loan_data.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "'Interest.Rate' variable is the one we are going to predict, so let's make that the 'Y' variable in both training and test data and drop it from the X variable in both training and test data. \n",
    "\n",
    "Also the variable 'Amount.Funded.By.Investors' won't be available to us at this point in real life. So, let's drop this also from the training and test data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train=train_loan_data.drop(labels=['Interest.Rate','Amount.Funded.By.Investors'],\n",
    "                             axis=1)\n",
    "y_train=train_loan_data['Interest.Rate']\n",
    "x_test=test_loan_data.drop(labels=['Interest.Rate','Amount.Funded.By.Investors'],\n",
    "                           axis=1)\n",
    "y_test=test_loan_data['Interest.Rate']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "linreg=LinearRegression()\n",
    "#Initializing a linear regression object"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None,\n",
       "         normalize=False)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "linreg.fit(X=x_train, y=y_train)\n",
    "#fitting the linear model on train data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions=linreg.predict(X=x_test)\n",
    "#Making predictions on test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "error=predictions-y_test\n",
    "#calculating the errors in prediction by fiding the difference in actual 'y' and \n",
    "# predicted 'y'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/latex": [
       "$rmse = 1/m*\\sum_{i=0}^{m-1}\\sqrt{(predictions(i)-ytest(i))^2}$\n"
      ],
      "text/plain": [
       "<IPython.core.display.Latex object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%%latex\n",
    "$rmse = 1/m*\\sum_{i=0}^{m-1}\\sqrt{(predictions(i)-ytest(i))^2}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "rmse=np.sqrt((np.sum(error**2))/len(x_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.0225656998568877"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rmse"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "RMSE can be used to compare our linear regression model with other techniques and eventually pick the model with the least error. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's see how to extract coefficients from the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "coefs=linreg.coef_\n",
    "#these are the beta or theta values(beta_1 to beta_n) that our model is predicting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "75.24596218863286"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "linreg.intercept_ \n",
    "# this is the intercept value"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "features = x_train.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Amount.Requested', 0.00016318429645184338),\n",
       " ('Debt.To.Income.Ratio', 0.0005724866425414895),\n",
       " ('Monthly.Income', -4.1088848603575156e-05),\n",
       " ('Open.CREDIT.Lines', -0.0393078462756966),\n",
       " ('Revolving.CREDIT.Balance', -2.472474215323584e-06),\n",
       " ('Inquiries.in.the.Last.6.Months', 0.3870776386826336),\n",
       " ('Employment.Length', 0.004939976256298612),\n",
       " ('Loan_lenght_36', -3.082420573668312),\n",
       " ('lp_cem', -0.33812455519896284),\n",
       " ('lp_chos', -0.37631807997293887),\n",
       " ('lp_dm', -0.513743744137324),\n",
       " ('lp_hmvw', -0.640702578936149),\n",
       " ('ho_mortgage', -0.3785684674068913),\n",
       " ('ho_rent', -0.13052594354375308),\n",
       " ('fico', -0.08592703966767974)]"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(zip(features,coefs))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regularization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, linear regression gives the coefficient value for each feature. Ideally, the feature which does not contribute much in predicting our output variable should have a coefficient value of 0. But it is not the case. So, we use regularization to penalize and minimize the coefficients of variables that don't contribute much to our model. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ridge Regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "alpha or regularisation parameter is a hyperparameter and we'd look at multiple values of it and choose the best one through 10 fold cross validation. \n",
    "Note: In class notes, regularisation parameter is called lambda"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "#finding the best value of regularisation parameter with cross validation for ridge\n",
    "# regression. \n",
    "alphas=np.linspace(0.0001,10,100)\n",
    "# We need to reset index for cross validation to work without hitch. \n",
    "x_train.reset_index(drop=True, inplace=True)\n",
    "y_train.reset_index(drop=True, inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.000\t 2.070131\t\n",
      "0.101\t 2.070081\t\n",
      "0.202\t 2.070040\t\n",
      "0.303\t 2.070008\t\n",
      "0.404\t 2.069980\t\n",
      "0.505\t 2.069957\t\n",
      "0.606\t 2.069938\t\n",
      "0.707\t 2.069921\t\n",
      "0.808\t 2.069906\t\n",
      "0.909\t 2.069892\t\n",
      "1.010\t 2.069881\t\n",
      "1.111\t 2.069870\t\n",
      "1.212\t 2.069861\t\n",
      "1.313\t 2.069853\t\n",
      "1.414\t 2.069845\t\n",
      "1.515\t 2.069838\t\n",
      "1.616\t 2.069832\t\n",
      "1.717\t 2.069827\t\n",
      "1.818\t 2.069821\t\n",
      "1.919\t 2.069817\t\n",
      "2.020\t 2.069813\t\n",
      "2.121\t 2.069809\t\n",
      "2.222\t 2.069806\t\n",
      "2.323\t 2.069803\t\n",
      "2.424\t 2.069800\t\n",
      "2.525\t 2.069797\t\n",
      "2.626\t 2.069795\t\n",
      "2.727\t 2.069793\t\n",
      "2.828\t 2.069792\t\n",
      "2.929\t 2.069790\t\n",
      "3.030\t 2.069789\t\n",
      "3.131\t 2.069788\t\n",
      "3.232\t 2.069788\t\n",
      "3.333\t 2.069787\t\n",
      "3.434\t 2.069787\t\n",
      "3.535\t 2.069787\t\n",
      "3.636\t 2.069787\t\n",
      "3.737\t 2.069787\t\n",
      "3.838\t 2.069787\t\n",
      "3.939\t 2.069788\t\n",
      "4.040\t 2.069789\t\n",
      "4.141\t 2.069790\t\n",
      "4.242\t 2.069791\t\n",
      "4.343\t 2.069792\t\n",
      "4.444\t 2.069793\t\n",
      "4.546\t 2.069795\t\n",
      "4.647\t 2.069796\t\n",
      "4.748\t 2.069798\t\n",
      "4.849\t 2.069800\t\n",
      "4.950\t 2.069802\t\n",
      "5.051\t 2.069804\t\n",
      "5.152\t 2.069807\t\n",
      "5.253\t 2.069809\t\n",
      "5.354\t 2.069812\t\n",
      "5.455\t 2.069815\t\n",
      "5.556\t 2.069817\t\n",
      "5.657\t 2.069820\t\n",
      "5.758\t 2.069824\t\n",
      "5.859\t 2.069827\t\n",
      "5.960\t 2.069830\t\n",
      "6.061\t 2.069834\t\n",
      "6.162\t 2.069837\t\n",
      "6.263\t 2.069841\t\n",
      "6.364\t 2.069845\t\n",
      "6.465\t 2.069849\t\n",
      "6.566\t 2.069853\t\n",
      "6.667\t 2.069857\t\n",
      "6.768\t 2.069861\t\n",
      "6.869\t 2.069866\t\n",
      "6.970\t 2.069870\t\n",
      "7.071\t 2.069875\t\n",
      "7.172\t 2.069880\t\n",
      "7.273\t 2.069884\t\n",
      "7.374\t 2.069889\t\n",
      "7.475\t 2.069894\t\n",
      "7.576\t 2.069900\t\n",
      "7.677\t 2.069905\t\n",
      "7.778\t 2.069910\t\n",
      "7.879\t 2.069916\t\n",
      "7.980\t 2.069921\t\n",
      "8.081\t 2.069927\t\n",
      "8.182\t 2.069933\t\n",
      "8.283\t 2.069939\t\n",
      "8.384\t 2.069945\t\n",
      "8.485\t 2.069951\t\n",
      "8.586\t 2.069957\t\n",
      "8.687\t 2.069964\t\n",
      "8.788\t 2.069970\t\n",
      "8.889\t 2.069977\t\n",
      "8.990\t 2.069983\t\n",
      "9.091\t 2.069990\t\n",
      "9.192\t 2.069997\t\n",
      "9.293\t 2.070004\t\n",
      "9.394\t 2.070011\t\n",
      "9.495\t 2.070018\t\n",
      "9.596\t 2.070025\t\n",
      "9.697\t 2.070032\t\n",
      "9.798\t 2.070040\t\n",
      "9.899\t 2.070047\t\n",
      "10.000\t 2.070055\t\n",
      "Alpha with minimum 10CV error is 3.5354181818181822 and the corresponding RMSE is 2.069786624576659: \n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "We have 100 values of alpha between 0.001 and 10. \n",
    "We use these 100 values and 10 fold CV for each value\n",
    "to see which value has the least rmse value, that value\n",
    "will give the best alpha value. \n",
    "\n",
    "    In the for-loop below: \n",
    "    \n",
    "    1. For each value of alpha: \n",
    "    \n",
    "        1.1 We initiate a KFold object for 10 fold \n",
    "        cross-validation. \n",
    "        \n",
    "        We have initiated 10 fold cross validation object so \n",
    "        cross validation loop will run for 10 iterations and each time \n",
    "        our training data X_train will be split into 10 buckets. \n",
    "        Likewise if we use 20 fold cross validation our cross validation\n",
    "        loop will run for 20 times and the training data will be split \n",
    "        into 20 buckets.\n",
    "        \n",
    "        1.2. We initiate an object called xval_err as 0.\n",
    "        \n",
    "        1.3. For each alpha and first iteration of the KFold validation:\n",
    "               \n",
    "            1.3.1. The k-fold will split our training data [x_train] into \n",
    "            9 buckets of training data and 1 bucket of validation data. \n",
    "            It will use 9 buckets of the training data [X_train] to train \n",
    "            the Ridge model and 1 bucket of the training data [X_train] to \n",
    "            preidct the output [Y_cap]. \n",
    "\n",
    "            1.3.2. This predicted 'y_cap' is subtracted from the actual \n",
    "            y values from the training data [X_train] corresponding to the \n",
    "            10th set (validation set) to calculate the validation\n",
    "            error.\n",
    "\n",
    "            1.3.3. The validation error for this iteration is squared and \n",
    "            added to the variable xval_err for the next 10 iterations.\n",
    "\n",
    "        1.4. The total validation error for the past 10 iterations \n",
    "        are squared, divided by the length of the \n",
    "        training dataset(x_train) and a squared root is taken on the \n",
    "        quotient. \n",
    "        This is the RMSE for this value of alpha using 10 fold CV. \n",
    "        \n",
    "        1.5. This RMSE value is stored in the rmse_list object.\n",
    "        Once we have completed iterating through all the alpha values,\n",
    "        the rmse_list object will have a length of 100, since we\n",
    "        use 100 different alpha values. \n",
    "        \n",
    "        1.6. Finally the alpha value which corresponds to the minimum \n",
    "        rmse value is returned as the optimal alpha value.         \n",
    "    \"\"\"  \n",
    "rmse_list=[]\n",
    "\n",
    "for a in alphas:\n",
    "    ridge=Ridge(fit_intercept=True, alpha=a)\n",
    "    # ridge is a linear regression model with L2 regularization. \n",
    "    \n",
    "    kf = KFold(n_splits=10)\n",
    "    xval_err=0\n",
    "    i =0\n",
    "    for train_index, validation_index in kf.split(x_train):\n",
    "        #print(len(train_index), len(validation_index))\n",
    "        ridge.fit(x_train.iloc[train_index,:], y_train[train_index])\n",
    "        p=ridge.predict(x_train.iloc[validation_index,:])\n",
    "        err = p-y_train[validation_index] #iloc doesn't work here for some reason\n",
    "        xval_err = xval_err+np.dot(err, err)\n",
    "    rmse_10cv = np.sqrt(xval_err/len(x_train))\n",
    "    rmse_list.append(rmse_10cv)\n",
    "    print('{:.3f}\\t {:.6f}\\t'.format(a, rmse_10cv))\n",
    "best_alpha= alphas[rmse_list==min(rmse_list)]\n",
    "print('Alpha with minimum 10CV error is {} and the corresponding RMSE is {}: '.\n",
    "      format(best_alpha[0], min(rmse_list)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will use the best alpha value that we just determined to fit a ridge regression object on the entire dataset and predict the rmse value for the entire dataset. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.0236668696412723"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ridge=Ridge(fit_intercept=True, alpha=best_alpha)\n",
    "ridge.fit(x_train, y_train)\n",
    "predictionsOnTestData = ridge.predict(x_test)\n",
    "predictionErrorOnTestData = predictionsOnTestData - y_test\n",
    "rmse_ridge = np.sqrt(np.dot(predictionErrorOnTestData,\n",
    "                            predictionErrorOnTestData)/len(predictionsOnTestData))\n",
    "rmse_ridge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Amount.Requested', 0.0001641799190080323),\n",
       " ('Debt.To.Income.Ratio', 0.0006487949921075561),\n",
       " ('Monthly.Income', -4.169344863753166e-05),\n",
       " ('Open.CREDIT.Lines', -0.039616383599302744),\n",
       " ('Revolving.CREDIT.Balance', -2.5209565104184987e-06),\n",
       " ('Inquiries.in.the.Last.6.Months', 0.3870049295626152),\n",
       " ('Employment.Length', 0.004888838204666946),\n",
       " ('Loan_lenght_36', -3.041894854788477),\n",
       " ('lp_cem', 0.011310143101232472),\n",
       " ('lp_chos', -0.0321514576874333),\n",
       " ('lp_dm', -0.1690620008874746),\n",
       " ('lp_hmvw', -0.2882910897632634),\n",
       " ('ho_mortgage', -0.3626507937625516),\n",
       " ('ho_rent', -0.11681552700277421),\n",
       " ('fico', -0.08593974410496531)]"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(zip(x_train.columns,ridge.coef_))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Results of Ridge Regression: \n",
    "We can see that ridge even though regression shrinks the coefficient value for each variable, it never really makes them 0. Which means it never shrinks the size of our model. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Lasso Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.000\t 2.070110\t\n",
      "0.010\t 2.069490\t\n",
      "0.020\t 2.069924\t\n",
      "0.030\t 2.071233\t\n",
      "0.041\t 2.073187\t\n",
      "0.051\t 2.075622\t\n",
      "0.061\t 2.077515\t\n",
      "0.071\t 2.079609\t\n",
      "0.081\t 2.082291\t\n",
      "0.091\t 2.085403\t\n",
      "0.101\t 2.088892\t\n",
      "0.111\t 2.092756\t\n",
      "0.121\t 2.096993\t\n",
      "0.131\t 2.101651\t\n",
      "0.141\t 2.106692\t\n",
      "0.152\t 2.112098\t\n",
      "0.162\t 2.117865\t\n",
      "0.172\t 2.123991\t\n",
      "0.182\t 2.130519\t\n",
      "0.192\t 2.137403\t\n",
      "0.202\t 2.144636\t\n",
      "0.212\t 2.152212\t\n",
      "0.222\t 2.160129\t\n",
      "0.232\t 2.168383\t\n",
      "0.242\t 2.176970\t\n",
      "0.253\t 2.185886\t\n",
      "0.263\t 2.195126\t\n",
      "0.273\t 2.204688\t\n",
      "0.283\t 2.214567\t\n",
      "0.293\t 2.224758\t\n",
      "0.303\t 2.235257\t\n",
      "0.313\t 2.246061\t\n",
      "0.323\t 2.257168\t\n",
      "0.333\t 2.268569\t\n",
      "0.343\t 2.280261\t\n",
      "0.354\t 2.292239\t\n",
      "0.364\t 2.304500\t\n",
      "0.374\t 2.317037\t\n",
      "0.384\t 2.329848\t\n",
      "0.394\t 2.342926\t\n",
      "0.404\t 2.356269\t\n",
      "0.414\t 2.369871\t\n",
      "0.424\t 2.382382\t\n",
      "0.434\t 2.391821\t\n",
      "0.444\t 2.397942\t\n",
      "0.455\t 2.399453\t\n",
      "0.465\t 2.400998\t\n",
      "0.475\t 2.402576\t\n",
      "0.485\t 2.404186\t\n",
      "0.495\t 2.405833\t\n",
      "0.505\t 2.407512\t\n",
      "0.515\t 2.409228\t\n",
      "0.525\t 2.410977\t\n",
      "0.535\t 2.412759\t\n",
      "0.545\t 2.414465\t\n",
      "0.556\t 2.415905\t\n",
      "0.566\t 2.417304\t\n",
      "0.576\t 2.418402\t\n",
      "0.586\t 2.419524\t\n",
      "0.596\t 2.420670\t\n",
      "0.606\t 2.421792\t\n",
      "0.616\t 2.422704\t\n",
      "0.626\t 2.423412\t\n",
      "0.636\t 2.423918\t\n",
      "0.646\t 2.424262\t\n",
      "0.657\t 2.424449\t\n",
      "0.667\t 2.424487\t\n",
      "0.677\t 2.424484\t\n",
      "0.687\t 2.424482\t\n",
      "0.697\t 2.424464\t\n",
      "0.707\t 2.424446\t\n",
      "0.717\t 2.424429\t\n",
      "0.727\t 2.424410\t\n",
      "0.737\t 2.424384\t\n",
      "0.747\t 2.424358\t\n",
      "0.758\t 2.424331\t\n",
      "0.768\t 2.424302\t\n",
      "0.778\t 2.424273\t\n",
      "0.788\t 2.424244\t\n",
      "0.798\t 2.424217\t\n",
      "0.808\t 2.424189\t\n",
      "0.818\t 2.424163\t\n",
      "0.828\t 2.424136\t\n",
      "0.838\t 2.424113\t\n",
      "0.848\t 2.424115\t\n",
      "0.859\t 2.424118\t\n",
      "0.869\t 2.424120\t\n",
      "0.879\t 2.424123\t\n",
      "0.889\t 2.424125\t\n",
      "0.899\t 2.424128\t\n",
      "0.909\t 2.424130\t\n",
      "0.919\t 2.424133\t\n",
      "0.929\t 2.424136\t\n",
      "0.939\t 2.424138\t\n",
      "0.949\t 2.424141\t\n",
      "0.960\t 2.424144\t\n",
      "0.970\t 2.424147\t\n",
      "0.980\t 2.424149\t\n",
      "0.990\t 2.424152\t\n",
      "1.000\t 2.424155\t\n",
      "Alpha with minimum 10CV error is 0.010199999999999999 and the corresponding RMSE is 2.069490402334332: \n"
     ]
    }
   ],
   "source": [
    "\"\"\"\n",
    "We have 100 values of alpha between 0.001 and 1. \n",
    "We use these 100 values and 10 fold CV for each value\n",
    "to see which value has the least rmse, that value\n",
    "will be the best alpha. \n",
    "\n",
    "    In the for-loop below: \n",
    "    \n",
    "    1. For each value of alpha: \n",
    "    \n",
    "        1.1 We initiate a KFold object for 10 fold \n",
    "        cross-validation. \n",
    "        \n",
    "        We have initiated 10 fold cross validation object so \n",
    "        cross validation loop will run for 10 iterations and each time \n",
    "        our training data X_train will be split into 10 buckets. \n",
    "        Likewise if we use 20 fold cross validation our cross validation\n",
    "        loop will run for 20 times and the training data will be split \n",
    "        into 20 buckets.\n",
    "        \n",
    "        1.2. We initiate an object called xval_err which is equal to 0.\n",
    "        \n",
    "        1.3. For each alpha and first iteration of the KFold validation:\n",
    "               \n",
    "            1.3.1. The k-fold will split our training data [x_train] into \n",
    "            9 buckets of training data and 1 bucket of validation data. \n",
    "            It will use 9 buckets of the training data [X_train] to train \n",
    "            the Lasso model and 1 bucket of the training data [X_train] to \n",
    "            preidct the output [Y_cap]. \n",
    "\n",
    "            1.3.2. This predicted 'y_cap' is subtracted from the actual \n",
    "            y values from the training data [X_train] corresponding to the \n",
    "            10th set (validation set) to calculate the validation\n",
    "            error.\n",
    "\n",
    "            1.3.3. The validation error for this iteration is squared and \n",
    "            added to the variable xval_err for the next 10 iterations.\n",
    "\n",
    "        1.4. The total validation error for the past 10 iterations \n",
    "        are squared, divided by the length of the \n",
    "        training dataset(x_train) and a squared root is taken on the \n",
    "        quotient. \n",
    "        This is the RMSE for this value of alpha using 10 fold CV. \n",
    "        \n",
    "        1.5. This RMSE value is stored in the rmse_list object.\n",
    "        Once we have completed iterating through all the alpha values,\n",
    "        the rmse_list object will have a length of 100, since we\n",
    "        use 100 different alpha values. \n",
    "        \n",
    "        1.6. Finally the alpha value which corresponds to the minimum \n",
    "        rmse value is returned as the optimal alpha value.\"\"\"\n",
    "\n",
    "alphas=np.linspace(0.0001,1,100)\n",
    "rmse_list=[]\n",
    "\n",
    "for a in alphas:\n",
    "    lasso=Lasso(fit_intercept=True, alpha=a, max_iter=10000)\n",
    "    # Lasso is a linear regression model with L1 regularization. \n",
    "    \n",
    "    kf = KFold(n_splits=10)\n",
    "    xval_err=0\n",
    "    for train_index, validation_index in kf.split(x_train):\n",
    "        lasso.fit(x_train.iloc[train_index,:], y_train[train_index])\n",
    "        p=lasso.predict(x_train.iloc[validation_index,:])\n",
    "        err = p-y_train[validation_index] #iloc doesn't work here for some reason\n",
    "        xval_err = xval_err+np.dot(err, err)\n",
    "    rmse_10cv = np.sqrt(xval_err/len(x_train))\n",
    "    rmse_list.append(rmse_10cv)\n",
    "    print('{:.3f}\\t {:.6f}\\t'.format(a, rmse_10cv))\n",
    "best_alpha= alphas[rmse_list==min(rmse_list)]\n",
    "print('Alpha with minimum 10CV error is {} and the corresponding RMSE is {}: '.\n",
    "      format(best_alpha[0], min(rmse_list)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best alpha value we got throuh lasso regression is 0.010199. We will use this value to train our final Lasso model and calculate the rmse. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.023276788895483"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lasso=Lasso(fit_intercept=True, alpha=best_alpha)\n",
    "lasso.fit(x_train, y_train)\n",
    "predictionsOnTestData_lasso = lasso.predict(x_test)\n",
    "predictionErrorOnTestData_lasso = predictionsOnTestData_lasso - y_test\n",
    "rmse_lasso = np.sqrt(np.dot(predictionErrorOnTestData_lasso,\n",
    "                            predictionErrorOnTestData_lasso)/len(predictionsOnTestData_lasso))\n",
    "rmse_lasso"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Amount.Requested', 0.00016395821531931226),\n",
       " ('Debt.To.Income.Ratio', 0.00032219108556223947),\n",
       " ('Monthly.Income', -4.2720916538871394e-05),\n",
       " ('Open.CREDIT.Lines', -0.03907551699159986),\n",
       " ('Revolving.CREDIT.Balance', -2.5527196166969956e-06),\n",
       " ('Inquiries.in.the.Last.6.Months', 0.37860640070117296),\n",
       " ('Employment.Length', 0.002810166741741678),\n",
       " ('Loan_lenght_36', -3.0073811789972993),\n",
       " ('lp_cem', 0.0),\n",
       " ('lp_chos', 0.057359802778081066),\n",
       " ('lp_dm', -0.027293745553160276),\n",
       " ('lp_hmvw', -0.0562011482052836),\n",
       " ('ho_mortgage', -0.22708727621821279),\n",
       " ('ho_rent', -0.0),\n",
       " ('fico', -0.08593321837314784)]"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(zip(x_train.columns,lasso.coef_))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Results of Lasso Regression\n",
    "As, we can see Lasso regression not only reduces our RMSE value a little, it also reduces the size of our model by making the coefficients of some variables as zero. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Analysing the trend between the FICO score in our test data and the response variables from the Linear Regression model without regularisation and Lasso and Ridge regression models and the. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x19f4d8b16d8>"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD8CAYAAACRkhiPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnX+UXWV5qJ93Tk7CDK3MhB+VjIkBGuMSIRAihKa2CC0RakmKUJNVriz8kbaX2oJtLFmySujVQh1rl657yy2t9uo1jQhCjAJGK9BevSa9wSTEaKYkSEiGKKOQUMlIJpP3/rH3nuw5Z//e+5yzzznvs9asOWfvb3/fu/c5c77Zez/nfUVVMQzDMLqTnlYHYBiGYbQOmwQMwzC6GJsEDMMwuhibBAzDMLoYmwQMwzC6GJsEDMMwuhibBAzDMLoYmwQMwzC6GJsEDMMwuphprQ4A4LTTTtO5c+e2OgzDMIy24sknn/yJqp6ep49STAJz585l69atrQ7DMAyjrRCRfXn7sMtBhmEYXUyiSUBEbhWRXSLyPRFZLyInicg6ERl2l31GRKpuWxGRT4nIHhF5SkQWNnYXDMMwjKzETgIiMgj8MbBIVd8MVIAVwDrgjcB5QC/wPneTq4B57s8q4J7iwzYMwzCKIOnloGlAr4hMA/qA51X1EXUB/h14ndt2GfA5d9VmoF9Eziw8csMwDCM3sZOAqo4AHweeAw4Ch1X169569zLQfwG+5i4aBPb7ujjgLjMMwzBKRpLLQQM4/92fBcwCThaRG3xN/g74N1X9P94mAd3UVa4RkVUislVEto6OjqaPHNiwbYQldz/GWbc9zJK7H2PDtpFM/RiGYXQrSS4H/QbwQ1UdVdVx4EHgVwBE5A7gdOCDvvYHgNm+568Dnq/tVFXvVdVFqrro9NPTa64bto2w5sGdjBwaQ4GRQ2OseXCnTQSGYRgpSDIJPAcsFpE+ERHgCuAHIvI+YCmwUlWP+9pvBN7tWkKLcS4fHSw68KFNw4yNT0xZNjY+wdCm4aKHMgzD6FhivyymqltE5AHgu8AxYBtwL/AKsA/4jjM38KCq/iXwCHA1sAc4AtzUiMCfPzSWarlhGIZRT6JvDKvqHcAdSbZ1baGbc8YVy6z+XkYCPvBn9fc2emjDMIyOoW2/Mbx66Xx6q5Upy3qrFVYvnd+iiAzDMNqPUuQOysLyCx3rdGjTMM8fGmNWfy+rl86fXA7OzeOo9d2MHRvDMKCNJwFwJoKwDy7PHvJuHnv2kLddN2PHxjAMj7a9HBSH2UPh2LExDMOjYycBs4fCsWNjGIZHx04CYZaQ2UN2bAzDOEHHTgJmD4Vjx8YwDI+2vjEcRRJ7qFuxY2MYhoc43+1qLYsWLdJml5c0RdIwjHZHRJ5U1UV5+ujYM4EoTJE0DMNw6Nh7AlGYImkYhuHQlZOAKZKGYRgOXTkJmCJpGIbh0JWTgCmShmEYDl15YzhMkQRYcvdjiYyhMthFWWMoQ+yGYZSDrlVEa6k1hsA5O7jr2vPqPiDTtC1DvEVsZxhG+ShCEU10OUhEbhWRXSLyPRFZLyInichZIrJFRJ4WkftEZLrbdob7fI+7fm6eAJtFGmOoDHZR1hjKELthGOUhdhIQkUHgj4FFqvpmoAKsAP4a+FtVnQe8BLzX3eS9wEuq+svA37rtSk8aY6gMdlHWGMoQu2EY5SHpjeFpQK+ITAP6gIPA5cAD7vrPAsvdx8vc57jrr3AL1JeaNMZQGeyirDGUIXbDMMpD7CSgqiPAx4HncD78DwNPAodU9Zjb7ADgXVAeBPa72x5z259a26+IrBKRrSKydXR0NO9+5CaNMVQGuyhrDGWI3TCM8hBrB4nIAM5/92cBh4D7gasCmnp3mIP+66+7+6yq9wL3gnNjOGG8DcNvDI0cGqMiMuVauf+maa1d1N9XRRVuvW87Q5uGC7VtwkyerEngorbbsG2EtRt3cWhsHICBvip3/Pa5dsPYMDqYWDtIRK4H3q6q73Wfvxu4FLgeeK2qHhORS4G1qrpURDa5j7/jXj76EXC6RgxUBjvII60900jbppkmz4ZtI6y+fwfjx6e+TNWKMHTdApsIDKOENMsOeg5YLCJ97rX9K4DvA48D17ltbgS+7D7e6D7HXf9Y1ARQNtLaM420bZpp8gxtGq6bAADGJ9TMIcPoYJLcE9iCc4P3u8BOd5t7gT8HPigie3Cu+X/a3eTTwKnu8g8CtzUg7oaR1p5ppG3TTJMnqk8zhwyjc0n0jWFVvQO4o2bxM8DFAW1/jnOpqC2Z1d/LSMCHXpRVk6Z9I2NpxFiNGs8wjHLQlbmDokhrzzTStmmmybN66XyqPfX39KsVMXPIMDqYrswdFEWcPeO3hyZUGezv5Z0XDfL47tFUlo7f+vHsosNj44kNoNs37GT9lv1MqCLA9Gk9vHrsOAD9vVXWXpPO6vHaprGDLAeRYbQ/ljsoIUGmjkdaYyeqryT93b5hJ5/f/FzkGNUeYej6xlk9loPIMFpP03IHGcGmjkdaYyeqryT9rd+yP3aM8eONtXosB5FhdAY2CSQka06erG2j2kwkPHtrpNVjOYgMozOwSSAhWXPyZG0b1aaSMBVTI60ey0FkGJ2BTQIJCTJ1PNIaO1F9Jelv5SWzY8eo9jTW6rEcRIbRGZgdlJCg3EITqvT3VhEJzhvkN3gqIqy8ZDYfWX5eaO6hWjsojI8sPw+gUDsoz/FopB3UagOp1eMbRqMxOygHUYbM1n0vBho8NyyeM/khbkTTagOp1eMbRhxmB7WYKEMmzOBJYvYYDq02kFo9vmE0A7sclIMoQybs/Cqp2WO03kBq9fiG0QzsTCAHUYZMmMGT1OwxWm8gtXp8w2gGNgnkIMqQCTN4kpg9hkOrDaRWj28YzcAuB+UgypDx1gXZQVmIyzUEcOdXdvHSESfvTzMMoaRkNWwaYSCliaVZBpRhtBKzg9qAuFxD1R7hODBRWxWswfmDklAmw6ZMsRhGETTFDhKR+SKy3ffzsojcIiIXiMhmd9lWEbnYbS8i8ikR2SMiT4nIwjwBGvG5hsaPa90E4C1vtclSJsOmTLEYRlmIvRykqsPABQAiUgFGgIeAfwDuVNVHReRq4GPAZThF6Oe5P5cA97i/jYzksVFabbKUybApUyyGURbS3hi+AtirqvsABV7jLj8FeN59vAz4nDpsBvpF5MxCou1S8tgorTZZymTYlCkWwygLaSeBFcB69/EtwJCI7Ac+Dqxxlw8C/m9EHXCXGRmJyzVU7REqQVXBGpw/KAllMmzKFIthlIXEk4CITAeuAe53F/0hcKuqzgZu5USh+SARvu6CtYiscu8lbB0dHU0XdZex/MJB7rr2PAb7exGcil/9vVUEGOzvZej6BfzN9QsY6KtObtPfW235TWGoj32wv7dlN2LLFIthlIXEdpCILANuVtUr3eeHgX5VVRER4LCqvkZE/h54QlXXu+2GgctU9WBY32YHBVObgG7x2QM8+9OxUF2xXZKdJYkz7b40ur1hlJEi7KA03xNYyYlLQeDcA/h14AngcuBpd/lG4I9E5As4N4QPR00ARjC1JSQnVPn23hcnn48cGmPNgzsBJusf+/XH2vVlIUmcafel0e0No5NJdDlIRPqA3wQe9C1+P/A3IrID+Ctglbv8EeAZYA+OQfRfC4u2i0iSaM6vN7aL/pgkzrT70uj2htHJJDoTUNUjwKk1y74FXBTQVoGbC4mui0lbQrJd9Mckcabdl0YvN4xOxnIHlZS0JSTbRX9MEmfafWn0csPoZGwSKClJEs359cZ20R+TxJl2Xxrd3jA6GUsgVxKCbBUg0A7yylv6r2OHJTsDuPAvv16axHJJkrKlTdzW6PaG0clYArkSkCaxWdq2qx/YwfhE+RLLGYaRHysv2SGksVXStq2dAKAcieUMwygHNgmUgDS2ShFt49YZhtE92CRQAtLYKkW0jVtnGEb3YJNACUhjq6RtW62UM7GcYRjlwOygEpDGVknbduu+F/nnLc/h1ZzprfZw17Xnh94U9iwlz0CaUGWwy+0ZyzNkdDI2CZQEf13iotpu2DbCl54cYWrRsfAvodWaR963lrs5t47lGTI6Hbsc1MEUkVMnyXadjOUZMjodmwQ6mKJz53SjUWR5hoxOxyaBDqbo3DndaBRZniGj07FJoIMpIqdOku06GcszZHQ6ljaiw6k1W972xtN5+KmDk7mEeqs9nFStcOjI+OT6x3ePTrGDBvqqqMLhsXH6fY+zmDKtto+ymD5ls4PKFo/ROopIG2GTQBcRlkvIT20eoqBcRVHt48YP6ytNP1lJk3eprHTCPhjF0ZTcQSIyX0S2+35eFpFb3HUfEJFhEdklIh/zbbNGRPa465bmCdAojrBcQn6SVPmKah83fivto04wfTphH4xyEfs9AVUdBi4AEJEKMAI8JCJvA5YB56vqqyJyhtvmTcAK4FxgFvAvIvIGVQ3/JDGaQlKjJUmVr7z95uknK51g+nTCPhjlIu2N4SuAvaq6D/hD4G5VfRVAVV9w2ywDvqCqr6rqD3FqDV9cVMBGdpIaLUmqfOXtN08/WekE06cT9sEoF2kngRXAevfxG4C3isgWEflXEXmLu3wQ8FdJP+AuM1pMWC4hP0mqfEW1jxu/lfZRJ5g+nbAPRrlIfGNYRKYDzwPnquqPReR7wGPAnwBvAe4Dzgb+O/AdVf28u92ngUdU9Us1/a0CVgHMmTPnon379hWzR0YkG7aNcOdXdtXZQS8dGUcA790w0Fflt84/M9IU8lcvS2qrBNlBQX36t4+yYYLsp8d3j4bGksSsiWvTajun1eMb5aGpdpCILANuVtUr3edfw7kc9IT7fC+wGHgfgKre5S7fBKxV1e+E9W12UGvZsG2E1ffvYPx49HshyELJa6vEbR+1Hog0l9LGkjce+yA2mk2zK4ut5MSlIIANwOVuIG8ApgM/ATYCK0RkhoicBcwD/j1PkEZjGdo0HDsBQLCFktdWids+an2cuZQ2lrzxGEY7kiiLqIj0Ab8J/L5v8WeAz7iXhY4CN6pzWrFLRL4IfB84hnP2YGZQiUljltS2zWurxG1fhA1TRNsi4zGMMpHoTEBVj6jqqap62LfsqKreoKpvVtWFqvqYb91HVfUcVZ2vqo82InCjONKYJbVt89oqcdtHrS/KSioqHsNoRyx3kOFYQz3R1hAEWyh5bZW47aPWx5lLaWPJG49htCNWVMaYvKG5duMuDo051pDfDoqyUNJUOosaO2z7JP2nsYOaEY9htBOWO8iIxK901lKb+C2pulirqfb3Vll7zbmhqmbSZHOmThrdRhF2kJ0JGKHEJY/zl1oEEpVhDEpid2hsnNX375jSNm2pSysDaRjZsHsCRihpFMyk6mRYErvx45o4cV0jVFXD6FbsTMAIpYjEcEmV0tp1aZPNmbppGNmwMwEjlDQKZlJ1MqrPNInrilZVDaNbsUnACCWNgplUnQxLYlftkcSJ6xqhqhpGt2KXgzqYNLZMWNut+15k/Zb9kzdm/fT3VhGBW+/bzqz+Xt550eCknumVobz1vu0MbRqe7M8b328HAfzCSdPqYhkbn5i0gsLsoNs37JyMT4CTp1c4cnQi1k4qs0VUG9/cU3vZ/MxLk8dh5SWz+cjy8wLblm1fjPJjimiHkibRWVjbd140yJeeHEmcvC3JuqiylXFj1sZ9+4adfH7zc3X7fsPiOZMfknmOSyuIM7I8blg8h0Wvn1nqfTEaj9UYNkJZcvdjoW7/t2+7PFFb77/voD6A0P6j1nljZxmzNu5z1jwS2LYiwt67rq5bHjVuUP+tICy+WioivPaUk0q9L0bjse8JGKGksWXC2gZ9wEa1T7OuiDHD2oYtjxq3LBZR0jgmVEu/L0Z7YDeGO5Q0tkxY24oE5xOKs4GSjJ1lzKRtw5ZHjVsWiyhpHBWR0u+L0R7YJNChpLFlwtquvGR2quRtaUyhLGPWsvKS2XXLopZHjVsWiyiJkQXOPpZ9X4z2wC4HdShpEp1FtV30+pmJk7fVrvMnpDup2lM3pt88EoGJ48cnb/T6S13OmNbD1n0v1iWK++qOg1P6FHecdZuf4/Hdow1JeOcnTa4kr90prlF16EhwKc2g+Pqm9/D0C69Mtpl3xsk8vnuUdZuf45TeKidVe0L7M4w47Maw0RCylGksmkaaMkkto7j9jIsxyXEyI6h7aUp5SRGZLyLbfT8vi8gtvvV/JiIqIqe5z0VEPiUie0TkKRFZmCdAoz3JUqaxaBqZOyhNrqSo/YyLsRElNA3DT+zlIFUdBi4AEJEKMAI85D6fjVN20i9rX4VTV3gecAlwj/vb6CKylmlsVhyN6jdNrqQkbYrI32QYUaS9MXwFsFdV97nP/xb4ECcu3wIsAz6nDpuBfhE5M3+oRjuRtUxjs+JoVL9pciUladOIEpqG4SftJLACWA8gItcAI6q6o6bNILDf9/yAu2wKIrJKRLaKyNbR0dGUYRhlJ0uZxqJppCmTJldS1H7GxdiIEpqG4SfxjWERmQ48D5wL/CfwOHClqh4WkWeBRar6ExF5GLhLVb/lbvdN4EOq+mRY33ZjuDMJqgzm/R7oq/Lq+ARHxo8DU20g7/m0HnBXA9AjcFydb8R6dpBnH/VVexARXjnqXD/v763yjgXx5THjYo/aNo8d9NKR8UTV0mBqfqSKCIvPHuDZn44Vni/I8hC1H01NGyEiy4CbVfVKETkP+CZwxF39OpwJ4mLgTuAJVfXOGIaBy1T1YEC3gE0CnUzRFlC1IgxdtyDWwKlWBNQpVuOR1KJpdH6hIvI6FW0DlT2nkhFMU+wgHytxLwWp6k5VPUNV56rqXJxLPgtV9UfARuDdriW0GDgcNQEYnU3RFtD4hCYycMYndMoEAMktmkZXKUvTf7Mqpllltu4l0ZfFRKQPxwL6/QTNHwGuBvbgnCnclDk6o+1phLWSxcBJ07bROXmKyOtU9HG1PETdS6IzAVU9oqqnqurhkPVzVfUn7mNV1ZtV9RxVPU9V7TpPF9MIayWLgZOmbaNz8hSR16no42p5iLoXyx1kNJSiLaBqRRIZONWKUO2ZmkguqUXT6Jw8ReR1KtoGsjxE3YuljTAykaVqmd8SGuir8rOfj0+xf/x4do/fAIIThtCAW7ns8Nj4ZC6hx3eP1o3hb+N9oPmrmnlWUq2pU9uuv7fK2mvOLcQQ8sdbu12tCbTyktmx+ZvykCavUdHjmYGUHysqY7SEIkySsKpgHp4FBPVVyoJIUpVsw7YRVj+wg/GJ6Pd80gpnefIHBbXLUiktD802gsxAKp5m20GGARRjkqzfsj9yvWcBJbWLxsYnWL9lf2y+orgJIGlfXn9Z8wcFtQs7JnHHKivNNoLMQConlkraSE0RJklU9a8s/UX1mSVfUZIKZ3nzB9Uuz1IpLQ/NNoLMQCondiZgpKYIkySq+pe/vyL6zJKvKEmFs7z5g2qXZ6mUlodmG0FmIJUTmwSM1BRhkkRV/4ITFlBSuyhJVbLVS+c73yQuoC+vv6z5g4LaZamUlodmG0FmIJUTuzFsZCKP5eG3hYKYMa2Ho8eOT8klVJt7yLOE4IS5A+FGj39M/7aeHSQC6uvvHQvO5OGnDk7pKygXEQRXKQuzgeLsJX+1tSx5goLsoqibykHmVlwuozyYHVQsZgcZbUecIRJlDUVZO1E2D9QbRmmrnKXJRRS2j0Ex1u5fFvvII6tdZNZO+2J2kNF2xBkiUSZMlLUTZfMUUeUsTS6isPGCYozqL61Nk9UuMmunuzE7yGgqcYZInAmT1qBJUrUrj52SJt9PWiMqrU2T1S4ya6e7sTMBo6nEGSJxJkxagybKMCqiylmafD9Jjai4ftL2HzeuWTvdjU0CRlOJM0SiTJgoayfK5imiylmaXERh4wXFGNVfWpsmq11k1k53Y5eDjKbi3WgMM0S8G5ie4eLhN1bCcunE5dgJWxcUU1Bun6g+ku6jP8b+ADvI31/csaql9tglsYOyjGN0FmYHGW2HXzP0Pkj9SeY8/Cpob7WHk6qVuuRoG7aN8KEHdnDUl05iyTkzWff+S6eMlUWh3LBthLUbd03GNtBX5Y7frk9Cl2f/gz6wsyT3y/Phb9pn62iKIioi84H7fIvOBv4Cp3j8bwNHgb3ATap6yN1mDfBeYAL4Y1XdFDWGTQJGUooqV+kpm+s2P0fQX8CSc2Zy/aI5oWPFKZQbto2w+v4ddUZRUHnMNMTpnM0uXWl6aWtpiiKqqsOqeoGqXgBchFMt7CHgG8CbVfV84D+ANW5QbwJW4BSkfzvwdyJSXEJ5o6spqlylp2yG/Qv07b0vRo4Vp1AObRqumwAguDxmGrLoro0sXWl6afuT9sbwFcBeVd2nql9X1WPu8s04xeYBlgFfUNVXVfWHOGUmLy4mXKPbKVJbzKpOJlmfdV0ccTpns0tXml7a/qSdBFbgFpuv4T3Ao+7jQcD/7ZQD7jLDyE2R2mJWdTLJ+qzr4siquzaqdKXppe1P4klARKYD1wD31yz/MHAMWOctCti87l8uEVklIltFZOvo6GjyiI2upqhylZ6yGTYNLDlnZuRYcQrl6qXz65RSCC6PmYYsumsjS1eaXtr+pFFErwK+q6o/9haIyI3AO4Ar9MQd5gOAX0x+HfB8bWeqei9wLzg3hlPGbXQwcbbJjGk9k9ehT55eoVrpCbSDwqiIsHDOKTy+ezTyprB3vbs2eV2/W4bx1vu2M7RpODRp3Lsunj2lPGYRdlCczplG9/SW+ZPuzZiW7uKA6aXtT2JFVES+AGxS1X9yn78d+ATw66o66mt3LvDPOPcBZgHfBOapaujdPLODDI8o2wTCE8EBgTZOUpIaNmExJClHWUbM7mlvmpZFVET6cK7zn62qh91le4AZwE/dZptV9Q/cdR/GuU9wDLhFVR+t7/UENgkYHkvufiwwxfSge4057bo0DPb38u3bLs8Ug3eWENZnWYna1zLHbTgUMQkkuhykqkeAU2uW/XJE+48CH80TmNGdZLFNijJRshg2HlkS2JUBs3sMyx1klIoo2yTLuixjZxknSTnKMmJ2j2GTgFEqomyTuHVBNk5Skho2aZLDtYMlY3aPYbmDjNIRZQeFlW308gj9fHyCsfHjk3357SF/7p+5p/ay+ZmXAi/jiMAvn34ye154ZdIe8spQDgYklwsqHekvVwkn8hgF5R1KkgvIn4MIppbObNTxzps3Kc4YspxD+bHykkZXk9VsyZt/KG1pyrTb164Ps56qPcLQ9dnzEEURtV9J8ibFvS5mJRWDlZc0upqseWvy5h9KW5oy7fa168O01/Hj+fIQRZE3b1Lc62I5h8qDTQJG25LVbCnCfMlbmjKpidSMfcnSbxZbK0/pTKNx2CRgtC1ZzZZmmER5t0/af6MsnkbkTcpTOtNoHDYJGG1LVrMlb/6htKUp025fuz7Meqr25MtDFEXevElxr4tZSeXBbgwbbU1Ww+T2DTvrSlj6qQhMxPxpeDmG/OZOX7UHEeGVo8HX02vzENV+0zioJGSUHQTBeXuiLKqocplA3Vi19PdWeceCM+uMKC+n0qEj47GlM739SvPamU1Uj9lBhpGBoqqTNYqklkyYYROUx6iWakVAmXLTuVoRJiaU46FbpaNI28dsomDMDjKMDBRVnaxRJLVkwgyb9Vv2x+7f+ITWWUfjBU4AXixF2T5mEzUOmwSMrqMdDJQkMYa1iauY1kyKzuvUqP67GZsEjK6jHQyUJDGmzWPUCoo61mYTNQ6bBIyuo6jqZI0iqSWTJo9RLdWK1FlH1YoU+oFQpO1jNlHjSFNZzDA6Bn91slpq8/6EMe+Mk/nJz45OVuXy8gvloSJSd63bX/nLnzPIX9XLs3TGxid4fPco77xosM7e8f8en1AGAgwemGoHecfCy33kxRBlB710ZHwylrUbd3HnV3Zx6Ej4GFEV1/xGUH9flRnTeiKNo6x0s3lkdpDRVRSVNwjqK4wVTbUiTBxXarNG1OYMymMJFZEHKK593T4F3ICuVoSh6xbUJc5rhhHUzuZRU+wgEZkvItt9Py+LyC0iMlNEviEiT7u/B9z2IiKfEpE9IvKUiCzME6BhFElReYOaYRiNT9RPAFCfMyiPJVREHqC49lNiDzGQxifq8yA1ywjqdvModhJQ1WFVvUBVLwAuAo4ADwG3Ad9U1Xk4dYRvcze5Cpjn/qwC7mlE4IaRhaLyBrXaSkmShyepJZQ3D1DSvtLG0SwjqNvNo7T3ga4A9qrqPmAZ8Fl3+WeB5e7jZcDn1GEz0C8iZxYSrWHkpKi8Qa22UpLk4UlqCeXNA5S0r7RxNMsI6nbzKO0ksAJY7z7+JVU9COD+PsNdPohTlN7jgLvMMFpOUXmDmmEYVStCUNqg2pxBeSyhIvIAxbWfEnuIgVSt1OdBapYR1O3mUWI7SESmA9cAa+KaBiyrOy8VkVU4l4uYM2dO0jAMIzcnVYPNoDDrxV9V621vPH2KrZOFMItoxrQeXj124or5eEDyor5qD3917fls3fcif/rFHZPxLT57gGd/OlZntyx6/czIHEJve+PpDG0a5pb7ttft58NPHawzn+Iqi/mNpecPjXGKL5+QN95Xdxysy030CzPqP4q8vvzHe8a0qVNIEVZPbcxmB4U1FFkG3KyqV7rPh4HLVPWge7nnCVWdLyJ/7z5eX9surG+zg4xmkKdalrf96gd2BH44N4tqj3DxWQN8e++LdetuWDxnSuK5ONKaUnkrmcWNF/QaRJk7UG9otYvVUxTNzh20khOXggA2Aje6j28Evuxb/m7XEloMHI6aAAyjWeSpluVt38oJABwzKGgCAFi/ZX/g8jDSGk55K5nFjRf0GkSZO91u9RRFostBItIH/Cbw+77FdwNfFJH3As8B17vLHwGuBvbgmEQ3FRatYeQgb5WustsiaXMGZdmfIu2fJG2ymDtlf53KRqJJQFWPAKfWLPspji1U21aBmwuJzjAKZFZ/LyMRHxBJqmlFbd9q0uYMyrI/ee2fuPGCDKGgbbx2UeuMZFjuIKNryFMty9u+WmltcrZqj7DknJmB61ZeMjtVX2kNp7yVzOLGC3oNosydbrd6isImAaNrWH7hIHe0iXmuAAARG0lEQVRdex6D7n+K3n/Og/29iW4mLr9wkKHrFjDQV61bd/L0+A9Tb/oY7O/lhsVzpvTTW+2ZfO7/h94/5fT3Vhm6fgHr3n8pNyyeMxl/RST1TWFvf8KOR2183th5brj6xxO3z4G+KkL4a1C7jb9d1DojOZY7yDAKojbZ2c/HJxgbP6F8eonSoL6EY7UHjh2PT0AXprEOBuifUSUda7cNausvwRlU9rIokmieZUzwVoaYrLykYZSEpLplpUfQ48VW8AqjVpdMo8jevmEnn9/8XF27LGccUSRJ3lbGBG9licnKSxpGSUiqW040aQKAel0yjSIbppum1VDjSKJ5llEFLWNMWbFJwDAKoKxaYpJEc0Hrw3TToktXJlFAy5jgrYwxZcUmAcMogLJqiUkSzQWtD9NNiy5dmSR5WxkTvJUxpqzYJGAYBZBUt6z0FFvCMYpaXTKNIhumm6bVUONIonmWUQUtY0xZsfKShpGSKCskqR205sGnpqxLilfusbYE5kBfld86/8w6Owhgyd2PTS5bOOcUNj/z0pTLOl5ZyFvv287QpmFWL50/efO30XZQkuRteRO8FWnx+Ps6pbfKSdWeKaUzW20sZcHsIMNIQRFWSNZEdJUep9xkEEmTr9VS7RGQqRlLW23eFEmRFk9ZjCA/ZgcZRpMpwgrJmogubAIIiyGJsTR+XOtiaVfLJYgiLZ5OMoL82CRgGCkowgpplEGSNPlalr7alSItnk4ygvzYJGAYKSjCCmmUQZK0PGOWvtqVIi2eTjKC/NgkYBgpKMIKyZqIrhJUazIihiTGUrVH6mJpV8sliCItnk4ygvyYHWQYKYgzVTZsG5lSDrG/t8q5s36R/7v3xdi8QGFlJz3i7gnc+ZVdU2KE8FKaHuPHFcFJgHfk6ESk5VJ0rpyw/rKOU7vd3FN760woZx7VKSYURJtHtTmhZkzr4fBYextBfswOMoyCKEX5yYowdN0CoL70YhxReYGKNmPC+nvnRYN86cmR1OOkLZXpUa0IqDMZBo1XRiPIT9PsIBHpF5EHRGS3iPxARC4VkQtEZLOIbBeRrSJysdtWRORTIrJHRJ4SkYV5AjSMdqEU5ScnNLT0YhxReYGKNmPC+lu/ZX+mcbLsLzjHa/x4uB3VqUaQn6SXgz4JfE1VrxOR6UAf8EXgTlV9VESuBj4GXAZcBcxzfy4B7nF/G0ZHUxZLJGscUXmBijZjwrYLi6HZpT+9/jrVCPITeyYgIq8Bfg34NICqHlXVQziXL1/jNjsFeN59vAz4nDpsBvpF5MzCIzeMklEWS2RWf2+mWKLyAhVtxoRtFxZDmrxHReD116lGkJ8kl4POBkaBfxKRbSLyjyJyMnALMCQi+4GPA2vc9oOA/7zygLtsCiKyyr2MtHV0dDTXThhGGShF+cmKhJZejCMqL1DRZkxYfysvmZ1pnCz7C87xqvaE21GdagT5ib0xLCKLgM3AElXdIiKfBF7G+e//X1X1SyLyu8AqVf0NEXkYuEtVv+Vu/03gQ6r6ZNgYdmPY6BTS2EFhVcGC8g5lJcg46q32sHBO/5SYZkzrobda4dDY+GRc3m8vt9BLR8brliXJm5PWAkpT/czff237uaf2Bh5375h4fcIJO+iUgP3yrw+ywfzrklR3K5KmVBYTkdcCm1V1rvv8rcBtwK8C/aqqIiLAYVV9jYj8PfCEqq532w8Dl6nqwbAxbBIwjHA2bBvhT+/fEamIJiHOeklKmnxDWe2apNtFtYNwQyptX2GTT9wxbLRJ1BQ7SFV/BOwXEe/85wrg+zj3AH7dXXY58LT7eCPwbtcSWowzOYROAIZhRDO0aTj3BADx1ktS0uQbymrXJN0uql2aSmpZYk1yDNvBJEpqB30AWOeaQc8ANwFfBj4pItOAnwOr3LaPAFcDe4AjblvDMDJSpIkSZ70U0XfcsiTjJ90uj72Tt6+kx7DsJlGiSUBVtwO1pxzfAi4KaKvAzflDMwwDHBNlpKAPEr/1UlSftX3XLgsaJ4ntk2S7uHZR+5i2r6QxJt2+LFjuIMMoOauXzo/MG5SUOOslKWnyDWW1a5JuF9UuTSW1LLEmOYbtYBJZ2gjDaAFJ8+PU2katpq/aw4xqZYopFGXuANy+YedkhTI/XqW1MFMIgq2d/r4qqkzm7/GMnCCbCGDtxl0cGjtx/Pp7q6y95txE+ZHe9sbTefipg1NsL/+2XWEHNQObBIxuIo35EpSLqAeoVKTlKSog3n6JM2iqFeFdb5kdmS8oro+onENZcxF5sQcd/2qPMHT9gu7KHWQYRnGkMV+CPuiPQykmAIi3X+IMmvEJjc0XFNdHVM6hrLmIvHGDjvP4cS298ZMGmwQMo8nkNV/KRlScSfYhLl9Qnj6y5iKKa9Mur00SbBIwjCaTNB9N2a0Sj6g4k+xDXL6gPH1kzUUU16ZdXpsk2CRgGE0mjfkSlIuoB1qeo8gjzn6JM2iqFYnNFxTXR1TOoay5iLxxg45ztUdKb/ykwSYBw2gyyy8c5K5rz2OwvxfByWETdKNy+YWDDF23gIG+6uSy/t4qn3jXBQxdt4BB979R77/dwf5elpwzM1UsYVPJQF+VGxbPqRv7hsVzYuMO29fa8Qb6qgxdt4CPLD8v8njUHq+BvqqTu8jXNqyPuL6jCDv+ZbkpXBRmBxlGm+FPmBZVktJTMKE+AZq3bOTQGCLgfQx4CuihI+P0uSUna/sf6KvyW+efyVd3HJxUL71ltarmQI3OmbeEZNZjFTXO7/3Dd/j23hcnny85Zybr3n9p4eM1Yp9NETWMLiNt4rdKj9DD1PKJQSUVm0VebTMNSVTc2gnAI8tEkDaZXRH7bIqoYXQZaRO/TRyvL58YVFKxWeTVNtOQRMUNmgCilmcdr8xlKpMmkDMMowR0gpqYR9tMQ7NLQ2YZrwyvp50JGEYb0QlqYh5tMw3NLg0ZNV6Zy1TaJGAYbUTaxG+VnvryiUElFZtFXm0zDUlU3DCbKq1lFTdemctU2uUgw2gjvJuIaeygrfteZN3m5ybbjU8oS86Zyfb9h3nlaLbCMrWcPL3C7ywcnLSDPPzxVURYOOcUHt89ytj4RGgCurhkcmkStZ1U7Zm8Fh+UOG7d+y+NtYOSWj3+1yasbe0+DG0a5tb7tjcl2VwYZgcZRgezYdsIH7xvO/mrFUfjJVWD8JKOQdQaMkGGTVA5yyz9ZLFxiuqnUf02zQ4SkX4ReUBEdovID0TkUnf5B0RkWER2icjHfO3XiMged93SPAEahpGdoU3DDZ8A4ERStbT2Uq0hE7R9UDnLLP1ksXEaZfWUyRZKejnok8DXVPU6t8Rkn4i8DVgGnK+qr4rIGQAi8iZgBXAuMAv4FxF5g6oWc95pGEZimmmfZB3Lv12eeJP0k7b/RhlGzTaXoog9ExCR1wC/BnwaQFWPquoh4A+Bu1X1VXf5C+4my4AvqOqrqvpDnFrDFzcieMMwommmfRJlwcRtF/S4Ef2k7b9RVk+ZbKEkl4POBkaBfxKRbSLyjyJyMvAG4K0iskVE/lVE3uK2HwT2+7Y/4C6bgoisEpGtIrJ1dHQ0524YhhHE6qXzm6IAeknV0tpLtYZM0PZB5Syz9JPFxmmU1VMmWyjJ+2MasBC4R1UvBF4BbnOXDwCLgdXAF0VECM5JVXdBT1XvVdVFqrro9NNPzxq/YRgRLL9wkE+86wJ6qyf+1HsEblg8hxsWz6lz9uPMUa+5v50wtdCKP2Fbf2+Vgb4Tyd7iEtAFJdcbun7BZMK8PP1kuZlbVD/N6jcLsXaQiLwW2Kyqc93nb8WZBCo4l4OecJfvxZkQ3gegqne5yzcBa1X1O2FjmB1kGO1Ho8wZIzlNsYNU9UfAfhHxzlOuAL4PbAAudwN5AzAd+AmwEVghIjNE5CxgHvDveYI0DKN8lMlwMbKT1A76ALDONYOeAW7CuSz0GRH5HnAUuFGd04pdIvJFnIniGHCzmUGG0XmUyXAxspNoElDV7UDQKccNIe0/Cnw0R1yGYZScWf29U74d7F9utA+WO8gwjEyUyXAxsmO5gwzDyESSXDlG+bFJwDCMzCy/cNA+9NscuxxkGIbRxdgkYBiG0cXYJGAYhtHF2CRgGIbRxdgkYBiG0cWUorKYiIwC+5o87Gk4aS7KSFljs7jSYXGlw+JKx2nAyaqaKwNnKSaBViAiW/MmXmoUZY3N4kqHxZUOiysdRcVll4MMwzC6GJsEDMMwuphungTubXUAEZQ1NosrHRZXOiyudBQSV9feEzAMwzC6+0zAMAyj6+noSUBEZovI4yLyAxHZJSJ/EtDmMhE5LCLb3Z+/aFJsz4rITnfMutqa4vApEdkjIk+JyMImxDTfdxy2i8jLInJLTZumHS8R+YyIvOAWLvKWzRSRb4jI0+7vgZBtb3TbPC0iNzYhriER2e2+Vg+JSH/ItpGvewPiWisiI77X6+qQbd8uIsPu++22JsR1ny+mZ0Vke8i2jTxegZ8PrX6PRcTVmPeYqnbsD3AmsNB9/IvAfwBvqmlzGfDVFsT2LHBaxPqrgUdx6ngvBrY0Ob4K8CPg9a06XsCvAQuB7/mWfQy4zX18G/DXAdvNxKmANxMYcB8PNDiuK4Fp7uO/DooryevegLjWAn+W4LXeC5yNUyZ2R+3fSdFx1az/G+AvWnC8Aj8fWv0ei4irIe+xjj4TUNWDqvpd9/F/Aj8A2iXv7TLgc+qwGegXkTObOP4VwF5VbfaX+CZR1X8DXqxZvAz4rPv4s8DygE2XAt9Q1RdV9SXgG8DbGxmXqn5dVY+5TzcDrytqvDxxJeRiYI+qPqOqR4Ev4BznhsclIgL8LrC+qPGSEvH50NL3WFhcjXqPdfQk4EdE5gIXAlsCVl8qIjtE5FERObdJISnwdRF5UkRWBawfBPb7nh+guRPYCsL/MFtxvDx+SVUPgvPHApwR0KbVx+49OGdxQcS97o3gj9xLCJ8JubTRyuP1VuDHqvp0yPqmHK+az4fSvMciPrcKe491RVEZEfkF4EvALar6cs3q7+Jc8viZe710AzCvCWEtUdXnReQM4Bsistv9j2ky7IBtmqJyich04BpgTcDqVh2vNLTy2H0YOAasC2kS97oXzT3Af8PZ//+Gc+nlPTVtWna8gJVEnwU0/HjVfj44JyfxmwUsK/SYhX1uFf0e6/gzARGp4hzIdar6YO16VX1ZVX/mPn4EqIrIaY2OS1Wfd3+/ADyEc0ru5wAw2/f8dcDzjY7L5Srgu6r649oVrTpePn7sXRZzf78Q0KYlx869OfgO4PfUvThbS4LXvVBU9ceqOqGqx4F/CBmvVcdrGnAtcF9Ym0Yfr5DPh5a/x8I+txrxHuvoScC93vhp4Aeq+omQNq912yEiF+Mck582OK6TReQXvcc4N3y+V9NsI/BucVgMHPZOUZtA6H9nrTheNWwEPBPjRuDLAW02AVeKyIB7+eNKd1nDEJG3A38OXKOqR0LaJHndi47Lfx/pd0LG+3/APBE5yz0LXIFznBvNbwC7VfVA0MpGH6+Iz4eWvsfC4mrYe6yIu9ll/QF+FecU7Slgu/tzNfAHwB+4bf4I2IVjRGwGfqUJcZ3tjrfDHfvD7nJ/XAL8DxxrYyewqEnHrA/nQ/0U37KWHC+cieggMI7zn9d7gVOBbwJPu79num0XAf/o2/Y9wB7356YmxLUH5xqx9z77n27bWcAjUa97g+P63+775ymcD7cza+Nyn1+NY6HsbUZc7vL/5b2vfG2bebzCPh9a+h6LiKsh7zH7xrBhGEYX09GXgwzDMIxobBIwDMPoYmwSMAzD6GJsEjAMw+hibBIwDMPoYmwSMAzD6GJsEjAMw+hibBIwDMPoYv4/LeYEdBg8JHMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\" Trend between LR model without regularization and FICO score in the data set\"\"\"\n",
    "\n",
    "plt.scatter(x=predictions, y=x_test.fico)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x19f4cbde518>"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAD8CAYAAACRkhiPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnX2UXWV56H/POTkJM6hMwoeGIRigGJfIZyKEprYKrQhVwgWpySpXrl9pe6m90DaWLF0CrV7Qse3VtW69pdVeuaUxiBijgNEqtleviQ0kAaNMSdCQTKKMhkRrRjKZPPePvfew58z+ePfHOWefOc9vrVk5593vfvez9znZ7+y9f/M8oqoYhmEYvUmt0wEYhmEYncMmAcMwjB7GJgHDMIwexiYBwzCMHsYmAcMwjB7GJgHDMIwexiYBwzCMHsYmAcMwjB7GJgHDMIweZlanAwA46aSTdOHChZ0OwzAMo6t49NFHf6KqJxcZoxKTwMKFC9myZUunwzAMw+gqRGR30THsdpBhGEYP4zQJiMgtIrJDRL4rImtF5DgRuVdEhv22T4lIw+8rIvJxEdkpIo+LyEWt3QXDMAwjL6mTgIgMAn8ELFHVVwN1YAVwL/BK4FygD3iXv8qVwNn+zyrgE+WHbRiGYZSB6+2gWUCfiMwC+oF9qvqQ+gDfAU7z+y4H7vEXbQIGRGR+6ZEbhmEYhUmdBFR1BPgo8AywHzikql8Jlvu3gf4z8GW/aRDYExpir99mGIZhVAyX20Fz8X67PwM4FTheRG4Idfkb4F9V9f8Gq0QMM61yjYisEpEtIrJldHQ0e+TA+q0jLLvr65xx64Msu+vrrN86kmscwzCMXsXldtBvAj9Q1VFVHQceAH4VQERuA04G/jjUfy+wIPT+NGBf86CqereqLlHVJSefnF1zXb91hDUPPMHIwTEUGDk4xpoHnrCJwDAMIwMuk8AzwFIR6RcRAS4Hvi8i7wKuAFaq6rFQ/w3A23xLaCne7aP9ZQc+tHGYsfGJKW1j4xMMbRwue1OGYRgzltQ/FlPVzSJyP/AYcBTYCtwN/ALYDXzbmxt4QFX/HHgIuArYCRwG3t6KwPcdHMvUbhiGYUzH6S+GVfU24DaXdX1b6KaCcaVy6kAfIxEn/FMH+lq9acMwjBlD1/7F8OorFtHXqE9p62vUWX3Fog5FZBiG0X1UIndQHq650LNOhzYOs+/gGKcO9LH6ikWT7eA9PE5a3uvY8TEMo2snAfAmgriTVmAPBQ+PA3soWK/XseNjGAZ08e2gNMweSsaOj2EYMIMnAbOHkrHjYxgGzOBJIM4SMnvIw46PYRgwgycBs4eSseNjGAZ0+YPhJFzsoV7Gjo9hGADi/W1XZ1myZIm2u7yk6ZGGYXQ7IvKoqi4pMsaMvRJIwvRIwzAMjxn7TCAJ0yMNwzA8enISMD3SMAzDoycnAdMjDcMwPHpyEjA90jAMw6MnHwxH6ZGvf+XJDG0c5pZ125xsoU7bRXm33+m4DcOoFj2riIZptoXAuzK489pzI0+QWft3Ot6i6xmGUU3KUESdbgeJyC0iskNEvisia0XkOBE5Q0Q2i8hTIrJORGb7fef473f6yxcWCbAdZLWFOm0X5d1+p+M2DKN6pE4CIjII/BGwRFVfDdSBFcCHgb9W1bOB54B3+qu8E3hOVX8F+Gu/X6XJagt12i7Ku/1Ox20YRvVwfTA8C+gTkVlAP7AfuAy431/+aeAa//Vy/z3+8sv9AvWVJast1Gm7KO/2Ox23YRjVI3USUNUR4KPAM3gn/0PAo8BBVT3qd9sLBDeVB4E9/rpH/f4nNo8rIqtEZIuIbBkdHS26H4XIagt12i7Ku/1Ox20YRvVItYNEZC7eb/dnAAeBzwJXRnQNnjBH/dY/7emzqt4N3A3eg2HHeFtCsy10Ql8DEbhl3TaGNg5PM2ia+w/0N1CN71+EJJsnq+WTtt76rSPcvmEHB8fGAZjb3+C2N59jD40NYwaTageJyPXAG1X1nf77twGXAtcDL1PVoyJyKXC7ql4hIhv919/2bx/9CDhZEzbUaTsoTJVMoXbaPOu3jrD6s9sZPzb1Y2rUhaG3nG8TgWFUkHbZQc8AS0Wk37+3fznwPeAR4C1+nxuBL/ivN/jv8Zd/PWkCqBpVMoXaafMMbRyeNgEAjE+o2UOGMYNxeSawGe8B72PAE/46dwN/BvyxiOzEu+f/SX+VTwIn+u1/DNzagrhbRpVMoXbaPEljmj1kGDMXp78YVtXbgNuamp8GLo7o+0u8W0VdyakDfYxEnPSSzJos/VsZSyu21artGYZRDXoyd1ASVTKF2mnzrL5iEY3a9Gf6jbqYPWQYM5iezB2URJJBE5g6IwfHqIswocrgQB/XLR7kkSdHM5k6YesnsIsOjY1PWT8plvevf4K1m/cwoYoAs2fVeP7oMQAG+hrcfnU2qyfom8UOsjxEhtH9WO4gR6JMnYCsxk7SWC7jvX/9E/zjpmcSt9GoCUPXt87qsTxEhtF52pY7yIg2dQKyGjtJY7mMt3bzntRtjB9rrdVjeYgMY2Zgk4AjefPy5O2b1GfC8eqtlVaP5SEyjJmBTQKO5M3Lk7dvUp+6YyqmVlo9lofIMGYGNgk4EmXqBGQ1dpLGchlv5SULUrfRqLXW6rE8RIYxMzA7yJGwqRO2gwYS8gyFDZ66CCsvWcAHrzk3NvdQsx0UxwevORegVDuoyPFolR1UBfuoCjEYRisxO6gASYbMlt0HIg2eG5aePnkSN+Kpgn1UhRgMIwmzgzpMkiETZ/C4mD1GNeyjKsRgGK3GbgcVIMmQibu+cjV7ep0q2EdViMEwWo1dCRQgyZCJM3hczZ5epwr2URViMIxWY5NAAZIMmTiDx8XsMaphH1UhBsNoNXY7qABJhkywLMoOykNariGAO764g+cOe3l/2mEIuZLHsGmVfZQllnYYUIbRacwO6gLScg01asIxYKK5KliL8we5UCXDpkqxGEYZtMUOEpFFIrIt9PMzEblZRC4QkU1+2xYRudjvLyLycRHZKSKPi8hFRQI00nMNjR/TaRNA0N5pk6VKhk2VYjGMqpB6O0hVh4ELAESkDowAnwf+DrhDVR8WkauAjwCvwytCf7b/cwnwCf9fIydFbJROmyxVMmyqFIthVIWsD4YvB3ap6m5AgZf47ScA+/zXy4F71GMTMCAi80uJtkcpYqN02mSpkmFTpVgMoypknQRWAGv91zcDQyKyB/gosMZvHwTCfxG1128zcpKWa6hRE+pRVcFanD/IhSoZNlWKxTCqgvMkICKzgauBz/pNfwDcoqoLgFt4odB8lAg/7Ya1iKzynyVsGR0dzRZ1j3HNhYPcee25DA70IXgVvwb6GggwONDH0PXn85fXn8/c/sbkOgN9jY4/FIbpsQ8O9HXsQWyVYjGMquBsB4nIcuAmVX2D//4QMKCqKiICHFLVl4jI3wLfUNW1fr9h4HWquj9ubLODogknoKsJzJlV45fjx2JVxW5JduYaZ5b9ybrv3XKsDCOJMuygLH8nsJIXbgWB9wzgN4BvAJcBT/ntG4A/FJHP4D0QPpQ0ARjRNJeQPKYwNu5lCR05OMaaB54AXnDZm/XHqD5VwDXOLPuTdd+75VgZRjtwuh0kIv3AbwEPhJrfDfyliGwH/juwym9/CHga2IlnEP3X0qLtIdISzTWrjd2iP7rGmWV/su57txwrw2gHTlcCqnoYOLGp7ZvA4oi+CtxUSnQ9jEuiubDa2C36o2ucWfYn6753y7EyjHZguYMqikuiubDa2C36o2ucWfYn6753y7EyjHZgk0BFSUs016w2dov+6Bpnlv3Juu/dcqwMox1YArmKEGWrAJF20AkJJS2jxrjwz79SmcRyrknZsiRvy5rozRLDGcYLWAK5CpAlsVnWvqvv3874RPUSyxmGURwrLzlDaJUJM7RxeNoEANVILGcYRjWwSaACtMqESbJdzIQxDANsEqgErTJhkmwXM2EMwwCbBCpBq0yY1VcsolGvZmI5wzCqgdlBFaBVJsw1Fw6yZfcB/mnzMwQ1Z/oaNe689rzUHDwjB8eoizChymAP2zOWY8iY6dgkUBHCdYnL6rt+6wife3SEqUXH4v8Irdk8Cv5quVdz61iOIaMXsNtBM5gycuq4rDdTsRxDRi9gk8AMpuzcOb1mFFmOIaMXsElgBlN27pxeM4osx5DRC9gkMIMpI6eOy3ozFcsxZPQCljZihtNst7z+lSfz4OP7p+QSetP583nkyVH2HRxjoL+BKhwcG5+0g+b6bYfGxifHCPpnNWaqYB+1smJZO6hiTEZnKCNthE0CPURcLqFmwrmIonIVJfVP237cWK5jFCVL7qUq0u3xG+XSltxBIrJIRLaFfn4mIjf7y94jIsMiskNEPhJaZ42I7PSXXVEkQKM84nIJNRM2YJKMoaj+advvtH3U7cZPt8dvVI/UvxNQ1WHgAgARqQMjwOdF5PXAcuA8VX1eRE7x+7wKWAGcA5wK/LOIvEJVk88kRsvJYrUEfV3XcelXBfuo242fbo/fqB5ZHwxfDuxS1d3AHwB3qerzAKr6rN9nOfAZVX1eVX+AV2v44rICNvKTxWoJ+rqu49KvCvZRtxs/3R6/UT2yTgIrgLX+61cArxWRzSLyLyLyGr99EAhXSd/rtxkdJi6XUDNhAybJGIrqn7b9TttH3W78dHv8RvVwThshIrOBq4E1oXXnAkuB1wD3iciZROclmHYjWkRWAasATj/99GxRG7kIHhze8cUd0+ygsDE0Nj7BHV/cwZbdB3jkyVHGxiemmDzNdtDrX3kyQxuHuWXdNue8R812UNIYaTZMFuMoLfeSi3nTSTvHqqIZZeNsB4nIcuAmVX2D//7LeLeDvuG/34U3IbwLQFXv9Ns3Arer6rfjxjY7qLOs3zrC6s9uZ/yYuzUUXreorZI0BpA4fpnGkcu+mJ1jVIl2VxZbyQu3ggDWA5f5gbwCmA38BNgArBCROSJyBnA28J0iQRqtZWjjcOoEANEWShm2StIYaeOXaRy57IvZOcZMw+l2kIj0A78F/F6o+VPAp0Tku8AR4Eb1Lit2iMh9wPeAo3hXD2YGVZg81lDaukXGdBnD1V4qOw6zc4yZhtOVgKoeVtUTVfVQqO2Iqt6gqq9W1YtU9euhZR9S1bNUdZGqPtyKwI3yyGMNpa1bZMxwe9r4ZRpHLvtido4x07DcQYZnDdWyWUPhdYvaKkljpI1fpnHksi9m5xgzDSsqY0w+0Lx9ww4OjnmG0Nz+Br993vzUHEFl2CouY8QtSzKOWhGH2TnGTMNyBxmJhPXLZppPtK7qpKvSmTXZnCVWM3qNMuwguxIwYklLHhcutwg4lWJ0LWGZtdSllYI0jHzYMwEjlizJ41zVSVelM6v6aeqmYeTDrgSMWMpIHueqlDYvz6p+mrppGPmwKwEjlizJ41zVSVelM6v6aeqmYeTDJgEjlizJ41zVSVelM6v6aeqmYeTDbgfNYMooo7hl9wHWbt4z+WA2zEBfAxEmk75dt3hwWpnKW9ZtY2jj8OR4zUpnQF2E6xYPTolvzqza5H3+msAxnW4kvX/9E5PxCXD87DqHj0w42UlVtYia41t4Yh+bnn6OCVXqIqy8ZAEfvObcyL5V2xej+pgiOkPJkugsru91iwf53KMjmRO7JS1zScTmsj54E8A/bnpm2r7fsPT0yZNkkePSCVzKeYK3j0tePq/S+2K0HqsxbMSy7K6vx7r937r1Mqe+gZsfNQYQO37SsmDbSfG5rA9w1pqHIuOri7Drzqumtadtt/m4dIK4+Jqpi/CyE46r9L4Yrcf+TsCIJYstE9c36gSb1D/LsiJJ49Lii2vPu9124hrHhGrl98XoDuzB8Awliy0T17cu0fmE0mygoonYXGOPiy+uPW27VcA1jrpI5ffF6A5sEpihZLFl4vquvGRBrsRuRROxuca+8pIFUbse25623SrgYmSBt49V3xejO7DbQTOULInOkvouefm8XIndYGpCuuMatWnbDJtHIjBx7Bg3r9sGQKMGIqDq/dZ70eknTCk/GZS4bKZRg3s3PcMjT462LOFdQNZcSfsOjnGCb1QdPDzunKCuf3aNp579xWSfs085nkeeHOXeTc9wQl+D4xq12PEMIw17MGy0hDQLx9WCKUIrTRlXyyhtP9NidDlOZgT1Lm0pLykii0RkW+jnZyJyc2j5n4qIishJ/nsRkY+LyE4ReVxELioSoNGdFCkLWRatzB1URq4klxiz5G8yjDyk3g5S1WHgAgARqQMjwOf99wvwyk6GZe0r8eoKnw1cAnzC/9foIdLMlXYZLK3ajquZ47L9vLZVnn6G0UzWB8OXA7tUdbf//q+B9wLhe0rLgXvUYxMwICLzi4dqdBNFy0K2Oo5WjZs1V1Janyz5mwwjD1kngRXAWgARuRoYUdXtTX0GgT2h93v9timIyCoR2SIiW0ZHpz/gM7qbImUhy6KVpkwZuZJcYsySv8kw8uD8YFhEZgP7gHOAnwOPAG9Q1UMi8kNgiar+REQeBO5U1W/6630NeK+qPho3tj0YnplEVQYL/p3b3+D58QkOjx8DQJh6OSnArBr4i4Gp+YMCOyg8drAcvLxGt199DpDPBHIxf4raQUF+pUNjyWZPOD9SXYSlZ87lhz8dKz1fkOUh6j7a/RfDVwKPqeqPReRc4Axgu3h/mHMa8JiIXIz3m39Y1D4Nb/IweozgBBJVIey5w+NT+jb/KqJMnQDAO8E36hJZhrLZoHn+6DG27D4wJfeRa7Ux1ypl4YR4SUT1c93G+q0jfO7RkcnjNqHKY88cKt0GsspsvUuW20Er8W8FqeoTqnqKqi5U1YV4J/6LVPVHwAbgbb4ltBQ4pKr7yw7c6A7KtoDGJ9S5qtjazXtyVRtrR5WyInZRK2wgq8zWuzhdCYhIP54F9HsO3R8CrgJ2AoeBt+eOzuh6WmGtuBo4eXIfJS0vc1+K2kVlH1fLQ9S7OF0JqOphVT1RVQ/FLF+oqj/xX6uq3qSqZ6nquapqN/t7mFZYK64GTlLuoyzju66XhaJ2UdnH1fIQ9S6WO8hoKWVbQMEzgbRtpOU+SqIdOXmK2EWtsIEsD1HvYmkjjFzkqVoWNnnm9jf4j1+OT3v4G9DfqHHt4tP40vb9k/mHwgZRf6PG7Fn1SbMmyhYKW0RBnK9/5ck8+Pj+yQfT/Y0aCoz5gcztb3Dbmz2r6I4v7pjsF9hGZRhCQT+Ybi6F8ykFVcTS8jcVIUteo7K3ZwZScayojNERyqjOFVcVLExQIWz91hFW37+d8Yn472paJbQgX1HaOOBdbbz1NQsSx4Ji+YPKqpRWhHZXWat6VbdupC25gwyjmTJMkrWb9zj3Gdo4nHridrGBXMYBz0ByMYvKNnzijonLscpDu40gM5CqiU0CRmbKMEmSqn8198lSbSsprjLiy1MdzbVfnkppRWi3EWQGUjWxScDITBkmSVL1r+Y+WaptJcVVRnyu1dGS3se156mUVoR2G0FmIFUTmwSMzJRhkiRV/2rus/qKRTTqySdCFxvIZRzwngm4mEVlGz55KqUVod1GkBlI1cQeDBu5KGJ5hG2hgLD505xHCDw7Z3ziGL84MjGtTzhPUJzRE95mOMdQsx0Uzk+08MQ+Nj39XGLOHojPTRRl3jx3eDzRXgpvU4D+2XUOH5lwPsbNeYZWXrIg8aFylLk12EJrx+ygcjE7yOg60gyRJGsoyQBKMoOAVCslT6WzJLMlarxGTUBItZxcY24mr11k1k73YpOA0XUsu+vrU64AAgYH+vjWrZdx1pqHEh+EBr+turYP+vebk7aZFFca4THC5B0vGBPSY24m7tjVRdh151Wx20v7TIzq0u4sooZRmDRDJM2EyWrQuFbtymuotMJ4yVtpLK9dZNZOb2MPho22kmaIpJkwWQ2aUwf6nKyUvIZKK4wX15ibyWsXmbXT29gkYLSVNEMkyYRJMoCSbB4XKyVPjqMksyVqvEZNnCwn15ibyWsXmbXT29jtIKOtBA8a4wyR4AFmYLgEhI2VuFw6aTl2kpZFxRWVdyj8PslsidvPtG1kibmZ5mPnYge5fCbGzMYeDBtdR1gzDEo0BknmwjSXm3zT+fOnnXAB3nv/do6EjJ1lZ83j3ndfOmVbeRTKVuiXZZa9zNq3SExGa2iLHSQii4B1oaYzgQ/gFY9/M3AE2AW8XVUP+uusAd4JTAB/pKobk7Zhk4DhSh6VM45GXWJ1zWVnzeP6JafHbitNoUyKM69+6aJyZtE9y1BDTS/tLG1JIKeqw6p6gapeACzGqxb2eeCrwKtV9Tzg34E1flCvAlbgFaR/I/A3IlJeQnmjpymzXGWSr/+tXQcSt5WW+KzIulnGzJvULmvfIjEZ1Sbrg+HLgV2qultVv6KqR/32TXgF5QGWA59R1edV9Qd4ZSYvLidco9dpp7aYtwxl0XWzrpMnqV3WvkViMqpN1klgBX6x+SbeATzsvx4Ewrlv9/pthlGYdmqLectQFl036zp5ktpl7VskJqPaOE8CIjIbuBr4bFP7+4CjwL1BU8Tq0667RWSViGwRkS2jo6PuERs9TZnlKpN0zWVnzUvcVppCWWTdLGPmTWqXtW+RmIxqk0URvRJ4TFV/HDSIyI3Am4DL9YUnzHuBsJh8GrCveTBVvRu4G7wHwxnjNmYwabbJnFm1yfvQx8+u06jXIu2gJJLWCx4KB/e7A7OnOfHb0MZhblm3bUqMzebSnFneNsqwg1xUziy6Z9AWTro3Z1a2mwOml3Y/zoqoiHwG2Kiq/+C/fyPwV8BvqOpoqN85wD/hPQc4FfgacLaqxj7NMzvICEiyTSA+qRrA6s9uZ/xYvt8nmstQJhkvccvTyltWEbN7upu2JZATkX68+/xnquohv20nMAf4qd9tk6r+vr/sfXjPCY4CN6vqw9NHfQGbBIyApGRmEJ9ULW5ZFoKEaWkJ1eKWJyWxq2oiNkse1920LYGcqh4GTmxq+5WE/h8CPlQkMKM3yWOblGWipJWhTFueJ4ldpzG7x7DcQUalSLJN8izLs+004yVuuUtJyqphdo9hk4BRKZJsk7RljVr+WrzNZSjTylRmTWJXVczuMSx3kFE5kuyg5mXhBGwD/Q1+OT4xWSoSYG5/g98+bz5f2r5/0gSKamvmpS+ezbM/PzKt5GVg9wDT8hcdGhvnhKYymAHhspXN9kyaDbV+68gUg6c5lqIPcKO2H96/oDTmwcPjzvZP2TmOjGisspjR0xQxW9ZvHcltE6WZQkXXb16++v7tsSkuWmHypO1TnrxJRXIcGfG0JXeQYVSVInlrhjYO59ZJw9vIk8sobf3m5Uk5jlqRpydtn/LkTSqS48hoLTYJGF1LEbOlqP2SZgoVXT/L+GWbPEW3WXaOI6O12CRgdC1FzJai9kuaKVR0/Szjl23yFN1m2TmOjNZik4DRtRQxW4rYRGmmUNH1m5cn5ThqhcmTtk958iYVyXFktBYrL2l0LUXz1rzouFmTxk0zL5lT5+fPT0zPfIh37/rmddtY88DjXLf4tCm5jPobNURkmh0UIP76f3Lfdm5et23aXxnXRbhu8eDkPkTl9wnbQS45jOLKWMYduy27D/D80fhnAnNm1diy+8C0qmnh/EjXLR5MLJuZ57Mzm6g1mB1k9BxlVidrBVkMJ9ccRs006gLKlIfjfY06F51+At/adaBt++CK2UTRmB1kGDkoszpZK8hiOEUZNms370ndv/EJnWZHjY1PlDIBBGOVafqYTdQ6bBIweo5uMFCKGDpxOYzaTZnH2Wyi1mGTgNFzdIOBUsTQicth1G7KPM5mE7UOmwSMnqPM6mStIIvh5JrDqJlGXabZUX2NOsvOmpc94AjKNn3MJmodZgcZPUnY6Gkmri5AM40azJ5VnzSBAmtHBPLekamLTLnXHZg+YTtooK/B7VefE1sZbMnL57Hk5fNi7Z26COMTytxQzqOwbfP+9U+wdvMeJlSpCdQFQumYGOhr8Kbz5/PIk6OR4wfj3rJuG7dv2DEt7xDA7Rt2TMnldNubz4l8wBtVqa053jLoZfPI7CCjpyhqBiVVOSubwPRZ9297pqWOaNSEoevPj4wjS4xl5AFK6z8l7rowMaEci2gfesv50xLntcMI6mbzqC0J5ERkEbAu1HQm8AHgHr99IfBD4HdU9TkREeBjwFXAYeC/qOpjSduwScBoF3GVtLJQViUzF5KuSsqqtpZURSxr5bEix7d5zHZVPevm6mptUURVdVhVL1DVC4DFeCf2zwO3Al9T1bPx6gjf6q9yJXC2/7MK+ESRAA2jTMqwSfYdHGublZJ0WyopjiwxFs0D5DpW1jjaZQT1unmU9cHw5cAuVd0NLAc+7bd/GrjGf70cuEc9NgEDIjK/lGgNoyBlVSBrl5WSZPqUVW2taB4g17GyxtEuI6jXzaOsk8AKYK3/+qWquh/A//cUv30Qryh9wF6/zTA6TlEzKKnKWdkEpk9U7qBGTZyqraXFWEYeoLT+U+KuS+RJp1GXaWO2ywjqdfPI2Q4SkdnA1cCatK4RbdOuaUVkFd7tIk4//XTXMAyjMMc1os2gNOslnKunFc8D5syq8fzRFx6Zjo1PcN+/7eFo00PhIM6wYZNUucwlh9AFd3xlcqz+Ro05jTrPHR6PPAZJ5kxzTqDmqmSvf+XJkRXdXjRn+qkozn4KU4bVUzQHVbfjbAeJyHLgJlV9g/9+GHidqu73b/d8Q1UXicjf+q/XNveLG9seDBvtIMlccbFBqpJzaNlZ8/jOD56blvYhyrBJI0uFtaLGTJ6KZUnmDsSbUT1zAm9z7qCVvHArCGADcKP/+kbgC6H2t4nHUuBQ0gRgGO0iKWeQSx6aquQc+tauA5En7fEJzZxLJ0uFtaK5evJULEvKGWT5hMrB6XaQiPQDvwX8Xqj5LuA+EXkn8Axwvd/+EJ4euhPPJHp7adEaRgHSbI+iy6tA1hhb3T/rumUYQt3wOVUJp0lAVQ8DJza1/RTPFmruq8BNpURnGCVy6kBf4r38NBskbf0qkNVoybpPRe2ftG1FGUJR6wT9kpYZbljuIKNnSDJXXGyQquQcWnbWvMiqaFGGTRpZKqwVNWbyVCwziqqWAAARAUlEQVTLaj/1ktVTFpY7yOgZwhZI2P5JM17i1g/TbPZE0deocVyjzsHD4wz0N3h+fILDflKeYNlzh8en5B6aXRfGjymq3t8MrLxkAR+85lzWbx1xzr/jsk/hsaLsINdj5LKtOHMoanwXc6dXrZ6ysNxBhlESzcnOfjk+wVgo81pwoobpE0mQfC5MVFswjiocHJt+kg7GjjsphmN0OQmHk8mFJ6EycdE8q5jgrQoxtSV3UDuwScDodlz10XrN+2MpVyMnC3ElIwNlMqui+f71T/CPm56Z1u+GpaeXNhG4JG+rYoK3qsRk5SUNoyK46qMTx6aXdSyLuJKRgTKZVdFcu3lPZL+49jy4aJ5VVEGrGFNebBIwjBKospYYxJZV0YxLXldm+UoXBbSKCd6qGFNebBIwjBKospYYxJa1ZGVc8royy1e6JG+rYoK3KsaUF5sEDKMEXPXRem16WceyiCsZGTwwzqporrxkQWS/uPY8uGieVVRBqxhTXkwRNYyMJFkhrnZQWMnMQ3MJy7oIb33NgsmykuGEcUMbh7ll3TZOHejjusWDPPj4/smEbME4A74pdMu6bQxtHGb1FYsmH/620g5yUUCLJngr0+JptquOa9QS7apuwOwgw8hAGVbI+q0jrL5/+7SSkWnUa8JEwkNlF6smyiBq1ASEKfF02r4pizItnqoYQWHMDjKMNlOGFTK0cTjzBAAkTgBRcUTFGmUQjR/TafF0q+nSTJkWz0wygsLYJGAYGSjDCmmlQeJi1eQZq1sp0+KZSUZQGJsEDCMDZVghrTRIXKyaPGN1K2VaPDPJCApjk4BhZKAMK2T1FYsiS0amUU+xilysmiiDqFGTafF0q+nSTJkWz0wygsKYHWQYGUgzVQJ7JJygrq9R45fjxyLzAIWJyxUUkPZMoHmOuObCQbbsPsC9m56ZHHd8QmnUvIR1gbk0fsyL8UVzZiWaLmXnyokbL+92mtdbeGIfm55+bsoft3nHSKdYUOCeb2mgv8GcWTUOjXW3ERTG7CDDKIkqlJ8Ml5hcv3WEP/ns9tTJY3LdmjB0fXR5yrLNmLjxrls8yOceHcm8nbzHPk++pU4bQWHaZgeJyICI3C8iT4rI90XkUhG5QEQ2icg2EdkiIhf7fUVEPi4iO0XkcRG5qEiAhtEtVKH8ZLjE5NDGYecJALwTYZzpUrYZEzfe2s17cm0n77HPk29pJhhBYVxvB30M+LKqvkVEZgP9wH3AHar6sIhcBXwEeB1wJXC2/3MJ8An/X8OY0VTFEsmSKyhu3aLtebcTl5eo3aU/045hVT7rMki9EhCRlwC/DnwSQFWPqOpBvNuXL/G7nQDs818vB+5Rj03AgIjMLz1yw6gYVbFEsuQKilu3aHve7cTlJXIp/VkmacewKp91GbjcDjoTGAX+QUS2isjfi8jxwM3AkIjsAT4KrPH7DwLhXLN7/bYpiMgq/zbSltHR0UI7YRhVoArlJ8MlJldfsSjVKJqybi2+PGXZZkzceCsvWZBrO3mPfZ58SzPBCAqT+mBYRJYAm4BlqrpZRD4G/Azvt/9/UdXPicjvAKtU9TdF5EHgTlX9pr/+14D3quqjcduwB8PGTCGLHVQTOKYw6Of4eeTJ0di8Q3lp1KB5mL5GjesWn8YDj+6dLG8pfvvh8WOTcUdVMAv+HXCoStZ8TFwtoKhjmFbeMqrc5qvmv5j/t+tAbMW2qGpsUdXWwsvjbLBwrqbgc2yHPdSWymIi8jJgk6ou9N+/FrgV+DVgQFVVRAQ4pKovEZG/Bb6hqmv9/sPA61R1f9w2bBIwjHjy5hqKIjBbgNwmU5ZcQ3ntmizrZTGNksbKGquLkdRqk6gtdpCq/gjYIyLB9c/lwPfwngH8ht92GfCU/3oD8DbfElqKNznETgCGYSSTN9dQFIHZUsRkypJrKK9dk2W9LKZR0lhZY3U5ht1gErnaQe8B7vXNoKeBtwNfAD4mIrOAXwKr/L4PAVcBO4HDfl/DMHLSKvOlbKLGzWvXZFkvq2kUt16rzKiqm0ROk4CqbgOaLzm+CSyO6KvATcVDMwwDPBNlpMQTSWC2lDlmeNzmtqjtuNg+ruvF9Q2eJ7jGmzVW18+l6iaR5Q4yjIqTN9dQFIHZUsRkypJrKK9dk2W9LKZR0lhZY3U5ht1gElnaCMPoAK75cdZvHeGOL+6YrARWFjWgHmEOhWnU4OixqfmM+hs15jTqPHd43Nncef/6JyarkwU0rwvTDZygrdkSijJwgGl2UFwVt4G+BrdffU7s8Y6yfeJMpZ6wg9qBTQJGL+FqocRZQTWgXpfSHhYXIc1+cTFokvL3QLrFlJRzKG8uorTYq5I/yCqLGUYX4mqhxFlBx6ASEwCk2y8uBk1S/h5XAycu51DeXERpsXeD9eOKpZI2jDbjaptU3SoJSIqzyD5kWTfuAXDeXEQufbrl80nDrgQMo8245qOpulUSkBRnkX04daDPef24nEN5cxG59OmWzycNmwQMo824WihxVlANSrOFipJmv7gYNEn5e1wNnLicQ3lzEaXF3g3Wjyt2O8gw2kxadbLmfmE7KDBbwuuH8930z67ziyPpfwlcA47zcwVFMbe/wW+fN58HH98/ZdtvOn9+JvslvK9hw8bFDgqP62LgLHn5vMgx4trTiIs9zYbqNswOMowuI5xgTQTi/gsHmmSSytg8RqCABhPK4SMTUxTRZlXzS9v3T+qXx8+u06jXpiScS1I6yyxVmXaskrbzu3/3bb6168Dk+2VnzePed19a+vbKLs8JpogaRs+RtYxioy689TULYhOpdYIsCeiK4KLiNk8AAXkmgqTtwXTVtYx9NkXUMHqMrInfxic0MZFaJ8iSgK4ILipu1ASQ1J53e1UuU2nPBAyji8ijJaYlUqsK7UqU184Eemnbq4JmalcChtFF5NES4zTJqtGqEpGt3o7L9qpcptImAcPoIrImfmvUJTGRWifIkoCuCC4q7rKz5kWuG9eed3tVLlNpt4MMo4to1hZd7aDNT/+Up579xeSyl754NkcmtNTEdGE7KKAuwpkn9/P06OFJNfTiM+ayY9/PpyV7S7No8pSdPK5Rm7wXH5U47t53X5pqB7laPS7qb7OhNbRxmFvWbWtLsrk4zA4yjBlOnAFTJo2aMHT9+UD2spVhS6ZTpSLjKGucVo3bNjtIRAZE5H4ReVJEvi8il/rt7xGRYRHZISIfCfVfIyI7/WVXFAnQMIxitHoCAM/4yVu2MmzJdKpUZBytsnqqZAu53g76GPBlVX2LX2KyX0ReDywHzlPV50XkFAAReRWwAjgHOBX4ZxF5hapWx1EzDKN0ykgW16lSka7j5h2nXePmIfVKQEReAvw68EkAVT2iqgeBPwDuUtXn/fZn/VWWA59R1edV9Qd4tYYvbkXwhmFUhywJ36LWDf/bTJrh5Jp8L2t8rbJ6qmQLudwOOhMYBf5BRLaKyN+LyPHAK4DXishmEfkXEXmN338Q2BNaf6/fNgURWSUiW0Rky+joaMHdMAwjjjymS1YaNcldtjJsyXSqVGQcrbJ6qmQLuUwCs4CLgE+o6oXAL4Bb/fa5wFJgNXCfiAgQNWVPu5ZT1btVdYmqLjn55JPzxm8YRgr3vvvSaRPBsrPm8T/eegEDfY0p7VH/eYMEn7WYX8aFF54JANx57bkMDvQhwOBAHzcsPT3xffhh6DUXDk5b/85rz+WD15w72Q4vXBk0rx8QN07Wh7lljdOucfOQageJyMuATaq60H//WrxJoI53O+gbfvsuvAnhXQCqeqffvhG4XVW/HbcNs4MMo/tolTljuNMWO0hVfwTsEZHgOuVy4HvAeuAyP5BXALOBnwAbgBUiMkdEzgDOBr5TJEjDMKpHlQwXIz+udtB7gHt9M+hp4O14t4U+JSLfBY4AN6p3WbFDRO7DmyiOAjeZGWQYM48qGS5GfpwmAVXdBkRdctwQ0/9DwIcKxGUYRsU5daCPkYgTfhXy4RjuWO4gwzByUSXDxciP5Q4yDCMXrmUyjWpjk4BhGLm55sJBO+l3OXY7yDAMo4exScAwDKOHsUnAMAyjh7FJwDAMo4exScAwDKOHqURlMREZBXa3ebMn4aW5qBpVjQuqG5vFlQ2LKxtVjut4VS2UgbMSk0AnEJEtRRMvtYKqxgXVjc3iyobFlY2ZHpfdDjIMw+hhbBIwDMPoYXp5Eri70wHEUNW4oLqxWVzZsLiyMaPj6tlnAoZhGEZvXwkYhmH0PDN6EhCRBSLyiIh8X0R2iMh/i+jzOhE5JCLb/J8PtCm2H4rIE/42p9XWFI+Pi8hOEXlcRC5qQ0yLQsdhm4j8TERuburTtuMlIp8SkWf9wkVB2zwR+aqIPOX/Ozdm3Rv9Pk+JyI1tiGtIRJ70P6vPi8hAzLqJn3sL4rpdREZCn9dVMeu+UUSG/e/brW2Ia10oph+KyLaYdVt5vCLPD53+jiXE1ZrvmKrO2B9gPnCR//rFwL8Dr2rq8zrgSx2I7YfASQnLrwIexqvjvRTY3Ob46sCPgJd36ngBvw5cBHw31PYR4Fb/9a3AhyPWm4dXAW8eMNd/PbfFcb0BmOW//nBUXC6fewviuh34U4fPehdwJl6Z2O3N/0/Kjqtp+V8CH+jA8Yo8P3T6O5YQV0u+YzP6SkBV96vqY/7rnwPfB7ol7+1y4B712AQMiMj8Nm7/cmCXqrb7j/gmUdV/BQ40NS8HPu2//jRwTcSqVwBfVdUDqvoc8FXgja2MS1W/oqpH/bebgNPK2l6RuBy5GNipqk+r6hHgM3jHueVxiYgAvwOsLWt7riScHzr6HYuLq1XfsRk9CYQRkYXAhcDmiMWXish2EXlYRM5pU0gKfEVEHhWRVRHLB4E9ofd7ae8EtoL4/5idOF4BL1XV/eD9ZwFOiejT6WP3DryruCjSPvdW8If+LYRPxdza6OTxei3wY1V9KmZ5W45X0/mhMt+xhPNWad+xnigqIyIvAj4H3KyqP2ta/BjeLY//8O+XrgfObkNYy1R1n4icAnxVRJ70f2OaDDtinbaoXCIyG7gaWBOxuFPHKwudPHbvA44C98Z0Sfvcy+YTwF/g7f9f4N16eUdTn44dL2AlyVcBLT9ezecH7+IkfbWItlKPWdx5q+zv2Iy/EhCRBt6BvFdVH2herqo/U9X/8F8/BDRE5KRWx6Wq+/x/nwU+j3dJHmYvsCD0/jRgX6vj8rkSeExVf9y8oFPHK8SPg9ti/r/PRvTpyLHzHw6+Cfhd9W/ONuPwuZeKqv5YVSdU9RjwdzHb69TxmgVcC6yL69Pq4xVzfuj4dyzuvNWK79iMngT8+42fBL6vqn8V0+dlfj9E5GK8Y/LTFsd1vIi8OHiN98Dnu03dNgBvE4+lwKHgErUNxP521onj1cQGIDAxbgS+ENFnI/AGEZnr3/54g9/WMkTkjcCfAVer6uGYPi6fe9lxhZ8j/aeY7f0bcLaInOFfBa7AO86t5jeBJ1V1b9TCVh+vhPNDR79jcXG17DtWxtPsqv4Av4Z3ifY4sM3/uQr4feD3/T5/COzAMyI2Ab/ahrjO9Le33d/2+/z2cFwC/E88a+MJYEmbjlk/3kn9hFBbR44X3kS0HxjH+83rncCJwNeAp/x/5/l9lwB/H1r3HcBO/+ftbYhrJ9494uB79r/8vqcCDyV97i2O6//435/H8U5u85vj8t9fhWeh7GpHXH77/w6+V6G+7TxeceeHjn7HEuJqyXfM/mLYMAyjh5nRt4MMwzCMZGwSMAzD6GFsEjAMw+hhbBIwDMPoYWwSMAzD6GFsEjAMw+hhbBIwDMPoYWwSMAzD6GH+P1W4IyhTc39EAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\" Trend between LR model with Ridge regularization and FICO score in the data set\"\"\"\n",
    "\n",
    "plt.scatter(x=predictionsOnTestData, y=x_test.fico)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x19f4d332048>"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJztnX2cXXV54L/PvbmBGapMgtiSIbwWw0dEkcRAm2oFFJRaQkULaV1Z1KYv1BbcZktWPhX2oxWNrau7W1uqdnVlY3gdo6LRCrZbtwkNJiFGSUnUvAxRR8PE1gzkZvLsH+ecyZlzz/s9595z732+n898Mvec38tzzkx+c+d3vvM8oqoYhmEY/Uut2wEYhmEY5WILvWEYRp9jC71hGEafYwu9YRhGn2MLvWEYRp9jC71hGEafYwu9YRhGn2MLvWEYRp9jC71hGEafM6fbAQC84AUv0LPOOqvbYRiGYfQUjz/++I9V9dSkdpVY6M866yw2b97c7TAMwzB6ChHZk6adbd0YhmH0OakWehG5VUR2iMi3RGStiJwoIveIyE732CdFpOG2FRH5qIjsEpEnROTici/BMAzDiCNxoReRUeCPgCWq+hKgDtwA3AOcD1wIDAHvcLu8HjjP/VgJfKz4sA3DMIy0pN26mQMMicgcYBh4WlUfVhfgMeB0t+1y4NPuqY3AiIicVnjkhmEYRioSF3pVHQc+BOwFDgCHVPUr3nl3y+Y/AF92D40C+3xD7HePGYZhGF0gzdbNPJx36WcDC4CTROQtviZ/Bfyjqv5fr0vIMC3VTURkpYhsFpHNExMT2SMHxraMs+yuRzj7ti+y7K5HGNsynmscwzCMfibN1s1rgO+p6oSqNoEHgV8GEJH3AKcC7/K13w8s9L0+HXg6OKiq3q2qS1R1yamnJmqgLYxtGWf1g9sZn5xCgfHJKVY/uN0We8MwjABpFvq9wKUiMiwiAlwBfEdE3gFcBaxQ1WO+9uuBt7r2zaU4Wz0Hig58zYadTDWnZx2bak6zZsPOoqcyDMPoaRL/YEpVN4nI/cA3gaPAFuBu4GfAHuCfnfWfB1X1vwIPA1cDu4DDwE1lBP705FSm44ZhGINKqr+MVdX3AO9J09e1cG5uM65EFowMMR6yqC8YGSp7asMwjJ6iZ/8ydtVVixhq1GcdG2rUWXXVoi5FZBiGUU0qkesmD9e+3DE212zYydOTUywYGeKy809lzYad3LpuKwtGhlh11aKZdsZsxraMz7p3dq8Mo3/p2YUenMXeW5w8C8d7QOtZOF474zh2rwxjsOjZrZsgZuGkx+6VYQwWfbPQm4WTHrtXhjFY9M1CH2XbmIXTit0rwxgs+mahNwsnPXavDGOw6OmHsX7CLBwzScKxe2UYg4U4f9/UXZYsWaKdKCVoSqFhGP2EiDyuqkuS2vXNO/okTCk0DGNQ6Zs9+iRMKTQMY1AZmIXelELDMAaVgVnoTSk0DGNQGZiF3pRCwzAGlYF5GBulFAIsu+uR1CZON8ydIuc088gwBo+B0iuDBE0ccN7lv/+NF4YuflnbdyPGTo1lGEb3SatXptq6EZFbRWSHiHxLRNaKyIkicraIbBKRp0RknYjMddue4L7e5Z4/q71LKY+sJk43zJ0i5zTzyDAGk8SFXkRGgT8ClqjqS4A6cAPwAeDDqnoe8AzwdrfL24FnVPUXgQ+77SpJVhOnG+ZOkXOaeWQYg0nah7FzgCERmQMMAweAy4H73fOfAq51P1/uvsY9f4VbVLxyZDVxumHuFDmnmUeGMZgkLvSqOg58CNiLs8AfAh4HJlX1qNtsP+Bt8o4C+9y+R932pwTHFZGVIrJZRDZPTEy0ex25yGridMPcKXJOM48MYzBJtG5EZB7Ou/SzgUngPuD1IU29p7ph795bnviq6t3A3eA8jE0Zb6H4TZzxySnqIrP2rIMPKIPmzshwA1W4dd1W1mzYWajB4rdjTh5qcGKjxuThZlumTFwys7Et49z5+R08c7gJwMhQgzuuucAe0hpGH5BGr3wN8D1VnQAQkQeBXwZGRGSO+679dOBpt/1+YCGw393qORk4WHjkBeEtZGnz4HjlC8vMnRMce3KqyVCjzoevv6jtsf3lF/3zrbp/G83p4z9vJ6earLpv20wfwzB6lzR79HuBS0Vk2N1rvwL4NvAo8Ca3zY3A59zP17uvcc8/olVwOGPIY6OUabB02o5Zs2HnrEXeo3lMzcgxjD4gzR79JpyHqt8Etrt97gb+FHiXiOzC2YP/hNvlE8Ap7vF3AbeVEHeh5LFRyjRYOm3H5LlOwzB6h1R/Gauq7wHeEzj8XWBpSNtngTe3H1rnWDAyxHjIghZno+TpU2Y8ZcxX5pyGYXSOgcl1E0ceG6VMg6XTdsyqqxbRqLc+Q2/UxIwcw+gDBibXTRxZ8uD42+WxYcJyzYTN/f43Xhhqx9w+tp21m/YxrYoAw3Pr/OzINHURplUZzWHleG3zWDeWO8cwqs9A57qJIywvTKMmIMx6cJklV0zomHUBdR58Jo15+9h2PrNxb+I8ncpfY7lzDKO7FJrrZhAJM1+ax7TFTsliw4SOOa2zFvm4Mddu2pdqnk7lr7HcOYbRG9hCH0EW2yRt23bHnM7w21cnbBnLnWMYvYEt9BFksU3Stm13zHqGlEGdsGUsd45h9Aa20EcQZr40atJip2SxYULHrIuz959izBWXLEw1T6fy11juHMPoDcy6iSDKxNm85+CM9VIX4brFs1MKBC2Uy84/lUefnJh5fd3i0Vmvo6ybsIeZ7732QoDCrZui71ERc1fJ5qlSLIaRB7NuMpBkmYSdD2JWSjJVsnmqFIthBDHrpgSSLJOw80HMSkmmSjZPlWIxjLzYQp+BJMukDPtmEKmSzVOlWAwjL7bQZyDJMinDvhlEqmTzVCkWw8iLLfQZSLJMws4HMSslmSrZPFWKxTDyYtZNBpIsk7DzQesmrbERZXok5crxql4dmmqvGlVR5DFWyrR5ssZUdiyG0QnMuqkgUabHdYtHeeDx8cT8O366aYhU0VipYkyGkZfCrBsRWSQiW30fPxWRW0TkIhHZ6B7bLCJL3fYiIh8VkV0i8oSIXFzEBQ0SUabH2k37UuXfCfbrliFSRWOlijEZRtkkbt2o6k7gIgARqQPjwEPA3wJ3quqXRORq4IPAq3EKh5/nflwCfMz910hJlNGRJddNmvHKporGShVjMoyyyfow9gpgt6ruARR4vnv8ZI4XB18OfFodNuIUET+tkGgHhCijI0uumzTjlU0VjZUqxmQYZZN1ob8BWOt+fguwRkT2AR8CVrvHRwF/Pt397jEjJVGmx4pLFqbKvxPs1y1DpIrGShVjMoyySW3diMhc4BqOL+i/D9yqqg+IyG/iFAV/DRC26rTsOYjISmAlwBlnnJEx7P4mzvRYcub8nrFuqmisVDEmwyib1NaNiCwHblbVK93Xh4ARVVUREeCQqj5fRP4G+LqqrnXb7QReraoHosY26+Y4nvo3PjlFTcCrSRJX2q/Xkm4lxZvnetL26bV7ZRhxpLVusnj0Kzi+bQPOnvyvAl8HLgeeco+vB/5QRD6L8xD2UNwibxwnqP75C09NTjVZdd82gJZF0d9nfHKK1Q9ub2lXFZLizXM9afv02r0yjKJItUcvIsPAa4EHfYd/B/gLEdkG/DnuNgzwMPBdYBeOmfMHhUXb5yQlRWse0xYNsNd0wTyJ4ZKuJ22fXrtXhlEUqd7Rq+ph4JTAsX8CFoe0VeDmQqIbMNIofsE2vaYL5k0MF3c9afv02r0yjKKwXDcVIo3iF2zTa7pg3sRwcdeTtk+v3SvDKApb6CtEUlK0Rk1aNMBe0wXzJIZLup60fXrtXhlGUVhSsy4RZ3+ksW78/U8eanBio8bk4ValcmzLOHd+fgfPHG6GjtNp8iSGSzJj0vYxtdIYVCypWRdoN7FW2v5jW8ZZdf+2llw4jZqw5s0vswXOMHocKyVYYdq1P7JYJmEJz8LsHcMw+hdb6LtAu/ZHu5ZJlrkMw+h9bKHvAu3aH+1aJlnmMgyj97GFvgu0a39ksUzCEp6F2TuGYfQvZt10gXbtjyz9T5o7h8mp5szrMOsmaADlLX/Yj1huHKMfMOumT8li5gTbBRnUUntWdtCoOmbdDDjt5H8JMqj5YCw3jtEv2ELfpxRh5uRp109YbhyjX7CFvk8pwszJ066fsNw4Rr9gC32f0k7+lyCDmg/GcuMY/YI9jO1jgvlwROCZw03qIkyrzvw7L1B+0G/deKUJJ6eO9xttwz65fWw7azftm5l/xSULee+1F5Zw9a2UWbmqU1QtHqO7pH0Yawv9AJDXrInrl8c+uX1sO5/ZuLfl+FsuPaP0xb4fDJp+uAajWAqzbkRkkYhs9X38VERucc+9U0R2isgOEfmgr89qEdnlnruqvUsx2iWvWRPXL499snbTvkzHi6QfDJp+uAajOyT+wZSq7gQuAhCROjAOPCQilwHLgZeq6nMi8kK3zYuBG4ALgAXA34vIi1Q1fqUxSiOvWZPUL6t9Mh3x22PU8SLpB4OmH67B6A5ZH8ZeAexW1T3A7wN3qepzAKr6I7fNcuCzqvqcqn4Pp3bs0qICNrKT16xJ6pfVPqlLazqGuONF0g8GTT9cg9Edsi70NwBr3c9fBLxSRDaJyD+IyCvc46OA/3fx/e4xo0vkNWvi+uWxT1ZcsjDT8SLpB4OmH67B6A6pc92IyFzgGmC1r+884FLgFcC9InIOEPb2rOV3cxFZCawEOOOMM7JFbWQiWLkqaN2MuqbNHet3cMu6rQDMG27wnl+/gOsWj85YMh5B6yatCeI9cPXGE4GhOTXu2biXR5+cCO0XNXbW/Dxp8gOluY5uWi9WIcvIS2rrRkSWAzer6pXu6y/jbN183X29G2fRfweAqr7fPb4BuENV/zlqbLNuusvYlnFW3beN5rHZ3ws1gXpNZhUvCVoeeU2QNP2i2ly3eJQHHh8vND9PO/GY9WJ0izJy3azg+LYNwBhwuTvZi4C5wI+B9cANInKCiJwNnAc8lmEeo8Os2bCzZZEHp15tsEJV0PLIa4Kk6RfVZu2mfYXn52knHrNejKqTautGRIaB1wK/6zv8SeCTIvIt4Ahwozq/HuwQkXuBbwNHcX4LMOOmwmS1Nvzt85ogafpFtUlr6WS5rnbiMevFqDqp3tGr6mFVPUVVD/mOHVHVt6jqS1T1YlV9xHfufap6rqouUtUvlRG4URxZrQ1/+7wmSJp+UW3SWjpZrqudeMx6MaqO5boxnEpUtdbFsya0VKgKWh55TZA0/aLarLhkYeH5edqJx6wXo+pYhSlj5kHiHet3zFSj8qwbiLc88pogafrFtVly5vxCq2K1G49hVBnLdWNE4qmE45NT1MR5OOvh/SDIoyfe+fkdPHPY+YESVtowOJaXkG3ycLOS2qNhdIu01o29ozdCCaqEQSnnmcNNVt2/DWDGa/e3H5+cYvWD22fOe2Ouun/bLJNncqrJqvu2tbTzj+WveRs1btLchjHI2B69EUqaRGjNaZ1RC9PqiUFdE6B5TBM1xqRxTXs0jGhsoTdCyZoIrR09MUu7LO1NezQMB1vojVCyJkJrR0/M0i5Le9MeDcPBFnojlDSJ0Bp1mVEL0+qJQV0ToFGTRI0xaVzTHg0jGnsY28ekNVGiEoRNNadnEp+FWTe/9tLTWLNhJ7eu28qCkSGuWzzKF584MGPUnDBn9vsIb26/dSMSvUfvzT0SYt0ALLvrkcgkbWmvtWp2jj++uDKOYJqnkR7TK/uUtAm4iiwz2KgLKLPy5kQl/Wqnf94Sh1VPSpbmawHZ7rPR35SR1MzoIdKaKEWWGWxOa0tytCj7pZ3+eUscVt3OSfO1gGz32TDAFvq+Ja2JUlaZwaS27fQvImFa3ljKpN04qnIdRvWwhb5PSWuilFVmMKltO/2LSJiWN5YyaTeOqlyHUT1soe9T0pooRZYZbNSlJTlalP3STv+8JQ6rbuek+VpAtvtsGGAPY/uadq2brP2CNkhSjprbx7a3lCkUYHhunZ8dmW4pdeiP6axThvh/uw/OqlF50tw6jXqNQ1OdyYmTt/Sg/x4F+wWtm2eb00w1j82MN9yoMXdOPdTEsQexg0fah7G20BulkGS4JBkmNeBY6Jn0lGmi5C092KgJCLHlGePmCGK2zWBTmHUjIotEZKvv46cicovv/J+IiIrIC9zXIiIfFZFdIvKEiFzc3qUYvUiS4ZJkmLS7yAfnK5q8pQebxzSxPGNc/yBm2xhpSPyDKVXdCVwEICJ1YBx4yH29EKfE4F5fl9fj1Ik9D7gE+Jj7rzFAJBkunTJEypqn3dw+acbLa0QZRpCsD2OvAHar6h739YeB/wyztkqXA59Wh43AiIic1n6oRi+RZLh0yhApa552c/ukGS+vEWUYQbIu9DcAawFE5BpgXFW3BdqMAvt8r/e7x2YhIitFZLOIbJ6YmMgYhlF1kgyXJMOkCB2sTBMlb+nBRk0SyzPG9Q9ito2RhtS5bkRkLnANsFpEhoF3A1eGNQ051vLEV1XvBu4G52Fs2jiM3uDal4+yec/BWVaNt5+8ec9BvrDtQOT+87Jz53P2qT/XYuQELZwvbDswU5TEy8Xjz3vj37/O+rAyyahpp/Sgd8zL0xMVp79/1pw+RV6r0fuktm5EZDlws6peKSIXAl8DDrunTweeBpYCdwJfV1Xvnf9O4NWqeiBqbLNu+o+0eVvCCMvl4pHGbEnqkyf2ou2WLHOUGU/V8/8Y8ZSR62YF7raNqm5X1Req6lmqehbO9szFqvoDYD3wVte+uRQ4FLfIG/1J2rwtYYTlcvFIY7Yk9UmiEzlxssxRZjxVz/9jFEOqrRt3q+a1wO+maP4wcDWwC+cd/025ozN6ljJNkDxmS7sGTNYxipyjzHiqnv/HKIZU7+hV9bCqnqKqhyLOn6WqP3Y/V1W9WVXPVdULVdX2ZAaQMk2QPGZLuwZM1jGKnKPMeKqe/8coBst1Y5RC2rwtYYTlcvFIY7Yk9UmiEzlxssxRZjxVz/9jFINVmDJyE2drRBkjnjGz7rG9NEP+/LUuQnNamRdSXUlw9o9vWbeVW+/diiqMupWt/AZOXM4bf8xeTh2Pk+bWed9vXNhiDNVFuG7xaO68OXHtgvbNnZ/fwR3rd4TGXpYZc8Kc2sw+/bzhBu/59QtKexBrhk93sIXeyEXQ1hifnGL1g9sBZi32Yf+Jbx/bHrrIAzM65TOHmww16vy36y8CYNV922Y9oPVksfHJKdY9tm+W1PuzI9MMNeDD118UW5nKv8h7r//TfdvYvOcgDzw+PhPLtCoPPD7OkjPnR9o+Ydeftp3/vFdiMaxt0QtimHHzbNQXpoT5ou6ZUTy2dWPkoh1bY+2mfYlt/OOt2bAz0sKB9Plj0hg608eUtZv25cpjk3bOLDl/up2vp5fnM45j7+iNXLRja0xnyJjajv2RtypWVHxpbJ+0c2bJ+dPNfD29PJ9xHHtHb+SiHVujLuEPWqPGy2uA5K2KFRVfGtsn7ZxZcv50M19PL89nHMcWeiMX7dgaKy5ZmGoOb7xVVy2KtHAgff6YNIZOvSasuGRhrjw2aefMkvOn2/l6enk+4zi2dWPkoh0T5L3XXsj3Jv6db+w+OHOsUYPrl57BF584MPNAcqo5za3rtqLAyFCDI0enORx4WFgX4fqlC1ly5nzuWL9jxrw5sRH+HsZvmJwwp8ZzR4+PJzh79I8+OdFi8pzYqLF5z8FZ13vd4tHYSlyeYTLVnA7NwbN5z0EefXJi1vnhRo2po8dQN29PmO0Tx9iW8Vn3Ic6iiTJ/8uYHSqJse8iIxipMGR0nKr/KdYtHWfcv+1oerAbbPPD4eGjfsONxFa2880CqSlBh8UTlhMmT6ydL9akwxraMt9hJ4Pxdwpo3vSxyDMt307tYKUGjsiy76xHGQx7Aee9q44hqE3V8dGSIb9x2eeSco+7+cNi5NHjjB4mar8g5sswZN0bcvUkzr9E90i70tnVjdJwoyyKNjRPVJsmUKcv46GZenCzt8pwzG6Z/sIexRseJsizS2DhRbZJMmTjjox3ro5t5cbK0y3PObJj+wRZ6o+NE2RcrLlnYYs+EtYnqm9Vu8Vs9aSpBhcUTZYzkyfWTpfpU1JxhdlKjLrFjmA3T/9jWjdFx4uyLJWfO587P75gxbwSnPNlooE1U3zS5d6KMj6hKUN6xy84/NdaySbrGYP+w8ZJiTHNf01o3ab4eRn9gD2ONnsOfGGvEl/wsiroIl54zj+//ZKplUX33Q9sjE5uFzROWbCwpxpOHGojA5OF0fbNcf5zWmWbRLiLJmCUq6x6FWTcisghY5zt0DvBnOAW/fx04AuwGblLVSbfPauDtwDTwR6q6IW4OW+iNtLRTotBPoy4cndbWYsa4fzS1dGGLrumnHbWyHXUxSYXsdIlCUzO7S2GlBFV1p6pepKoXAYtxqkY9BHwVeImqvhT4V2C1O/GLgRuAC4DXAX8lIvkSkxtGgHZKFPppRizyEJ3YzE9cMq4yE5XlSZJWZolCS1TWG2R9GHsFsFtV96jqV1T1qHt8I06BcIDlwGdV9TlV/R5OScGlxYRrDDqdUv7SqJ7taIl5ryNvkrSyShSamtkbZF3ob8AtEB7gbcCX3M9HAX8e2v3uMcNom04pf2lUz3a0xKIStQWPd7pEoamZvUHqhV5E5gLXAPcFjr8bOArc4x0K6d7y9khEVorIZhHZPDExkT5iY6Bpp0Shn0ZdQr9RITqxmZ921Mp21MU8SdLKLFFoamZvkEWvfD3wTVX9oXdARG4E3gBcocef6u4H/OkJTweeDg6mqncDd4PzMDZj3EafE2VyeA/4whTMLDSnlZGhBlPN6VmJzU6YU2OoUeeejXs5eajBiY0ak4ebs+yeuORfYYnMRgq0bpJUyCyqZNi9PGFOtl/yTc3sDbIs9CvwbduIyOuAPwV+VVUP+9qtB/6PiPwlsAA4D3isgFiNASFNyTl/yTvFeYeOEluJKsjk1PFyhX5jxVM1vfMfDpyPiit4flqVoUadO64ptgZrUlnBrGUH/fdycqqZubxfGWUOjWJJ9eNbRIaB1wIP+g7/D+B5wFdFZKuI/DWAqu4A7gW+DXwZuFlV29ckjIEhj1nSnNZMi3zacYs8X0V6MWYjO6ne0bvv2E8JHPvFmPbvA97XXmjGoFJE+b0887VrtPSigdKLMRvZsVw3RuUoovxenvnaNVp60UDpxZiN7NhCb1SOPGZJoy6x5QajyGKsFGm8VIVejNnIjuW6MSpJ2nwuXgm8aVXmheS9EYGhOTUON49FFieZWxeO+Ko6BV/XxLFRnm0ei8xbM7ZlfJa94jHcqPHGxacnJkOLu97g2GGJ3tohbG44btLkydWTJv+N5chpH6swZfQ9WfOsFJUnJzjH2JZxVt2/LbbsYFL/uDKHcWOXkVem3Vw9ab4uliOnGArLdWMYVSWrMVJUnpzgHGs27Ey9yEf1j7qOpLHLMGTazdWT5utitk9nsXz0Rs+S1Rgpq7xfnnHT9C+ihGAe2s3Vk+Z6zPbpLPaO3uhZshojZZX3yzNumv5pyxyWZSHlbZPm/pvt01lsoTd6lqzGSFF5coJzrLpqUWLZwaT+cWUOk8orFm3ItJurJ83XxWyfzmIPY42eJqu5MbZlfFapvTCWnTsfgG/sPpg4/3Cjxtw5dSanmohA8L9TowZz59RnqlgNNWoIcNhNO3DS3DqNes3pz/GcPf4SgFHWjWcRBe2b4D2JKoEYZ9uMx2yhjAw1eMPLTuPRJydmWU/+nD5pKnLl+dqZpTMbs24MI4SxLeOsum9bZLqEoUad6xaPxlaX6hR5KkMBiWZR1DXmyReUJ/48mKUTji30hhHCsrseiX23CkT69t1gdGSIb9x2+axjUdcw6u5vJ10fdO4aw+LPQ9w1FzF+r5J2oTfrxhgo0lgdVVnkobzKUJ26xqIsGrN02sMexhoDRRqrI011qU6RtTJUWmulU9dYlEVjlk572EJvDBSrrloUmxNnqFFPrC7VKfJUhkpjFkVdY958QXHzFGXRmKXTHrZHbwwcY1vG+S8PPjFjvngI8NuXnsGSM+fz7oe2z5gycQhwYqPGlC+XTp6KV95YDV+eHc+82bznIGs37WNalbo4ZQ6XnDl/xkAJGi6XnX8qX3ziwIylM9SocWKj3mLDhJkxQEsOoaBNlGTdPHO4GVldKxjbyFAjtjCL37RJY/JkpddNHnsYaxgRxJk3NUBqwnSEedKoCQiz0hIUbav4qUfE8pZLz+C9114YaqOExRhl2hSRtyapfRKNmrDmzS/LZBeZyeNQWK4bEVnkVpDyPn4qIreIyHwR+aqIPOX+O89tLyLyURHZJSJPiMjFRVyQYRTFmg07IxflYxC5yIOzmAdzz+StbpWGqFjWbtoHRFTbColxqjnN2k37MueX6UQ+oeYxDR2v7Hw4g5RvJ3GhV9WdqnqRql4ELAYOAw8BtwFfU9XzgK+5r8EpIn6e+7ES+FgZgRtGXvrB1PCsmSJMm3bz1qQdK46y7KKscxY5fpXI+jD2CmC3qu4BlgOfco9/CrjW/Xw58Gl12AiMiMhphURrGAXQD6aGZ81kuZYo06bdvDVpx4ojq11UBINk8mRd6G8A1rqf/7yqHgBw/32he3wU2Ofrs989ZhiVIM68qeHsi0fRqElL7pmibRU/UbGsuGQhEFFtKyTGKNOmiLw1Se2TaNQks11UBINk8qT+gykRmQtcA6xOahpyrOV3RhFZibO1wxlnnJE2DMPIjb8qVdj66RkgwOzcMq51Mm+4wXPN6RZbpzmtod/0eZhTE4769uWDe/Q1gd+6xPn/cu7qh2esmGHX/AmrEOW3Sfy2jnd8856D3Hrv1hmzZrhR48/f+NKZB5KOVTQ96x5FPaz0jkdVpzrrlKFZOYQEuH7pwtDxvGP+r8UJc2a/N23HmgnG2ovWTVpSWzcishy4WVWvdF/vBF6tqgfcrZmvq+oiEfkb9/O1wXZRY5t1Y5RNnA2SxrTIWkWqLBo1YenZ80ITrnkmThZuH9vOZzbubTnu/UDJaunEEWU7NerCmje1Wjdenyx5fXrNmmmXMipMreCCCayxAAASYUlEQVT4tg3AeuBG9/Mbgc/5jr/VtW8uBQ7FLfKG0QnibJA0pkXWKlJl0TymkVk1PRMnC1F9jim5LJ04omyn5nS4deP1iau+NSjWTLuk2roRkWHgtcDv+g7fBdwrIm8H9gJvdo8/DFwN7MIxdG4qLFrDyEmSSdHu+SqQJ39NXJ88lk4cnbJ7euFr1WlSLfSqehg4JXDsJzgWTrCtAjcXEp1hFMSCkaHYrI5JpkVS/yqQJ39NXBbLqHPtmDVR9zDOgAnr47WPO2ccx3LdGANBnA2SxrTIWkWqLBo1mSmMEsQzcbIQ1acm5LJ04oiynRr1cOvG65Mlr0+/WjPtYmmKjYHAb1j487MEqzMl9fcbIH5q4uxrh+HPfeN9PurmffnCtgMz1a6GGzVEZFaOHX9fv/Fy+9j2lvw3WR/EAjN97tm0N9S6CbN08j7o9Pr5K3z5K2nF9YmLYRCsmXaxXDeGUSDBUoVh5QGBlnKGjRoErM2WY17SNS/HTdiCCckLX1TfuFJ/ZSQUC5snbuwqJiDrdkyW1MwwOkxSmUJw/gBKjynHIlsks+zc+Tz2vWda5qnXhBqzk6sFdcO0imNScrKiNMa0icWqmICsCjGVoVcahhFDXLI0j+k2F3lwipaHzTN9rDW5WlA3TKs4JiUnK0pjTKtIVlGlrGJMUdhCbxgFUVWtzx9XWi0xzbUUcb1p9ckqJiCrYkxR2EJvGAVRVa3PH1faBGZprqWI602bWKyKCciqGFMUttAbRkEklSmE4/vo7bDs3Pmh89RrrcnVgrphWsUxKTlZURpjWkWyiiplFWOKwvRKw8hBnG2RxrpZ/eATTAU1m5R8/ydTXL90IQ88vn9mjJrAiqULW3TIy84/lTUbdnLruq0zcV6/dCH3bNw7E9dJc+u87zcczXLZXY/M9L1u8SiPPjlRqnWTNrFYuwnIirRj/GOdPNTgxEZtJmlbFUygMMy6MYyMtGtbFJEgLax8YZhh01JmMKJfnjKDvUKRdkwVTBs/Zt0YRkm0a1sUkSAtrHxhmGHTUmYwol/RCcyqRJF2TC+ZNn5soTeMjLRrW5RpZWQ1ZzyKTmBWJYq0Y3rJtPFjC71hZKRd26JMKyOrOeORp8xgr1CkHdNLpo0fW+gNIyPt2hZFJEgLK18YZti0lBmM6Fd0ArMqUaQd00umjR+zbgwjI1EGCBy3Vk4eatCcPjYrQdlJc+tMHZmO/MvYE+bUOHL0WGvdzRDCyhf694qvffko1758lPs2751VqKQ5rQw3apx0whwOTTUZatSYOjrNZzbuRdwYDx+ZbjFIysjpEjdm1vmC7S87/1QefXJiVgK7uHuVZj6vzVRzOnNSvG5j1o1hFEBSbpg4lp07nzcvOSN3/yCeBbJ5z8HQMoGQrSRhGaZJkSUC27n3aY2jqtk2HoUmNROREeDjwEtwtOC3AVPAXwMnAkeBP1DVx0REgI/gVJk6DPxHVf1m3Pi20Bu9zrK7HmmrMMlowYVNRkeG+MGhZ3NVnaqLsPv9V8+8jrq20ZEhvnHb5bniixsTwguKRM3X7r2PKrDin6+Me1AEaRf6tFs3HwG+rKpvEpG5wDBwL3Cnqn5JRK4GPgi8Gng9cJ77cQnwMfdfw+hb2rUuirY2np6cSrUFFEZw0SvDNCmyRGC79y6NcdSrto1H4sNYEXk+8CrgEwCqekRVJ3He2T/fbXYy8LT7+XLg0+qwERgRkdMKj9wwKkS71kXR1saCkaFcpQWh1cApwzSJGzPrfO3euzTGUa/aNh5prJtzgAng70Rki4h8XEROAm4B1ojIPuBDwGq3/SjgLy2/3z02CxFZKSKbRWTzxMREWxdhGN0mKTdMHMvOnd9W/yCeBRJXWjBLScIyTJMiSwS2c+/SGke9att4pNm6mQNcDLxTVTeJyEeA23Dexd+qqg+IyG/ivON/DbTIAEDrb5GqejdwNzh79DnjN4xKEDRx0lo3y86dzz2/80szr71Sh/4cOVmZak5zy7qtkee9koQAW/cdmhVjowaf2biXz2zcy0lz6zTqNSanmrP+U/tz9vhz42Q1Y/y5dKJKBHrWTNCS8RNW5nG4UWPunPqsKl5B/JW1lpw5f1aOohMbs98DpzGt/MeqVgkrzUK/H9ivqpvc1/fjLPS/Avyxe+w+nIe1Xnv/W4LTOb6tYxh9i6c0Ft1/bMs477p3a2RN2iwMNeozi3yYqeLPs+b8AHDO+6d+tnmMzXsOzjJVxienWP3g9pnrCLsG/3zjk1M88Ph4pLXiHQv2iZvjWV/wh5vHaB5TGjWJLAbzbCCp3HNHj79+5nCzZa7g1yfsmlbdtw2EmRQXSTF3isStG1X9AbBPRLzfUa4Avo2zeP+qe+xy4Cn38/XAW8XhUuCQqh4oNmzDGBzWbNhZyCIPx/3xpApSSWNkzY2TJ0dMlj5p8/pEjVVUfM1j2pLHqAq5cNJaN+8E7nGNm+8CNwGfAz4iInOAZ4GVbtuHcdTKXTh65U2FRmwYA0YZRk67ZM2NU7Zlk/eavH5Fxtdu2zJItdCr6lYg6Gr+E7A4pK0CN7cfmmEY4JgdRTr2C2Jc9bREuedxdkrYfEkVr9L2yXuPvLGKjC9unm5huW4Mo+KsumoRCYWrUhNntmQZI2tunDzWSpY+afP6RI1VVHyNmrTkMaqCnWO5bgyjC2TN5fL8ExuxBklakoycJOYNN/i1l57Go09OpM754s8RUxNmnjdMNae58/M7ZtpF3Y8w+2bznoMtxs51i0dZu2kf06rURbj+FQtbbBoRUKUl3qQKVrePbZ819opLFs6kiQgzcapm3ViuG8PoMFnyprSTx6UMakC9LrMeOLabh8aroxtVLSvNGI2azLJdvDGKqJx1+9j20JxBwZxA3cAqTBlGRWnXJukmxyCTVZIm/ulj8dWy0owRZbsUUTlr7aZ9mY5XEVvoDaPDdMIm6TRl5sNpZ4wiKmdFjZEnYVy3sIXeMDpMlrwp3bY10lJmPpx2xiiiclbUGHlzCXUDW+gNo8O0a5N0kxpkskrSxF+vxVfLSjNGlO1SROWsqJxBcbmEqoZZN4bRYZIMj6i2/mpJftvFq6b09OQUw26FqLybCp6V4o3vt2T8OXLSWiXBax0ZbvBsc5opN/2AP29O1Jhh98t/zUm2y5Iz57dlwXgPXKOsm17ArBvD6DH8ql9U8jPv+GjEInjZ+afyhW0HZrRDfwKz4A8T71/vhwDM1iz9P4DmpVzIwxbqshTEJJXVsXqemIm5JvBbl+Q3aooskZhEoRWmysYWesNIR5TqF0ejLqDE5n0pkzB9MkhZZfmSVNaxLeO8a93W0Dq+efTJIkskpsH0SsPoQ/IofUnJvcomTJ8MUlbirySVdc2GnZHF2vPc67j58iROKwrbozeMHqKXlL6slKGSJmmfcXPmuddlJ0fLi72jN4weopeUvqyUoZImaZ9xc+a510WWSCwSW+gNo4fIo/QlJfcqmzB9MkhZib+SVNZVVy2KXATz3OsiSyQWiW3dGEYPEVT9kqybecMNVGlJiOZVymtGbVBnxNMwg9YNwPNOmMMFC57Hxu8+M2PvzJHjc/tL+kG0meId91s+UcnU/GOMDDc4YU6NQ1PNSHUzybpJa8ukUWf95SZF4NZ1W1mzYWep5pFZN4bRp3QyIVrWJGRZ+kYlJwvrD9mSxqWh6PGKHLNQ60ZERkTkfhF5UkS+IyK/5B5/p4jsFJEdIvJBX/vVIrLLPXdV6qgNwyiMTiZEy5qELEvfqORkYf3jxshrt5Rhy3TawEm7dfMR4Muq+ia3nOCwiFwGLAdeqqrPicgLAUTkxcANwAXAAuDvReRFqlqdFHyGMQB0OiFaO0nIkvomGTD+fkUnWCszYVuRY8aR+I5eRJ4PvAr4BICqHlHVSeD3gbtU9Tn3+I/cLsuBz6rqc6r6PZzasUvLCN4wjGg6nRCtnSRkSX2TDBh/v6LtljITthU5Zhxptm7OASaAvxORLSLycRE5CXgR8EoR2SQi/yAir3DbjwL+vzTY7x6bhYisFJHNIrJ5YmKizcswDCNIJxOiZU1ClqVvVHKysP5xY+S1W8qwZTpt4KTZupkDXAy8U1U3ichHgNvc4/OAS4FXAPeKyDk4D/yDtPzepap3A3eD8zA2X/iGYUQRTIjmz1UzMtTgDS87bSbfzMlDDf7t2SbTIf8TBUfRPBI46Zk9/hJ/wXnDctuMuCZQnAUTl5wsybrJkjQu630sKkdNGWPGkWjdiMgvABtV9Sz39StxFvo6ztbN193ju3EW/XcAqOr73eMbgDtU9Z+j5jDrxjB6jzJsFCMbhVk3qvoDYJ+IeL9TXAF8GxgDLncnexEwF/gxsB64QUROEJGzgfOAx3JdhWEYlaWbuVuMbKS1bt4J3OMaN98FbgJ+BnxSRL4FHAFuVOfXgx0ici/OD4OjwM1m3BhG/9Fpc8TIT6qFXlW3AmG/Hrwlov37gPe1EZdhGBVnwcgQ4yGLeq+UPxwkLNeNYRi56GbuFiMbluvGMIxcdNocMfJjC71hGLm59uWjtrD3ALZ1YxiG0efYQm8YhtHn2EJvGIbR59hCbxiG0efYQm8YhtHnVKLClIhMAHs6NN0LcFI1VJGqxlbVuKC6sVU1LrDY8lDVuM5U1VOTGlVioe8kIrI5TRKgblDV2KoaF1Q3tqrGBRZbHqoaV1ps68YwDKPPsYXeMAyjzxnEhf7ubgcQQ1Vjq2pcUN3YqhoXWGx5qGpcqRi4PXrDMIxBYxDf0RuGYQwUfbnQi8hCEXlURL4jIjtE5I9D2rxaRA6JyFb34886GN/3RWS7O29LDUVx+KiI7BKRJ0Tk4g7EtMh3L7aKyE9F5JZAm47dMxH5pIj8yC1s4x2bLyJfFZGn3H/nRfS90W3zlIjc2IG41ojIk+7X6iERGYnoG/t1Lym2O0Rk3Pc1uzqi7+tEZKf7PXdbh2Jb54vr+yKyNaJvafctaq2owvdaoahq330ApwEXu58/D/hX4MWBNq8GvtCl+L4PvCDm/NXAl3DqL18KbOpwfHXgBziOblfuGfAqnKL03/Id+yBwm/v5bcAHQvrNx6mCNh+neP13gXklx3UlMMf9/ANhcaX5upcU2x3An6T4eu8GzsEpCbot+P+ljNgC5/8C+LNO37eotaIK32tFfvTlO3pVPaCq33Q//zfgO0Av5VJdDnxaHTYCIyJyWgfnvwLYraqd+iO2FlT1H4GDgcPLgU+5n38KuDak61XAV1X1oKo+A3wVeF2ZcanqV1T1qPtyI3B6UfNlIeKepWEpsEtVv6uqR4DP4tzrjsQmIgL8JrC2yDnTELNWdP17rUj6cqH3IyJnAS8HNoWc/iUR2SYiXxKRCzoYlgJfEZHHRWRlyPlRYJ/v9X46+4PqBqL/03XrngH8vKoeAOc/KPDCkDbdvndvw/ltLIykr3tZ/KG7rfTJiC2Ibt+zVwI/VNWnIs535L4F1ope+F5LTV8v9CLyc8ADwC2q+tPA6W/ibE28DPjvwFgHQ1umqhcDrwduFpFXBc5LSJ+O6FHiFIC/Brgv5HQ371launnv3g0cBe6JaJL0dS+DjwHnAhcBB3C2SIJ07Z65rCD+3Xzp9y1hrYjsFnKskhpj3y70ItLA+cLdo6oPBs+r6k9V9d/dzx8GGiLygk7EpqpPu//+CHgI51dnP/uBhb7XpwNPdyI2nP9M31TVHwZPdPOeufzQ28Jy//1RSJuu3Dv3QdwbgN9WdwM3SIqve+Go6g9VdVpVjwF/GzFn177fRGQO8EZgXVSbsu9bxFpR2e+1PPTlQu/u+X0C+I6q/mVEm19w2yEiS3HuxU86ENtJIvI873OcB3nfCjRbD7zVtW8uBQ55v0Z2gMh3V926Zz7WA57ZcCPwuZA2G4ArRWSeu01xpXusNETkdcCfAteo6uGINmm+7mXE5n+28xsRc/4LcJ6InO3+RncDzr3uBK8BnlTV/WEny75vMWtFJb/XctPtp8FlfAC/gvMr1BPAVvfjauD3gN9z2/whsAPHMNgI/HKHYjvHnXObO/+73eP+2AT4nzgmxHZgSYdiG8ZZuE/2HevKPcP5YXMAaOK8c3o7cArwNeAp99/5btslwMd9fd8G7HI/bupAXLtw9mq977W/dtsuAB6O+7p3ILb/7X4PPYGzeJ0WjM19fTWOcbK7U7G5x/+X9/3la9ux+xazVnT9e63ID/vLWMMwjD6nL7duDMMwjOPYQm8YhtHn2EJvGIbR59hCbxiG0efYQm8YhtHn2EJvGIbR59hCbxiG0efYQm8YhtHn/H8JZbJXfctoGAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\"\"\" Trend between LR model with Lasso regularization and FICO score in the data set\"\"\"\n",
    "\n",
    "plt.scatter(x=predictionsOnTestData_lasso, y=x_test.fico)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The trend between our response variables and FICO score is linear as we assumed in the beggining. This linearity assumption is what made us choose Linear Regression model. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}