Random-Forests.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c0b549e1",
   "metadata": {
    "papermill": {
     "duration": 0.011104,
     "end_time": "2022-06-21T12:21:22.632848",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.621744",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Random Forest with Breast Cancer Dataset\n",
    "![](https://img.freepik.com/free-photo/dna_1048-4880.jpg?t=st=1655813117~exp=1655813717~hmac=4a3594af68a2d7d9ef718c3411624f28e6dcc6489d31cf706481c8a3dc9e0fe7&w=1060)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ecf94f93",
   "metadata": {
    "papermill": {
     "duration": 0.006715,
     "end_time": "2022-06-21T12:21:22.647045",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.640330",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Hi Guys 😀\n",
    "\n",
    "In this notebook, I'm going to talk about random forest.\n",
    "\n",
    "- First, I'm going to cover what random forest is\n",
    "- Next, I'm going to mention the some advantages and disadvantages of random forest.\n",
    "- Finally, I'm going to show you how to implement random forest with a real world dataset.\n",
    "\n",
    "Please don't forget to follow us on [Tirendaz Academy](https://youtube.com/c/tirendazacademy) YouTube channel \n",
    "\n",
    "Happy learning 🐱‍🏍"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca8ae2a3",
   "metadata": {
    "papermill": {
     "duration": 0.006504,
     "end_time": "2022-06-21T12:21:22.660512",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.654008",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# What is Random Forest?\n",
    "![](https://img.freepik.com/free-photo/dense-forest-with-tall-pine-trees-fog-it_181624-11826.jpg?t=st=1655813320~exp=1655813920~hmac=e7d8537257c1c4844b31aa6d831f28225019833106ba96630aae60b5741bb248&w=1060)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1cadfc2c",
   "metadata": {
    "papermill": {
     "duration": 0.006704,
     "end_time": "2022-06-21T12:21:22.674131",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.667427",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "You can think of random forest as an ensemble of decision trees. Each tree in random forest is slightly different from the others. These trees are selected a different subset of features. When building the model since we use many trees, we can reduce the amount of overfitting."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7ee4f8c",
   "metadata": {
    "papermill": {
     "duration": 0.006749,
     "end_time": "2022-06-21T12:21:22.687786",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.681037",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Some Advantages of Random Forest\n",
    "![](https://img.freepik.com/free-photo/glad-dark-haired-young-woman-says-sounds-good-confirms-something-everything-control-going-great-approves-promo-has-glad-expression-agrees-with-person-wears-yellow-sweatshirt_273609-42865.jpg?t=st=1655813420~exp=1655814020~hmac=1e0fce9eff4ffbc5385b758adf88e5d67ea1949ef6e5c9441ec2725bb374893a&w=1060)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb7d9977",
   "metadata": {
    "papermill": {
     "duration": 0.006644,
     "end_time": "2022-06-21T12:21:22.701414",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.694770",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "- You can use random forest for both classification and regression tasks.\n",
    "- Random forest often work well without heavy tuning of the hyperparameters.\n",
    "- You don't need to scale the data.\n",
    "- Random forest may provide better accuracy than decision tree since it overcomes the overfitting problem."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "44d33a95",
   "metadata": {
    "papermill": {
     "duration": 0.006675,
     "end_time": "2022-06-21T12:21:22.715062",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.708387",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Some Disadvantages Of Random Forest\n",
    "![](https://img.freepik.com/free-photo/photo-thoughtful-handsome-adult-european-man-holds-chin-looks-pensively-away-tries-solve-problem_273609-45891.jpg?t=st=1655813585~exp=1655814185~hmac=2a5cb9a9972936443d261541efc6d48d8cb60a5d75259e6223729df99b3155c9&w=1060)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01beec2f",
   "metadata": {
    "papermill": {
     "duration": 0.006668,
     "end_time": "2022-06-21T12:21:22.728707",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.722039",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "- Random forest cannot be perform well on very high dimensional, and sparse data such as text data.\n",
    "- Random forest is not simple to interpret since it uses deeper tree than decision trees."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1090a0b",
   "metadata": {
    "papermill": {
     "duration": 0.007471,
     "end_time": "2022-06-21T12:21:22.743263",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.735792",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Loading Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "39cdd43d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-21T12:21:22.760812Z",
     "iopub.status.busy": "2022-06-21T12:21:22.760117Z",
     "iopub.status.idle": "2022-06-21T12:21:22.834212Z",
     "shell.execute_reply": "2022-06-21T12:21:22.833023Z"
    },
    "papermill": {
     "duration": 0.086849,
     "end_time": "2022-06-21T12:21:22.837520",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.750671",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>diagnosis</th>\n",
       "      <th>radius_mean</th>\n",
       "      <th>texture_mean</th>\n",
       "      <th>perimeter_mean</th>\n",
       "      <th>area_mean</th>\n",
       "      <th>smoothness_mean</th>\n",
       "      <th>compactness_mean</th>\n",
       "      <th>concavity_mean</th>\n",
       "      <th>concave points_mean</th>\n",
       "      <th>...</th>\n",
       "      <th>texture_worst</th>\n",
       "      <th>perimeter_worst</th>\n",
       "      <th>area_worst</th>\n",
       "      <th>smoothness_worst</th>\n",
       "      <th>compactness_worst</th>\n",
       "      <th>concavity_worst</th>\n",
       "      <th>concave points_worst</th>\n",
       "      <th>symmetry_worst</th>\n",
       "      <th>fractal_dimension_worst</th>\n",
       "      <th>Unnamed: 32</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>842302</td>\n",
       "      <td>M</td>\n",
       "      <td>17.99</td>\n",
       "      <td>10.38</td>\n",
       "      <td>122.80</td>\n",
       "      <td>1001.0</td>\n",
       "      <td>0.11840</td>\n",
       "      <td>0.27760</td>\n",
       "      <td>0.3001</td>\n",
       "      <td>0.14710</td>\n",
       "      <td>...</td>\n",
       "      <td>17.33</td>\n",
       "      <td>184.60</td>\n",
       "      <td>2019.0</td>\n",
       "      <td>0.1622</td>\n",
       "      <td>0.6656</td>\n",
       "      <td>0.7119</td>\n",
       "      <td>0.2654</td>\n",
       "      <td>0.4601</td>\n",
       "      <td>0.11890</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>842517</td>\n",
       "      <td>M</td>\n",
       "      <td>20.57</td>\n",
       "      <td>17.77</td>\n",
       "      <td>132.90</td>\n",
       "      <td>1326.0</td>\n",
       "      <td>0.08474</td>\n",
       "      <td>0.07864</td>\n",
       "      <td>0.0869</td>\n",
       "      <td>0.07017</td>\n",
       "      <td>...</td>\n",
       "      <td>23.41</td>\n",
       "      <td>158.80</td>\n",
       "      <td>1956.0</td>\n",
       "      <td>0.1238</td>\n",
       "      <td>0.1866</td>\n",
       "      <td>0.2416</td>\n",
       "      <td>0.1860</td>\n",
       "      <td>0.2750</td>\n",
       "      <td>0.08902</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>84300903</td>\n",
       "      <td>M</td>\n",
       "      <td>19.69</td>\n",
       "      <td>21.25</td>\n",
       "      <td>130.00</td>\n",
       "      <td>1203.0</td>\n",
       "      <td>0.10960</td>\n",
       "      <td>0.15990</td>\n",
       "      <td>0.1974</td>\n",
       "      <td>0.12790</td>\n",
       "      <td>...</td>\n",
       "      <td>25.53</td>\n",
       "      <td>152.50</td>\n",
       "      <td>1709.0</td>\n",
       "      <td>0.1444</td>\n",
       "      <td>0.4245</td>\n",
       "      <td>0.4504</td>\n",
       "      <td>0.2430</td>\n",
       "      <td>0.3613</td>\n",
       "      <td>0.08758</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>84348301</td>\n",
       "      <td>M</td>\n",
       "      <td>11.42</td>\n",
       "      <td>20.38</td>\n",
       "      <td>77.58</td>\n",
       "      <td>386.1</td>\n",
       "      <td>0.14250</td>\n",
       "      <td>0.28390</td>\n",
       "      <td>0.2414</td>\n",
       "      <td>0.10520</td>\n",
       "      <td>...</td>\n",
       "      <td>26.50</td>\n",
       "      <td>98.87</td>\n",
       "      <td>567.7</td>\n",
       "      <td>0.2098</td>\n",
       "      <td>0.8663</td>\n",
       "      <td>0.6869</td>\n",
       "      <td>0.2575</td>\n",
       "      <td>0.6638</td>\n",
       "      <td>0.17300</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>84358402</td>\n",
       "      <td>M</td>\n",
       "      <td>20.29</td>\n",
       "      <td>14.34</td>\n",
       "      <td>135.10</td>\n",
       "      <td>1297.0</td>\n",
       "      <td>0.10030</td>\n",
       "      <td>0.13280</td>\n",
       "      <td>0.1980</td>\n",
       "      <td>0.10430</td>\n",
       "      <td>...</td>\n",
       "      <td>16.67</td>\n",
       "      <td>152.20</td>\n",
       "      <td>1575.0</td>\n",
       "      <td>0.1374</td>\n",
       "      <td>0.2050</td>\n",
       "      <td>0.4000</td>\n",
       "      <td>0.1625</td>\n",
       "      <td>0.2364</td>\n",
       "      <td>0.07678</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 33 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         id diagnosis  radius_mean  texture_mean  perimeter_mean  area_mean  \\\n",
       "0    842302         M        17.99         10.38          122.80     1001.0   \n",
       "1    842517         M        20.57         17.77          132.90     1326.0   \n",
       "2  84300903         M        19.69         21.25          130.00     1203.0   \n",
       "3  84348301         M        11.42         20.38           77.58      386.1   \n",
       "4  84358402         M        20.29         14.34          135.10     1297.0   \n",
       "\n",
       "   smoothness_mean  compactness_mean  concavity_mean  concave points_mean  \\\n",
       "0          0.11840           0.27760          0.3001              0.14710   \n",
       "1          0.08474           0.07864          0.0869              0.07017   \n",
       "2          0.10960           0.15990          0.1974              0.12790   \n",
       "3          0.14250           0.28390          0.2414              0.10520   \n",
       "4          0.10030           0.13280          0.1980              0.10430   \n",
       "\n",
       "   ...  texture_worst  perimeter_worst  area_worst  smoothness_worst  \\\n",
       "0  ...          17.33           184.60      2019.0            0.1622   \n",
       "1  ...          23.41           158.80      1956.0            0.1238   \n",
       "2  ...          25.53           152.50      1709.0            0.1444   \n",
       "3  ...          26.50            98.87       567.7            0.2098   \n",
       "4  ...          16.67           152.20      1575.0            0.1374   \n",
       "\n",
       "   compactness_worst  concavity_worst  concave points_worst  symmetry_worst  \\\n",
       "0             0.6656           0.7119                0.2654          0.4601   \n",
       "1             0.1866           0.2416                0.1860          0.2750   \n",
       "2             0.4245           0.4504                0.2430          0.3613   \n",
       "3             0.8663           0.6869                0.2575          0.6638   \n",
       "4             0.2050           0.4000                0.1625          0.2364   \n",
       "\n",
       "   fractal_dimension_worst  Unnamed: 32  \n",
       "0                  0.11890          NaN  \n",
       "1                  0.08902          NaN  \n",
       "2                  0.08758          NaN  \n",
       "3                  0.17300          NaN  \n",
       "4                  0.07678          NaN  \n",
       "\n",
       "[5 rows x 33 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "df = pd.read_csv(\"../input/breast-cancer-wisconsin-data/data.csv\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d25396ad",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-21T12:21:22.855475Z",
     "iopub.status.busy": "2022-06-21T12:21:22.854811Z",
     "iopub.status.idle": "2022-06-21T12:21:22.860022Z",
     "shell.execute_reply": "2022-06-21T12:21:22.859382Z"
    },
    "papermill": {
     "duration": 0.016558,
     "end_time": "2022-06-21T12:21:22.861959",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.845401",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(569, 33)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51ba6884",
   "metadata": {
    "papermill": {
     "duration": 0.007323,
     "end_time": "2022-06-21T12:21:22.877602",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.870279",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Data Preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "823f71c1",
   "metadata": {
    "papermill": {
     "duration": 0.007975,
     "end_time": "2022-06-21T12:21:22.893473",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.885498",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's create the target and features variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "60f9afe7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-21T12:21:22.912392Z",
     "iopub.status.busy": "2022-06-21T12:21:22.911603Z",
     "iopub.status.idle": "2022-06-21T12:21:22.923347Z",
     "shell.execute_reply": "2022-06-21T12:21:22.922411Z"
    },
    "papermill": {
     "duration": 0.024219,
     "end_time": "2022-06-21T12:21:22.925686",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.901467",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "y = df.loc[:,\"diagnosis\"].values\n",
    "X = df.drop([\"diagnosis\",\"id\",\"Unnamed: 32\"], axis=1).values"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f85aea32",
   "metadata": {
    "papermill": {
     "duration": 0.007762,
     "end_time": "2022-06-21T12:21:22.941230",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.933468",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's encode the target variable with label encoder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "845c8362",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-21T12:21:22.959354Z",
     "iopub.status.busy": "2022-06-21T12:21:22.958715Z",
     "iopub.status.idle": "2022-06-21T12:21:24.096719Z",
     "shell.execute_reply": "2022-06-21T12:21:24.095480Z"
    },
    "papermill": {
     "duration": 1.150085,
     "end_time": "2022-06-21T12:21:24.099300",
     "exception": false,
     "start_time": "2022-06-21T12:21:22.949215",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import LabelEncoder\n",
    "le = LabelEncoder()\n",
    "y = le.fit_transform(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c0536c8",
   "metadata": {
    "papermill": {
     "duration": 0.008172,
     "end_time": "2022-06-21T12:21:24.115649",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.107477",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's split the dataset into the training and test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "953a2a54",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-21T12:21:24.133324Z",
     "iopub.status.busy": "2022-06-21T12:21:24.132930Z",
     "iopub.status.idle": "2022-06-21T12:21:24.206304Z",
     "shell.execute_reply": "2022-06-21T12:21:24.205240Z"
    },
    "papermill": {
     "duration": 0.085439,
     "end_time": "2022-06-21T12:21:24.209066",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.123627",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test=train_test_split(X,y,stratify=y, random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38dddbc2",
   "metadata": {
    "papermill": {
     "duration": 0.007708,
     "end_time": "2022-06-21T12:21:24.224674",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.216966",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Building the Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d92441e",
   "metadata": {
    "papermill": {
     "duration": 0.007609,
     "end_time": "2022-06-21T12:21:24.240222",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.232613",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "First, I'm going to build the model with the default hyperparameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "12d0706e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-21T12:21:24.257679Z",
     "iopub.status.busy": "2022-06-21T12:21:24.257312Z",
     "iopub.status.idle": "2022-06-21T12:21:24.732959Z",
     "shell.execute_reply": "2022-06-21T12:21:24.731191Z"
    },
    "papermill": {
     "duration": 0.487722,
     "end_time": "2022-06-21T12:21:24.735798",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.248076",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RandomForestClassifier(random_state=0)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.ensemble import RandomForestClassifier\n",
    "rf = RandomForestClassifier(random_state=0)\n",
    "rf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dc68031",
   "metadata": {
    "papermill": {
     "duration": 0.007989,
     "end_time": "2022-06-21T12:21:24.752661",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.744672",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's predict the values of the training and test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "82713111",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-21T12:21:24.770841Z",
     "iopub.status.busy": "2022-06-21T12:21:24.770466Z",
     "iopub.status.idle": "2022-06-21T12:21:24.806296Z",
     "shell.execute_reply": "2022-06-21T12:21:24.805244Z"
    },
    "papermill": {
     "duration": 0.0477,
     "end_time": "2022-06-21T12:21:24.808832",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.761132",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "y_train_pred = rf.predict(X_train)\n",
    "y_test_pred = rf.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7583e690",
   "metadata": {
    "papermill": {
     "duration": 0.007757,
     "end_time": "2022-06-21T12:21:24.824713",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.816956",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Let's take a look at the performances of the model on trainin and test set with the accuracy score function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "8bb3f7cd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-21T12:21:24.842511Z",
     "iopub.status.busy": "2022-06-21T12:21:24.842039Z",
     "iopub.status.idle": "2022-06-21T12:21:24.851991Z",
     "shell.execute_reply": "2022-06-21T12:21:24.850462Z"
    },
    "papermill": {
     "duration": 0.022333,
     "end_time": "2022-06-21T12:21:24.855028",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.832695",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random forest train/test accuracies:1.000/0.958\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "rf_train = accuracy_score(y_train, y_train_pred)\n",
    "rf_test = accuracy_score(y_test, y_test_pred)\n",
    "print(f\"Random forest train/test accuracies:{rf_train:.3f}/{rf_test:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf119ae2",
   "metadata": {
    "papermill": {
     "duration": 0.008329,
     "end_time": "2022-06-21T12:21:24.872051",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.863722",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "As you can see, the score on the training set is 100%, and the score on the test set is 95%. This means that the model has the overfitting problem. Note that this random forest model learned the training set so well. So, it simply memorized the outcome. But, the model cannot generalize. To overcome the overfitting problem, we control the complexity of the model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "819e3d38",
   "metadata": {
    "papermill": {
     "duration": 0.008063,
     "end_time": "2022-06-21T12:21:24.888416",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.880353",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Building the Model with Grid Search Technique"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8e64299",
   "metadata": {
    "papermill": {
     "duration": 0.007908,
     "end_time": "2022-06-21T12:21:24.904596",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.896688",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The grid search helps us to improve the performance of a model by finding the optimal combination of hyperparameter values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "4cf83e9f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-21T12:21:24.923970Z",
     "iopub.status.busy": "2022-06-21T12:21:24.922683Z",
     "iopub.status.idle": "2022-06-21T12:26:42.072073Z",
     "shell.execute_reply": "2022-06-21T12:26:42.070730Z"
    },
    "papermill": {
     "duration": 317.169346,
     "end_time": "2022-06-21T12:26:42.082364",
     "exception": false,
     "start_time": "2022-06-21T12:21:24.913018",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'criterion': 'entropy', 'max_depth': 5, 'max_features': 'log2', 'min_samples_leaf': 3, 'n_estimators': 10}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "# Creating an object from RandomForestClassifier class.\n",
    "rf = RandomForestClassifier(random_state=0)\n",
    "# Specifing the values of the parameters.\n",
    "parameters = {\"max_depth\":[5,10,20],\n",
    "              \"n_estimators\":[i for i in range(10,100,10)],\n",
    "              \"min_samples_leaf\":[i for i in range(1,10)],\n",
    "              \"criterion\":[\"gini\",\"entropy\"],\n",
    "              \"max_features\":[\"auto\",\"sqrt\",\"log2\"]}\n",
    "clf = GridSearchCV(rf, parameters, n_jobs=-1)\n",
    "# Building the model.\n",
    "clf.fit(X_train, y_train)\n",
    "# Seeing the best parameters.\n",
    "print(clf.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6a6914f",
   "metadata": {
    "papermill": {
     "duration": 0.008574,
     "end_time": "2022-06-21T12:26:42.099337",
     "exception": false,
     "start_time": "2022-06-21T12:26:42.090763",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Evaluationg the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "fbe593a3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-06-21T12:26:42.119836Z",
     "iopub.status.busy": "2022-06-21T12:26:42.119296Z",
     "iopub.status.idle": "2022-06-21T12:26:42.134072Z",
     "shell.execute_reply": "2022-06-21T12:26:42.132254Z"
    },
    "papermill": {
     "duration": 0.028622,
     "end_time": "2022-06-21T12:26:42.136622",
     "exception": false,
     "start_time": "2022-06-21T12:26:42.108000",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Random forest train/test accuracies:0.993/0.965\n"
     ]
    }
   ],
   "source": [
    "y_train_pred = clf.predict(X_train)\n",
    "y_test_pred = clf.predict(X_test)\n",
    "rf_train = accuracy_score(y_train, y_train_pred)\n",
    "rf_test = accuracy_score(y_test, y_test_pred)\n",
    "print(f\"Random forest train/test accuracies:{rf_train:.3f}/{rf_test:.3f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac6e82bf",
   "metadata": {
    "papermill": {
     "duration": 0.008339,
     "end_time": "2022-06-21T12:26:42.153752",
     "exception": false,
     "start_time": "2022-06-21T12:26:42.145413",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "The performance of the model is better on both the training and test set. Notice that the score of our model on the training set is close to the score on the test set.In addition, both accuracy scores are close to 1."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65a9fc69",
   "metadata": {
    "papermill": {
     "duration": 0.008191,
     "end_time": "2022-06-21T12:26:42.170694",
     "exception": false,
     "start_time": "2022-06-21T12:26:42.162503",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Please don't forget to follow us on [YouTube](http://youtube.com/tirendazacademy) | [Medium](http://tirendazacademy.medium.com) | [Twitter](http://twitter.com/tirendazacademy) | [GitHub](http://github.com/tirendazacademy) | [Linkedin](https://www.linkedin.com/in/tirendaz-academy) | [Kaggle](https://www.kaggle.com/tirendazacademy) 😎"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "556eb6fc",
   "metadata": {
    "papermill": {
     "duration": 0.008585,
     "end_time": "2022-06-21T12:26:42.187832",
     "exception": false,
     "start_time": "2022-06-21T12:26:42.179247",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "![](https://img.freepik.com/free-vector/thank-you-lettering_1262-7412.jpg?t=st=1655813654~exp=1655814254~hmac=86c396cf2062d906540b475205797d297f2e540109d094d3087394e37d847e94&w=996)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 332.654624,
   "end_time": "2022-06-21T12:26:44.820316",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-06-21T12:21:12.165692",
   "version": "2.3.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}