02.Our Dataset.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# (Collect Data), Visualize and Explore\n",
    "* Well, the collection has already been done for us and this dataset is included with `sklearn`\n",
    "* In reality, many datasets will need to go through a preprocessing and exploratory data analysis step.  `sklearn` has many tools for this.\n",
    "\n",
    "<img src='imgs/ml_process_by_micheleenharris.png' alt=\"Smiley face\" width=\"400\"><br>\n",
    "\n",
    "## The Dataset - Fisher's Irises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most machine learning algorithms implemented in scikit-learn expect data to be stored in a\n",
    "**two-dimensional array or matrix**.  The arrays can be\n",
    "either ``numpy`` arrays, or in some cases ``scipy.sparse`` matrices.\n",
    "The size of the array is expected to be `n_samples x n_features`.\n",
    "\n",
    "- **n_samples:**   The number of samples: each sample is an item to process (e.g. classify).\n",
    "  A sample can be a document, a picture, a sound, a video, an astronomical object,\n",
    "  a row in database or CSV file,\n",
    "  or whatever you can describe with a fixed set of quantitative traits.\n",
    "- **n_features:**  The number of features or distinct traits that can be used to describe each\n",
    "  item in a quantitative manner.  Features are generally real-valued, but may be boolean or\n",
    "  discrete-valued in some cases.<br><br>\n",
    "<font color = \"lightgrey\">\n",
    "The number of features must be fixed in advance. However it can be very high dimensional\n",
    "(e.g. millions of features) with most of them being zeros for a given sample. This is a case\n",
    "where `scipy.sparse` matrices can be useful, in that they are\n",
    "much more memory-efficient than numpy arrays.\n",
    "</font><br><br>\n",
    "If there are labels or targets, they need to be stored in **one-dimensional arrays or lists**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Today we are going to use the <b>`iris`</b> dataset which comes with `sklearn`.  It's fairly small as we'll see shortly."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> <b>Remember our ML TIP:  Ask sharp questions.</b><br><br>e.g. What type of flower is this (pictured below) closest to of the three given classes?\n",
    "\n",
    "<img border=\"0\" alt=\"iris species\" src=\"imgs/iris-setosa.jpg\" width=\"200\">\n",
    "<p align=\"right\">from http://www.madlantern.com/photography/wild-iris</p>\n",
    "\n",
    "### Labels (species names/classes):\n",
    "<img border=\"0\" alt=\"iris species\" src=\"imgs/irises.png\" width=\"500\" height=\"500\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color = \"lightgrey\"> TIP: Commonly, machine learning algorithms will require your data to be standardized, normalized or even reguarlized and preprocessed.  In `sklearn` the data must also take on a certain structure.</font>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Imports for python 2/3 compatibility\n",
    "\n",
    "from __future__ import absolute_import, division, print_function, unicode_literals\n",
    "\n",
    "# For python 2, comment these out:\n",
    "# from builtins import range"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "QUICK QUESTION:\n",
    "1.  What do you expect this data set to be if you are trying to recognize an iris species?\n",
    "* For our `[n_samples x n_features]` data array, what do you think\n",
    "    * the samples are?\n",
    "    * the features are?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "print(type(iris.data))\n",
    "print(type(iris.target))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let's Dive In!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Features (aka columns in data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "# converting to dataframe for clearer printing\n",
    "pd.DataFrame({'feature name': iris.feature_names})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Targets (aka labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn import datasets\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "# converting to dataframe for clearer printing\n",
    "pd.DataFrame({'target name': iris.target_names})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> `sklearn` TIP: all included datasets for have at least `feature_names` and sometimes `target_names`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get to know the data - explore\n",
    "* Features (columns/measurements) come from this diagram\n",
    "<img border=\"0\" alt=\"iris data features\" src=\"imgs/iris_petal_sepal.png\" width=\"200\" height=\"200\">\n",
    "* Shape\n",
    "* Peek at data\n",
    "* Summaries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Shape and representation<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn import datasets\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "# How many data points (rows) x how many features (columns)\n",
    "print(iris.data.shape)\n",
    "print(iris.target.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Sneak a peek at data (and a reminder of your `pandas` dataframe methods)<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# convert to pandas df (adding real column names) to use some pandas functions (head, describe...)\n",
    "iris.df = pd.DataFrame(iris.data, \n",
    "                       columns = iris.feature_names)\n",
    "\n",
    "\n",
    "# first few rows\n",
    "iris.df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<b>Describe the dataset with some summary statitsics<b>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# summary stats\n",
    "iris.df.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* We don't have to do much with the `iris` dataset.  It has no missing values.  It's already in numpy arrays and has the correct shape for `sklearn`.  However we could try <b>standardization</b> and/or <b>normalization</b>. (later, in the transforms section, we will show one hot encoding, a preprocessing step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize\n",
    "* There are many ways to visualize and here's a boxplot:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x10f9c58d0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAAFeCAYAAACy1qeuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3X90VOWdx/HPkAkZyAxLAjN0j1LAWMoP0WpigcPhx4Ld\n8iu7ipk2AQKU6NYIdhVpaWg1kJ5SiBXdEmHJqVuKilTs6LHIEesiorSBwGERUiKIEH+tJEggmSQa\nhtz9g+2UlB8ZMjM8k8n79Rdz780z32Qe5jPPnec+12ZZliUAAHDNdTFdAAAAnRUhDACAIYQwAACG\nEMIAABhCCAMAYAghDACAIYQwEKN2796tzMzMkLdHwoEDB1RYWNiu53n88ce1c+fOsGtobGzUvffe\nq+bm5rDbAmIdIQwg6MiRIzpx4sRV/9z+/ft19OhRjRo1KuwaunfvrqlTp+qJJ54Iuy0g1tlNFwB0\nNI2NjSooKNCHH34om82mm266SUVFRZKkN998U2vWrFEgEJDD4dCiRYt0yy23qKSkRIcPH9apU6dU\nU1OjwYMH6+c//7mSk5P15ptvau3atQoEAjp16pT+9V//Vf/+7/8eUi1nz57VL3/5S5WXl6ulpUWD\nBw/WT3/6UyUnJ2v8+PGaNm2a/vznP+t///d/NWnSJP3whz+UJJWWlur3v/+9kpOTlZGRoTfeeEPP\nP/+8Vq1aJb/fr8WLF+vOO+9UQ0ODFixYoA8++EDNzc362c9+pvT09IvqWLVqlXJzc4OPX3zxRa1b\nt04JCQlKSUnR8uXL9eGHH2rlypXyeDw6cuSIunXrpgceeEDPPPOMjh8/rm9961sqKCiQJE2aNEmP\nPfaY7r33XqWmpob7kgGxywJwVV5++WXrnnvusSzLss6dO2c98sgj1ocffmgdP37cmjp1qnX69GnL\nsizryJEj1qhRo6ympiZr1apV1ujRo63PP//csizLWrBggbVixQrLsixr1qxZVlVVlWVZlnXixAlr\nyJAhVm1trbVr1y5r6tSpFz3/hdtLSkqs4uLi4L6VK1daS5cutSzLsv7pn/4p+ByfffaZdfPNN1sf\nf/yxtWPHDmvSpElWfX29ZVmWtXjxYmv8+PGWZVmWz+ezvv/97wefZ+jQoda7775rWZZl/eY3v7Hm\nzJlzUT11dXXWN77xDevs2bOWZVnWoUOHrBEjRlifffaZZVmW9dvf/tYqLCwMtnfo0CHLsizrnnvu\nsbKzs61AIGCdOnXKGjp0qFVdXR1s9wc/+IHl8/lCfl2AjoiRMHCV0tPT9eSTTyo3N1ejRo3S7Nmz\n1bdvX23YsEEnT57UnDlzZP3/arB2u11VVVWSpG9/+9vBUV1WVpZ+8Ytf6Ec/+pHWrFmj7du365VX\nXtEHH3wgSWpqagqplu3bt6u+vj74XWwgEFCvXr2C+ydMmCBJ6tOnj3r16qUzZ85ox44dmjhxopxO\npyRpxowZKisru2T7ffv21bBhwyRJgwcPls/nu+iYqqoqeTwe2e3n307Kyso0evRo9enTR5I0a9Ys\nSee/Y77uuus0aNAgSdJXv/pVuVyu4GjZ6XTqzJkzcrvdwf3Hjh0L6e8AdFSEMHCVrr/+er3++uva\nvXu3ysrKNHv2bD3yyCNqaWnRyJEjtXLlyuCxn376qb7yla/oj3/8YzCkJMmyLCUkJKipqUl33nmn\n/vmf/1kZGRnKysrSG2+8EQzxtpw7d04/+clPNHr0aEnnT5V/+eWXwf0Oh6PV8ZZlyW63t2q/S5fL\nTw25sGabzXbJurp06aJz584FHyckJMhmswUfNzc369NPP5Ukde3a9bLtX+p3+/vjgXjDxCzgKj3/\n/PP68Y9/rFGjRunhhx/W6NGjdeTIEY0YMUI7d+4Mjmbffvtt3XXXXcFZvtu2bZPf71dLS4teeOEF\njR8/XlVVVWpsbNSDDz6ocePGadeuXTp79myrULuS0aNH67nnnlNzc7NaWlr06KOPtjmhaezYsXr9\n9dfl9/slnf/+9q+hmZCQoEAgcFV/j759++rzzz8P/p7Dhw/Xn/70J508eVKStGHDBhUXF19Vm5L0\n8ccf64YbbrjqnwM6EkbCwFW68847VV5ersmTJ6tbt2667rrrNHv2bLlcLhUVFWnBggWSzgfamjVr\ngqPR3r1769/+7d906tQpZWRk6Pvf/74SExM1btw4TZo0SR6PR7fddpuGDh2qDz/8UImJiW3Wcv/9\n96u4uFh33XWXLMvS4MGDtWjRIklqNRq98PGIESPk9XqVnZ0th8Ohr33ta+rWrZsk6dZbb9WTTz6p\nBx54oNVEqytxuVzKyMjQrl27NHr0aA0cOFA/+tGPlJeXJ5vNJrfbrWXLlrV5avnvR8//8z//o2XL\nloVUA9BR2axQz3sBaLeSkhKdPHlSS5YsMV2KDh48qH379gVDdt26dXr33XdbnUa/Wvv27dN//ud/\nau3atRGp8aWXXtL7778fnM0NxKs2T0dblqXFixcrJydHM2fOvOjT7LZt25SVlaXs7Gxt2rQpaoUC\niIz+/ftrz549yszMVGZmpsrKyvTjH/84rDZvvfVW3XDDDXrnnXfCrq+hoUGbN2/WAw88EHZbQKxr\ncyT89ttvy+fz6YknntCf/vQnbdy4Ub/61a8knZ+JOXnyZPl8PiUlJSknJ0elpaVc1wcAQAjaHAkn\nJSWpvr5elmWpvr6+1fdUR48eVb9+/eR0OpWYmKj09HSVl5dHtWAAAOJFmxOz0tPT9eWXX2rixIk6\nffp0q+98/H6/XC5X8HFycrLq6+ujUykAAHGmzZHwr3/9a912223aunWrXnnlFS1atCh4KYLT6Qxe\n5iCd/y6nR48eV2wvEAjt0gsAAOJdmyPhxsbG4Mo6LpdLgUBALS0tkqS0tDRVVVWprq5ODodD5eXl\nysvLu2J7tbWNESi7c3C7Xaqp4cwCIoP+hEijT4XO7XZdcnubE7Pq6upUUFCg2tpanTt3TrNmzZJl\nWWpqapLX69X27dtVUlIiy7KUlZWlnJycKxbCCxY6Ojgiif6ESKNPha7dIRxpvGCho4MjkuhPiDT6\nVOguF8IsWwkAgCGEMAAAhhDCAAAYQggDAGAId1ECAASNGTNclZWHIt7uoEGDtWPHroi329ERwgCA\noKsJSo/HpepqZkeHg9PRAAAYQggDAGAIIQwAaJfCQtMVdHyEMACgXZYsMV1Bx0cIAwBgCCEMAIAh\nhDAAAIYQwgAAGEIIAwDahYlZ4SOEAQDtsnSp6Qo6PkIYAABDCGEAAAwhhAEAMIQQBgDAEEIYANAu\nrB0dPkIYANAuXKIUPkIYAABDCGEAAAwhhAEAMIQQBgDAEEIYANAuTMwKHyEMAGgX1o4OHyEMAIAh\nhDAAAIbY2zrgpZdeks/nk81m05dffqnKykrt3LlTTqdTkrRu3Tq9+OKLSk1NlSQVFRWpf//+US0a\nAIB40GYI33XXXbrrrrsknQ/YrKysYABLUkVFhYqLizVkyJDoVQkAQBwK+XT0gQMH9P7778vr9bba\nXlFRobVr12r69OkqLS2NeIEAgNjE2tHhCzmES0tLNX/+/Iu2T5kyRUuXLtX69eu1d+9evfXWWxEt\nEAAQm7hEKXw2y7Kstg6qr6/X9OnT9Yc//OGifX6/P3h6esOGDTpz5ozy8/Mv21YgcE52e0IYJQMA\nEB/a/E5YksrLyzVixIiLtvv9fmVmZmrLli1yOBwqKytTVlbWFduqrW1sX6WdkNvtUk1NvekyECfo\nT4g0+lTo3G7XJbeHFMLHjh1T3759g483b96spqYmeb1eLVy4ULm5uUpKStLIkSM1ZsyYyFQMAECc\nC+l0dCTxqSl0fMpEJNGfEGn0qdBdbiTMYh0AgHZhYlb4CGEAQLuwdnT4CGEAAAwhhAEAMIQQBgDA\nEEIYAABDCGEAQLuwdnT4CGEAQLtwiVL4CGEAAAwhhAEAMIQQBgDAEEIYAABDCGEAQLswMSt8hDAA\noF1YOzp8hDAAAIYQwgAAGEIIAwBgCCEMAIAhhDAAoF1YOzp8hDAAoF24RCl8hDAAAIYQwgAAGEII\nAwBgCCEMAIAhhDAAoF2YmBU+QhgA0C6sHR0+QhgAAEMIYQAADCGEAQAwhBAGAMAQe1sHvPTSS/L5\nfLLZbPryyy9VWVmpnTt3yul0SpK2bdum1atXy2636+6775bX64160QAA81g7Onw2y7KsUA8uKirS\n4MGDg0EbCAQ0efJk+Xw+JSUlKScnR6WlpUpNTb1sGzU19eFX3Um43S7+XogY+hMijT4VOrfbdcnt\nIZ+OPnDggN5///1WI92jR4+qX79+cjqdSkxMVHp6usrLy8OvFgCATiDkEC4tLdX8+fNbbfP7/XK5\n/pbuycnJqq/nUxEAAKEIKYTr6+t1/PhxffOb32y13el0yu/3Bx83NDSoR48eka0QAIA41ebELEkq\nLy/XiBEjLtqelpamqqoq1dXVyeFwqLy8XHl5eVdsKyWlu+z2hPZV2wld7nsEQJJuuukmVVRURKXt\noUOH6uDBg1FpG/GD96jwhBTCx44dU9++fYOPN2/erKamJnm9XhUUFGju3LmyLEter1cej+eKbdXW\nNoZXcSfCpAe05c03/xzysR6PS9XVV9ef6H+4kqeecmnePPpIKC73YeWqZkdHAv+pQ0cII5LaE8LA\nldCnQhf27GgAABBZhDAAAIYQwkAnwepGQOwhhIFOghuwA7EnpNnRAICOa+BAp06ftkWlbY8nOpco\n9exp6fBhf9sHdnCEMADEudOnbVGZxRzNKziiFe6xhtPRAAAYQggDAGAIIQx0EkzMAmIPIQx0EkuX\nmq4AwN8jhAEAMIQQBgDAEEIYAABDCGEAAAwhhIFOgrWjgdhDCAOdBJcoAbGHEAYAwBBCGAAAQwhh\nAAAMIYQBADCEEAY6CSZmAbGHEAY6CdaOBmIPIQwAgCGEMAAAhhDCAAAYQggDAGAIIQx0EqwdDcQe\nQhjoJLhECYg9hDAAAIYQwgAAGEIIAwBgiD2Ug0pLS7Vt2zYFAgHNnDlTd955Z3DfunXr9OKLLyo1\nNVWSVFRUpP79+0elWAAA4kmbIbx7927t27dPGzduVGNjo55++ulW+ysqKlRcXKwhQ4ZErUgA4Vuy\nRJo3z3QVAC7U5unod955RwMHDtT999+v/Px8jR8/vtX+iooKrV27VtOnT1dpaWnUCgUQHtaOBmJP\nmyPh2tpaffrpp1q7dq0++ugj5efn67XXXgvunzJlimbMmCGn06l58+bprbfe0tixY6NaNAAA8aDN\nEO7Zs6fS0tJkt9s1YMAAJSUl6dSpU8HvgGfPni2n0ylJGjt2rP7yl79cMYRTUrrLbk+IUPnxz+12\nmS4BcYT+1Dkd0E1yeyqi0rY7Kq1KBzRUbvfBKLUeO9oM4fT0dD3zzDOaM2eOTpw4oS+++EIpKSmS\nJL/fr8zMTG3ZskUOh0NlZWXKysq6Ynu1tY2RqbwTcLtdqqmpN10G4gb9qbMapoOqro78ax/N96hh\nHpeq46i/Xu4DcJshPG7cOO3Zs0dZWVmyLEuPPvqoXn31VTU1Ncnr9WrhwoXKzc1VUlKSRo4cqTFj\nxkS8eAAA4pHNsizrWj4hn8RDx0gYkfTUUy7Nm0d/6ow8HleHGwlHq2ZTLjcSZrEOoJNg7Wgg9hDC\nAAAYQggDAGAIIQwAgCGEMAAAhhDCQCfBxCwg9hDCQCfB2tFA7CGEAQAwhBAGAMAQQhgAAEMIYQAA\nDGnzBg4Arq2BA506fdoWlbY9nujcyrBnT0uHD/uj0jYQzwhhIMacPm3rkIvtA7h6nI4GAMAQQhgA\nAEMIYQAADCGEAQAwhIlZANAJRG/yXPRm3HcGhDAAxLlozLaXzgd7tNruLDgdDQCAIYQwAACGEMIA\nABhCCAMAYAghDABol8JC0xV0fIQwAKBdliwxXUHHRwgDAGAIIQwAgCGEMAAAhhDCAAAYQggDANqF\niVnhCymES0tLlZ2draysLL388sut9m3btk1ZWVnKzs7Wpk2bolIkACD2LF1quoKOr80bOOzevVv7\n9u3Txo0b1djYqKeffjq4LxAIaPny5fL5fEpKSlJOTo4mTJig1NTUqBYNAEA8aHMk/M4772jgwIG6\n//77lZ+fr/Hjxwf3HT16VP369ZPT6VRiYqLS09NVXl4e1YIBAIgXbY6Ea2tr9emnn2rt2rX66KOP\nlJ+fr9dee02S5Pf75XL97V6SycnJqq/ntlYAAISizRDu2bOn0tLSZLfbNWDAACUlJenUqVNKTU2V\n0+mU3+8PHtvQ0KAePXpcsb2UlO6y2xPCr7yTcLujdSNuxLJove7R7E/01c6J1z08bYZwenq6nnnm\nGc2ZM0cnTpzQF198oZSUFElSWlqaqqqqVFdXJ4fDofLycuXl5V2xvdraxshU3gm43S7V1HBmofOJ\nzuse3f5EX+2MCgt53UN1uQ8rbYbwuHHjtGfPHmVlZcmyLD366KN69dVX1dTUJK/Xq4KCAs2dO1eW\nZcnr9crj8US8eABA7FmyRKqpMV1Fx2azLMu6lk/Ip6bQMRLunDwel6qrO9ZIOFo1I7bxHhW6y42E\nWawDAABDCGEAAAwhhAEAMIQQBgC0C2tHh4+JWTGMSQ+d0wnPSN2kCtNlXJWDGqo+1X82XQauMSbk\nha7dlygBuLaG6WCHmx09zONStXgzBq4Wp6MBADCEEAYAwBBCGAAAQwhhAEC7FBaarqDjI4QBAO3C\nJUrhI4QBADCES5SuoTFjhquy8lBU2h40aLB27NgVlbYBANFBCF9DVxuSLNYBAPGN09EAABhCCAMA\n2oWJWeEjhGMYHRxALFu61HQFHR8hHMPo4AAQ3whhAAAMIYQBADCEEAYAwBBCGADQLqwdHT5COIbR\nwQHEMq7gCB8hHMPo4AAQ3whhAAAMIYQBADCEEAYAwBDuogTEII/HFaWWo9Nuz55WVNrFtRetW65y\nu9VLs1mWdU3/93BrvtA99ZRL8+bx90JkeDwuVVfTnxA53G41dG73pT8Aczo6hrF2NADEN0IYAABD\nQvpOeNq0aXI6nZKk66+/XsuWLQvuW7dunV588UWlpqZKkoqKitS/f//IVwoAQJxpM4Sbm5slSevX\nr7/k/oqKChUXF2vIkCGRrQwAgDjX5unoyspKNTY2Ki8vT3PmzNH+/ftb7a+oqNDatWs1ffp0lZaW\nRq1QAOFhGVQg9rQ5EnY4HMrLy5PX69Xx48d17733auvWrerS5Xx+T5kyRTNmzJDT6dS8efP01ltv\naezYsVEvvDPgTRORtGSJVFNjugoAF2rzEqXm5mZZlqWkpCRJktfrVUlJifr06SNJ8vv9we+LN2zY\noDNnzig/P/+y7QUC52S3J0SqfgAAOqw2R8I+n0/vvfeeCgsLdeLECTU0NMjtdks6H8CZmZnasmWL\nHA6HysrKlJWVdcX2amsbI1N5J8A1eIgk+hMijT4VustdJ9zmSDgQCGjx4sX65JNPZLPZtHDhQn38\n8cdqamqS1+vVq6++qt/85jdKSkrSyJEjNX/+/CsWwgsWOjo4Ion+hEijT4Wu3SEcabxgoaODI5Lo\nT4g0+lToWDEL6OS4PzUQewjhGMabJiKJZVCB2EMIxzDeNAEgvhHCAAAYQggDAGAIIQwAgCGEMNBJ\nsAwqEHsI4RjGmyYiidn2QOwhhGMYb5oAEN8IYQAADCGEAQAwhBAGAMAQQhjoJJhjAMQeQjiG8aaJ\nSGIZVCD2EMIxjDdNAIhvhDAAAIYQwgAAGEIIAwBgCCEMdBIsgwrEHptlWda1fMKamvpr+XRRN3Cg\nU6dP20yXcVV69rR0+LDfdBm4xtxuV9z9/4NZ9KnQud2uS263X+M64s7p0zZVV0enE0arg3s8l+4M\nAIBri9PRAAAYQggDAGAIIQwAgCGEMNBJsAwqEHsIYaCTYBlUIPYQwgAAGEIIAwBgCCEMAIAhhDAA\nAIaEtGLWtGnT5HQ6JUnXX3+9li1bFty3bds2rV69Wna7XXfffbe8Xm90KgUQFtaOBmJPmyHc3Nws\nSVq/fv1F+wKBgJYvXy6fz6ekpCTl5ORowoQJSk1NjXylAMKyZIlUU2O6CgAXavN0dGVlpRobG5WX\nl6c5c+Zo//79wX1Hjx5Vv3795HQ6lZiYqPT0dJWXl0e1YAAA4kWbI2GHw6G8vDx5vV4dP35c9957\nr7Zu3aouXbrI7/fL5frbzQCSk5NVX88dNQAACEWbIdy/f3/169cv+O+ePXuqpqZGffr0kdPplN//\nt1viNTQ0qEePHldsLyWlu+z2hDDLjh0HdJPcnoqote+OQpsHNFRu98EotIxYd7nbqQHtRZ8KT5sh\n7PP59N5776mwsFAnTpxQQ0OD3O7z0ZCWlqaqqirV1dXJ4XCovLxceXl5V2yvtrYxMpXHiGE62OFu\nZTjM41I19wDtdLj3KyKNPhW6dt9POCsrS4sXL9aMGTNks9m0bNkybdmyRU1NTfJ6vSooKNDcuXNl\nWZa8Xq88Hk/EiwcQviVLpHnzTFcB4EI2y7Ksa/mE8fapyeNxdbiRcDRrRuzidUekMRIO3eVGwizW\nAQCAIYQwAACGEMIAABhCCAMAYEhIa0cDiE1jxgxXZeWhkI+/mosXBg0arB07drWjKgChIoSBDuxq\nQpKZrEDs4XQ0AACGEMIAABhCCAMAYAghDACAIYQwAACGMDs6AjyeaN7KK/Jt9+x5TZcLBwBcBiEc\npmguiM+C+wAQ3zgdDQCAIYQwAACGEMIAABhCCAMAYAghHMMKC01XAACIJkI4hi1ZYroCAEA0EcIA\nABhCCAMAYAghDACAIYQwAACGEMIxjIlZABDfCOEYtnSp6QoAANFECAMAYAghDACAIYQwAACGEMIA\nABhCCMcw1o4GgPgWUgh//vnnGjdunI4dO9Zq+7p16zR16lTNmjVLs2bN0vHjx6NRY6fFJUoAEN/s\nbR0QCARUWFgoh8Nx0b6KigoVFxdryJAhUSkOAIB41uZIeMWKFcrJyZHH47loX0VFhdauXavp06er\ntLQ0KgUCABCvrhjCPp9PvXr10qhRo2RZ1kX7p0yZoqVLl2r9+vXau3ev3nrrragVCgBAvLFZl0rX\n/zdz5kzZbDZJUmVlpQYMGKA1a9aoV69ekiS/3y+n0ylJ2rBhg86cOaP8/PwrPmEgcE52e0Kk6gcA\noMO64nfCzz77bPDfubm5KioqahXAmZmZ2rJlixwOh8rKypSVldXmE9bWNoZZcufx1FMuzZtXb7oM\nxAm326WaGvoTIoc+FTq323XJ7SFfovTXEfHmzZu1adMmOZ1OLVy4ULm5uZo5c6YGDhyoMWPGRKZa\nSGLtaACId1c8HR0NfGoKncfjUnU1fy9EBqMWRBp9KnRhj4QBAEBkEcIAABhCCAMAYAghHMNYOxoA\n4hshHMNYOxoA4hshDACAIYQwAACGEMIAABhCCAMAYAghHMOYmAUA8Y0QjmGsHQ0A8Y0QBgDAkCve\nyhCRNWbMcFVWHrqqn/F4Qjtu0KDB2rFjVzuqAgCYQghfQ1cbktyhBADiG6ejAQAwhBAGAMAQQhgA\nAEMIYQAADCGEAQAwhBAGAMAQQhgAAEMIYQAADCGEAQAwhBAGAMAQQhgAAEMIYQAADCGEAQAwhBAG\nAMAQQhgAAEMIYQAADCGEAQAwJKQQ/vzzzzVu3DgdO3as1fZt27YpKytL2dnZ2rRpU1QKBAAgXtnb\nOiAQCKiwsFAOh+Oi7cuXL5fP51NSUpJycnI0YcIEpaamRq1YAADiSZsj4RUrVignJ0cej6fV9qNH\nj6pfv35yOp1KTExUenq6ysvLo1YoAADx5ooh7PP51KtXL40aNUqWZbXa5/f75XK5go+Tk5NVX18f\nnSoBAIhDVzwd7fP5ZLPZtHPnTlVWVmrRokVas2aNevXqJafTKb/fHzy2oaFBPXr0aPMJ3W5Xm8fg\nb/h7IZLoT4g0+lR4rhjCzz77bPDfubm5KioqUq9evSRJaWlpqqqqUl1dnRwOh8rLy5WXlxfdagEA\niCNtTsz6K5vNJknavHmzmpqa5PV6VVBQoLlz58qyLHm93ou+NwYAAJdns/7+y14AAHBNsFgHAACG\nEMIAABhCCAMAYAghHCMOHz6sPXv2mC4Dcebtt9++6iVlS0pK9Lvf/S5KFSEWXU0/OXnypIqKii67\nv7KyUqtXr45UaXGPiVkxoqSkRL1791Z2drbpUtDJlZSUyO1267vf/a7pUoC4F/IlSmif48ePq6Cg\nQHa7XZZl6Ze//KU2bNigvXv36ty5c/re976nb3zjG/L5fOratauGDh2quro6/cd//IeSkpKUkpKi\nZcuWqbm5WQ899JAsy1Jzc7OWLFmiQYMGaeXKlaqoqFBtba0GDRqkZcuWmf6VEQEPPPCAZs+erYyM\nDB08eFCrVq1S7969VVVVJcuy9OCDD+r2229XZmam+vfvr65du2rGjBlasWKFEhMT5XA49Ktf/Upb\nt27VBx98oIcfflirV6/Wf//3f6ulpUU5OTn6zne+o//6r//Sli1bZLfbdfvtt+vhhx9uVceKFSu0\nd+9e2Ww2TZ06Vbm5uSooKFBtba3OnDmj0tLSVivnoWO4sH8dOHBA3/ve9zR9+nR997vf1X333aeU\nlBSNHTtWt99+u4qKiuR0OpWamqqkpCTNnz9fCxYs0O9+9zv9y7/8i775zW/qvffek81m0+rVq/WX\nv/xFGzdu1MqVK7Vp0yZt3LhRlmVp/Pjxmj9/vp577jm9/vrr+uKLL5SSkqKSkhLZ7Z03ijrvb36N\n7Ny5U7fccot++MMfqry8XG+88YY++eQTPffcc2pubtZ3vvMdPfvss5o2bZrcbreGDRumCRMmaOPG\njXK73XrmmWf01FNPacSIEUpJSVFxcbGOHDmipqYm+f1+/cM//IOefvppWZalKVOmqLq6muu144DX\n65XP51NGRoZ8Pp/GjBmjzz77TD//+c91+vRpzZw5U5s3b1ZDQ4PmzZunQYMGqbi4WJMmTdLs2bO1\nbds21dXVSTp/jf+hQ4f0zjvv6Pe//70CgYAef/xxHT58WFu3btULL7ygLl266Ac/+IG2b98erGH7\n9u365JNgac+5AAAEiklEQVRP9MILLygQCGjGjBkaPny4JGnkyJGaPXu2iT8NIuDC/vXSSy/poYce\n0okTJySdv2veyy+/rISEBE2bNk2PPfaY0tLS9MQTT6i6ulrS39aN8Pv9yszM1E9/+lMtXLhQO3bs\nUO/evWWz2XTq1Cn9+te/1h/+8Ad17dpVK1euVENDg06fPq3f/va3kqS8vDwdOHBAt956q5k/RAwg\nhKPM6/WqtLRUeXl56tGjh77+9a/r4MGDmjVrlizL0rlz5/Txxx8Hjz916pRcLpfcbrckKSMjQ088\n8YQWLVqk48ePKz8/X4mJicrPz5fD4dDJkyf18MMPq3v37mpqalIgEDD1qyKCRo8erccee0xnzpzR\nnj171NLSor1792r//v3BflNbWytJGjBggCTpvvvu05o1azR79mx95Stf0c033xxs79ixY8HHdrtd\nixYt0muvvaZbbrlFXbqcnxpy22236ciRI8GfOXr0qNLT04M/c/PNN+v9999v9ZzomP6+fw0dOjS4\n7/rrr1dCQoIkqbq6WmlpaZLOvxdt2bLlorYGDx4sSfrHf/xHNTc3B7d/9NFHGjhwoLp27SpJWrBg\ngSQpMTFRCxYsULdu3VRdXd3p37OYmBVlb7zxhjIyMrRu3Tp9+9vfls/n0/Dhw7V+/XqtW7dOEydO\n1Fe/+lXZbDa1tLQoNTVVfr9fJ0+elCTt3r1b/fv3165du+R2u/X000/rvvvu08qVK7Vjxw599tln\nevzxx/XQQw+pqanpohttoGOy2WyaOHGilixZom9961u68cYblZmZqfXr12vNmjWaPHmyevbsGTxW\nkl555RXdfffdWr9+vW688Ua98MILwfZuuOEGVVRUSJLOnj2rvLw8DRgwQO+++65aWlpkWZb27NnT\nKlxvvPFG7d27N/gz+/btC+7/a3CjY/r7/nXh6/nX/iSdD9ajR49Kkvbv339Vz9G3b1998MEHOnv2\nrCTpwQcfDJ4NXLlypR555BGdO3eu079nMRKOsmHDhgVvfNHS0qJVq1bplVde0YwZM9TU1KQ77rhD\n3bt310033RQ87fOzn/1M8+fPV5cuXdSjRw8tX75c0vlPks8//7xaWlo0f/58fe1rXwuOfHr37q1b\nbrlF1dXVuu666wz/1oiEu+++W3fccYf++Mc/qlevXnrkkUeUm5urhoYG5eTkyGaztXrDvPnmm/WT\nn/xE3bp1U0JCgoqKirR7925J0qBBgzR69GhlZ2fLsizl5OTo61//uiZOnBjclpGRoTvuuEOVlZWS\npLFjx6qsrEzZ2dk6e/asJk+eHBz1oOP7a/96/fXXtWvXruD2C/vUo48+qsWLFys5OVmJiYnq06dP\nqzYuPPbCf0tSamqq7rnnHs2cOVM2m03jx4/XsGHD1L17d82cOVMpKSkaMmRI8BR3Z8XsaADAJT33\n3HOaPHmyUlJS9OSTT6pr1666//77TZcVVxgJAwAuqXfv3po7d666d+8ul8ulFStWmC4p7jASBgDA\nEGZXAABgCCEMAIAhhDAAAIYQwgAAGEIIAwBgCCEMAIAh/weWZ/IADA/0rAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10f2b77f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import pandas as pd\n",
    "from sklearn import datasets\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "iris.df = pd.DataFrame(iris.data, \n",
    "                       columns = iris.feature_names)\n",
    "\n",
    "iris.df['target'] = iris.target\n",
    "\n",
    "df = iris.df.loc[:, ['sepal length (cm)', 'target']]\n",
    "\n",
    "df['idx'] = list(range(0, 50))*3\n",
    "df = df.pivot(index = 'idx', columns = 'target')\n",
    "df = np.array(df)\n",
    "plt.boxplot(df, labels = iris.target_names)\n",
    "plt.title('sepal length (cm)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Using `pairplot` from `seaborn` is a quick way to see which features separate out our data ()**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import seaborn as sb\n",
    "sb.pairplot(pd.DataFrame(iris.data, columns = iris.feature_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preprocessing (Bonus Material)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<p>What you might have to do before using a learner in `sklearn`:</p>\n",
    "1. Non-numerics transformed to numeric (tip: use applymap() method from `pandas`)\n",
    "* Fill in missing values\n",
    "* Standardization\n",
    "* Normalization\n",
    "* Encoding categorical features (e.g. one-hot encoding or dummy variables)\n",
    "\n",
    "<b>Features should end up in a numpy.ndarray (hence numeric) and labels in a list.</b>\n",
    "\n",
    "Data options:\n",
    "* Use pre-processed [datasets](http://scikit-learn.org/stable/modules/classes.html#module-sklearn.datasets) from scikit-learn\n",
    "* [Create your own](http://scikit-learn.org/stable/datasets/index.html#sample-generators)\n",
    "* Read from a file\n",
    "\n",
    "If you use your own data or \"real-world\" data you will likely have to do some data wrangling and need to leverage `pandas` for some data manipulation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Standardization - make our data look like a standard Gaussian distribution (commonly needed for `sklearn` learners)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> FYI: you'll commonly see the data or feature set (ML word for data without it's labels) represented as a capital <b>X</b> and the targets or labels (if we have them) represented as a lowercase <b>y</b>.  This is because the data is a 2D array or list of lists and the targets are a 1D array or simple list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Standardization aka scaling\n",
    "from sklearn import preprocessing, datasets\n",
    "\n",
    "# make sure we have iris loaded\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "X, y = iris.data, iris.target\n",
    "\n",
    "# scale it to a gaussian distribution\n",
    "X_scaled = preprocessing.scale(X)\n",
    "\n",
    "# how does it look now\n",
    "pd.DataFrame(X_scaled).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# let's just confirm our standardization worked (mean is 0 w/ unit variance)\n",
    "pd.DataFrame(X_scaled).describe()\n",
    "\n",
    "# also could:\n",
    "#print(X_scaled.mean(axis = 0))\n",
    "#print(X_scaled.std(axis = 0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> PRO TIP: To save our standardization and reapply later (say to the test set or some new data), create a transformer object like so:\n",
    "```python\n",
    "scaler = preprocessing.StandardScaler().fit(X_train)\n",
    "# apply to a new dataset (e.g. test set):\n",
    "scaler.transform(X_test)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Normalization - scaling samples <i>individually</i> to have unit norm\n",
    "* This type of scaling is really important if doing some downstream transformations and learning (see sklearn docs [here](http://scikit-learn.org/stable/modules/preprocessing.html#normalization) for more) where similarity of pairs of samples is examined\n",
    "* A basic intro to normalization and the unit vector can be found [here](http://freetext.org/Introduction_to_Linear_Algebra/Basic_Vector_Operations/Normalization/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Standardization aka scaling\n",
    "from sklearn import preprocessing, datasets\n",
    "\n",
    "# make sure we have iris loaded\n",
    "iris = datasets.load_iris()\n",
    "\n",
    "X, y = iris.data, iris.target\n",
    "\n",
    "# scale it to a gaussian distribution\n",
    "X_norm = preprocessing.normalize(X, norm='l1')\n",
    "\n",
    "# how does it look now\n",
    "pd.DataFrame(X_norm).tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# let's just confirm our standardization worked (mean is 0 w/ unit variance)\n",
    "pd.DataFrame(X_norm).describe()\n",
    "\n",
    "# cumulative sum of normalized and original data:\n",
    "#print(pd.DataFrame(X_norm.cumsum().reshape(X.shape)).tail())\n",
    "#print(pd.DataFrame(X).cumsum().tail())\n",
    "\n",
    "# unit norm (convert to unit vectors) - all row sums should be 1 now\n",
    "X_norm.sum(axis = 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> PRO TIP: To save our normalization (like standardization above) and reapply later (say to the test set or some new data), create a transformer object like so:\n",
    "```python\n",
    "normalizer = preprocessing.Normalizer().fit(X_train)\n",
    "# apply to a new dataset (e.g. test set):\n",
    "normalizer.transform(X_test) \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Created by a Microsoft Employee.\n",
    "\t\n",
    "The MIT License (MIT)<br>\n",
    "Copyright (c) 2016 Micheleen Harris"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}