Naive Bayes Implementation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Naive Bayes Classifier Implementation using numpy\n",
    "\n",
    "Naive Bayes is anoother supervised machine laerning algorithm for classification problem. It makes a strong assumption about the data that **each feature is independent of the value of any other feature**. For example, a fruit may be considered to be an apple if it is red, round, and about 10 cm in diameter. A naive Bayes classifier considers each of these features to contribute independently to the probability that this fruit is an apple, regardless of any possible correlations between the color, roundness, and diameter features.\n",
    "\n",
    "In Naive bayes classifier what we are trying to find is the probability that a given data point belogs to a specific class, we are going to have prediction for all the class in our target.\n",
    "\n",
    "\n",
    "![title](figures/bayes-theorem.png)\n",
    "\n",
    "### Our motivation\n",
    "To gain better understand on how the algorithm works"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "digits = datasets.load_digits()\n",
    "X = digits.data\n",
    "y = digits.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p(y|x) = p(x|y) * p(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MultinomialNB:\n",
    "    \n",
    "    def fit(self, X, y):\n",
    "        unique_y, y_counts = np.unique(y, return_counts=True)\n",
    "        self.prob_y = y_counts / len(y)\n",
    "        self.unique_y = unique_y\n",
    "        self.unique_x = np.array([np.unique(x_t) for x_t in X.T])\n",
    "        self.prob_x = self.get_prob_x(X, y)\n",
    "        return self\n",
    "    \n",
    "    def get_prob_x(X, y):\n",
    "        return []\n",
    "    \n",
    "    def predict(self, X):\n",
    "        pass\n",
    "    \n",
    "    def score(self, X, y):\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = MultinomialNB().fit(X, y)\n",
    "model.score(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  5.],\n",
       "       [ 0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.],\n",
       "       [ 0.,  0.,  7.],\n",
       "       [ 0.,  0.,  0.],\n",
       "       [ 0.,  0., 12.],\n",
       "       [ 0.,  0.,  0.],\n",
       "       [ 0.,  0.,  7.],\n",
       "       [ 0.,  0.,  9.],\n",
       "       [ 0.,  0., 11.]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X[:10, :3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y[:2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}