Lemmatization.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Lemmatization.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyPqUPmmp1nbyUoCVZk5yhR2",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/DataMinati/NLP-Legion/blob/main/Lemmatization.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vpcGmPYqemKg"
      },
      "source": [
        "### Downloading packages"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5pBjoCoNdjr3",
        "outputId": "e6904133-10f0-4e09-ce06-5a349de55e9c"
      },
      "source": [
        "nltk.download('punkt')\n",
        "nltk.download('stopwords')\n",
        "nltk.download('wordnet')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n",
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Package stopwords is already up-to-date!\n",
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/wordnet.zip.\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YN89SADeepHC"
      },
      "source": [
        "### Importing necessary libraries"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "evW4SfZuddCx"
      },
      "source": [
        "import nltk\n",
        "from nltk.stem import WordNetLemmatizer\n",
        "from nltk.corpus import stopwords"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PquYfPhNesu2"
      },
      "source": [
        "### Storing a sentence"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "pnQPYIhCdhbY"
      },
      "source": [
        "paragraph = \"\"\"Thank you all so very much. Thank you to the Academy. \n",
        "               Thank you to all of you in this room. I have to congratulate \n",
        "               the other incredible nominees this year. The Revenant was \n",
        "               the product of the tireless efforts of an unbelievable cast\n",
        "               and crew. First off, to my brother in this endeavor, Mr. Tom \n",
        "               Hardy. Tom, your talent on screen can only be surpassed by \n",
        "               your friendship off screen … thank you for creating a t\n",
        "               ranscendent cinematic experience. Thank you to everybody at \n",
        "               Fox and New Regency … my entire team. I have to thank \n",
        "               everyone from the very onset of my career … To my parents; \n",
        "               none of this would be possible without you. And to my \n",
        "               friends, I love you dearly; you know who you are. And lastly,\n",
        "               I just want to say this: Making The Revenant was about\n",
        "               man's relationship to the natural world. A world that we\n",
        "               collectively felt in 2015 as the hottest year in recorded\n",
        "               history. Our production needed to move to the southern\n",
        "               tip of this planet just to be able to find snow. Climate\n",
        "               change is real, it is happening right now. It is the most\n",
        "               urgent threat facing our entire species, and we need to work\n",
        "               collectively together and stop procrastinating. We need to\n",
        "               support leaders around the world who do not speak for the \n",
        "               big polluters, but who speak for all of humanity, for the\n",
        "               indigenous people of the world, for the billions and \n",
        "               billions of underprivileged people out there who would be\n",
        "               most affected by this. For our children’s children, and \n",
        "               for those people out there whose voices have been drowned\n",
        "               out by the politics of greed. I thank you all for this \n",
        "               amazing award tonight. Let us not take this planet for \n",
        "               granted. I do not take tonight for granted. Thank you so very much.\"\"\""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hU-DCnKhewEx"
      },
      "source": [
        "### Making an object of Lemmatizer"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OaadDnYGdvT7"
      },
      "source": [
        "sentences = nltk.sent_tokenize(paragraph)\n",
        "lemmatizer = WordNetLemmatizer()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "up7CHyC1e0Sd"
      },
      "source": [
        "### Lemmatization of the text"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MjI2MdPFdxff"
      },
      "source": [
        "# Lemmatization\n",
        "for i in range(len(sentences)):\n",
        "    words = nltk.word_tokenize(sentences[i])\n",
        "    words = [lemmatizer.lemmatize(word) for word in words if word not in set(stopwords.words('english'))]\n",
        "    sentences[i] = ' '.join(words)      "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0O6nYjkVe31B"
      },
      "source": [
        "### Display the final result"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "X0OuMYIydzqL",
        "outputId": "2583d0ec-72d4-410f-cabf-fa287fc1bcb0"
      },
      "source": [
        "sentences"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['Thank much .',\n",
              " 'Thank Academy .',\n",
              " 'Thank room .',\n",
              " 'I congratulate incredible nominee year .',\n",
              " 'The Revenant product tireless effort unbelievable cast crew .',\n",
              " 'First , brother endeavor , Mr. Tom Hardy .',\n",
              " 'Tom , talent screen surpassed friendship screen … thank creating ranscendent cinematic experience .',\n",
              " 'Thank everybody Fox New Regency … entire team .',\n",
              " 'I thank everyone onset career … To parent ; none would possible without .',\n",
              " 'And friend , I love dearly ; know .',\n",
              " \"And lastly , I want say : Making The Revenant man 's relationship natural world .\",\n",
              " 'A world collectively felt 2015 hottest year recorded history .',\n",
              " 'Our production needed move southern tip planet able find snow .',\n",
              " 'Climate change real , happening right .',\n",
              " 'It urgent threat facing entire specie , need work collectively together stop procrastinating .',\n",
              " 'We need support leader around world speak big polluter , speak humanity , indigenous people world , billion billion underprivileged people would affected .',\n",
              " 'For child ’ child , people whose voice drowned politics greed .',\n",
              " 'I thank amazing award tonight .',\n",
              " 'Let u take planet granted .',\n",
              " 'I take tonight granted .',\n",
              " 'Thank much .']"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lkpby05ieQVC"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}