C3W1_Assignment.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/RLWH/tensorflow-certification-labs/blob/main/C3W1_Assignment.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "juvenile-enforcement",
      "metadata": {
        "id": "juvenile-enforcement"
      },
      "source": [
        "# Week 1: Explore the BBC News archive\n",
        "\n",
        "Welcome! In this assignment you will be working with a variation of the [BBC News Classification Dataset](https://www.kaggle.com/c/learn-ai-bbc/overview), which contains 2225 examples of news articles with their respective categories (labels).\n",
        "\n",
        "Let's get started!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "combined-brooklyn",
      "metadata": {
        "id": "combined-brooklyn",
        "tags": [
          "graded"
        ]
      },
      "outputs": [],
      "source": [
        "import csv\n",
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "from tensorflow.keras.preprocessing.sequence import pad_sequences"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "dependent-power",
      "metadata": {
        "id": "dependent-power"
      },
      "source": [
        "Begin by looking at the structure of the csv that contains the data:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "finite-panic",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "finite-panic",
        "outputId": "0953ca7d-091c-4ab9-b0be-c50d1a50538e"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "First line (header) looks like this:\n",
            "\n",
            "category,text\n",
            "\n",
            "Each data point looks like this:\n",
            "\n",
            "tech,tv future in the hands of viewers with home theatre systems  plasma high-definition tvs  and digital video recorders moving into the living room  the way people watch tv will be radically different in five years  time.  that is according to an expert panel which gathered at the annual consumer electronics show in las vegas to discuss how these new technologies will impact one of our favourite pastimes. with the us leading the trend  programmes and other content will be delivered to viewers via home networks  through cable  satellite  telecoms companies  and broadband service providers to front rooms and portable devices.  one of the most talked-about technologies of ces has been digital and personal video recorders (dvr and pvr). these set-top boxes  like the us s tivo and the uk s sky+ system  allow people to record  store  play  pause and forward wind tv programmes when they want.  essentially  the technology allows for much more personalised tv. they are also being built-in to high-definition tv sets  which are big business in japan and the us  but slower to take off in europe because of the lack of high-definition programming. not only can people forward wind through adverts  they can also forget about abiding by network and channel schedules  putting together their own a-la-carte entertainment. but some us networks and cable and satellite companies are worried about what it means for them in terms of advertising revenues as well as  brand identity  and viewer loyalty to channels. although the us leads in this technology at the moment  it is also a concern that is being raised in europe  particularly with the growing uptake of services like sky+.  what happens here today  we will see in nine months to a years  time in the uk   adam hume  the bbc broadcast s futurologist told the bbc news website. for the likes of the bbc  there are no issues of lost advertising revenue yet. it is a more pressing issue at the moment for commercial uk broadcasters  but brand loyalty is important for everyone.  we will be talking more about content brands rather than network brands   said tim hanlon  from brand communications firm starcom mediavest.  the reality is that with broadband connections  anybody can be the producer of content.  he added:  the challenge now is that it is hard to promote a programme with so much choice.   what this means  said stacey jolna  senior vice president of tv guide tv group  is that the way people find the content they want to watch has to be simplified for tv viewers. it means that networks  in us terms  or channels could take a leaf out of google s book and be the search engine of the future  instead of the scheduler to help people find what they want to watch. this kind of channel model might work for the younger ipod generation which is used to taking control of their gadgets and what they play on them. but it might not suit everyone  the panel recognised. older generations are more comfortable with familiar schedules and channel brands because they know what they are getting. they perhaps do not want so much of the choice put into their hands  mr hanlon suggested.  on the other end  you have the kids just out of diapers who are pushing buttons already - everything is possible and available to them   said mr hanlon.  ultimately  the consumer will tell the market they want.   of the 50 000 new gadgets and technologies being showcased at ces  many of them are about enhancing the tv-watching experience. high-definition tv sets are everywhere and many new models of lcd (liquid crystal display) tvs have been launched with dvr capability built into them  instead of being external boxes. one such example launched at the show is humax s 26-inch lcd tv with an 80-hour tivo dvr and dvd recorder. one of the us s biggest satellite tv companies  directtv  has even launched its own branded dvr at the show with 100-hours of recording capability  instant replay  and a search function. the set can pause and rewind tv for up to 90 hours. and microsoft chief bill gates announced in his pre-show keynote speech a partnership with tivo  called tivotogo  which means people can play recorded programmes on windows pcs and mobile devices. all these reflect the increasing trend of freeing up multimedia so that people can watch what they want  when they want.\n",
            "\n"
          ]
        }
      ],
      "source": [
        "with open(\"./data/bbc-text.csv\", 'r') as csvfile:\n",
        "    print(f\"First line (header) looks like this:\\n\\n{csvfile.readline()}\")\n",
        "    print(f\"Each data point looks like this:\\n\\n{csvfile.readline()}\")     "
      ]
    },
    {
      "cell_type": "markdown",
      "id": "aggregate-calvin",
      "metadata": {
        "id": "aggregate-calvin"
      },
      "source": [
        "As you can see, each data point is composed of the category of the news article followed by a comma and then the actual text of the article."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "rocky-credit",
      "metadata": {
        "id": "rocky-credit"
      },
      "source": [
        "## Removing Stopwords\n",
        "\n",
        "One important step when working with text data is to remove the **stopwords** from it. These are the most common words in the language and they rarely provide useful information for the classification process.\n",
        "\n",
        "Complete the `remove_stopwords` below. This function should receive a string and return another string that excludes all of the stopwords provided."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "permanent-privilege",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "permanent-privilege"
      },
      "outputs": [],
      "source": [
        "# GRADED FUNCTION: remove_stopwords\n",
        "def remove_stopwords(sentence):\n",
        "    \"\"\"\n",
        "    Removes a list of stopwords\n",
        "    \n",
        "    Args:\n",
        "        sentence (string): sentence to remove the stopwords from\n",
        "    \n",
        "    Returns:\n",
        "        sentence (string): lowercase sentence without the stopwords\n",
        "    \"\"\"\n",
        "    # List of stopwords\n",
        "    stopwords = [\"a\", \"about\", \"above\", \"after\", \"again\", \"against\", \"all\", \"am\", \"an\", \"and\", \"any\", \"are\", \"as\", \"at\", \"be\", \"because\", \"been\", \"before\", \"being\", \"below\", \"between\", \"both\", \"but\", \"by\", \"could\", \"did\", \"do\", \"does\", \"doing\", \"down\", \"during\", \"each\", \"few\", \"for\", \"from\", \"further\", \"had\", \"has\", \"have\", \"having\", \"he\", \"he'd\", \"he'll\", \"he's\", \"her\", \"here\", \"here's\", \"hers\", \"herself\", \"him\", \"himself\", \"his\", \"how\", \"how's\", \"i\", \"i'd\", \"i'll\", \"i'm\", \"i've\", \"if\", \"in\", \"into\", \"is\", \"it\", \"it's\", \"its\", \"itself\", \"let's\", \"me\", \"more\", \"most\", \"my\", \"myself\", \"nor\", \"of\", \"on\", \"once\", \"only\", \"or\", \"other\", \"ought\", \"our\", \"ours\", \"ourselves\", \"out\", \"over\", \"own\", \"same\", \"she\", \"she'd\", \"she'll\", \"she's\", \"should\", \"so\", \"some\", \"such\", \"than\", \"that\", \"that's\", \"the\", \"their\", \"theirs\", \"them\", \"themselves\", \"then\", \"there\", \"there's\", \"these\", \"they\", \"they'd\", \"they'll\", \"they're\", \"they've\", \"this\", \"those\", \"through\", \"to\", \"too\", \"under\", \"until\", \"up\", \"very\", \"was\", \"we\", \"we'd\", \"we'll\", \"we're\", \"we've\", \"were\", \"what\", \"what's\", \"when\", \"when's\", \"where\", \"where's\", \"which\", \"while\", \"who\", \"who's\", \"whom\", \"why\", \"why's\", \"with\", \"would\", \"you\", \"you'd\", \"you'll\", \"you're\", \"you've\", \"your\", \"yours\", \"yourself\", \"yourselves\" ]\n",
        "    \n",
        "    # Sentence converted to lowercase-only\n",
        "    sentence = sentence.lower()\n",
        "    \n",
        "    ### START CODE HERE\n",
        "    sentence = sentence.split(\" \")\n",
        "    sentence = [word for word in sentence if word not in stopwords]\n",
        "    sentence = \" \".join(sentence)\n",
        "    ### END CODE HERE\n",
        "    return sentence"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "northern-third",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "northern-third",
        "outputId": "07eaf364-e3c4-417c-d4ba-6056dd43637e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'go store get snack'"
            ]
          },
          "execution_count": 12,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Test your function\n",
        "remove_stopwords(\"I am about to go to the store and get any snack\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "pressed-excellence",
      "metadata": {
        "id": "pressed-excellence"
      },
      "source": [
        "***Expected Output:***\n",
        "```\n",
        "'go store get snack'\n",
        "\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "animal-photography",
      "metadata": {
        "id": "animal-photography"
      },
      "source": [
        "## Reading the raw data\n",
        "\n",
        "Now you need to read the data from the csv file. To do so, complete the `parse_data_from_file` function.\n",
        "\n",
        "A couple of things to note:\n",
        "- You should omit the first line as it contains the headers and not data points.\n",
        "- There is no need to save the data points as numpy arrays, regular lists is fine.\n",
        "- To read from csv files use [`csv.reader`](https://docs.python.org/3/library/csv.html#csv.reader) by passing the appropriate arguments.\n",
        "- `csv.reader` returns an iterable that returns each row in every iteration. So the label can be accessed via row[0] and the text via row[1].\n",
        "- Use the `remove_stopwords` function in each sentence."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "monthly-beginning",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "monthly-beginning"
      },
      "outputs": [],
      "source": [
        "# GRADED FUNCTION: parse_data_from_file\n",
        "def parse_data_from_file(filename):\n",
        "    \"\"\"\n",
        "    Extracts sentences and labels from a CSV file\n",
        "    \n",
        "    Args:\n",
        "        filename (string): path to the CSV file\n",
        "    \n",
        "    Returns:\n",
        "        sentences, labels (list of string, list of string): tuple containing lists of sentences and labels\n",
        "    \"\"\"\n",
        "    sentences = []\n",
        "    labels = []\n",
        "    with open(filename, 'r') as csvfile:\n",
        "        ### START CODE HERE\n",
        "        reader = csv.reader(csvfile, delimiter=\",\")\n",
        "        \n",
        "        next(reader, None)\n",
        "        \n",
        "        for row in reader:\n",
        "            labels.append(row[0])\n",
        "            \n",
        "            sentence = remove_stopwords(row[1])\n",
        "            sentences.append(sentence)\n",
        "        \n",
        "        ### END CODE HERE\n",
        "    return sentences, labels"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "listed-saturn",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "listed-saturn",
        "outputId": "5cc25675-a2de-43bb-e8ca-06095f264c1c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "ORIGINAL DATASET:\n",
            "\n",
            "There are 2225 sentences in the dataset.\n",
            "\n",
            "First sentence has 436 words (after removing stopwords).\n",
            "\n",
            "There are 2225 labels in the dataset.\n",
            "\n",
            "The first 5 labels are ['tech', 'business', 'sport', 'sport', 'entertainment']\n",
            "\n",
            "\n",
            "MINIATURE DATASET:\n",
            "\n",
            "There are 5 sentences in the miniature dataset.\n",
            "\n",
            "First sentence has 436 words (after removing stopwords).\n",
            "\n",
            "There are 5 labels in the miniature dataset.\n",
            "\n",
            "The first 5 labels are ['tech', 'business', 'sport', 'sport', 'entertainment']\n"
          ]
        }
      ],
      "source": [
        "# Test your function\n",
        "\n",
        "# With original dataset\n",
        "sentences, labels = parse_data_from_file(\"./data/bbc-text.csv\")\n",
        "\n",
        "print(\"ORIGINAL DATASET:\\n\")\n",
        "print(f\"There are {len(sentences)} sentences in the dataset.\\n\")\n",
        "print(f\"First sentence has {len(sentences[0].split())} words (after removing stopwords).\\n\")\n",
        "print(f\"There are {len(labels)} labels in the dataset.\\n\")\n",
        "print(f\"The first 5 labels are {labels[:5]}\\n\\n\")\n",
        "\n",
        "# With a miniature version of the dataset that contains only first 5 rows\n",
        "mini_sentences, mini_labels = parse_data_from_file(\"./data/bbc-text-minimal.csv\")\n",
        "\n",
        "print(\"MINIATURE DATASET:\\n\")\n",
        "print(f\"There are {len(mini_sentences)} sentences in the miniature dataset.\\n\")\n",
        "print(f\"First sentence has {len(mini_sentences[0].split())} words (after removing stopwords).\\n\")\n",
        "print(f\"There are {len(mini_labels)} labels in the miniature dataset.\\n\")\n",
        "print(f\"The first 5 labels are {mini_labels[:5]}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "favorite-shanghai",
      "metadata": {
        "id": "favorite-shanghai"
      },
      "source": [
        "***Expected Output:***\n",
        "```\n",
        "ORIGINAL DATASET:\n",
        "\n",
        "There are 2225 sentences in the dataset.\n",
        "\n",
        "First sentence has 436 words (after removing stopwords).\n",
        "\n",
        "There are 2225 labels in the dataset.\n",
        "\n",
        "The first 5 labels are ['tech', 'business', 'sport', 'sport', 'entertainment']\n",
        "\n",
        "\n",
        "MINIATURE DATASET:\n",
        "\n",
        "There are 5 sentences in the miniature dataset.\n",
        "\n",
        "First sentence has 436 words (after removing stopwords).\n",
        "\n",
        "There are 5 labels in the miniature dataset.\n",
        "\n",
        "The first 5 labels are ['tech', 'business', 'sport', 'sport', 'entertainment']\n",
        "\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "narrative-illness",
      "metadata": {
        "id": "narrative-illness"
      },
      "source": [
        "_**Note: The test above should take less than 5 seconds to run. If your implementation runs longer than that, the grader might time out when you submit. Try to improve it and if it still takes a while to execute, you can click the text below for some hints.**_\n",
        "\n",
        "<br>\n",
        "\n",
        "<details>\n",
        "    <summary><font size=\"3\" color=\"darkgreen\"><b><u>Click for hints</u></b></font></summary>\n",
        "    \n",
        "    # Read the CSV file and specify the comma character as the delimiter\n",
        "    reader = csv.reader(None, delimiter=None)\n",
        "        \n",
        "        # skip the first line using the next() function (1 line of code)\n",
        "        None\n",
        "    \n",
        "        # start for loop and iterate over each row of the reader\n",
        "        for row in reader:\n",
        "    \n",
        "            # append the first element of the row to the `labels` list (1 line of code)\n",
        "            None\n",
        "    \n",
        "            # use the remove_stopwords() function on the second element of the \n",
        "            # row then append to the `sentences` list (1 to 3 lines of code)\n",
        "            None\n",
        "</details>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "diverse-basket",
      "metadata": {
        "id": "diverse-basket"
      },
      "source": [
        "## Using the Tokenizer\n",
        "\n",
        "Now it is time to tokenize the sentences of the dataset. \n",
        "\n",
        "Complete the `fit_tokenizer` below. \n",
        "\n",
        "This function should receive the list of sentences as input and return a [Tokenizer](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text/Tokenizer) that has been fitted to those sentences. You should also define the \"Out of Vocabulary\" token as `<OOV>`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "cultural-virginia",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "cultural-virginia"
      },
      "outputs": [],
      "source": [
        "# GRADED FUNCTION: fit_tokenizer\n",
        "def fit_tokenizer(sentences):\n",
        "    \"\"\"\n",
        "    Instantiates the Tokenizer class\n",
        "    \n",
        "    Args:\n",
        "        sentences (list): lower-cased sentences without stopwords\n",
        "    \n",
        "    Returns:\n",
        "        tokenizer (object): an instance of the Tokenizer class containing the word-index dictionary\n",
        "    \"\"\"\n",
        "    ### START CODE HERE\n",
        "    # Instantiate the Tokenizer class by passing in the oov_token argument\n",
        "    tokenizer = Tokenizer(oov_token=\"<OOV>\")\n",
        "    # Fit on the sentences\n",
        "    tokenizer.fit_on_texts(sentences)\n",
        "    \n",
        "    ### END CODE HERE\n",
        "    return tokenizer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "tracked-hostel",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "tracked-hostel",
        "outputId": "e92c8096-bfef-4a29-db89-0556bed13025"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Vocabulary contains 29714 words\n",
            "\n",
            "<OOV> token included in vocabulary\n"
          ]
        }
      ],
      "source": [
        "tokenizer = fit_tokenizer(sentences)\n",
        "word_index = tokenizer.word_index\n",
        "\n",
        "print(f\"Vocabulary contains {len(word_index)} words\\n\")\n",
        "print(\"<OOV> token included in vocabulary\" if \"<OOV>\" in word_index else \"<OOV> token NOT included in vocabulary\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "moderate-pollution",
      "metadata": {
        "id": "moderate-pollution"
      },
      "source": [
        "***Expected Output:***\n",
        "```\n",
        "Vocabulary contains 29714 words\n",
        "\n",
        "<OOV> token included in vocabulary\n",
        "\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "golden-flash",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "golden-flash"
      },
      "outputs": [],
      "source": [
        "# GRADED FUNCTION: get_padded_sequences\n",
        "def get_padded_sequences(tokenizer, sentences):\n",
        "    \"\"\"\n",
        "    Generates an array of token sequences and pads them to the same length\n",
        "    \n",
        "    Args:\n",
        "        tokenizer (object): Tokenizer instance containing the word-index dictionary\n",
        "        sentences (list of string): list of sentences to tokenize and pad\n",
        "    \n",
        "    Returns:\n",
        "        padded_sequences (array of int): tokenized sentences padded to the same length\n",
        "    \"\"\"\n",
        "    \n",
        "    ### START CODE HERE\n",
        "    # Convert sentences to sequences\n",
        "    sequences = tokenizer.texts_to_sequences(sentences)\n",
        "    \n",
        "    # Pad the sequences using the post padding strategy\n",
        "    padded_sequences = pad_sequences(sequences, padding='post')\n",
        "    ### END CODE HERE\n",
        "    \n",
        "    return padded_sequences"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "spanish-entrepreneur",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "spanish-entrepreneur",
        "outputId": "de3ee3bb-6e01-413b-dc21-b1975aa7c954"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "First padded sequence looks like this: \n",
            "\n",
            "[  96  176 1157 ...    0    0    0]\n",
            "\n",
            "Numpy array of all sequences has shape: (2225, 2438)\n",
            "\n",
            "This means there are 2225 sequences in total and each one has a size of 2438\n"
          ]
        }
      ],
      "source": [
        "padded_sequences = get_padded_sequences(tokenizer, sentences)\n",
        "print(f\"First padded sequence looks like this: \\n\\n{padded_sequences[0]}\\n\")\n",
        "print(f\"Numpy array of all sequences has shape: {padded_sequences.shape}\\n\")\n",
        "print(f\"This means there are {padded_sequences.shape[0]} sequences in total and each one has a size of {padded_sequences.shape[1]}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "wired-brief",
      "metadata": {
        "id": "wired-brief"
      },
      "source": [
        "***Expected Output:***\n",
        "```\n",
        "First padded sequence looks like this: \n",
        "\n",
        "[  96  176 1157 ...    0    0    0]\n",
        "\n",
        "Numpy array of all sequences has shape: (2225, 2438)\n",
        "\n",
        "This means there are 2225 sequences in total and each one has a size of 2438\n",
        "\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "unknown-optimization",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "unknown-optimization"
      },
      "outputs": [],
      "source": [
        "# GRADED FUNCTION: tokenize_labels\n",
        "def tokenize_labels(labels):\n",
        "    \"\"\"\n",
        "    Tokenizes the labels\n",
        "    \n",
        "    Args:\n",
        "        labels (list of string): labels to tokenize\n",
        "    \n",
        "    Returns:\n",
        "        label_sequences, label_word_index (list of string, dictionary): tokenized labels and the word-index\n",
        "    \"\"\"\n",
        "    ### START CODE HERE\n",
        "    \n",
        "    # Instantiate the Tokenizer class\n",
        "    # No need to pass additional arguments since you will be tokenizing the labels\n",
        "    label_tokenizer = Tokenizer()\n",
        "    \n",
        "    # Fit the tokenizer to the labels\n",
        "    label_tokenizer.fit_on_texts(labels)\n",
        "    \n",
        "    # Save the word index\n",
        "    label_word_index = label_tokenizer.word_index\n",
        "    \n",
        "    # Save the sequences\n",
        "    label_sequences = label_tokenizer.texts_to_sequences(labels)\n",
        "\n",
        "    ### END CODE HERE\n",
        "    \n",
        "    return label_sequences, label_word_index"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "streaming-conviction",
      "metadata": {
        "tags": [
          "graded"
        ],
        "id": "streaming-conviction",
        "outputId": "ff2347b1-40d6-4f3b-8fbb-f72c20cae141"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Vocabulary of labels looks like this {'sport': 1, 'business': 2, 'politics': 3, 'tech': 4, 'entertainment': 5}\n",
            "\n",
            "First ten sequences [[4], [2], [1], [1], [5], [3], [3], [1], [1], [5]]\n",
            "\n"
          ]
        }
      ],
      "source": [
        "label_sequences, label_word_index = tokenize_labels(labels)\n",
        "print(f\"Vocabulary of labels looks like this {label_word_index}\\n\")\n",
        "print(f\"First ten sequences {label_sequences[:10]}\\n\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "endangered-poultry",
      "metadata": {
        "id": "endangered-poultry"
      },
      "source": [
        "***Expected Output:***\n",
        "```\n",
        "Vocabulary of labels looks like this {'sport': 1, 'business': 2, 'politics': 3, 'tech': 4, 'entertainment': 5}\n",
        "\n",
        "First ten sequences [[4], [2], [1], [1], [5], [3], [3], [1], [1], [5]]\n",
        "\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "cross-chess",
      "metadata": {
        "id": "cross-chess"
      },
      "source": [
        "**Congratulations on finishing this week's assignment!**\n",
        "\n",
        "You have successfully implemented functions to process various text data processing ranging from pre-processing, reading from raw files and tokenizing text.\n",
        "\n",
        "**Keep it up!**"
      ]
    }
  ],
  "metadata": {
    "dlai_version": "1.2.0",
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.8"
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}