00_tensorflow_fundamentals.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "00_tensorflow_fundamentals.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyMMBz9zMPig65LZbKOLg5ON",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/prachuryanath/TF-Learning/blob/main/00_tensorflow_fundamentals.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EEoFzYXsY-Xa"
      },
      "source": [
        "**In this notebook, we're going to cover some of the most fundamental concepts of tensors using Tensorflow**\n",
        "\n",
        "More specifically, we're going to cover:\n",
        "* Introduction to tensors\n",
        "* Getting information from tensors\n",
        "* Manipulating tensors\n",
        "* Tensors & NumPy\n",
        "* Using @tf.function ( a way to speed up your regular Python functions )\n",
        "* Using GPUs with Tensorflow \n",
        "* Exercises to try for yourself !\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EvIuBmzKZ_ZM"
      },
      "source": [
        "# Introduction to Tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0c_9VsaFZAgo",
        "outputId": "e1efc973-c83c-4bff-b6ae-3f022da1e1ea"
      },
      "source": [
        "import tensorflow as tf\n",
        "print(tf.__version__)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.4.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5SpzGBJva1x4",
        "outputId": "fd5201ba-e7c8-4bc1-966f-283d66d24f86"
      },
      "source": [
        "# Create tensors with tf.constant()\n",
        "scalar = tf.constant(7)\n",
        "scalar"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=7>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PFtfpdRFdS4i",
        "outputId": "7ea730f5-5f1e-4d91-d0c7-d387685fe597"
      },
      "source": [
        "# Check the number of dimensions of a tensor ( ndim stands for number of dimensions)\n",
        "scalar.ndim"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RCJmLCMdd261",
        "outputId": "5e7714fc-044a-40fa-a072-61ca42486896"
      },
      "source": [
        "# Create a vector\n",
        "vector = tf.constant([10,10])\n",
        "vector"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([10, 10], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ofvrv3dfd_-C",
        "outputId": "46571256-d354-4290-cf77-faf2c4573cd1"
      },
      "source": [
        "# Check the dimension of our vector\n",
        "vector.ndim"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yPFiNUX8eLLP",
        "outputId": "fa163579-15a6-475c-99f9-63d5677e82be"
      },
      "source": [
        "# Create a matrix ( has more than 1 dimension)\n",
        "matrix = tf.constant([[10,7],\n",
        "                     [7,10]])\n",
        "matrix"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 7, 10]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gCNEqLk5eWkU",
        "outputId": "d44d22d0-86ba-42c3-f5e7-c51e3c9a2041"
      },
      "source": [
        "# Create another matrix\n",
        "another_matrix = tf.constant([[10.,7.],\n",
        "                              [3.,2.],\n",
        "                              [8.,9.]], dtype = tf.float16) # specify the data type with dtype parameter\n",
        "another_matrix"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float16, numpy=\n",
              "array([[10.,  7.],\n",
              "       [ 3.,  2.],\n",
              "       [ 8.,  9.]], dtype=float16)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_U4r0s8ufUnv",
        "outputId": "b8cc1c96-2bdd-436b-8f0d-0c39d6654b5e"
      },
      "source": [
        "# Dimension of another_matrix\n",
        "another_matrix.ndim"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5312dJtPfol-",
        "outputId": "0c9c9af2-999f-4116-a8e6-c0ef4b2e4742"
      },
      "source": [
        "# Let's create a tensor\n",
        "tensor = tf.constant([[[1,2,3],\n",
        "                      [4,5,6]],\n",
        "                     [[7,8,9],\n",
        "                      [10,11,12]],\n",
        "                     [[13,14,15],\n",
        "                      [16,17,18]]])\n",
        "tensor"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2, 3), dtype=int32, numpy=\n",
              "array([[[ 1,  2,  3],\n",
              "        [ 4,  5,  6]],\n",
              "\n",
              "       [[ 7,  8,  9],\n",
              "        [10, 11, 12]],\n",
              "\n",
              "       [[13, 14, 15],\n",
              "        [16, 17, 18]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jmz7QnlJgQME",
        "outputId": "73cdfc5a-ee7d-4529-e901-515fc6465692"
      },
      "source": [
        "# Dimension of tensor\n",
        "tensor.ndim"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2O5nFnTSgqcO"
      },
      "source": [
        "What we've created so far :\n",
        "* Scalar : a single number\n",
        "* Vector : a number with direction \n",
        "* Matrix : a 2-dimensional array of numbers\n",
        "* Tensor : an n-dimensional array of numbers (when n can be any number, a 0-dimensional tensor is a scalar, a 1-dimensional tensor is a vector)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G12caNMbhUUX"
      },
      "source": [
        "## Creating tensors with `tf.Variable`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "bw4nmDBPgaQu",
        "outputId": "f7e6735b-8e3e-4d77-a998-eae5e9c7bc03"
      },
      "source": [
        "# Create the same tensor with tf.Variable() as above\n",
        "changeable_tensor = tf.Variable([10,7])\n",
        "unchangeable_tensor = tf.constant([10,7])\n",
        "changeable_tensor, unchangeable_tensor"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([10,  7], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2,), dtype=int32, numpy=array([10,  7], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 195
        },
        "id": "QL-l0FpYiM5j",
        "outputId": "ac33d749-97aa-4bba-8c10-e8970986a788"
      },
      "source": [
        "# Let's try change one of the elements in our changeable tensor\n",
        "changeable_tensor[0] = 7\n",
        "changeable_tensor"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "TypeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-12-55cb07e37ee9>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Let's try change one of the elements in our changeable tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mchangeable_tensor\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m7\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0mchangeable_tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mTypeError\u001b[0m: 'ResourceVariable' object does not support item assignment"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oEkLu5nxigRb",
        "outputId": "81d4040e-89ca-4416-eb45-9d4681874c69"
      },
      "source": [
        "# How about we try .assign()\n",
        "changeable_tensor[0].assign(7)\n",
        "changeable_tensor"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([7, 7], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 212
        },
        "id": "vT1yP3zYiuv2",
        "outputId": "3f3c908d-5c81-4a13-fcff-49c1346af01e"
      },
      "source": [
        "# Let's try change our unchangeable tensor\n",
        "unchangeable_tensor[0].assign(7)\n",
        "unchangeable_tensor"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "AttributeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-14-e5c33979885b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Let's try change our unchangeable tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0munchangeable_tensor\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0massign\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m7\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      3\u001b[0m \u001b[0munchangeable_tensor\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mAttributeError\u001b[0m: 'tensorflow.python.framework.ops.EagerTensor' object has no attribute 'assign'"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "X-8308yejSFW"
      },
      "source": [
        "### Creating random tensors\n",
        "\n",
        "Random tensors are tensors of some arbitrary size which contains random numbers"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "w_1gpl9Vi5_n",
        "outputId": "6ab84031-b17c-4090-cf3d-159ab999bac1"
      },
      "source": [
        "# Create two random(but the same) tensors\n",
        "random_1 = tf.random.Generator.from_seed(42) # set seed for reproducibility\n",
        "random_1 = random_1.normal(shape =(3,2))\n",
        "random_2 = tf.random.Generator.from_seed(42)\n",
        "random_2 = random_2.normal(shape = (3,2))\n",
        "\n",
        "# Are they equal ?\n",
        "random_1, random_2 , random_1 == random_2"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-0.7565803 , -0.06854702],\n",
              "        [ 0.07595026, -1.2573844 ],\n",
              "        [-0.23193763, -1.8107855 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-0.7565803 , -0.06854702],\n",
              "        [ 0.07595026, -1.2573844 ],\n",
              "        [-0.23193763, -1.8107855 ]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
              " array([[ True,  True],\n",
              "        [ True,  True],\n",
              "        [ True,  True]])>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xAeeSagVTWnP"
      },
      "source": [
        "### Shuffle the order of elements in a tensor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e9AzSyNwTQkN"
      },
      "source": [
        "# Shuffle a tensor (valuable for when you want to shuffle your data so the inherent order doesn't effect learning)\n",
        "not_shuffled = tf.constant([[10,7],\n",
        "                            [3,4],\n",
        "                            [2,5]])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CHXkTzKeWHX9",
        "outputId": "e1f51717-6637-4ecc-fbb7-7a7e804640de"
      },
      "source": [
        "# Shuffle our non-shuffled tensor\n",
        "tf.random.set_seed(42)    # global level random seed \n",
        "tf.random.shuffle(not_shuffled, seed = 42)    # operation level random seed"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4],\n",
              "       [ 2,  5]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "R4tI3JrrXUZ8"
      },
      "source": [
        "### Other ways to make tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "szwgMrk0U5dE",
        "outputId": "8fbaf935-0fe2-410c-dac3-7a27af6cb71d"
      },
      "source": [
        "# Create a tensor of all ones\n",
        "tf.ones([10, 7])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(10, 7), dtype=float32, numpy=\n",
              "array([[1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XEw9HydyXvkb",
        "outputId": "5d0732ee-2c4e-459a-c753-10dca099e783"
      },
      "source": [
        "# Create a tensor of all zero#es\n",
        "tf.zeros(shape = (3,3))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=float32, numpy=\n",
              "array([[0., 0., 0.],\n",
              "       [0., 0., 0.],\n",
              "       [0., 0., 0.]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 28
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-dyQdYMqYgEW"
      },
      "source": [
        "### Turn NumPy arrays into tensors\n",
        "The main difference between NumPy arrays and TensorFlow tensors is that tensors can be run on a GPU(for faster numerical computing)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TGZw-pfKYA5G",
        "outputId": "d504537a-0f07-4db3-d3bf-e804bd983df8"
      },
      "source": [
        "# You can also turn NumPy arrays into tensors\n",
        "import numpy as np\n",
        "numpy_A = np.arange(1,25, dtype = np.int32)   # create a NumPy array between 1 and 25\n",
        "numpy_A\n",
        "\n",
        "# X = tf.constant(some_matrix)    capital for matrix or tensor\n",
        "# y = tf.constant(vector)       non-capital for vector"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "       18, 19, 20, 21, 22, 23, 24], dtype=int32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rIYDvm7gZSGs",
        "outputId": "4648d3ba-e178-4fb3-8412-60f0b1e4fadc"
      },
      "source": [
        "A = tf.constant(numpy_A)\n",
        "B = tf.constant(numpy_A, shape = (2,3,4))\n",
        "A,B"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(24,), dtype=int32, numpy=\n",
              " array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "        18, 19, 20, 21, 22, 23, 24], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2, 3, 4), dtype=int32, numpy=\n",
              " array([[[ 1,  2,  3,  4],\n",
              "         [ 5,  6,  7,  8],\n",
              "         [ 9, 10, 11, 12]],\n",
              " \n",
              "        [[13, 14, 15, 16],\n",
              "         [17, 18, 19, 20],\n",
              "         [21, 22, 23, 24]]], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JiUNi2khooGL"
      },
      "source": [
        "### Getting information from tensors\n",
        "\n",
        "* Shape -> tensor.shape\n",
        "* Rank  -> tensor.ndim\n",
        "* Axis or dimension -> tensor[0], tensor[:,-1]\n",
        "* Size  -> tf.size(tensor)\n",
        " "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "neVgoYc8ZpZN",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "de8ff580-d1f9-4048-f62e-94d64fda70c6"
      },
      "source": [
        "# Create a rank 4 tensor ( 4 dimensions )\n",
        "rank_4_tensor = tf.zeros([2,3,4,5])\n",
        "rank_4_tensor"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3, 4, 5), dtype=float32, numpy=\n",
              "array([[[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KARKvpUMqL9Q",
        "outputId": "d5895736-855d-494a-e283-ab66992a435b"
      },
      "source": [
        "rank_4_tensor.shape, rank_4_tensor.ndim, tf.size(rank_4_tensor)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([2, 3, 4, 5]), 4, <tf.Tensor: shape=(), dtype=int32, numpy=120>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "J9pD4KKvqkqI",
        "outputId": "2bb0e03b-9346-46c1-bbdf-190e2d0d9a4f"
      },
      "source": [
        "# Get various attributes of our tensor\n",
        "print(\"Datatype of every element\", rank_4_tensor.dtype)\n",
        "print(\"Number of dimensions (rank):\", rank_4_tensor.ndim)\n",
        "print(\"Shape of tensor:\", rank_4_tensor.shape)\n",
        "print(\"Elements across the 0 axis:\", rank_4_tensor.shape[0])\n",
        "print(\"Elements across the last axis:\", rank_4_tensor.shape[-1])\n",
        "print(\"Total no of elements in our tensor:\", tf.size(rank_4_tensor))\n",
        "print(\"Total no of elements in our tensor:\", tf.size(rank_4_tensor).numpy())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Datatype of every element <dtype: 'float32'>\n",
            "Number of dimensions (rank): 4\n",
            "Shape of tensor: (2, 3, 4, 5)\n",
            "Elements across the 0 axis: 2\n",
            "Elements across the last axis: 5\n",
            "Total no of elements in our tensor: tf.Tensor(120, shape=(), dtype=int32)\n",
            "Total no of elements in our tensor: 120\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jbNFjGB-xfhs"
      },
      "source": [
        "### Indexing Tensors\n",
        "\n",
        "Tensors can be indexed just like Python lists."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rkLTSMxSrXEP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b788d0fe-2e84-4bbd-aa87-23e6c76a7043"
      },
      "source": [
        "# Get the first 2 elements of each dimension\n",
        "rank_4_tensor[:2,:2,:2,:2]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 2, 2), dtype=float32, numpy=\n",
              "array([[[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CzLa686-yIPn",
        "outputId": "33b12e8b-3b93-4126-ca9c-9b8c9ef5e4cf"
      },
      "source": [
        "# Get the first element from each dimension from each index except from the final one\n",
        "rank_4_tensor[:1,:1,:1,:]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 5), dtype=float32, numpy=array([[[[0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fcf2gCqrynvv",
        "outputId": "acc2d863-0f3c-46b6-877f-b5205c8fdee4"
      },
      "source": [
        "# Create a rank 2 tensor ( 2 dimensions )\n",
        "rank_2_tensor = tf.constant([[10,7],\n",
        "                             [3,4]])\n",
        "rank_2_tensor.shape, rank_2_tensor.ndim"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([2, 2]), 2)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UWUHn7LnzdWR",
        "outputId": "c43a3cd9-1e90-477f-8941-32a7cf3b7670"
      },
      "source": [
        "# Get the last item of each row of our rank 2 tensor\n",
        "rank_2_tensor[:,-1]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([7, 4], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9nml7DIszyFO",
        "outputId": "1956c0fb-8053-4d57-ab81-4277adaa1ca0"
      },
      "source": [
        "# Add in extra dimension to our rank 2 tensor\n",
        "rank_3_tensor = rank_2_tensor[..., tf.newaxis]\n",
        "rank_3_tensor"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3bcfsq2m0JLQ",
        "outputId": "8353e5dd-2dc6-4617-ce7c-e2eef59a2adf"
      },
      "source": [
        "# Alternative to tf.newaxis\n",
        "tf.expand_dims(rank_2_tensor, axis=-1)    # \"-1\" means expand the final axis"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iW4g5kyS0xiI",
        "outputId": "c9ea4ce3-212f-47f8-e109-3d34b1ff77f1"
      },
      "source": [
        "# Expand the 0-axis\n",
        "tf.expand_dims(rank_2_tensor, axis=0) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 2, 2), dtype=int32, numpy=\n",
              "array([[[10,  7],\n",
              "        [ 3,  4]]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JyGYm7HB1suO"
      },
      "source": [
        "### Manipulating tensors ( Tensor operations )\n",
        "\n",
        "**Basic Operations**\n",
        "\n",
        "`+` , `-` , `*` , `/`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NM1-Ok9b14IQ",
        "outputId": "271b783e-9119-4b51-8e9c-f8b2b5e95785"
      },
      "source": [
        "# You can add values to a tensor using the addition operator\n",
        "tensor = tf.constant([[10,7],[3,4]])\n",
        "tensor + 10"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[20, 17],\n",
              "       [13, 14]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oC93OstX3A-E",
        "outputId": "b36084f7-a8f8-49db-ba2f-aea9039b4822"
      },
      "source": [
        "# Original tensor is unchanged\n",
        "tensor"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Vlg60fB73VxJ",
        "outputId": "d07c15fa-51fe-4dde-bdc3-4670861624bd"
      },
      "source": [
        "# Multiplication also works\n",
        "tensor * 10"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[100,  70],\n",
              "       [ 30,  40]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8v7uuUGv3d3Q",
        "outputId": "c7298ebd-d61e-4e4b-bf6c-ef1d0b159712"
      },
      "source": [
        "# Subtraction if you want\n",
        "tensor - 10"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 0, -3],\n",
              "       [-7, -6]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qkdGN3Va3mND",
        "outputId": "54450fb6-e02b-4d31-82f8-b7c44a3dfbef"
      },
      "source": [
        "# We can use the tensorflow built-in function too\n",
        "tf.multiply(tensor,10)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[100,  70],\n",
              "       [ 30,  40]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M1x6Y_Lt4OWy"
      },
      "source": [
        "### **Matrix Multiplication**\n",
        "\n",
        "In machine learning, matrix multiplication is one of the most common tensor operations.\n",
        "\n",
        "There are two rules our tensors(or matrices) need to fulfil if we're going to matrix multiply them :\n",
        "\n",
        "1. The inner dimensions must match.\n",
        "2. The resulting matrix has the shape of the outer dimensions."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZfFJIaZh3ztR",
        "outputId": "da4ddb98-990f-4d8d-d715-0bcd951661f9"
      },
      "source": [
        "# Matrix multiplication in tensorflow\n",
        "print(tensor)\n",
        "tf.matmul(tensor, tensor)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tf.Tensor(\n",
            "[[10  7]\n",
            " [ 3  4]], shape=(2, 2), dtype=int32)\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[121,  98],\n",
              "       [ 42,  37]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "deMiXEXY50SU",
        "outputId": "2a0dec30-af6c-4f03-a802-65c373c14952"
      },
      "source": [
        "# Matrix multiplication with Python operator \"@\"\n",
        "tensor @ tensor"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[121,  98],\n",
              "       [ 42,  37]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 47
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BoEI_SFt6Jiq",
        "outputId": "b770c54a-d06e-4bf3-f8d7-d13ad95f7ab7"
      },
      "source": [
        "# Create a tenspr (3,2) tensor\n",
        "X = tf.constant([[1,2],\n",
        "                 [3,4],\n",
        "                 [5,6]])\n",
        "Y = tf.constant([[7,8],\n",
        "                 [9,10],\n",
        "                 [11,12]])\n",
        "X, Y"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>, <tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[ 7,  8],\n",
              "        [ 9, 10],\n",
              "        [11, 12]], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 253
        },
        "id": "A1YYKDSN6myL",
        "outputId": "e2864618-218b-4bcf-802c-31c3a148efc0"
      },
      "source": [
        "# Try to matrix multiply tensors of same shape\n",
        "tf.matmul(X, Y)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "InvalidArgumentError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mInvalidArgumentError\u001b[0m                      Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-49-32b138ac07cc>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Try to matrix multiply tensors of same shape\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mtf\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmatmul\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mY\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py\u001b[0m in \u001b[0;36mwrapper\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    199\u001b[0m     \u001b[0;34m\"\"\"Call target, and fall back on dispatchers if there is a TypeError.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    200\u001b[0m     \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 201\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mtarget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    202\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mTypeError\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    203\u001b[0m       \u001b[0;31m# Note: convert_to_eager_tensor currently raises a ValueError, not a\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/math_ops.py\u001b[0m in \u001b[0;36mmatmul\u001b[0;34m(a, b, transpose_a, transpose_b, adjoint_a, adjoint_b, a_is_sparse, b_is_sparse, name)\u001b[0m\n\u001b[1;32m   3313\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3314\u001b[0m       return gen_math_ops.mat_mul(\n\u001b[0;32m-> 3315\u001b[0;31m           a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)\n\u001b[0m\u001b[1;32m   3316\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   3317\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/gen_math_ops.py\u001b[0m in \u001b[0;36mmat_mul\u001b[0;34m(a, b, transpose_a, transpose_b, name)\u001b[0m\n\u001b[1;32m   5530\u001b[0m       \u001b[0;32mreturn\u001b[0m \u001b[0m_result\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5531\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0m_core\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 5532\u001b[0;31m       \u001b[0m_ops\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from_not_ok_status\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   5533\u001b[0m     \u001b[0;32mexcept\u001b[0m \u001b[0m_core\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_FallbackException\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   5534\u001b[0m       \u001b[0;32mpass\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py\u001b[0m in \u001b[0;36mraise_from_not_ok_status\u001b[0;34m(e, name)\u001b[0m\n\u001b[1;32m   6860\u001b[0m   \u001b[0mmessage\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmessage\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0;34m\" name: \"\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m \u001b[0;32melse\u001b[0m \u001b[0;34m\"\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6861\u001b[0m   \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 6862\u001b[0;31m   \u001b[0msix\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mraise_from\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_status_to_exception\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0me\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmessage\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   6863\u001b[0m   \u001b[0;31m# pylint: enable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   6864\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.7/dist-packages/six.py\u001b[0m in \u001b[0;36mraise_from\u001b[0;34m(value, from_value)\u001b[0m\n",
            "\u001b[0;31mInvalidArgumentError\u001b[0m: Matrix size-incompatible: In[0]: [3,2], In[1]: [3,2] [Op:MatMul]"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_STLsJQP7WZG",
        "outputId": "4cd7aa25-ba35-480c-cbad-8e8f179a537c"
      },
      "source": [
        "# Let's change the shape of Y\n",
        "tf.reshape(Y, shape=(2,3))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              "array([[ 7,  8,  9],\n",
              "       [10, 11, 12]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yqFPaM03-UlD",
        "outputId": "4d03673f-dd55-4a99-fc5e-53c37900913e"
      },
      "source": [
        "# Try to matrix multiply X by reshaped Y\n",
        "X @ tf.reshape(Y, shape=(2,3))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lqnFX8BP-1vu",
        "outputId": "11aed8d8-5134-4b5e-ce8a-f71e6e5d1e8d"
      },
      "source": [
        "# Try change the shape of X instead of Y\n",
        "tf.matmul(tf.reshape(X, shape=(2,3)),Y)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 58,  64],\n",
              "       [139, 154]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eGt06q6E_m5c",
        "outputId": "7136426b-7204-4fc8-d42a-d00104d9aa61"
      },
      "source": [
        "# Can do the same with transpose\n",
        "X, tf.transpose(X), tf.reshape(X, shape =(2,3))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>, <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 3, 5],\n",
              "        [2, 4, 6]], dtype=int32)>, <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 2, 3],\n",
              "        [4, 5, 6]], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KNjQzYlNAMJ4",
        "outputId": "6c3f240a-12e3-487d-98bb-96af22a7707c"
      },
      "source": [
        "# Try matrix multiplication with transpose rather than reshape\n",
        "tf.matmul(tf.transpose(X), Y)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 89,  98],\n",
              "       [116, 128]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hj7c4GXsAzyy"
      },
      "source": [
        "**The dot product**\n",
        "\n",
        "Matrix multiplication is also referred to as the dot product.\n",
        "\n",
        "You can perform matrix multiplication using :\n",
        "* `tf.matmul()`\n",
        "* `tf.tensordot()`\n",
        "* `@`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XM2FM1B7AiPv",
        "outputId": "9ab199d3-c4f5-479d-faae-7d4384ee4b08"
      },
      "source": [
        "# Perform the dot product on X and Y (requires X or Y to be transposed)\n",
        "tf.tensordot(tf.transpose(X), Y, axes=1)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 89,  98],\n",
              "       [116, 128]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h7GinmtPBnLB",
        "outputId": "4012c952-b106-4cf2-c69a-2c4226ce1242"
      },
      "source": [
        "# Perform the matrix between X and Y(transposed)\n",
        "tf.matmul(X, tf.transpose(Y)) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 23,  29,  35],\n",
              "       [ 53,  67,  81],\n",
              "       [ 83, 105, 127]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "sBmxAOrSB5tW",
        "outputId": "6c681720-7900-47da-a5b0-1f8f6a9f0f02"
      },
      "source": [
        "# Perform the matrix between X and Y(reshaped)\n",
        "tf.matmul(X, tf.reshape(Y, shape=(2,3))) "
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KtnhroCmCDzZ",
        "outputId": "e245b214-dce2-4ccd-941b-c1cc23118ffa"
      },
      "source": [
        "# Check the values of Y, reshape Y and transposed Y\n",
        "print(\"Normal Y:\")\n",
        "print(Y, \"\\n\")\n",
        "\n",
        "print(\"Y reshaped to (2,3):\")\n",
        "print(tf.reshape(Y, shape=(2,3)), \"\\n\")\n",
        "\n",
        "print(\"Y transposed:\")\n",
        "print(tf.transpose(Y))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Normal Y:\n",
            "tf.Tensor(\n",
            "[[ 7  8]\n",
            " [ 9 10]\n",
            " [11 12]], shape=(3, 2), dtype=int32) \n",
            "\n",
            "Y reshaped to (2,3):\n",
            "tf.Tensor(\n",
            "[[ 7  8  9]\n",
            " [10 11 12]], shape=(2, 3), dtype=int32) \n",
            "\n",
            "Y transposed:\n",
            "tf.Tensor(\n",
            "[[ 7  9 11]\n",
            " [ 8 10 12]], shape=(2, 3), dtype=int32)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7wZBIpXMDg3P"
      },
      "source": [
        "### Changing the datatype of a tensor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "843FVuoGCu3M",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "093d6b07-9822-4b24-81c4-4976bcf209f8"
      },
      "source": [
        "# Create a new tensor with default datatype (float32)\n",
        "B = tf.constant([1.7,2.4])\n",
        "B, B.dtype"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float32, numpy=array([1.7, 2.4], dtype=float32)>,\n",
              " tf.float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 59
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i8ksjdyXH6TH",
        "outputId": "e4ecf5e1-54b9-403e-d214-0a6730d1cc38"
      },
      "source": [
        "# Change from float32 to float16 ( reduced precision )\n",
        "D = tf.cast(B, dtype = tf.float16)\n",
        "D, D.dtype"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float16, numpy=array([1.7, 2.4], dtype=float16)>,\n",
              " tf.float16)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "uYKo4R0jLPzB"
      },
      "source": [
        "### Aggregating Tensors\n",
        "\n",
        "Aggregating tensors = condensing them from multiple values down to a smaller amount of values."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VjRBZ3geIROO",
        "outputId": "cc136c58-4ca7-4f44-9ea8-1d7d84812e9f"
      },
      "source": [
        "# Get the absolute values\n",
        "D = tf.constant([-7,-10])\n",
        "D, tf.abs(D)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ -7, -10], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2,), dtype=int32, numpy=array([ 7, 10], dtype=int32)>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QuhtOKJBMdwB"
      },
      "source": [
        "Let's go through the following forms of aggregation :\n",
        "\n",
        "* Get the minimum\n",
        "* Get the maximum\n",
        "* Get the mean of a tensor\n",
        "* Get the sum of a tensor"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Q21w0uFWMSC5",
        "outputId": "2213be99-f5bd-4573-e2f8-7f2ef512ad29"
      },
      "source": [
        "# Create a random tensor with values between 0 and 100 of size 50\n",
        "E = tf.constant(np.random.randint(0,100, size = 50))\n",
        "E"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=int64, numpy=\n",
              "array([42, 68, 30, 53, 28, 97, 37, 62, 23, 97,  3, 48, 16, 13, 44, 18, 90,\n",
              "       60, 55,  7, 30,  9, 95,  9, 54, 81,  7, 78, 89, 45, 83, 16, 54, 46,\n",
              "       66, 56, 89, 13, 54, 38,  4, 79, 38, 54, 69,  4, 53, 16, 79, 84])>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 62
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Kg_iB3_sNCQs",
        "outputId": "e2c3daeb-75d9-4503-a998-2e6571f0f773"
      },
      "source": [
        "tf.size(E), E.shape, E.ndim"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=int32, numpy=50>, TensorShape([50]), 1)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "vBcKEzcENKao",
        "outputId": "ce97f314-66f4-4403-a025-aab5ebc27734"
      },
      "source": [
        "# Find the minimum\n",
        "tf.reduce_min(E)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=3>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 64
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FwViIswANXkd",
        "outputId": "edbc9587-1f72-42a9-db25-f097a02f6661"
      },
      "source": [
        "# Find the maximum\n",
        "tf.reduce_max(E)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=97>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6aueRkBkNi4o",
        "outputId": "4cfad47b-54fb-4545-e943-c17e16846df5"
      },
      "source": [
        "# Find the mean\n",
        "tf.reduce_mean(E)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=47>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TDCTJ6EdNl6e",
        "outputId": "1cd94a38-d53a-450d-9f6f-02ca1eee9792"
      },
      "source": [
        "# Find the sum\n",
        "tf.reduce_sum(E)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=2383>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LZn0LxuuNpRO",
        "outputId": "56bfb25c-c81d-4991-8ea5-5681335ea532"
      },
      "source": [
        "# To find the variance, we need access to tensorflow_probability \n",
        "import tensorflow_probability as tfp\n",
        "tfp.stats.variance(E)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=823>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 68
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8KS4ewyBQdea",
        "outputId": "5118fde9-577d-4c84-8aa1-a9ba2c0c9482"
      },
      "source": [
        "tf.math.reduce_variance(tf.cast(E, dtype = tf.float16))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float16, numpy=823.5>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iR9VsWVuN1qd",
        "outputId": "a7d9077b-8999-4620-fb6d-7bc565e05ad8"
      },
      "source": [
        "# To find standard variance\n",
        "tf.math.reduce_std(tf.cast(E, dtype = tf.float32))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=28.690495>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 69
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H5pO0c5dPymV"
      },
      "source": [
        "### Find the positional maximum and minimum"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ipjUrBahOhFk",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9f1327e2-4081-43b1-ef59-364696ec7fe6"
      },
      "source": [
        "# Create a new tensor for finding positional minimum and maximum\n",
        "tf.random.set_seed(42)\n",
        "F = tf.random.uniform(shape=[50])\n",
        "F"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              "array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "       0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "       0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "       0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "       0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "       0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "       0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "       0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "       0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "       0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 76
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "S1C_eChPTOUU",
        "outputId": "88f92e30-c6a0-4d19-df44-7d4a4df616ae"
      },
      "source": [
        "# Find the positional maximum\n",
        "tf.argmax(F)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=42>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 77
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "N9x2rJHDTffz",
        "outputId": "4a4db0bc-085e-4f91-b050-4eb9e41d9349"
      },
      "source": [
        "# Index on our largest value position\n",
        "F[tf.argmax(F)]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZLULTIqTTo5M",
        "outputId": "fec96daa-3328-4c69-ded6-b2602447fbd1"
      },
      "source": [
        "# Find the max value of F\n",
        "tf.reduce_max(F)\n",
        "F[tf.argmax(F)] == tf.reduce_max(F)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=bool, numpy=True>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 82
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zCutCMUfUHm5",
        "outputId": "71e0ffbd-5f1a-4f85-8ec9-fa047685a620"
      },
      "source": [
        "# Find the positional minimum\n",
        "tf.argmin(F)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=16>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 83
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RcrhT2hkUif3",
        "outputId": "0be16d8a-9ad2-4f7f-da56-cdd04dc26888"
      },
      "source": [
        "# Find the minimum using the positional minimum index\n",
        "F[tf.argmin(F)]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 84
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FPsSFHttUuP5"
      },
      "source": [
        "### Squeezing a tensor ( removing all single dimensions )"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Y-soxQcRU6Jm",
        "outputId": "24f0375b-c471-40bf-a42f-855c2cbcbd12"
      },
      "source": [
        "# Create a tensor to get started\n",
        "tf.random.set_seed(42)\n",
        "G = tf.constant(tf.random.uniform(shape=[50]), shape = (1,1,1,1,50))\n",
        "G"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 1, 50), dtype=float32, numpy=\n",
              "array([[[[[0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "           0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "           0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "           0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "           0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "           0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "           0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "           0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "           0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "           0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043]]]]],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 86
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VfJEuZbNVdUB",
        "outputId": "a4761609-dd39-4201-86e7-6dc1f17d3e69"
      },
      "source": [
        "G.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([1, 1, 1, 1, 50])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 87
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nS5zXk-rWaaK",
        "outputId": "2c2d93b5-ca5b-4730-e1cd-beb9246721d9"
      },
      "source": [
        "G_squeezed = tf.squeeze(G)\n",
        "G_squeezed, G_squeezed.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              " array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "        0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "        0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "        0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "        0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "        0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "        0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "        0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "        0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "        0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "       dtype=float32)>, TensorShape([50]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 88
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N5H0DtqrW0os"
      },
      "source": [
        "### One-hot encoding tensors"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3ZRKcqKRXPb6",
        "outputId": "770a546b-75bb-4830-dc62-09bdffb6fc67"
      },
      "source": [
        "# Create a list of indices\n",
        "some_list = [0,1,2,3]   # could be red, green, blue, purple\n",
        "\n",
        "# One hot encode our list of indices\n",
        "tf.one_hot(some_list, depth=4)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=float32, numpy=\n",
              "array([[1., 0., 0., 0.],\n",
              "       [0., 1., 0., 0.],\n",
              "       [0., 0., 1., 0.],\n",
              "       [0., 0., 0., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 89
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "M6jji81fXxEK",
        "outputId": "cb4fa05a-0570-4603-df09-5da4de6ede5e"
      },
      "source": [
        "# Specify custom values for one hot encoding\n",
        "tf.one_hot(some_list, depth = 4, on_value=\"yo I love deep learning\", off_value=\"I also like to dance\")"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=string, numpy=\n",
              "array([[b'yo I love deep learning', b'I also like to dance',\n",
              "        b'I also like to dance', b'I also like to dance'],\n",
              "       [b'I also like to dance', b'yo I love deep learning',\n",
              "        b'I also like to dance', b'I also like to dance'],\n",
              "       [b'I also like to dance', b'I also like to dance',\n",
              "        b'yo I love deep learning', b'I also like to dance'],\n",
              "       [b'I also like to dance', b'I also like to dance',\n",
              "        b'I also like to dance', b'yo I love deep learning']],\n",
              "      dtype=object)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 91
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zrE3bBSWYXAj"
      },
      "source": [
        "### Squaring, log, square root"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i6OTsRCvZEps",
        "outputId": "c2860cc0-64c0-43c9-b1c6-6c1dc7092f0e"
      },
      "source": [
        "# Create a new tensor\n",
        "H = tf.range(1, 10)\n",
        "H"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 92
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lI85rRYYZLTV",
        "outputId": "5fbf41bb-8724-476a-a3e3-2bc21957cc31"
      },
      "source": [
        "# Square it\n",
        "tf.square(H)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([ 1,  4,  9, 16, 25, 36, 49, 64, 81], dtype=int32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 93
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v07k1Ef1ZN8X",
        "outputId": "616f3e50-5763-4fb7-9b09-7d3aad4fbe82"
      },
      "source": [
        "# Find the squareroot\n",
        "tf.sqrt(tf.cast(H, dtype=tf.float32))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
              "array([0.99999994, 1.4142134 , 1.7320508 , 1.9999999 , 2.236068  ,\n",
              "       2.4494896 , 2.6457512 , 2.8284268 , 3.        ], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 95
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "I9KV-XgZZZRK",
        "outputId": "b0f00e8a-fc1d-4cc2-9d17-7c25b914074e"
      },
      "source": [
        "# Find the log\n",
        "tf.math.log(tf.cast(H, dtype=tf.float32))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
              "array([0.       , 0.6931472, 1.0986123, 1.3862944, 1.609438 , 1.7917595,\n",
              "       1.9459102, 2.0794415, 2.1972246], dtype=float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 96
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4btv__jJZtDo"
      },
      "source": [
        "### Tensors and NumPy\n",
        "\n",
        "TensorFlow interacts beautifully with NumPy arrays."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CUHhqkDnZ8wr",
        "outputId": "d1a6027d-0735-4c20-802c-d84059d805d8"
      },
      "source": [
        "# Create a tensor directly from a NumPy array\n",
        "J = tf.constant(np.array([3.,7.,10.]))\n",
        "J"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([ 3.,  7., 10.])>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 98
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OjoJlLcwafV3",
        "outputId": "ade36d31-0387-42d2-fb1e-4a54ba335d1f"
      },
      "source": [
        "# Convert our tensor back to a NumPy array\n",
        "np.array(J), type(np.array(J))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 3.,  7., 10.]), numpy.ndarray)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 99
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iysyRseaan9r",
        "outputId": "0acb64c6-052d-43aa-bc2f-4078f06306f3"
      },
      "source": [
        "# Convert tensor J to a NumPy array\n",
        "J.numpy(), type(J.numpy())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 3.,  7., 10.]), numpy.ndarray)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 100
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b2_g_rGtavwx",
        "outputId": "f235f20e-d62b-4d3f-9ebb-e9fd2db74aa2"
      },
      "source": [
        "# The default types of each are slightly different\n",
        "numpy_J = tf.constant(np.array([3.,7.,10.]))\n",
        "tensor_J = tf.constant([3.,7.,10.])\n",
        "# Check the datatypes of each\n",
        "numpy_J.dtype, tensor_J.dtype"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tf.float64, tf.float32)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 103
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xY_7nbpja4_5"
      },
      "source": [
        "1+2"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bjBrJ5z9hdxt"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}