w2/for-loops_and_list_comprehensions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8bcf6d2f-ce7a-48c2-93db-82cf216d5e31",
   "metadata": {},
   "source": [
    "Before we start, let's import the library we need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "6ad9309e-b542-4726-845f-2af72b3c941e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ddd2e9b-52e5-4ae1-9abd-9c377f698ea6",
   "metadata": {},
   "source": [
    "# Definition: for-loops\n",
    "The for-loop is a simple concept. Over a range of numbers or items, a for-loop will go through each number or item sequentially and terminates when it reaches the end of the range. In python, the for-loop is initialised with a `for <item> in <range>:`. The code snippet below provides a few examples.\n",
    "\n",
    "**Note**: Unlike in many other programming languages, we want to avoid using for-loops in Python as they may be excruciatingly slow here. In future exercises, we will encounter more concrete scenarios where using a for-loop may be the easiest solution, and we will learn how to efficiently *vectorise* our code to avoid using unnecessary for-loops.\n",
    "\n",
    "In situations such as numerical intergration over time or over dimensions, a for-loop may be unavoidable. For unavoidable for-loops that are in critical parts of the code, you may want to use a *just-in-time compiler* such as [numba](https://numba.pydata.org/). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7e2665a6-ad64-4d96-9d18-06670e06c399",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "2\n",
      "3\n",
      "4\n",
      "5\n",
      "6\n",
      "7\n",
      "8\n",
      "9\n",
      "\n",
      "============\n",
      "\n",
      "1\n",
      "2\n",
      "3\n",
      "a\n",
      "dog\n",
      "item\n",
      "10.0\n",
      "55.5\n",
      "1e-06\n",
      "<function sample_func at 0x7ff85c6b95f0>\n",
      "[[ 0  1  2  3  4]\n",
      " [ 5  6  7  8  9]\n",
      " [10 11 12 13 14]\n",
      " [15 16 17 18 19]\n",
      " [20 21 22 23 24]]\n",
      "\n",
      "============\n",
      "\n",
      "index = 0, item = 1\n",
      "index = 1, item = 2\n",
      "index = 2, item = 3\n",
      "index = 3, item = a\n",
      "index = 4, item = dog\n",
      "index = 5, item = item\n",
      "index = 6, item = 10.0\n",
      "index = 7, item = 55.5\n",
      "index = 8, item = 1e-06\n",
      "index = 9, item = <function sample_func at 0x7ff85c6b95f0>\n",
      "index = 10, item = [[ 0  1  2  3  4]\n",
      " [ 5  6  7  8  9]\n",
      " [10 11 12 13 14]\n",
      " [15 16 17 18 19]\n",
      " [20 21 22 23 24]]\n"
     ]
    }
   ],
   "source": [
    "# A basic for-loop loops over a given range:\n",
    "r = 10 # we want to count 0,1,2,...,9\n",
    "for number in range(r):\n",
    "    print(number)\n",
    "    \n",
    "print(\"\\n============\\n\")\n",
    "\n",
    "# A for-loop works for a collection of items in a list too:\n",
    "def sample_func(): return None # initialise an arbitrary function\n",
    "array = np.arange(25).reshape(5,5) # and an array\n",
    "lst = [1,2,3,'a','dog','item',10.0,55.5,1e-6,sample_func,array]\n",
    "\n",
    "for item in lst:\n",
    "    print(item)\n",
    "\n",
    "print(\"\\n============\\n\")\n",
    "\n",
    "# The enumerate function provides a pythonic way of looping over items and keeping count of the loop:\n",
    "for index, item in enumerate(lst):\n",
    "    print(\"index = %i, item = %s\" %(index, str(item)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9764cf7-1967-40d5-8a50-c32890308840",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5aea3e10-fd5d-4654-a60e-f272be1db149",
   "metadata": {},
   "source": [
    "# List comprehensions\n",
    "A powerful feature of Python is *list comprehension*. In the last tutorial, we were introduced to the basics of lists in Python. Now, we see how we can make use of list comprehensions to write compact code.\n",
    "\n",
    "A common operation in Python is zipping. Say you have two lists, A and B, and you want to create a third list, C that has the first element of A with the first element of B, second element of A with the second element of B and so on. To do this, you use the `zip` function. Let's see how we can do this in a for-loop:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "5a53b4cc-2373-473f-b883-8a8fd4d1e18e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "list A is: [410 686  33 580 381 550 406 233]\n",
      "list B is: [100 101 102 103 104 105 106 107]\n",
      "list C is: [(410, 100), (686, 101), (33, 102), (580, 103), (381, 104), (550, 105), (406, 106), (233, 107)]\n",
      "list D is: [(410, 100), (686, 101), (33, 102), (580, 103), (381, 104), (550, 105), (406, 106), (233, 107)]\n"
     ]
    }
   ],
   "source": [
    "# Let's use the help of numpy to generate some random lists A and B:\n",
    "\n",
    "# What does numpy.random.seed() do?\n",
    "# Try commenting out this line below and re-run this cell multiple times. Print out the list A.\n",
    "# What happens to the entries of A if the seed is enabled versus not specifying the seed?\n",
    "np.random.seed(555)  \n",
    "A = np.random.randint(0,1000,size=(8))\n",
    "print(\"list A is: %s\" %A)\n",
    "\n",
    "# What is numpy.arange?\n",
    "B = np.arange(100,108,1)\n",
    "print(\"list B is: %s\" %B)\n",
    "\n",
    "# we create C as an empty list.\n",
    "C = []\n",
    "\n",
    "# now we loop through lists A and B and zip the items in the list together:\n",
    "for zipped_item in zip(A,B):\n",
    "    C.append(zipped_item)    \n",
    "print(\"list C is: %s\" %C)\n",
    "\n",
    "# If we were to use list comprehension...\n",
    "D = [zipped_item for zipped_item in zip(A,B)]\n",
    "print(\"list D is: %s\" %D)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "611089a0-87af-4dbc-9a26-0a7fad7f5b4d",
   "metadata": {},
   "source": [
    "You may have noticed that inside each list, the elements are now grouped pair-wise between (parenthesis) brackets, i.e. `()`. We know that lists are defined by the square brackets `[]`. These parenthesis are *not* lists, but tuples. If you do not already know, find out the difference between a list and a tuple.\n",
    "\n",
    "Now let's play around a little more with the list D."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "7a38a7ee-557d-44d2-b285-1b860321648c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the entries of the list D are:\n",
      "\n",
      "entry index 0 is: (410, 100)\n",
      "entry index 1 is: (686, 101)\n",
      "entry index 2 is: (33, 102)\n",
      "entry index 3 is: (580, 103)\n",
      "entry index 4 is: (381, 104)\n",
      "entry index 5 is: (550, 105)\n",
      "entry index 6 is: (406, 106)\n",
      "entry index 7 is: (233, 107)\n",
      "\n",
      "list E is: [410, 100, 686, 101, 33, 102, 580, 103, 381, 104, 550, 105, 406, 106, 233, 107]\n"
     ]
    }
   ],
   "source": [
    "# Let's loop the entries in D. We see that each entry is a tuple!\n",
    "# In other words, the size of list D is actually 8 rows by 2 columns, or 8x2.\n",
    "print(\"the entries of the list D are:\\n\")\n",
    "for idx, entry in enumerate(D):\n",
    "    print(\"entry index %i is: %s\" %(idx, entry))\n",
    "    \n",
    "print(\"\")\n",
    "\n",
    "# What if we want to \"flatten\" D? I.e. to make the list a flat list of 16 entries?\n",
    "# Let's try it with list comprehension:\n",
    "E = [item for item_pair in D for item in item_pair]\n",
    "print(\"list E is: %s\" %E)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0547351-9be4-4ddc-8280-0c25d8a42b7e",
   "metadata": {},
   "source": [
    "\n",
    "Try to make sense of the list comprehension for `E` above. How would you write it in terms of for-loops?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97500139-db1a-4d0c-af46-c4f79dab603b",
   "metadata": {},
   "source": [
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32d9e14f-4d85-4506-bb94-1e3c4c968006",
   "metadata": {},
   "source": [
    "# Warning!\n",
    "\n",
    "Python lists are different from numpy lists:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "4b0600ee-4c93-4f81-9fef-f957cf54004f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the list numpy_t is: [0.  0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9]\n",
      "the list python_t is: [10, 10, 10, 10, 10, 10, 10, 10, 10, 10]\n",
      "\n",
      "<class 'numpy.ndarray'>\n",
      "<class 'list'>\n",
      "\n",
      "<class 'numpy.ndarray'>\n",
      "<class 'list'>\n"
     ]
    }
   ],
   "source": [
    "# We did the following the code above with a numpy list:\n",
    "numpy_lst = np.arange(10)*0.1\n",
    "# Let's see what the results are\n",
    "print(\"the list numpy_t is: %s\" %numpy_lst)\n",
    "\n",
    "# On the other hand, if we were to muliply a Python list:\n",
    "python_lst = [10]*10\n",
    "print(\"the list python_t is: %s\" %python_lst)\n",
    "\n",
    "print(\"\")\n",
    "# To check the type of list you have,\n",
    "print(type(numpy_lst))\n",
    "print(type(python_lst))\n",
    "\n",
    "print(\"\")\n",
    "# To convert a python list to a numpy list:\n",
    "new_numpy_lst = np.array(python_lst)\n",
    "print(type(new_numpy_lst))\n",
    "\n",
    "# And to convert a numpy list to a python list:\n",
    "new_python_lst = list(numpy_lst)\n",
    "print(type(new_python_lst))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83bf81f3-ff8e-4020-96ac-922f3c01711b",
   "metadata": {},
   "source": [
    "There are a few other subtle differences between the two types of list. For example, Python lists can be just a collection of items, while numpy lists have stricter rules on what *types* its entries are."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "8d7e5c3d-07d2-4c03-b00d-a678f5d7b16b",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ray/anaconda3/envs/playground/lib/python3.7/site-packages/ipykernel_launcher.py:7: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.\n",
      "  import sys\n"
     ]
    }
   ],
   "source": [
    "# Let's initialise the Python list we have above:\n",
    "def sample_func(): return None # initialise an arbitrary function\n",
    "array = np.arange(25).reshape(5,5) # and an array\n",
    "lst = [1,2,3,'a','dog','item',10.0,55.5,1e-6,sample_func,array]\n",
    "\n",
    "# This is entirely valid as a Python list. However, if we were to convert it to a numpy list...\n",
    "numpy_lst = np.array(lst)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "31ee3b8b-3b3b-491e-a583-1e0c1c7e4134",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Numpy complains that the entries of the list are not of the same type.\n",
    "# To tell numpy that the list is meant to contain entries of different types, we need to explicitly specify that:\n",
    "numpy_lst_ok = np.array(lst, dtype='object')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d87c267-1fed-405b-917f-b3e416e77a10",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}