Tutorial_06b_MatrixAlgebra.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic Matrix Algebra\n",
    "\n",
    "This notebook presents some basic linear algebra and some matrix reshuffling/computations often needed in finance and econometrics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "using Printf, LinearAlgebra\n",
    "\n",
    "include(\"src/printmat.jl\");"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding and Multiplying: An Array and a Scalar\n",
    "\n",
    "With an array $A$ (for instance, a vector or a matrix, but also higher-dimensional arrays) and a scalar $c$, do\n",
    "\n",
    "1. `A*c` (textbook: $Ac$) to multiply each element of $A$ by $c$\n",
    "\n",
    "2. `A .+ c` (textbook: $A+cJ$, where $J$ is an array of ones) to add $c$ to each element of $A$, and similarly `A .- c` (textbook: $A-cJ$). Notice the dot.\n",
    "\n",
    "Watch out when the number comes first: `2.+A` is not allowed since it is ambiguous. However, `2.0.+A` and `2 .+ A` both work."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mA:\u001b[22m\u001b[39m\n",
      "     1         3    \n",
      "     3         4    \n",
      "\n",
      "\u001b[34m\u001b[1mc:\u001b[22m\u001b[39m\n",
      "    10    \n",
      "\n",
      "\u001b[34m\u001b[1mA*c:\u001b[22m\u001b[39m\n",
      "    10        30    \n",
      "    30        40    \n",
      "\n",
      "\u001b[34m\u001b[1mA .+ c:\u001b[22m\u001b[39m\n",
      "    11        13    \n",
      "    13        14    \n",
      "\n"
     ]
    }
   ],
   "source": [
    "A = [1 3;3 4]\n",
    "c = 10\n",
    "\n",
    "printblue(\"A:\")\n",
    "printmat(A)\n",
    "printblue(\"c:\")\n",
    "printmat(c)\n",
    "\n",
    "printblue(\"A*c:\")\n",
    "printmat(A*c)\n",
    "\n",
    "printblue(\"A .+ c:\")\n",
    "printmat(A .+ c)          #notice the dot in .+"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Adding and Multiplying Two Arrays\n",
    "\n",
    "With two arrays of the same dimensions ($A$ and $B$), do\n",
    "\n",
    "`A+B` (textbook: $A+B$) to add them (element by element), and similarly `A-B` (textbook: $A-B$).\n",
    "\n",
    "To multiply arrays ($A$ and $B$) of conformable dimensions: `A*B` (textbook: $AB$)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mA:\u001b[22m\u001b[39m\n",
      "     1         3    \n",
      "     3         4    \n",
      "\n",
      "\u001b[34m\u001b[1mB:\u001b[22m\u001b[39m\n",
      "     1         2    \n",
      "     3        -2    \n",
      "\n",
      "\u001b[34m\u001b[1mA+B:\u001b[22m\u001b[39m\n",
      "     2         5    \n",
      "     6         2    \n",
      "\n",
      "\u001b[34m\u001b[1mA*B:\u001b[22m\u001b[39m\n",
      "    10        -4    \n",
      "    15        -2    \n",
      "\n"
     ]
    }
   ],
   "source": [
    "A = [1 3;3 4]               #A and B are 2x2 matrices\n",
    "B = [1 2;3 -2]\n",
    "printblue(\"A:\")\n",
    "printmat(A)\n",
    "printblue(\"B:\")\n",
    "printmat(B)\n",
    "\n",
    "printblue(\"A+B:\")\n",
    "printmat(A+B)\n",
    "\n",
    "printblue(\"A*B:\")\n",
    "printmat(A*B)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transpose\n",
    "\n",
    "You can transpose a numerical vector/matrix `A` by `A'`. \n",
    "\n",
    "Notice that (in Julia) `A` and `B = A'` share the same elements (changing one changes the other). If you want an independent copy, use `B = permutedims(A)` or perhaps `B = copy(A')`.\n",
    "\n",
    "For an array of other elements (for instance, strings), use `permutedims(A)` to swap the dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mA: \u001b[22m\u001b[39m\n",
      "     1         2         3    \n",
      "     4         5         6    \n",
      "\n",
      "\u001b[34m\u001b[1mA': \u001b[22m\u001b[39m\n",
      "     1         4    \n",
      "     2         5    \n",
      "     3         6    \n",
      "\n"
     ]
    }
   ],
   "source": [
    "A = [1 2 3;4 5 6]\n",
    "printblue(\"A: \")\n",
    "printmat(A)\n",
    "printblue(\"A': \")\n",
    "printmat(A')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vectors: Inner and Outer Products\n",
    "\n",
    "There are several different ways to think about a vector in applied mathematics: as a $K \\times 1$ matrix (a column vector), a $1 \\times K$ matrix (a row vector) or just a flat $K$ vector. Julia uses flat vectors but they are mostly interchangable with column vectors. \n",
    "\n",
    "The inner product of two (column) vectors with $K$ elements is calculated as `x'z` or `dot(x,y)` (textbook: $x'z$ or $x \\cdot z$) to get a scalar. (The dot is obtained by `\\cdot + TAB`, but this is sometimes hard to distinguish from  or things like `x.z`.)\n",
    "\n",
    "In contrast, the outer product of two (column) vectors with $K$ elements is calculated as `x*z'` (textbook: $xz'$) to get a $K\\times K$ matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mx and z\u001b[22m\u001b[39m\n",
      "    10         2    \n",
      "    11         5    \n",
      "\n",
      "\u001b[34m\u001b[1mx'z: \u001b[22m\u001b[39m\n",
      "    75    \n",
      "\u001b[34m\u001b[1mx*z':\u001b[22m\u001b[39m\n",
      "    20        50    \n",
      "    22        55    \n",
      "\n"
     ]
    }
   ],
   "source": [
    "x = [10,11]                  #[10;11] gives the same\n",
    "z = [2,5]\n",
    "printblue(\"x and z\")\n",
    "printmat([x z])\n",
    "\n",
    "printblue(\"x'z: \")\n",
    "printlnPs(x'z)               #dot(x,z) gives the same\n",
    "\n",
    "printblue(\"x*z':\")\n",
    "printmat(x*z')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vectors: Quadratic Forms\n",
    "\n",
    "A quadratic form ($A$ is an $n \\times n$ matrix and $x$ is an $n$ vector): `x'A*x` (textbook: $x'Ax$) to get a scalar. There is also the form `dot(x,A,x)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mx:\u001b[22m\u001b[39m\n",
      "    10    \n",
      "    11    \n",
      "\n",
      "\u001b[34m\u001b[1mA:\u001b[22m\u001b[39m\n",
      "     1         3    \n",
      "     3         4    \n",
      "\n",
      "\u001b[34m\u001b[1mx'A*x: \u001b[22m\u001b[39m\n",
      "  1244    \n"
     ]
    }
   ],
   "source": [
    "A = [1 3;3 4]\n",
    "x = [10,11]\n",
    "\n",
    "printblue(\"x:\")\n",
    "printmat(x)\n",
    "printblue(\"A:\")\n",
    "printmat(A)\n",
    "\n",
    "printblue(\"x'A*x: \")\n",
    "printlnPs(x'A*x)         #or dot(x,A,x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Kronecker Product\n",
    "\n",
    "The Kronecker product of two matrices \n",
    "\n",
    "$ \n",
    "A\\otimes B=\\left[\n",
    "\\begin{array}{ccc}\n",
    "a_{11}B & \\cdots & a_{1n}B\\\\\n",
    "\\vdots &  & \\vdots\\\\\n",
    "a_{m1}B & \\cdots & a_{mn}B\n",
    "\\end{array} \\right]\n",
    "$\n",
    "\n",
    "is calculated by using `kron()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mA:\u001b[22m\u001b[39m\n",
      "     1         3    \n",
      "     2         4    \n",
      "\n",
      "\u001b[34m\u001b[1mB:\u001b[22m\u001b[39m\n",
      "    10        11    \n",
      "\n",
      "\u001b[34m\u001b[1mkron(A,B):\u001b[22m\u001b[39m\n",
      "    10        11        30        33    \n",
      "    20        22        40        44    \n",
      "\n"
     ]
    }
   ],
   "source": [
    "A = [1 3;2 4]\n",
    "B = [10 11]\n",
    "z = kron(A,B)\n",
    "\n",
    "printblue(\"A:\")\n",
    "printmat(A)\n",
    "printblue(\"B:\")\n",
    "printmat(B)\n",
    "printblue(\"kron(A,B):\")\n",
    "printmat(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Matrix Inverse\n",
    "\n",
    "A matrix inverse of an $n \\times n$ matrix $A$:\n",
    "\n",
    "`inv(A)` or `A^(-1)` (textbook: $A^{-1}$)\n",
    "\n",
    "The inverse is such that $AA^{-1}=I$ and $A^{-1}A=I$, where $I$ is the identity matrix (ones along the main diagonal, zeros elsewhere)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mA:\u001b[22m\u001b[39m\n",
      "     1         3    \n",
      "     3         4    \n",
      "\n",
      "\u001b[34m\u001b[1minv(A):\u001b[22m\u001b[39m\n",
      "    -0.800     0.600\n",
      "     0.600    -0.200\n",
      "\n",
      "\u001b[34m\u001b[1minv(A)*A:\u001b[22m\u001b[39m\n",
      "     1.000    -0.000\n",
      "     0.000     1.000\n",
      "\n"
     ]
    }
   ],
   "source": [
    "A = [1 3;3 4]\n",
    "printblue(\"A:\")\n",
    "printmat(A)\n",
    "\n",
    "printblue(\"inv(A):\")\n",
    "printmat(inv(A))\n",
    "\n",
    "printblue(\"inv(A)*A:\")\n",
    "printmat(inv(A)*A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Solving Systems of Linear Equations\n",
    "\n",
    "If you have the system $A b = y$, where $A$ is a matrix and $b$ and $y$ are vectors, then we can solve as `b = inv(A)*y` or `b = A\\y`. The latter is typically more robust."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     sol 1     sol 2\n",
      "    -1.400    -1.400\n",
      "     3.800     3.800\n",
      "\n"
     ]
    }
   ],
   "source": [
    "y = [10,11]\n",
    "\n",
    "b1 = inv(A)*y\n",
    "b2 = A\\y\n",
    "\n",
    "printmat(b1,b2,colNames=[\"sol 1\",\"sol 2\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Identity Matrix\n",
    "\n",
    "The identity matrix $I_n$ can often be represented by `I` and then Julia will compare with the surrounding code to create the right dimension. For instance, if `A` is a square matrix, then `I + A` and `[A I]` both work. Notice that you need to do `using LinearAlgebra` before `I` works.\n",
    "\n",
    "If you still need to specify the dimension, then use `1I(3)` or `1I[1:3,1:3]`.\n",
    "\n",
    "You can create a basis vector as `1I[1:3,2]`. This 3-vector is filled with zeros, except that element 2 is 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mI + A\u001b[22m\u001b[39m\n",
      "     2         3    \n",
      "     3         5    \n",
      "\n",
      "\u001b[34m\u001b[1m1I[1:3,1:3]\u001b[22m\u001b[39m\n",
      "     1         0         0    \n",
      "     0         1         0    \n",
      "     0         0         1    \n",
      "\n",
      "\u001b[34m\u001b[1m1I[1:3,2]\u001b[22m\u001b[39m\n",
      "     0    \n",
      "     1    \n",
      "     0    \n",
      "\n"
     ]
    }
   ],
   "source": [
    "printblue(\"I + A\")\n",
    "printmat(I + A)\n",
    "\n",
    "printblue(\"1I[1:3,1:3]\")\n",
    "printmat(1I[1:3,1:3])\n",
    "\n",
    "printblue(\"1I[1:3,2]\")\n",
    "printmat(1I[1:3,2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vectors: Extracting Vectors from Matrices\n",
    "\n",
    "Notice that `A[1,:]` and `A[:,1]` both give flat vectors. In case you want a row vector use `A[1:1,:]`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Vector{Int64}:\n",
       " 1\n",
       " 3"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "A[1,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Julia 1.11.3",
   "language": "julia",
   "name": "julia-1.11"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}