berkeley-rl-notebook-tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CS294-112 Fall 2018 Tensorflow Tutorial"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial will provide a brief overview of the core concepts and functionality of Tensorflow. This tutorial will cover the following:\n",
    "\n",
    "0. What is Tensorflow\n",
    "1. How to input data\n",
    "2. How to perform computations\n",
    "3. How to create variables\n",
    "4. How to train a neural network for a simple regression problem\n",
    "5. Tips and tricks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.cm as cm\n",
    "import matplotlib.patches as mpatches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def tf_reset():\n",
    "    try:\n",
    "        sess.close()\n",
    "    except:\n",
    "        pass\n",
    "    tf.reset_default_graph()\n",
    "    return tf.Session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 0. What is Tensorflow\n",
    "\n",
    "Tensorflow is a framework to define a series of computations. You define inputs, what operations should be performed, and then Tensorflow will compute the outputs for you.\n",
    "\n",
    "Below is a simple high-level example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "c = 3.0\n"
     ]
    }
   ],
   "source": [
    "# create the session you'll work in\n",
    "# you can think of this as a \"blank piece of paper\" that you'll be writing math on\n",
    "sess = tf_reset()\n",
    "\n",
    "# define your inputs\n",
    "a = tf.constant(1.0)\n",
    "b = tf.constant(2.0)\n",
    "\n",
    "# do some operations\n",
    "c = a + b\n",
    "\n",
    "# get the result\n",
    "c_run = sess.run(c)\n",
    "\n",
    "print('c = {0}'.format(c_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. How to input data\n",
    "\n",
    "Tensorflow has multiple ways for you to input data. One way is to have the inputs be constants:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "c = 3.0\n"
     ]
    }
   ],
   "source": [
    "sess = tf_reset()\n",
    "\n",
    "# define your inputs\n",
    "a = tf.constant(1.0)\n",
    "b = tf.constant(2.0)\n",
    "\n",
    "# do some operations\n",
    "c = a + b\n",
    "\n",
    "# get the result\n",
    "c_run = sess.run(c)\n",
    "\n",
    "print('c = {0}'.format(c_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, having our inputs be constants is inflexible. We want to be able to change what data we input at runtime. We can do this using placeholders:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "c0 = [3.]\n",
      "c1 = [6.]\n"
     ]
    }
   ],
   "source": [
    "sess = tf_reset()\n",
    "\n",
    "# define your inputs\n",
    "a = tf.placeholder(dtype=tf.float32, shape=[1], name='a_placeholder')\n",
    "b = tf.placeholder(dtype=tf.float32, shape=[1], name='b_placeholder')\n",
    "\n",
    "# do some operations\n",
    "c = a + b\n",
    "\n",
    "# get the result\n",
    "c0_run = sess.run(c, feed_dict={a: [1.0], b: [2.0]})\n",
    "c1_run = sess.run(c, feed_dict={a: [2.0], b: [4.0]})\n",
    "\n",
    "print('c0 = {0}'.format(c0_run))\n",
    "print('c1 = {0}'.format(c1_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But what if we don't know the size of our input beforehand? One dimension of a tensor is allowed to be 'None', which means it can be variable sized:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"a_placeholder:0\", shape=(?,), dtype=float32)\n",
      "a shape: (?,)\n",
      "Tensor(\"b_placeholder:0\", shape=(?,), dtype=float32)\n",
      "b shape: (?,)\n",
      "c0 = [3.]\n",
      "c1 = [3. 6.]\n"
     ]
    }
   ],
   "source": [
    "sess = tf_reset()\n",
    "\n",
    "# inputs\n",
    "a = tf.placeholder(dtype=tf.float32, shape=[None], name='a_placeholder')\n",
    "b = tf.placeholder(dtype=tf.float32, shape=[None], name='b_placeholder')\n",
    "\n",
    "# do some operations\n",
    "c = a + b\n",
    "\n",
    "# get outputs\n",
    "c0_run = sess.run(c, feed_dict={a: [1.0], b: [2.0]})\n",
    "c1_run = sess.run(c, feed_dict={a: [1.0, 2.0], b: [2.0, 4.0]})\n",
    "\n",
    "print(a)\n",
    "print('a shape: {0}'.format(a.get_shape()))\n",
    "print(b)\n",
    "print('b shape: {0}'.format(b.get_shape()))\n",
    "print('c0 = {0}'.format(c0_run))\n",
    "print('c1 = {0}'.format(c1_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2. How to perform computations\n",
    "\n",
    "Now that we can input data, we want to perform useful computations on the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's create some data to work with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a:\n",
      "[[-1.]\n",
      " [-2.]\n",
      " [-3.]]\n",
      "b:\n",
      "[[1. 2. 3.]]\n"
     ]
    }
   ],
   "source": [
    "sess = tf_reset()\n",
    "\n",
    "# inputs\n",
    "a = tf.constant([[-1.], [-2.], [-3.]], dtype=tf.float32)\n",
    "b = tf.constant([[1., 2., 3.]], dtype=tf.float32)\n",
    "\n",
    "a_run, b_run = sess.run([a, b])\n",
    "print('a:\\n{0}'.format(a_run))\n",
    "print('b:\\n{0}'.format(b_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can do simple operations, such as addition:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b:\n",
      "[[1. 2. 3.]]\n",
      "c:\n",
      "[[2. 4. 6.]]\n"
     ]
    }
   ],
   "source": [
    "c = b + b\n",
    "\n",
    "c_run = sess.run(c)\n",
    "print('b:\\n{0}'.format(b_run))\n",
    "print('c:\\n{0}'.format(c_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Be careful about the dimensions of the tensors, some operations may work even when you think they shouldn't..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a:\n",
      "[[-1.]\n",
      " [-2.]\n",
      " [-3.]]\n",
      "b:\n",
      "[[1. 2. 3.]]\n",
      "c:\n",
      "[[ 0.  1.  2.]\n",
      " [-1.  0.  1.]\n",
      " [-2. -1.  0.]]\n"
     ]
    }
   ],
   "source": [
    "c = a + b\n",
    "\n",
    "c_run = sess.run(c)\n",
    "print('a:\\n{0}'.format(a_run))\n",
    "print('b:\\n{0}'.format(b_run))\n",
    "print('c:\\n{0}'.format(c_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also, some operations may be different than what you expect:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a:\n",
      "[[-1.]\n",
      " [-2.]\n",
      " [-3.]]\n",
      "b:\n",
      "[[1. 2. 3.]]\n",
      "c_elementwise:\n",
      "[[-1. -2. -3.]\n",
      " [-2. -4. -6.]\n",
      " [-3. -6. -9.]]\n",
      "c_matmul: \n",
      "[[-14.]]\n"
     ]
    }
   ],
   "source": [
    "c_elementwise = a * b\n",
    "c_matmul = tf.matmul(b, a)\n",
    "\n",
    "c_elementwise_run, c_matmul_run = sess.run([c_elementwise, c_matmul])\n",
    "print('a:\\n{0}'.format(a_run))\n",
    "print('b:\\n{0}'.format(b_run))\n",
    "print('c_elementwise:\\n{0}'.format(c_elementwise_run))\n",
    "print('c_matmul: \\n{0}'.format(c_matmul_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Operations can be chained together:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b:\n",
      "[[1. 2. 3.]]\n",
      "c0:\n",
      "[[2. 4. 6.]]\n",
      "c1:\n",
      "[[3. 5. 7.]]\n"
     ]
    }
   ],
   "source": [
    "# operations can be chained together\n",
    "c0 = b + b\n",
    "c1 = c0 + 1\n",
    "\n",
    "c0_run, c1_run = sess.run([c0, c1])\n",
    "print('b:\\n{0}'.format(b_run))\n",
    "print('c0:\\n{0}'.format(c0_run))\n",
    "print('c1:\\n{0}'.format(c1_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, Tensorflow has many useful built-in operations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b:\n",
      "[[1. 2. 3.]]\n",
      "c:\n",
      "2.0\n"
     ]
    }
   ],
   "source": [
    "c = tf.reduce_mean(b)\n",
    "\n",
    "c_run = sess.run(c)\n",
    "print('b:\\n{0}'.format(b_run))\n",
    "print('c:\\n{0}'.format(c_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3. How to create variables\n",
    "\n",
    "Now that we can input data and perform computations, we want some of these operations to involve variables that are free parameters, and can be trained using an optimizer (e.g., gradient descent)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, let's create some data to work with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b:\n",
      "[[1. 2. 3.]]\n"
     ]
    }
   ],
   "source": [
    "sess = tf_reset()\n",
    "\n",
    "# inputs\n",
    "b = tf.constant([[1., 2., 3.]], dtype=tf.float32)\n",
    "\n",
    "sess = tf.Session()\n",
    "\n",
    "b_run = sess.run(b)\n",
    "print('b:\\n{0}'.format(b_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll now create a variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<tf.Variable 'myvar:0' shape=(1, 3) dtype=float32_ref>\n"
     ]
    }
   ],
   "source": [
    "var_init_value = [[2.0, 4.0, 6.0]]\n",
    "var = tf.get_variable(name='myvar',\n",
    "                      shape=[1, 3],\n",
    "                      dtype=tf.float32,\n",
    "                      initializer=tf.constant_initializer(var_init_value))\n",
    "\n",
    "print(var)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and check that it's been added to Tensorflow's variables list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[<tf.Variable 'myvar:0' shape=(1, 3) dtype=float32_ref>]\n"
     ]
    }
   ],
   "source": [
    "print(tf.global_variables())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can do operations with the variable just like any other tensor:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"Const:0\", shape=(1, 3), dtype=float32)\n",
      "<tf.Variable 'myvar:0' shape=(1, 3) dtype=float32_ref>\n",
      "Tensor(\"add:0\", shape=(1, 3), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# can do operations\n",
    "c = b + var\n",
    "print(b)\n",
    "print(var)\n",
    "print(c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we can run any of these operations, we must first initalize the variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "init_op = tf.global_variables_initializer()\n",
    "sess.run(init_op)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and then we can run the operations just as we normally would."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "b:\n",
      "[[1. 2. 3.]]\n",
      "var:\n",
      "[[2.0, 4.0, 6.0]]\n",
      "c:\n",
      "[[3. 6. 9.]]\n"
     ]
    }
   ],
   "source": [
    "c_run = sess.run(c)\n",
    "\n",
    "print('b:\\n{0}'.format(b_run))\n",
    "print('var:\\n{0}'.format(var_init_value))\n",
    "print('c:\\n{0}'.format(c_run))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far we haven't said yet how to optimize these variables. We'll cover that next in the context of an example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4. How to train a neural network for a simple regression problem\n",
    "\n",
    "We've discussed how to input data, perform operations, and create variables. We'll now show how to combine all of these---with some minor additions---to train a neural network on a simple regression problem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we'll create data for a 1-dimensional regression problem:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x13b401160>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXt4VNW5/7977pdMJpcZdkiYMMAIE5oRm1LDI4QnLTn1PMnRaFpMBawpKEJ60hMqARQ1DkrLRQaOsVCxQ8EKXnDw6Dl4Do9tJ8eW2vrDo3ZGgpBwC4TZTEgRcr+9vz+SvZsgSJAkc1uf59kPc9nsvWZnrXe9613vhSMiMBgMBiO2kIW6AQwGg8EYfZjwZzAYjBiECX8Gg8GIQZjwZzAYjBiECX8Gg8GIQZjwZzAYjBiECX8Gg8GIQZjwZzAYjBiECX8Gg8GIQRShbsC1MJlMZLVaQ90MBoPBiCg++uijRiIyX++8sBX+VqsVhw4dCnUzGAwGI6LgOO7UUM5jZh8Gg8GIQZjwZzAYjBiECX8Gg8GIQZjwZzAYjBiECX8Gg8GIQZjwZzAYjBiECX8Gg8GIQZjwDwGCIIS6CQwGI8Zhwn+UEQQBpaWlbAJgMBghhQn/ELB161bwPB/qZjAYjBiGCf9RRBAELFy4MNTNYDAYDCb8R4KvMulwHIdgMDiKrWFEM1f2NWZOZAwVJvyHGVG7v9qg5Hke69atQ2VlJRukjJtm4P6RIAjw+/1X3U9ifY1xNZjwHwE4jpNeiwNzwYIFEAQBmZmZg2z+4sAcOEDZYGUMBZ7nsXXrVgSDQSxcuBCrVq2C0+mUvhf7HnMwYFwNJvyHGZ7nsXz5cvA8L2liBw8ehM/nQzAYHDQIq6urUVpaCr/fj0WLFrHByvhaLFu2DBUVFXC73TCbzYP6VDAYZA4GjKvChP8wIQprv9+P+fPno7q6GitXrsT8+fOxe/durFy5EgAwd+5cLFq0CNXV1Zg/fz7mz58Ps9kMIpKudbWVAYNxJYIgIBgM4pNPPsHatWsRDAZRU1ODsrIyAEB7ezuWLVsW4lYywhUm/IeBgdq62WzGgQMHYDKZ0NnZiW3btmHatGlYsWIF5s2bhyNHjqCoqAgZGRnYvXu3tGzfsWMHampqUFpaetXrMhhAn3IB/KNvNDY2wuv1YunSpVi6dCm+973v4e6778aDDz6IefPmQaVSSeezfsQYBBGF5fGtb32LIgmfz0eBQICKiorI5/NRUVERud1uuv3228loNBLHcWS328nlclFiYiLl5eWR1+ul+Ph46TXP8+T1egddNxAIhOYHMcIOn89HPM+Tz+cjIiKPx0OpqankcrkoNTWV3G43uVwu0uv1JJPJKDExkVwuF/l8PsrLy6O8vDzWn2IAAIdoCDJ2WDR/juN2cBx3nuM4/zW+5ziOe57juFqO4/7GcVzWcNw3XBAEAZWVlQgGgygrK0NmZibKysqwYsUKtLW1IT09HWq1GkePHsWmTZvQ3NyM/Px8fPzxx7h8+TLy8/PR1NSEq9UsZrZaxkAmT54Ms9kMQRCwdetWPPLII1i/fj3uuecerFy5Etu3b4dCocDkyZNhNBqxfPlyzJs3D5cvX8bly5dD3XxGODGUGeJ6B4DZALIA+K/xfT6A/wbAAZgB4K/Xu2Ykav5z5swhs9lMXq9X0uqNRiPZ7XYCIB3Tp08nk8lECQkJVFJSQllZWSSXy8lgMFBycjJ5vV4KBAJMS2NI+Hw+ys/Pp5ycHPL5fOTxeCguLo50Oh1ZrVZSKpXE8zwpFAqyWq2D+pxOpyO73U7JycnSqoERvWCImj9HAzYabwaO46wA/ouIMq/y3YsAqono1f73nwPIJaJz17re9OnTKdIKuO/YsQPbt2+HSqWCWq2GIAi4ePEi9Ho9jhw5Mujc6dOn49ChQ5DJZFCr1VAoFFiyZAk++OADaDQaAIBarYbb7Wbaf4wixoYIgoD58+fjiSeewNq1a3HmzBnU19ejpaUFAKDRaGA0GiEIAtLS0nDu3DkolUp0d3cjLS0NX3zxBQBg0qRJePfddwGwFWU0w3HcR0Q0/XrnjdaGbxqA+gHvz/R/FtEM9NHft28fli5dioaGBnR2duL+++/HsWPHUF9fj1OnTkGpVAIAkpKSkJiYCHFiMxqN6OjogMFgwObNm1FUVITNmzfjlVdegdvtDtlvY4QWQRAk999gMIjPPvsMAJCTk4MjR46gp6dH6lPt7e1SX2xubkZvby86OjqQmpoKo9GIxYsXY+zYsQCAgwcPSt5obAM4xhnK8mAoBwArrm32+S8Aswa8/z2A6Vc5bzGAQwAOpaenj9CiaHgQN3W9Xi/dfvvtlJOTQ6mpqQSAtFotTZgwgbRaLQGgtLQ0UqvVg0w/Vzs4jiOdTkc5OTkUCAQG3YMRWwQCAcrPz5dMgF6vl+bMmUOJiYmk0WikPjNmzBjief6q/UmlUn2pf+n1etLr9WQ0GtkGcJSCIZp9Rkv4vwjg/gHvPwcw9quuF842f9Grx+v10qxZs0ihUFB6ejoZjcZBA06hUHxpQFosFmlSuPIwm80EgBwOh2TjFT062AQQW4gC32QySULa4/GQw+GgtLQ0stlslJCQcFWBn5qaSnK5XPpMr9cTAFIqlaTVaikuLo7i4+PJ4/Ew4R+FDFX4j5bZ5x0AP+r3+pkB4Av6Cnt/uCOG1WdkZGDbtm2wWq04ffo04uLiBqV26O7uRnx8PABAq9ViwoQJUKvVAPpSQJjN5kHXbWxshFarRXNzM15++WVwHIfJkydj9+7dqKqqYsv0GEE0+ZhMJthsNmzevBkHDx7Ej370I/h8Ppw9exa1tbW4ePEi4uLiAED6t7OzEw0NDejp6ZE+a2lpgdFoRFdXF9rb22EymUBE2LJli2RaYsQew+Xq+SqADwBM4TjuDMdxiziOW8Jx3JL+U94FcBxALYCXAJRe41IRxcKFCyWBDQBnz55FR0cHAEhC/9KlSwD6VliCIKC2thZtbW0gokHZPQsLCzFlyhQQEU6cOIGNGzfCbDZj5cqVyMjIkKJ+2UCNDYgIjY2NOHHiBPbt24eSkhK0tLRAJvvHkNXr9WhubgbQZ+u3WCywWCyQy+VITEyUvgMgBXup1WoEg0H09PSgvLyc7SvFMkNZHoTiCHezTyAQoKysLLLb7WQ0GgkAqdVqUiqVV7W7OhwOys7OJp7nieM4KiwspLi4ONLr9VRcXExxcXFkNBrJYrGQTCajO++8k5KTkyX7v3jfoqIitlSPcsT+FQgEyO12k1wuH9SvkpKSCADZ7Xay2WzE8zzZ7XbKzs6mxMREcjqdZLVaSS6XU2FhIQGQ9qMSEhIks2NWVpa0r8T6VPSAMDP7RA1iWP3+/fvxySef4MiRI7j33nsBAB0dHejq6pLO1ev1+Ld/+zcoFAqUl5djxYoV+Pvf/44HH3wQf/zjH9Hd3Q2ZTIb//M//RHNzMzQaDQKBAP7pn/4Jhw4dQnd3N8rLy6UVgmhuYm560YuYElzM1AkAPT09Ur/Kzc1FWloabDYburu7IQgCrFYrzGYzFi9ejI6ODvzmN7/ByZMnoVAo8P7776OiokK6/sWLF9HZ2QmtVovW1lY0NjbC6XSyVWUMwoT/DcLzPJxOJzweD1JSUgAA//M//wMAsFgs0Ol00ue9vb34/e9/D61Wi1dffRWTJ0/Gbbfdhrq6OiQlJaGjowNLliyR9gnOnz8Pg8GAAwcOQKfTYdKkSdiwYQO++93vSjldGNFPZ2cnjh49Co7jkJCQMCjyu7q6epDdv62tDYsXL0ZnZyc2btyI1tZWnD9/HqWlpZg2bRo6Ozvx1ltvoaGhAQkJCQD6JpO2tjYcOXIEhYWFUhZQlkcqtmDC/wYQB0ZmZia+//3vSzbVQCAAjuOgUCigVCol/2ue56HX6zF16lSsXr0amZmZWL9+PTo7O5GcnIzMzEzk5+dDoVBAo9GAiNDU1ASgb/NX3C+wWCxobGxkid5ihEuXLmHx4sWYOnUqSkpKcO5cn28Ex3GS1r9q1SpkZ2fjxRdfxMSJE3H06FEYjUZYrVYoFAocPXoUP/zhD9HR0YGenh5otVpYLBbJ4UC83vjx43Hx4kWYzWa2qowxmPAfIgMFb3V1NZYvX47x48dDp9MB6IuyPHXqFJKSkjBlyhRUVFRg06ZNKC8vh0qlwtq1a+H3+/Hkk0+itrYWXV1d0kC79dZbMXXqVGnFAABtbW2ora3FhQsXcOzYMRQVFbHc7DGCWq2WNPnLly9LTgREhIaGBtTV1WHNmjUoLi6Gx+PBqlWrAAArVqyAxWKBUqnE0qVL8dZbb0meaB0dHdDr9Vi0aBEASB4/EydOhM/nw549ewCwFOIxxVA2BkJxhOOGr7gJl52dTXK5nCoqKqSNteLiYsm33+12U1ZWFikUCkpMTKTs7GzKy8sjn89HBQUF5PF4pH8HBorhKr7/YlxAXFwc8/WPEXw+H7lcLpLL5ZIvv0wmI5lMRhqNhqxWKxmNRsrOziaPx0Nz5syRHAPE4ECfz0cmk4lycnLI4/GQx+Mhr9dLycnJpNFoBsWaiMFfOTk5Uj9lRC4Y7SCv4T7CUfgTkSSoxcAZAJSSkkIKhYJkMhk5nU5p4GVlZUkRmuKAEr0qvF6vlP5ZfJ+dnU1Op5MqKiq+5DXEcRxlZ2czr4woR/Qis9lsZLPZyGq1SsJfVDQSEhLI6XSS2WyWhPXV+oX4eSAQkM6z2WxkMBiI4zjJA8hsNpNcLqfS0lLKy8ujgoIC1s8iGCb8RwCv10tms5mys7MlTb+kpIQSExOppKSEANCMGTMkzUscQFe60w3M+y++z8/Pp+zsbMrOziaFQkE6nW6Q8Fer1dJkIv4fRnQhRvFeuforLS2lWbNmSQqCuJL0eDxD0tLFOgAul4s4jqOSkhKSy+VSRPnAo6KigvWtCIcJ/2FGFNB2u50cDgdxHEfFxcXkdDpJLpdLWvv1BP3A6w3E6/VSQUHBoBWAuDRXqVSk1WrJaDRKKZ+Zb3Z0IfYvt9tNJpNpkEBOS0sjj8dDRCQpFmJxlqFq6eL/czgclJOT8yXlguM4UqlUpFQqpXux/hWZMOE/jIiDwOVykUqlotLS0kEDR6vVksPhoJSUFGmQXe3/f9X1xQnC5/NRTk4OJScnk1arJZlMRmlpaaRQKKRl+bWW+YzIRdxLUigUFBcXR6mpqaTVaikpKYnkcjklJiZKe0QDg8CG2g/EXFHiiiE7O3tQgjie58loNJLD4ZBWmEzBiEyY8B8mBiZxS05OJqvV+iWtSaVSUU5OjmTf/zqDRhzI4iTg8Xi+lChOLpeTy+Wi/Pz8QSsLRuQzMHFbYmIiud1ustls5HA4pOjdgoKCr7UZe6VyITobJCYmUmpqKul0OoqLiyOHw0Eul0uK/mUOBpEJE/7DiM/nI7fbTXq9ntRqtZSeOTc3l7RaLXEcR263Wzr/ZgTyQKE+sBqTSqWSNn0zMjIG1QxmE0BkEwgEpCpwV270chxHiYmJ5PV6b8oLZ2C/ys/PlzZ/lUolFRYWkkwmI51OJ+1d3XLLLWzjN0Jhwn+YCAQCNGvWrEEpckVTj9FoJLfbPUjwDycej4f0ej1pNBppwrnjjjsIANlsNlbqMYrwer2DzIlyuZwsFou0qTucOXjEdNFKpfJLOf8BUHFxMfE8zzT/CIUJ/2FCdNO02Wwkk8lIrVYTz/PkdDpJqVRKy/HhFsKBQIAKCgrI5XKRwWD40gCVyWTSxhwjsvF6vV/KzV9YWEgmk2mQKXG4/e+dTiclJyeTxWKRkr6JWj/rW5ELE/7DgDjoPB4PJScnU0VFhVQ1KSMjg1wu14hp36IpICcnhxwOhyQUxImA4zipmDcjchF98LOzs+nOO+8cNAHYbLZBwVvDic/no9TUVCkgUbyn6P55yy23sL4VoTDhP0yInhEGg4EMBoOk/btcLkpNTR3RATKwmpfRaJRcANPS0iguLo5cLhez+UcoA23ws2bNkmz8JpOJrFYrxcXFUVZWFpnN5hHrY+J1fT4f2e120mq1g9xMs7KyWN+KQIYq/Flun6/A7/dj4cKFOHz4MBISEpCcnAyNRoP4+Hi8++672L17NzIzM0fs/pmZmdixYwdmzpyJ9PR0KceLTCZDa2srnE4nCgoKWK6fCEPMEyVmag0EAjh58iQA4MKFC7hw4QImT56MXbt24Q9/+MOXKr4NFwP7rtFoRFtbGxobGwEAd955J4gGFxxiRBdM+H8FH374IS5fvozW1lbU19fj5MmTaG1tRTAYxPHjx5GRkTGi9xcEATzPIxgM4vz583jggQeg0+nQ1NQEtVqNnp4erFixgqV7jjDEtOCVlZXYtGkTamtrpe8mTZqEW265BUlJSTCbzTCbzSOayVUQBFRWVqK4uFj6LCkpCQcOHMDHH3+MBx98EIIgsIRv0chQlgehOEJt9hHz84heNujP4WOz2Sg9PX3EPSGudON0u91SpK9WqyWdTkdWq3VQpS9G5CD69atUKsmTTKVSSRG4A009I/339fl8NGfOHNLr9cRxHMXFxQ2qQMcCviILMJv/zePxeMhms0kDU6PRkEKhoKysrFHxgb7y+i6Xi3Q6HXEcJ01ITqdzRNvAGH4GTuwVFRWDPLjE7JqjLWhF98+BgWa33HIL5eTkfO3gMkZoYML/JgkEAuRwOEihUJBSqZRWADabjXw+36j7QIsbgwqFglJTU6X2JCQkfCljKCP8EevzDkytXFxcLOVuGu22iHEELpeLzGYzVVRUUF5enhRcxvpW5MCE/00gakFqtZqmT58+KLBLTOMw0p4+V0P0ypDJZMRxHOl0OnI4HINSQ7BBGhl4vV4pDbhCoRi0qhzNfiVG/BYUFEimxbS0NJLL5WSz2chut0tJ5FjfigyGKvzZhu8VCIKARYsWoampCffffz8OHToElUoFnU4HlUqF8vJy5Obm4sCBAyPq6XMtjEYj9Ho9xo8fj/T0dJw6dUryyGBVviILh8OBH/3oR+ju7oZMJsOtt96K1atXo7KyctQ2WHmex44dO+B2u1FQUIApU6bg3Llz6OnpQW1tLY4cOYJvfvObUslSRvSgCHUDwo1gMIjTp0/jvvvuk2qfjh8/HkajEV1dXZLr5WgLfkEQsHLlSqhUKlitVhw+fBgKhQIdHR14+eWXcfjwYbjd7lFtE+PrUV1djR/84Afo6OhAbW0ttFot1Go1FixYgB07dmDdunWjOokPvNd7772Hmpoa7N27F1u3bgUAvPbaa2hoaEBNTQ1TLqIIpvkPQPS/bmhoQGpqKoC+2rzBYBDFxcVISUkZ9YEpwvM81q9fj7179+L555+HRqOR/P6fe+451NXVMZ/sCECcxM1mM7q6utDT04Px48ejvLwczz77LC5dujRifv3XaxfQ189MJhN++9vfAgAUCgXq6+vR09OD48ePj3q7GCMHE/5XYDAY8Oabb+Khhx4CAPz973+HXq/Hhg0bUFFRERJTD/APf2wAyM3NxTPPPAOO4wD07dvU1tZKPtmM8OXgwYP4v//7P9TV1aGjowOtra04evQoXC4XiAhr1qwZdeVCVHrEvtPY2IiWlhYAQHd3N1QqFUpKSrB//37Wv6KJoWwMhOII1Yavz+eTauiKG15DLZc3Gm0T/50zZ84gT5E777xTSgTGCE/EzVUxlYPoS3/nnXfedMrm4WjbwNcOh2NQjItMJiOXyxWy9jGGDpi3z40TCATom9/8plQz12azkdvtJp7nQ57lcGBmR7EYR0JCwqAJQPTMYF4Z4YnoRTYwUZ9YpSuc/m5iVTG5XE46nY4SEhJIJpOFxA2VceMMVfgzs88AeJ6Hy+UCz/PSZty6deswZswY/OQnPwlpGgWe57F161aYzWYQEWbOnIlNmzZJJgKZTIbvfve7UKvVIWsj49oIgoCFCxfi0UcfxYkTJ1BSUoKUlBRMnToVmZmZ2Lx5c9hspvI8j7fffhu/+93vUFJSgosXL8JqteKnP/0pqqqqmOknWhjKDBGKIxSav+hHj36NDP0plEXf/nBBDMpJSkqSsoxqNBriOI4qKipC3TzGNfB6vaTX6welbdbpdGHVt0TE9BPiOFAqlaRUKpnpJwIA0/xvDEEQsHTpUhw7dgxKpRI9PT2wWCzYsmUL9u7di9zc3FA3UYLneZjNZrz00kv4xje+ga6uLhiNRqhUKmzZsgX79u1j2lkYkpGRgczMTCmJ2pgxY6BWq2EymULcssGIq5QdO3bgqaeeQnFxMbq6usBxHJ555hmWSDBKYMK/H57n8cwzz2D69OlISUkBAMjlcqxYsSLsXCgHDs6nn34aKpUKgiCgo6MDaWlp2LJlCxYsWMAmgDCjpqYGtbW1+PTTTyGTyTBu3DhkZmaGxLXzqxADvxYuXIhnn30W77zzDgCgs7MTPT09OHr0aIhbyBgOWJBXP4IgoKqqCrNnz8bGjRsBAOfPnw9LG7o4OIG+oDSZ7B9z+Pnz55GUlAS1Wo1gMBg2duRYRxAErFq1CmazGefOncPEiROxevVqTJ48OSz/RjzPY+bMmbDb7Th8+DDsdju6u7vxwAMPoKysDABQVFQU4lYybgam+ffD8zwmTZoEl8sFnU6H9PR07N+/H++//37IfPu/Cp7nJaHBcRw0Gg2APlNCUlIS8vPzRzVNAOPqiM//4MGD+Oijj3Dq1Cm0t7fjxIkTWLhwIZYtWxa2fyOe5/H8888jISEBM2bMwIkTJ7B+/XrMnz8fZWVlzPwT6QxlYyAUx2hu+AYCASopKZFq46akpJBSqQzLjbgr8Xq9pFQqpdqrcrmcSktLSaPRhNw9NdYR3XO9Xi/l5eVJ9Z8BUHp6esTUYHY6nYM2qRMSEqT61YzwA8zPf2gEAgHJw2fgkZGREREDk4iknPBqtZqUSiXp9Xrm9RMm+Hw+ysvLkwK74uPjSavVRozPvBiYZrfbpaC03NxcMpvNrJBQmDJU4T8sZh+O4/6Z47jPOY6r5Thu1VW+L+E4Lshx3Cf9x0PDcd/hYty4cUhISJDea7VaPPvssxFhNhEEAZ999hkcDgfGjBmDrq4utLS0YPPmzaiurg518xgAmpqaJKeBS5cuAQC2b98eVh5kX8WOHTuwePFidHZ2AuhLTKdSqcJyP4xxAwxlhviqA4AcQB2AiQBUAD4FMPWKc0oAvHAj1x1Ns4/P55OiLq1Wq2QuiRStxufzkc/no+zsbLJaraTVaslut0dM+6OVQCBAc+bMIYPBQBqNhnQ6HaWlpVF2dnZE/G0Gmq3y8/MHma3i4+MjYuUSi2AUNf/bAdQS0XEi6gTwGoDCYbjuqFJXVwcA6O3txY4dO6Ti6eGOmPDt6NGjqKurk7TJ06dPo6amJrSNi3F4nkdpaSm6u7vR3t6Orq4u8DyPrq6usHMfvhpiofmqqip8//vfR2Njo6TtsyyfUcBQZoivOgD8AMCvB7x/AFdo+ejT/M8B+BuANwFYrnGtxQAOATiUnp4+YjPjlVqX2+0eFHkZaXVxfT4f5efnS/WGxcPhcETMvkU0EggEKCcnh+Li4kgmkxEASk5OJqVSGVH2cjGi3Gg0ksVikfqXQqEgt9sd6uYxrgBhFuH7nwCsRHQrgPcA7LraSUS0nYimE9H0kQp8uTJ9bXV1NR566CF0dnZCr9djwoQJeOGFFyLKjS0zMxM7duzAn/70JzidTmi1WthsNiQkJGDVqlVhv28RrQSDQbS0tKCnpwe9vb2YOnUqmpqaUF5ejr1790bEyhL4R0S5w+GAWq2GRqMBx3FITU3FqlWr2N5ShDIcQV5nAVgGvB/X/5kEEV0Y8PbXADYMw32/FmKCNJ7nIQgCjh8/DiJCV1cXxowZgzFjxmDcuHFhF3V5PURBYjAY0NXVhcceewwJCQmYOXOm9F2kmLKiAUEQUF5ejrq6OshkMiiVShw+fFiqwhZp8DyPH//4x1i0aBGsVis0Gg1Onz6N9PR0PPfcc8jIyGB9K8IYDs3//wG4heO4CRzHqQD8EMA7A0/gOG7sgLd3AwipMVoU/IsWLcKZM2ekiNikpCQYDIaIrYVbXV2NVatWQalUYuPGjZg3bx72798P4MsrHsbIwvM8CgoKYLfbkZSUhN7eXsTFxWHTpk1wu90R2b8KCgrgcDjwxRdf4NKlS2htbcWZM2ewfPnyiPw9Mc9QbEPXOwDkAziKPq+f1f2frQFwd//rXwD4DH2eQF4A9utdc6S9fXw+H2VmZg6ykScmJkZ8YJTT6SSDwUAJCQlUWlpKJpNJsvtHio05GvB4PMRxnOQhI9aGKCgoiOi/w5Xj5pZbbmH7SmEGRtPmT0TvEtFkIppERGv7P3uKiN7pf/0YEX2DiKYR0XeI6Mhw3PfrInrIpKWlDfq8vLwcu3fvjljt2O/3o6qqCj09PWhra8POnTvR09Mjfc+0s9Fj5syZGD9+vNSXFAoFXn31VVy6dCkiPH2uhdlshkqlglarhVarhcFgQGlpaUTtkTH6iLncPqLde9q0aThw4AA4joPRaITL5cJTTz0VsSYfoG/j98UXXwTHcVJ92MLCwrDMTRTt1NTUIBAISO9PnTqFpUuXAkBEb8IHg0HU1taiq6sLNpsNly5dwuHDh5lJMQKJKeEv2r39fj9mz56NxMREEBHmz5+PefPmRcWGaFFREV5++WWoVCoAwM6dO7F582Y2MEcRQRDw7LPPIiUlBUqlEjzPo7u7G0Dfhvy6desitp+ZzWbcfvvtePLJJ5GTk4Pa2loUFxeL5l9GJDEU21AojpGy+YvFzw0Gw6Di1FlZWRFvjxUZGLGM/mpk4VQjNtoJBAKUlZUlJQqMj48nu91OPp8vKv4GbrdbilsAQHq9nhITE5ntP0xAmPn5hw2ZmZnYsmULli9fDrVaDY7jkJ6ejqSkpIjWyET8fj9WrlwJoC/Vs0qlQnJycljViI0FNm3aBJvNBq1Wi9TUVASDQSxbtizUzbppBEHAb37zG2g0GqjVasjlcowZMwaTJ0+OOPfoWCfmhL9YrtHpdKK7uxsPPvgg2tvbsXTp0oi3jYsb2RUVFUhISIBKpUJnZydOnjyJffv2hbp5MYEgCJg/fz6eeuopCIKAzs5OJCQk4M3R4w3zAAAgAElEQVQ338Qrr7wSFRMwx3GQy+XQarXIy8vD6dOnwXFcRG9kxyIxJ/zFYBW9Xo+UlBS89dZbeOCBByLay0dEDGDLzc1FeXm55JHB8zy2bdvGPDJGgWAwCI7jUFJSgttuuw0bNmzAL37xi6gJghL72G233YaEhAQcOHAAAHDfffeFdWEaxlUYim0oFMdI2vyNRiNptVrJZqlUKiPev38gYjZGp9NJOp2O5HI5lZSURIW9OZwRc99nZ2dTYmIipaamUmJiIiUnJ0fdnovP56OsrCziOI5sNhvl5OSQ2WyOqnEUqWCINv+Yq+F79OhRtLS0YMyYMWhvb8fy5csxY8aMqKpHyvM8ysrKcM8996CtrQ1EhJ07d+Kjjz7Ce++9FxUaaDjC8zwqKipw/PhxLFmyBH//+99xyy23YPv27VGj+YuYzWZ0dnZKXj7l5eXw+/34yU9+gsmTJ0e8CTUWiCnh7/f7sWXLFthsNpw+fRrjx49Hfn4+qqqqBuXAiQZMJhPkcjmmTJmC06dPIyEhIap+Xzji9/vxgx/8AK2trejq6kJSUhL27dsHs9kclc9+/PjxmD17Nl588UXMnz8f3d3dsNlsbOM3QogZm7+YaOujjz5CIBBAa2sr6uvr8dRTT8HpdEbd4MzMzMT//u//YvHixWhtbcX58+dx//33h7pZUY3ZbMbYsX1prEwmE5qamvDhhx9GZQAUz/NYt24d/va3v6Gnpwft7e0gIgSDQVZHIlIYim0oFMdI2PwHFqJWKpXkdrujyg57JV6vV6pToFaryeFwRJ3tOZzw+XzSXpJarZaqqUXz8/Z6vaTT6aT6vqWlpVRUVBTVvzncAfPzH4wgCPjrX/8KnueRlpaGuLg43H777VGn8YuIUaYqlQoWiwUdHR347LPPcObMGel7xvDS2NiIjo4OAJD+DQaDUdvHRDo7OzFv3jzo9Xr85S9/icqVdDQSM8IfALKzsyEIAmQyGcaNGxfq5owoPM9j9+7deP/99/HQQw8B6CtRefToUezZsycqTRGhJjc3F48++qj0vru7O6Lz+AyFpqYmdHd34+WXX4bBYMCnn36KDz/8MNTNYgyFoSwPQnEMp9lHdMHLyckhtVpNMpmM5HJ5RJXS+7p4vV5SKBSkVqsHpXtgxbeHH4/HI6UN4TiOPB5P1Pcvn89HVqv1S6nRWf8KHWBmn3/A8zx27NiBGTNmgIgwd+5cTJs2LaIzeA4Vk8kkheFbLBZotVpwHBfqZkUdfr8fjzzyCBISEsBxHKZMmRJ1HmRXIkaUb9q0CRbLP4r5GQwGPPfcc1G94okGYkL4A8CePXuwadMm3HrrrXj99ddx9OjRUDdpVGhsbIRCoZC8m8RZf+3atWxwDiNmsxnp6ek4c+YMtFotzGZz1Kc74HkeTqcTbrcbCQkJsNlssFqtCAQCKCoqiuqJLxqIeuEvCAKqq6uxfv16xMXF4dChQzCbzVAqldL30YogCKiqqkJlZaWk7Xd0dIDneaxevZoNzmFgYP+xWCwgIqSmpqK8vByVlZVR3b+APpfiRYsW4ciRIxAEAUlJSVi9ejU8Hk/U//aIZyi2oVAcw2HzF239BQUFlJubO8jmnZ2dTT6fL+rd0kRXQ5vNJrm4yuVyys7OjurfPRqIaTR8Pp/0fO+44w5KSkqivLy8mElxHAgEyOVykd1uJ71eTzqdjmbMmMH6V4jAEG3+IRfy1zqGa8M3EAhQaWkpASC5XE52u51mzJgRc3VtvV4vWa1WkslkpNVqyeFwxMxvH0nEyVWsnyAKvlgS/HPmzKHExMRBOf6tVmvUxziEK0MV/lFv9jl48CBee+016b1SqURcXJwUgh4Lpg9BELBx40YkJSWht7cXRITDhw/j4MGDoW5axMPzPILBIOrq6gAAbW1tePjhh2Mmt43oUrxv3z58+9vfht1uR3JyMurr67F//37mUhzODGWGCMUxHJq/z+ej1NRUKi4uJo7jyGq1Ul5eXky6oYlLc/RHn8bHx7No32FCfK4AKDc3N9TNCQmBQIA8Hg/pdDrpWTgcjphZAYUTYJp/nwfG8uXL8frrrwOAlN+mqqoqJnPbv/XWW9Bqteju7sajjz6KzZs3h7pJEU91dTVWrlwJjUYDoG+lWV1dHVPariAIWLhwIbZs2QK5XA6dTgeTyQSlUsmSvIUxUSv8xWLt586dAwDcfffdaG1txS9/+UuUlZXFhCfGQILBIFQqFWw2G3p6erBhwwY89NBDWLBgQUw9h5Fg/PjxsFqtsNvtsNvtMJlMMWXuEONoxCIv9913HxobG/Hpp5/i4MGDMfMcIo2oTenM8zzmz5+PH/7wh1CpVPj0008hk8mgUqmQkZEREwFeImIwzhNPPIGPP/4YP/vZz9DS0oILFy5ApVKFunkRS3V1Ne69915cvHgRQF9wk/g8Y6l/Af/YOyMi1NTUYMKECYiLi8OGDRtgMBiipoRlNBG1mj8ATJ48GSkpKVIdW51Oh1/84hfgeT6mOqIYjFNVVYXx48fDbrdDrVbjxIkT6OzsDHXzIhIxhuLhhx+WPnvggQdQXV2NzMzMmOpfA+nq6sKRI0cQFxcHADhy5IiU5I4RXkSt8Bfz91+4cEEKcFq6dClyc3ND27AQkZmZibKyMpSVleH+++9HR0cH1Go1VqxYEbOC6mbgeR4FBQX44IMPpIDB7du3x0zk+LVQqVTo6elBXV0d6urqQERYs2YN62NhSNQKf57nMXv2bLS2toKIkJubi927d8fkRq9Ibm4uDhw4gNmzZ8NmswEANm7ciOrq6tA2LALZt28fli5diuPHj6OnpwcA8OSTT2L37t0xa+PmeR5r1qyBXC5HW1sbWltbAfTll2KEH1Er/P1+P55//nmkpKQAAC5duoTdu3fHjP/1VzF37lycPXsWra2t+Nvf/ob77rsvpifFG0UQBGzbtg08z6OhoQF6vR5An80/1mz9V5Kbm4vKykokJycDAPR6PfP4CVOiVvg3Njaira0N58+fh9VqRX19PdNA0Gf+efzxx6X3ycnJeOONN9ikeAMEg0FcunQJzc3NiI+Px+XLl6HRaPDuu++GumkhZ9++fXjssccwduxYqFQqjB07FsFgMGZXQ+FMVAp/QRCwatUqEBF6e3vB8zwee+wxJuDQtyL6+c9/DpvNBo1Gg0uXLqGpqSnUzYoYxL0ktVoNlUqFS5cugeM4/PznP495jxZBELB7927s2bNHekZ1dXV48MEHMXfuXDYBhBlRKfyDwSCOHDmCtrY2AMC3vvUtPP7448y2jb7At9tuuw3l5eWwWq3o7OzEkiVL2LO5ATQaDfLy8gYJs7feeiuELQoPeJ7H1q1bkZSUhJUrV0Imk0Gv1+Pzzz+H3++P+hTXw4HYp0ZjooxK4W82mzFp0iTJC2POnDn47//+75j19BmImM65vLxcmiBLSkqk4htMO7s+69atw/vvvw+LxQKdToenn34ae/fujWmtfyBVVVV48cUXMX/+fDQ0NKCtrQ08zzPb/3UQA1P9fv+oBAlGpfAH+jZ4u7q6UFhYiJkzZyI3N5cJtgG0tLRALpejsLAQdrtdCs6JpcjUG0UcnEePHkVzczPOnj0LIsILL7zAtNp+xJiSDRs2YPfu3QAAuVyOM2fOsGd0HcSVk9lshtPpHHFlIiqFfzAYRH19PQBg//79mDt37qjNppFARkYGpk2bhgkTJuDtt9/GokWL8I1vfANVVVWj0ukiFZ7nUVZWhtLSUmRkZKC3txdyuRzr1q1j+0lXUFdXJ3lBdXV1obm5OeZjIIbK3Llz8fDDD0eG5s9x3D9zHPc5x3G1HMetusr3ao7jXu///q8cx1mH477X4r333pOiCp988kns3bsXmZmZMe+GN5DU1FS89NJLsFqtAIBt27ahoKCACbHrkJGRgbS0NOzatQsymQwmkwn79u1jSsUAMjMz8eKLL6KzsxMWiwUymQylpaWYPHkye07XIRgMwufz4dChQ6ipqRnZmw0l9edXHQDkAOoATASgAvApgKlXnFMK4Ff9r38I4PXrXfdmUjoHAgGyWq1kt9tZStlrEAgEpAIv6E/BazQa2fMaAmIK58LCQlKpVOR2u0PdpLDE5/OR2+0mjuNIq9WSwWCgnJwclkb8Orjd7pt6ThjFlM63A6glouNE1AngNQCFV5xTCGBX/+s3AczhxJwLI4Doh33u3DksW7aMaRtXIRgM4t577x0U/zCCf5Kowe/345VXXoFGo0F1dTUmTpzINP9rYDabsX37dqjVainit6urK9TNClsEQYDf78eqVatQXl4eETb/NAD1A96f6f/squcQUTeALwAkD8O9r0pmZiaeeOIJOBwObN68mZl6roLZbMbEiRPR09ODlpYWAMA999zDzD5fgSAIWLRoET7++GO0t7ejo6MDCQkJWLduHetjVyEYDKKmpkZSKh555BG89NJL7FldhYHOBB0dHdiwYUNk2PyHC47jFnMcd4jjuEM34xkgFthobW1l7mXXgOd5bNq0CUajEfHx8QCAnTt3Ys2aNUyL/Qra2tqgVqsB9G1kEhHrY9egsbERSqUSaWlpsFqt+O1vfyu5MjIGM9BLqrm5GZ999tmIe0cNh/A/C8Ay4P24/s+ueg7HcQoARgAXrrwQEW0noulENP1mBpTJZIJGo2G56q9DRkYGJk2aNEjYO51OFo15Dfbv34/Dhw+jq6sLubm5uOOOO/DrX/+aabJXQUx5/dOf/hTHjx/HuXPncPnyZVy8eJGZYq+B2WxGZ2cnNBoNWltb0djYOKL3Gw7h//8A3MJx3ASO41To29B954pz3gHwYP/rHwD4Q//GxIghl8ulIC/G1eF5HgsWLADQF7VqNpshk8lGxd4Yafj9fjz++OOYMGECEhMTUV1dDYfDwcxk10DUZN977z0AkLzv/vmf/xkjPPQjlpqaGpw8eRJjxoyB0Wgc8VxkNy38+234/wrgAIAaAG8Q0Wccx63hOO7u/tPcAJI5jqsF8DMAX3IHHU7MZjOmT5/OXDuvgyAIeO211wAA7e3tCAaD4HkeM2fODHHLwg+z2YyVK1dCEAR0dXWhuLgYJ0+eZBrsV5CZmYk333wTe/fulVw+AeZYcDUEQcCTTz6JtrY21NfX47nnnhtxxWJYbP5E9C4RTSaiSUS0tv+zp4jonf7X7UQ0l4hsRHQ7ER0fjvteC57nsXnzZqaVXQee57F+/Xp885vflPL7NzQ04ODBgyFuWXghCALmz5+PZ555BpcvXwYR4eTJk6wK2hAQla/6+nr09vZi48aNOHXqFIv2vQpffPEF2tvbQURISEgY8fuF1YbvcCEIgqSlMa6NaJd1uVzQarUA+uI+nnjiCfbsBsDzPEpLS/HQQw8B6NNoDQYD8yQbIjNnzkRmZib0ej04jsPx48dHJYI10hATUYqTZEx5+wwnbGl5fcRcIiaTCfX19dIz+/zzz1FTU8O8Mvqprq7GvHnz0NLSApPJhD//+c84efIk8/IZIjzP43e/+x3+67/+C5s2bcK0adPYftwABEHAwYMHcfx4n0FEq9WOyqoyKoU/z/Nwu91MKxsCPM8jMzMTf/zjH7F8+XLI5XJoNBo0NTXhzjvvZBMA+ryiLBYLtm7dCoVCAY7jWGGgr8GTTz6JJ554AgAkd9lYRxAELFiwAFu2bBnknTgaprGoFP6Mr8dvfvMbWCwWdHV1ISkpiZW9HMBLL70EjUaD8+fPY8qUKSxY6QYJBoMIBAJobW1FU1MTcnJy2PPrR6VSYc2aNbDZbJg4cSJeeOEFVFdXR8aGb7ghRssxm+KNMWnSJCQlJaG8vBwZGRmoqqqK+WcoRvXu3bsXHMeBiKBUKpnJ5wYQ9+DEfaWGhgZUVlZix44dIW5ZeFBRUYGPP/4Yfr8fJ06cwIoVK0bnxkNJABSK42YSuxERSx51AwQCAcrPzye73U4cxxEAcrlc5PV6Y/45Xpn8juM4SkxMZAnwbhCfz0cOh0N6jsXFxTH/DAOBAM2ZM4cMBoP0XHQ6HXk8npu6LoaY2C3kQv5ax80Kf8aN4fV6yWQySYLOYDBQcnIy5eXlxewE4PP5yGQyDRqcxcXF5PV6Q920iMPn8w16jmq1mk2i1DfuBj4Xp9N509ccqvCPSrMP48bJzc3F3r17odFoAADx8fFYvXo1Vq9eHbO2WTEv/ZIlSwAACQkJeOONN/DYY4/FvDnsRjGbzcjOzobT6UR8fDw6OjrQ1tY24ikMwp3jx49L40ur1eKPf/zj6NXTHsoMEYqDaf6jj8/no8TEREpNTZVMHLGc41/U/JOTkyklJYXkcjk5nc6YXQndLF6vl+x2+yATWnZ2dsw+T4/HIz0HABQXF0dut5tSU1NvasyBaf6Mr4NcLsf3vvc9AH3ueD09PSFuUegwm8148cUXcdddd0EQBPT09GDcuHExuxK6GQRBwMaNG6HT6SSXRoVCIb2OtZWUIAgoKipCaWkpkpP7stv39PRg4sSJOHDgwOh42Q1lhgjFwTT/0ODxeEin00n2baVSGZM2bnEzTnwWZrOZXC5XqJsV0fh8PsrLyyOPx0PFxcUEgEpLS8nn81FRUVHMrAACgQAVFRWR0+mUVkHo32fLy8u76ZU2mObP+Lq0trYCAN5++21MmjQJGRkZIW7R6MPzPObNm4fu7m4AfX7qRqMxxK2KbMxmM4gI27Ztw4ULF8DzPLZv346HH34YTqczZlZUPM9j/vz5eOGFF1BSUgKdTge73Y6dO3di6dKlWLVq1aishBQjfgdGRDF58mRYLBbU19ejvb0dp0+flrJ9xhJiQSCFQgGZTAaLxYKCgoJQNyui4Xkeu3fvRk1NDT7++GP84Q9/QG9vLy5evBhTcROCIGDLli1ob2/HG2+8gdbWVpw+fRpr165FfX097Hb7qLSDaf4MCUEQUF5ejosXLwIAkpKSYLfbY84jQ7RP//jHP4ZCoYDFYsH48eNZJsphIBgMYu7cuXjiiSfQ29sLAKitrY25TLIqlQpWqxVAXx6y9vZ2KJVKvPHGG9i7d+/oKFtDsQ2F4mA2/9AQCATI6/WSxWIhAKTVaikxMTGmAr4CgQBlZWWRUqkkjUZDcrmcXC4XFRQUxMwzGElcLhfJ5XLSarXEcRylpqZSTk5OTD1bn89H2dnZZDAYqLCwkOx2+7B51YHZ/BlfB57nYTKZcPnyZeh0OshkMnR1deHRRx/FggULYsYrIyUlBQ8//LCU0uGVV17BpUuXQt2siEcQBPzud7/DtGnT8K//+q8gIjQ0NODTTz+NuZVVZ2cnLl++jLfffhtHjhzB0aNHR/X+TPgzvoRYCa2qqgpTpkwBEaGurg5NTU2hbtqowPM88vLysG3bNrS1tSE9PR1KpZJlohwGxAJCer0ev/rVr6TPbTZbzNj9xdxjbW1tkMlkSE9Ph9vtRlFR0ai2gwl/xlXZvHkzXn31VXR1dYHjOIwdOzZmKjD5/X44nU5wHIfU1FTwPM+KtwwjmZmZ2Lp1K6xWK+x2OwwGAwoLC8HzfEysLPfv34+//e1vqK+vBxFh7NixIXEmYMKfMQhRKwGApqYmHDt2TKowNHXq1JjRzrq7u6HRaKSavaws6PDS2NiII0eOYMaMGejo6MDTTz+NzZs3R3023n379mHJkiXo6urC2LFjodPpsG7dupAoFUz4MwYhVvcCgMOHD6O9vR1yuRwNDQ0oLy8PcetGh8bGRnR2duK+++6DTqdjKZxHAJPJBKVSiZ07d6KzsxNmsxlvvfVWVPv7C4IAt9uNSZMmoaOjA+fOnUNHR0fI2sP8/Blfgud51NTUDCol19zcjNWrV2PcuHFRrwU3NTVBpVJh586d0Ol0rCToCGA2mzFlyhR88sknSE1NRXNzc9RXjeN5HhUVFQD6Erp99NFH+OCDD0JWFY4Jf8ZVycjIwPTp03H+/HmcPHkSAHDs2DHodDqsWrUqastk7tu3D4sXL4ZWq0VLSwuIiNWbHQF4nsfLL7+Mhx56CIsXL8bGjRujPoLa7/dLm7ptbW1ob2+HTqfDsmXL8Morr4z6eGJmH8aXEARBqoM8btw4WCwWKBQKPPDAA9i1a1fUCn6/348f//jHaG5ulgLbMjIysHXr1qj8vaHGbDZDpVKhrKwMR44cgSAIo5baIFTIZDIUFhZCJpNBpVKhqqoqJIIfYMKfcQUDS2CazWZ0dXXh4sWLMJlM2LlzJ+bNmxfqJo4YjY2NaG5uRldXF7RaLRwOB3bt2hXVJq5QU15eLjkU1NfX4/bbbw9xi0YWjUaDnTt3orW1FZ2dnfjlL38ZsrYw4c8YhLjhK2oinZ2daGlpQSAQANA3QKPV3TMjIwNTp05FSkoKbDYbM/eMIKKSMXnyZGRmZkKj0aCnpwdr1qzB3Llzo0r7FwQBgiBg3rx5OHv2LKZPnw6j0QiLxYJdu3aFbFXJhD/jSwzsjMnJyUhJSUFCQgKcTicmTpwYwpaNLDU1NaitrUVDQwM+//xzfPrpp1HvehgqRCXDbDZDq9Vi3LhxAACLxRJVZjZxkhOTI9555504dOgQZsyYAUEQQpo3iwl/xjXheR75+floaGhAfHw8fvvb3+KTTz6JSoHo9/vxs5/9DO3t7QCArq4u6HQ6lJeXR40gCjd4nkcwGMSJEydgMplQWlqK5uZm7Nu3L9RNGzYGTnLt7e348MMPwfM8fv/732PixIkhTZfOhD/jK5k3bx7sdjsuXLiA2tpa9Pb2Ii8vL9TNGlbEbKa1tbVSDWO1Wo0tW7Zg9+7dUTfRhROZmZl44403cPHiRezatQu9vb2orKzEmjVrQt20YUNUHp555hl0dnbiiy++gEqlQkJCQkjbxYQ/4yvheR579+7F2LFjpc/efvttLFq0KKr8smfPng2O41BYWAigL8J34sSJUWWCCFeamppw5MgRtLS0oKmpCUqlEi+88ELU9C+/348FCxbg+PHjaG1tRXt7O3p7e0O+p8T8/BlD4ty5c9BqtVJE4sKFC1FZWRnxwlEQBNx999348MMPAQCvv/464uLi8O///u/Izc0NbeNihMmTJ0Ov16OlpQVqtRoKhQKPPfZYqJs1LIiryk8++QSXL1/G0qVL4fF40N3djTVr1oR27Awl73MoDpbPP3zweDzEcRwVFxdTYmIiORwOysvLi5ravj6fj1JTU6VaqhUVFaFuUswh5vhXKBQEgOLi4ig5OXnYctyHkkAgQG63mwwGg9THSktLR+x+YPn8GcNFUVERnn76abz11lsoLy/HiRMncObMGTz33HNRYQ//8MMP0dDQAKCvwtKMGTOixuQQCQiCgD/96U/4l3/5F3R3dyMtLQ0TJkyIqkSCe/bskVKCm81m7Nu3L+R9jAl/xpAoKiqCwWDAn//8ZzQ3N+PYsWOYM2dORJt8gD57bFlZmfReoVBg4cKF+O53vxvywRkr8DyPsrIy/OUvf0FaWhoeffRRnD59GiUlJaFu2rAQDAZx9uxZNDY2wmQyYdq0aXj11VdDHzw4lOVBKA5m9gk/KioqCADl5uYSAEpOTo5404/H4yGZTEYAyGq1kt1up5ycnIj/XZGIy+UihUJBKpWKAJBOp6O8vLyINv34fD7Ky8sju91OACg+Pn7E+xZGw+zDcVwSx3HvcRx3rP/fxGuc18Nx3Cf9xzs3c09G6PjRj36EuLg4fOc734FMJsNPf/pTPPvssxFr+vH7/XC73UhPTwfQt6ldV1eH8vJyttk7yojlHceOHStlk+3s7MT999+PysrKiOxj4mavIAjo7u6GVqvF+PHjQ5bF80sMZYa41gFgA4BV/a9XAVh/jfOab/TaTPMPLwKBgFTYXKfTkUwmI7vdLhV3jzR8Ph8VFBSQ1+slp9NJaWlpJJPJyOl0EhHFVDHxcEEsaq5SqUgul0tFzSNV8xfHi7iyBEAGg2HEi9VjlDZ8CwHs6n+9C8A9N3k9RhgihqjX1NSAiNDa2gqlUon6+np0d3dj7dq1EaWZCYKAlStX4tKlS3j44YdRWVmJs2fPore3F2+//Taqq6ujMoo53DGbzSAi9PT0ICEhATqdDqWlpRGb6TMYDGLs2LG46667APQ5E2zZsgUGgyHELetnKDPEtQ4AFwe85ga+v+K8bgCHAPwFwD1DuTbT/MMLn89HRUVF5Ha7CQBpNBqKj48nh8NBbrc71M27YXw+H3k8HtJoNASA1Go1xcXFUXZ2NhUUFESsthnJBAIB8vl8xPO89DdxOBwR+bcQbf1Op1Pax5DJZOT1ekd8VYkhav5DEfC/A+C/ylF4pbAH8PdrXCOt/9+JAE4CmHSN8xb3TxKH0tPTR/QBMW6cQCBAXq9XWsKmpaWRwWAgpVIZUaafQCBA+fn5lJWVRUqlknJzc2nWrFnk9XrJ5/Mxk08ICAQCVFRURF6vl3iepzvuuIMAkFwuJ4/HE+rm3RCBQIDmzJlDer2e4uLiSKvVSpvYo/Fbhk34f+V/Bj4HMLb/9VgAnw/h/+wE8IPrncc0//DE5/ORwWAgrVZLAMhisVBWVlZECUyfz0der5ccDgdZLBYCQDabjbxeLxUVFUXUb4kmxOfucrkkO7lKpaKcnJyI0/4HepGp1WoCQOnp6aPSt4Yq/G/W5v8OgAf7Xz8I4O0rT+A4LpHjOHX/axOAmQAO3+R9GSFk/PjxuPvuuwH05fdvbW0NcYuGjt/vx3e+8x0sXboUPp9PqlNw4sQJPPXUU1FdQDzc4Xkefr8f+/fvh0qlAtDn8RMMBrFs2bKIsvsnJSVJSQLj4+MBIOzKVN6s8F8H4J84jjsGIK//PTiOm85x3K/7z8kAcIjjuE8BeAGsIyIm/CMQQRCkhG5vvPGG9PmpU6dQU1MTwpYNHbPZjPT0dJjNZlgsFnR1dSEtLQ3Tp0/H1q1bQx94E8OIZRy3bNmCn//851Cr1eA4DnfddZcUHRsJ+P1+PProo5LLajAYhPGzcUYAAB0aSURBVM1mw549e8JLsRjK8iAUBzP7hCeBQICcTqdk9pk+fTpxHEfZ2dkRYS7xer2UkpJCXq+XKioqSKPRRKRZIRoR92LcbjclJyeTwWAgjUZDcrk8YpwKrubeCUByIR4NwHL7MEaKv/71r8jIyIBarcahQ4dARDh//nzYl3cUBAFVVVV49dVXcfz4cWzcuBHt7e0oKSlhGn8YwPM8Kioq8Pjjj8Nms2HJkiWQyWQgIjz33HOorq4Oe9NPTU0N6urqkJ6eDo7joNVqYTab8cgjj4S6aV9mKDNEKA6m+YcvgUCAPB4PGQwG0ul0pFKpyOFw0IwZM8Je+xeDhrxeL2m1WtJqtRHlqRQLiG64SqVy0IZpfHw85eXlhW0fCwQCNGvWLJLL5WSxWEiv10tePqO5cgHT/BkjydatW6FQKGAymdDZ2Yljx47ho48+wp49e8JWOxMEAUuXLsXs2bOxdOlScByHzs5OPPXUU2Hb5lgkMzMTRUVFWL9+PeRyOdLT01FRUYFvf/vbWL16dXjZzQfA8zyKioqkAEidTofu7m7I5fLwrH09lBkiFAfT/MMbMUc5x3GSP3ZxcTGp1eoRD1//OogBRNnZ2VJgl06nI7vdHnF+5LFCIBAgq9VKHMeRwWAgl8sVlvszYl8f6N4pHqWlpVL8yGiB0fDzH8mDCf/wRhSmRqOR7HY7TZgwgfR6PblcrrAU/Pn5+ZSdnS1lVwRACoWCDAYDmc3msBMojL7NebG4C8dxpNPpSKFQhNUEIAan+Xw+mjVrFslkMkpPT5ei4BMTE8nj8VBqauqotXmowp+ZfRg3jN/vR2lpKRobG3Hrrbdi8eLFOHHiBFpaWrBly5awc/sUNxLr6urQ0dEBhaKveqlYOORXv/oV2/ANQ3Jzc7Fs2TIAgFKpRG9vLxYvXow1a9aETb4fnuexdetWmM1mPPPMM9BqtWhrawMApKSkQCaTYfLkyThw4EDY9TEm/Bk3hCAIqKysRFlZGTZu3Aigr+6t6Id95swZFBUVhV0hlNzcXJSVleHEiRPo7u4GAGi1Wuh0Omzbti0sBAljMIIg4IMPPgDQF+zV3t6OrVu34tFHH5VqSYcLixYtQlNTE1paWhAMBqFSqZCUlIQ333wTmZmZYSf4AVbAnXGDiJoOz/PIyMhAMBjEvffei46ODqSmpiI5ORl6vR6NjY2hbuogBEHAf/zHf0CpVKKrqwtarRYvvPAC9uzZE+qmMa6BuIH6pz/9CWPHjoVCoYBarcamTZtw/PjxsNn4DQaDICIkJSWhoqICzz//PJRKJU6dOhU+ufuvxlBsQ6E4mM0/MvD5fKTX6wkAabVaysrKIpvNFnZ29IGb08XFxZLrnVingBF+iAnSbDYbyWQySklJkepIyGSysNioF/eTnE4nJSQkSH1MzBUVCsBs/ozRIDMzE88884z0vqmpCbW1tUhOTg5hqwbj9/tRUVHR5+EAYO/evViyZAn27dsHnufDRoNkfBmNRoPS0lL09vYiEAigt7cXd911FyorK1FUVBTq5iEYDOLChQtYu3Yt9Ho9tFot0tLS8MUXX4S6addnKDNEKA6m+UcOYsoHo9FIcXFxZLX+//bOPjaq81zwv3e+x19jexhmbMfGpBMybj2NsHIxSeoVVzhLl15dVNQqyWarulCRjbvpjZQ4oSHBmaC2kCbje0sLaoSdUoU0LBl3d9u6y+Zqh6pSddsb7m6YEGhEEcUFPDEBh4R4xth+9g/7nB0DBvPlmfG8P+koM2eOz3lOeN/nPO9zno96CYfDOZGQY1w/EomIxWKRqqoqiUQiEo1Gc0I+zZXJTCi0Wq3mCtPhcGQ9smxgYMDs/WD0t2Yy7DkajcrKlSuzIh/a8tfMBslkkjVr1tDX14fP5+P8+fN88MEHHDx4kDNnzmRdtrVr19Lb28vLL7/M+Pg4p06dIhKJ0NnZyf79+3O+JEWhMzg4yGOPPcbXvvY1xsbGWLFiBVarlS9/+cs8/fTTfPWrX83qy3qZXE3+8pe/xGq1Ul9fT1lZGR6PB6VU1uSaCVr5a26Yjo4Ojhw5wsmTJ1FK4fV6cTqdFBcXAxNKOFsTNJVKsX37dhYsWIDb7cblctHZ2cldd93FokWL8Pl8WZFLMzMaGxvZtWsXiUQCp9PJ3r17+eSTT9izZw92u50XXngha247v9/Pli1b6O3t5fDhw/j9fgKBAI8//jjPPfccTz75ZE67FLXy11w3Rm/fhoYGXnzxRQDGx8fp7+9neHiYpUuXMjg4yJo1a1i7du2sPwQGBwd59tln6erqAmB4eJh0Os1vfvMbNm3alNuRGBqThoYGPvnkE0ZHR3G73cyfP5+amhpGRkY4evRo1gyLZDLJ17/+dV544QUABgYGOHDgAD/60Y/YtGkTDQ0NWZFrxszEN5SNTfv88wMjWsbIbiQjtV0pJUuXLjVbIxp9gGfDDxqLxaSiokK8Xq+0tbWJx+MRi8Vi9unVUT75w8DAgLS0tEg4HDaLvDHZRa6ioiJr5UQSiYR4vV6ZN2+eWXiuoqJCuru7pbW1Vb70pS9pn79m7mJEy2zfvp1FixYBYLFYWLVqFTBhXJw+fZrBwUE6OztnpVNWb28vjzzyCOPj49hsNn7605/icDjo7Oxk586dZiRSLi/JNVMpLS3l8ccf58KFCwA4nU5Onz5NKpWaVTkyVxk+n4+qqiozp+XcuXMUFRWxZMkSHA4Hmzdvzu0xNpMnRDY2bfnnFwMDA7JkyRKxWCzicrnM4mmrVq2SefPmSWtr66zE/ScSCamurpZYLCaRSGSKpWi1WqWlpUWXcM5DDAu6u7t7Sn0mJsslx+PxW76aM+r4GNcYGBiQaDRqlp4uLy8381uyuapEF3bTzDaJRELC4bBZgOvee+8VQEKh0Kwo3MzqiplNwMlwE+gm7fmLoXxXrFhh/psGg0HzIR8Oh2+pqyXz4TIwMCCLFy82k7psNpsUFRVJNBo1f88WWvlrZh3DN1tWVibBYFCsVqvYbDYJhUK33C9rKIZ4PC6VlZVisVhMq3/evHnicrnEZrOZFqImvzD+zaLR6JQHeiAQELvdLm1tbeL1em+ZkZFp9Q8MDEg8HpeKigpxuVxTVpclJSVZNzC08tdkhXg8Lj6fT9ra2qZM0vLycnNi3mrLLBaLSVFRkdllzHgQNDU1acWfh1yseOvq6qS0tHSK0i0uLpbS0tJb+nA3zt3c3CzNzc1mr4GysjLp6OiQUCgkXq9Xu31udNPKP3/p6OgQpZT4/X6pqKgQQGpqakx/u5H5eLMniFFnJRqNitVqFZfLZS7LAXNJrsk/jLESj8fFbrdLKBQSt9ttRpUZLTmbm5unZG7frDEWj8elurpaotGo2Gw289qA+P1+CQQCs960ZTpmqvx1tI/mprJv3z5+9rOfUVVVRTKZ5OzZs1RUVHDixAn279/Pt7/9bT7++GMOHTrEmjVrbmqMtlFn5fvf/z5+vx+lFA6HA7fbTXFxMffff/9Nu5ZmdjGiZhoaGli6dCnbt2+npqYGAK/Xy+joKMPDw1Nq6hh5KDc6xpLJJC+99BJbt26lr6+PhoYGbDYbDoeD8vJyzpw5w1NPPcWyZctysnTzdGjlr7lpJJNJtm7dyrZt2wgEAub+s2fPopRibGyMgwcPMjQ0xMaNGxkZGbmha2V+TiaTZuOPtrY2Tp48aSZ1jYyMsGnTpryamJrL4/f72bNnD/PmzWNoaAi/38/p06e5cOECy5YtY9++fWZSX2b58RslnU5TWVkJwPPPP4/T6WRkZIShoSEsFgt9fX151xNCK3/NTcOYbPfddx/FxcUEg0Gz3IPdbiedTjM+Pm7GahuT9FrJtOiMz4ODg5w7d47Dhw/zxhtvTDl+4cKFeTk5NZfH7/fT2NjIli1bOHfunLn/t7/9LS+//DLf/OY3zYxyQ/HfaHZ5KpVi48aNfPzxxzzxxBNmbL/f72dsbIyHHnoot2P6L4NW/pqbijEBHA4HXq+XqqoqPvzww0usfKPz19WW5Zf7LdOiMz6///77OJ1O0uk0J06coKamhkAggMfjwev15lznJ82NkUwmL2nEc88999DV1cX+/ftZvXq1WbTPKD5oPBCMfdeCy+Xi/PnzfPTRRxw7dszc73a7CQaD9Pb25lz3uqsykxcD2dj0C9/8Jh6PS0tLi4RCITMJhoySD5nNVKbj4qSai38zoipisZi4XC5ZtWqVeR2n0ynNzc3mS7hspdprbh3d3d1SUVEhtbW14na7xePxSH19vYRCIWlqapJAIGA2fLk4Rv9qoZiZvxmhnR6PR6qrq8XpdJpjuqSkRLxer8RisZzJH0FH+2iyhRF109TUJFarVSwWizgcDjPL9oEHHhCn0ymxWOySuOyLozQuF7WRSCRk+fLl4vV6zegOI7TUbreb17rjjjskkUhkPfROc/NJJBLi9/slHA5LRUWFdHR0iNvtFovFIkVFReLxeKS9vV0CgcBlI3CmGw8X16AyHhQX5xcYW2YCY66MMa38NVnFmETxeFwikYiUlZVJVVWVaZ3X1dVJfX292O32KZNn9erVlxSASyQSsnLlSlOJG8lcRrx1KBSSoqIic0LW1dVJMBg0Hw7TKQBNfpNIJEzr3wi9dDgcUlxcLHa7Xex2u4TDYXPcZG7TnW/lypXS0tJijt3MZjKZq1ellNTX1+dkQyCt/DVZJVNJG7H+mZvNZjPdM8Yky5ykmZ+NVYThujF+W7JkiQSDQbHZbOLz+cxzWywWiUajU86rmXsYdZwikciUsWWMt2XLlklLS4s0NTVJKBSS1tZWs9rmxatBwzXY3d0tfr/frNnT1NQk4XB4iuI3PofD4ZysE6WVvybrGMp7yZIlUlFRIe3t7VJbWztlklosFgmHw9LS0iI+n09isZgsX77cnKiGv9X4zfCtGtaYzWab8nApLy8XQDweT05aZZqbi+HWMzJujXFQVlYmgFRXV5sK22j7GIvFzIdALBa7xMCIRCKyfPlyKSoqkmAwaGYS22w2KSkpEZfLJe3t7bNaovxa0MpfkzPE43GJx+MSi8UkHA5PqbvDZGp+LBYzrbSWlhZzghqT2/jN4XBId3e3tLS0iMfjEb/fb57H6/VO8cVqi3/uY/TRNcZXZtZvbW2t2Gw2aW9vl6amJnMc+v1+aW5ulmg0Ki6XyzQqAoGAdHR0iNPpNLPUM1cUSimzdn95eXnOvkuaqfJXE8fmHnfffbe8/fbb2RZDc4O8++67tLa2UlNTw4EDBygpKWHjxo288sor9Pf3MzY2xsjICC+99BKnTp2iq6uLhoYGysvLATh//jx2u52nnnqKrq4uPv30U3bu3MlDDz1EKpXiz3/+MwA2m83s9FRXV8eePXt0UlcBkEwmefjhh1FKkU6n+eMf/2iG9a5atYpf/epXlJSUsG7dOnp6eliwYAHf+ta3ePXVV7lw4QLr1q3j9ddfRynF4sWL2bFjByUlJfh8Pt577z1SqRRWqxW73W5+vv3229m8eTOrV6/O8t1fHqXUfhG5+6oHzuQJkY1NW/5zB8N6X7x4sYTDYbP2js/nk9LSUrP2P5PvAILBoDQ1NUl3d7dYLBaxWCym77W1tVUeeOAB8+We2+2Wuro6M/QuHA6bYaSawsBwLxrjxXifpJSStrY2KS4uFkBcLpdYrVYJh8NmCLKx4qytrRWr1WqOw8wxyWRlWJvNJpFIxIw2y0WrX2Tmlr9O8tLcchobG/H5fDgcDg4dOkRnZydjY2MMDg7icDhMS81ms5FOpzly5AgHDhzgd7/7HS6XC5joknTo0CFGR0fZvXs3ixYtYmRkhOHhYY4fP86FCxdwuVx85Stfob29nd7eXuDak3k0+YeRWPjqq6+a3x0OByLC7t27zQRDESEQCHD06FFOnTpFKBSisrKS/v5++vv7KS8vp7i4mBUrVpiZ6TDRmW5sbIwXX3yRvr4+Tp8+zcGDB80ksrxlJk+I6Tbgq8BBYBy4+wrHfRH4E3AEWD+Tc2vLf+6RSCTMRitGaKbFYpGamhozRtvoAma8GHa73RIMBiUYDF4SMWS322XhwoXidrslEAiYoZ/Gi7tcfSGnuTXEYjHTym9raxOn0yk1NTVmXwlj3FgsFgmFQtLc3CzBYFCUUlMaxGRuxmrg9ttvl2g0Kkop6e7uzun3SczGC1+gAbgT2Ded8geswJ+B2wEH8A7w2audWyv/uYURUdHc3Cwej2dKXD6TL33dbrfZdMVw7QQCAXG73WKz2SQQCJiT1/g7I7rHWJL7/f4pNd214i8MjNDi7u5uCYfD4vF4zDFijBtA2tvbJRQKmZuxf/78+eZYM1w+TqdTysrKpK2tTTwejzQ3N0tpaWnOR5HNVPnfkNtHRA6JyJ+uctgS4IiIHBWREeANYNWNXFeTf/j9frZs2YLD4QAgFArR0dGB2+0G4Atf+AIiwsjICOXl5bz55ps4nU4GBgYYHh7G4/EwMDAAgN1ux2azATA0NER5eTk/+clPeOedd3jjjTdYtmyZ6QrIt2JbmuvD7/cTiUT49a9/zeuvv05PT4851s6ePYvL5UIpBcDhw4c5fPgwQ0NDWK1WAD744AMGBgYQER588EGUUiilGB0d5cCBA5w/f55169bx+9//ntdee21OjKvZ8PnXAP0Z3/86uU9TYDQ2NrJt2zY+85nPANDT02PWZN+7dy+pVIrx8XFOnz6NzWZj7dq1ps//ww8/NM8zMjKC3W43J+jQ0BAA27ZtY9myZbN7U5qcobGxkUgkQmNjI0NDQ2b12HQ6TSqVQkTYtm2befzAwABjY2NYLP9fDaZSKXbu3InT6SSVSpFOp7njjjtwu938/Oc/x+fzzQnFDzNQ/kqpf1ZKvXuZ7aZb70qpdUqpt5VSb+f9yxTNtBgvaBcsWEBVVRUrVqwAJl74ulwu/H4/6XSa7du3k0qlgIkqoHa7HZhwVQaDQZ5//nmUUrjdbjM0VFO4JJNJOjs72bdvH8888wx33nkn9fX1VFdXA5irxUwsFgvj4+O43W46OjoIBAKICF6vl6KiImw2G7t372Z8fJwNGzbMGcUP3JxQT67s878H2Jvx/TvAd652Tu3zn7vEYjGprKw0X7iREV6nlDL/W1lZKYBUVlZKTU2NmcHpdrulvr5eYrGYeDweCYfD4vP5ct4Xq7n1GP/+sVhMFi9eLBaLRaxW65SKr0z68y/uM11UVCRWq1Wqq6vNAnFNTU3S0dGR06GdF8MMff6XPgpvPv8K3KGUWgicAB4E/uMsXFeTo9x33300NTXx6KOP8o1vfMPcr5RCRExrf3h4GJvNxpkzZ8xjqqurOXnyJMeOHWP9+vV8/vOfN5fyc2lJrrk2jMYtfr+fZDJJd3c3JSUl1NXVcezYMfbu3Wsea7FYSKfTnDp1CrfbTSqVMkNDnU4nDoeD8fFx0x303nvv8eyzz869sTWTJ8R0G/BlJnz4aSDJpIUPVAN9GcetBN5nIupnw0zOrS3/uY2RGh+PxyUUCklJSYnU1dVJaWnplEig4uJiqa+vl0AgYJbqDYVC0t7eblb61BQ2l6vPn1kg0CjAFolExOVySUlJiRklllnKwWKxmH0BgsGgxGIxWblypcTj8bwKGWY2LH8R+QXwi8vsPzmp8I3vfUDfjVxLM3dIJpM8/fTTjIyM0NXVhcfjob+/n+PHjxONRgHYsGED6XSa2tpaTp06xfj4OHa7nbq6Ok6ePMn+/fvZsWOHLuGguWyv3szP8+fPp6GhgYaGBvr6+jh8+LCZ+DU6OgqA1Wrltttu46OPPsLv9zM4OEhlZSU9PT34/X4aGhrmnOWvM3w1s87FYZ9er5cf/vCHxGIx7r//fiKRCOl0mqKiIjo6OgiFQtTX1xONRnnrrbfYsGEDf/nLX8w+qhrNlRSzETEGsGPHDnp6erj33nuJRCJs3LiRXbt28dxzzxEMBtmxYwdlZWUEg0G++93vzuj8+cps+Pw1mktobGykp6cHgI6ODh5++GHTLysifO5zn+PChQu88sorfPrppxw6dIgnn3wSgC1btvC9732PH/zgB3PSItPcPPx+vzlu1qxZY1r8jz76KI888ghKKTZv3kwsFgOgsrKS48eP8+Mf/5ju7u4p58psCD8nmIlvKBub9vnPfTJ9tYbvPpFIiM/nk2g0KjabTTwej7S0tJhlnFtbW81OS7ovr+ZqGA1fjHdMRrcu4/PFDYSMvxG5tI9vvvj90fX8NfnA5SaTUZ6hpaVlSqmGzIk63d9qNBeTGRRwcRtHo/6TYUxcKYAgX8bbTJW/9vlrssrFy+hkMsnWrVsB2LNnj1mqwUjgyfybObUE19wyjKCAZDJJe3u7uf/dd99l/fr1RCIRfD4fqVSK9evXX1IJ1vg+18abbuaiyTmm863OOZ+rZtYxxlBmE5jXXnttyjGZY8x4YFwcTZTLzLSZi7b8NTnHdJMsXyafJnfJHEMul4uuri4Ac0Vw8Ri7XBjpXEFb/hqNpiDJXEnOpVWltvw1Go3mCkyXFFYoaOWv0Wg0BYhW/hqNRlOAaOWv0Wg0BYhW/hqNRlOAaOWv0Wg0BYhW/hqNRlOAaOWv0Wg0BUjOJnkppQaBv9yi088D8rkYvJY/++T7PeS7/JD/93Cr5F8gIr6rHZSzyv9WopR6eyYZcLmKlj/75Ps95Lv8kP/3kG35tdtHo9FoChCt/DUajaYAKVTl/0q2BbhBtPzZJ9/vId/lh/y/h6zKX5A+f41Goyl0CtXy12g0moKmYJW/UuoxpdRhpdRBpdSL2ZbnelFKPaGUEqXUvGzLci0opX4w+f//gFLqF0qp8mzLNBOUUl9USv1JKXVEKbU+2/JcK0qpWqVUXCn13uTY/4dsy3Q9KKWsSqn/o5T6VbZluR6UUuVKqTcn58AhpdQ9sy1DQSp/pdTfAquAu0Tkc8BLWRbpulBK1QL/HjiebVmug7eARhH5PPA+8J0sy3NVlFJW4MfAfwA+CzyklPpsdqW6ZkaBJ0Tks8BS4Ft5eA8A/wAcyrYQN8A/Af9TRELAXWThXgpS+QOPAptFJA0gIh9kWZ7rpQt4Csi7Fzci8r9EZHTy678At2VTnhmyBDgiIkdFZAR4gwkjIm8QkVMi8m+Tnz9mQunUZFeqa0MpdRvwJWBHtmW5HpRSHuDfAd0AIjIiIkOzLUehKv9FQItS6g9Kqd8qpf4m2wJdK0qpVcAJEXkn27LcBNYAv8m2EDOgBujP+P5X8kxxZqKUqgcWA3/IriTXzD8yYfSMZ1uQ62QhMAi8Oum62qGUKp5tIWyzfcHZQin1z0DgMj9tYOK+K5lY9v4N8F+VUrdLjoU+XeUenmHC5ZOzXEl+Efnvk8dsYMIVsWs2ZSt0lFIlQAx4XETOZVuemaKU+jvgAxHZr5Ralm15rhMb0AQ8JiJ/UEr9E7AeeG62hZiTiEjrdL8ppR4FeieV/R+VUuNM1NkYnC35ZsJ096CUCjNhPbyjlIIJl8m/KaWWiMjALIp4Ra70bwCglGoD/g5YnmsP3mk4AdRmfL9tcl9eoZSyM6H4d4lIb7bluUbuA/5eKbUScAFlSqnXROQ/ZVmua+GvwF9FxFhxvcmE8p9VCtXt89+AvwVQSi0CHORRgSgRSYjIfBGpF5F6JgZTUy4p/quhlPoiE0v3vxeRT7Mtzwz5V+AOpdRCpZQDeBD4H1mW6ZpQE9ZCN3BIRKLZludaEZHviMhtk+P+QeB/55niZ3Ke9iul7pzctRx4b7blmLOW/1XoAXqUUu8CI8DX88TynEv8CHACb02uXv5FRP5zdkW6MiIyqpT6L8BewAr0iMjBLIt1rdwHfA1IKKX+7+S+Z0SkL4syFSKPAbsmjYijwDdmWwCd4avRaDQFSKG6fTQajaag0cpfo9FoChCt/DUajaYA0cpfo9FoChCt/DUajaYA0cpfo9FoChCt/DUajaYA0cpfo9FoCpD/B4rn5JUJ0/GkAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# generate the data\n",
    "inputs = np.linspace(-2*np.pi, 2*np.pi, 10000)[:, None]\n",
    "outputs = np.sin(inputs) + 0.05 * np.random.normal(size=[len(inputs),1])\n",
    "\n",
    "plt.scatter(inputs[:, 0], outputs[:, 0], s=0.1, color='k', marker='o')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The below code creates the inputs, variables, neural network operations, mean-squared-error loss, gradient descent optimizer, and runs the optimizer using minibatches of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0000 mse: 0.700\n",
      "Model saved in path: /tmp/model.ckpt\n",
      "1000 mse: 0.038\n",
      "Model saved in path: /tmp/model.ckpt\n",
      "2000 mse: 0.029\n",
      "Model saved in path: /tmp/model.ckpt\n",
      "3000 mse: 0.014\n",
      "Model saved in path: /tmp/model.ckpt\n",
      "4000 mse: 0.010\n",
      "Model saved in path: /tmp/model.ckpt\n",
      "5000 mse: 0.002\n",
      "Model saved in path: /tmp/model.ckpt\n",
      "6000 mse: 0.003\n",
      "Model saved in path: /tmp/model.ckpt\n",
      "7000 mse: 0.002\n",
      "Model saved in path: /tmp/model.ckpt\n",
      "8000 mse: 0.002\n",
      "Model saved in path: /tmp/model.ckpt\n",
      "9000 mse: 0.002\n",
      "Model saved in path: /tmp/model.ckpt\n"
     ]
    }
   ],
   "source": [
    "sess = tf_reset()\n",
    "\n",
    "def create_model():\n",
    "    # create inputs\n",
    "    input_ph = tf.placeholder(dtype=tf.float32, shape=[None, 1])\n",
    "    output_ph = tf.placeholder(dtype=tf.float32, shape=[None, 1])\n",
    "\n",
    "    # create variables\n",
    "    W0 = tf.get_variable(name='W0', shape=[1, 20], initializer=tf.contrib.layers.xavier_initializer())\n",
    "    W1 = tf.get_variable(name='W1', shape=[20, 20], initializer=tf.contrib.layers.xavier_initializer())\n",
    "    W2 = tf.get_variable(name='W2', shape=[20, 1], initializer=tf.contrib.layers.xavier_initializer())\n",
    "\n",
    "    b0 = tf.get_variable(name='b0', shape=[20], initializer=tf.constant_initializer(0.))\n",
    "    b1 = tf.get_variable(name='b1', shape=[20], initializer=tf.constant_initializer(0.))\n",
    "    b2 = tf.get_variable(name='b2', shape=[1], initializer=tf.constant_initializer(0.))\n",
    "\n",
    "    weights = [W0, W1, W2]\n",
    "    biases = [b0, b1, b2]\n",
    "    activations = [tf.nn.relu, tf.nn.relu, None]\n",
    "\n",
    "    # create computation graph\n",
    "    layer = input_ph\n",
    "    for W, b, activation in zip(weights, biases, activations):\n",
    "        layer = tf.matmul(layer, W) + b\n",
    "        if activation is not None:\n",
    "            layer = activation(layer)\n",
    "    output_pred = layer\n",
    "    \n",
    "    return input_ph, output_ph, output_pred\n",
    "    \n",
    "input_ph, output_ph, output_pred = create_model()\n",
    "    \n",
    "# create loss\n",
    "mse = tf.reduce_mean(0.5 * tf.square(output_pred - output_ph))\n",
    "\n",
    "# create optimizer\n",
    "opt = tf.train.AdamOptimizer().minimize(mse)\n",
    "\n",
    "# initialize variables\n",
    "sess.run(tf.global_variables_initializer())\n",
    "# create saver to save model variables\n",
    "saver = tf.train.Saver()\n",
    "\n",
    "# run training\n",
    "batch_size = 32\n",
    "for training_step in range(10000):\n",
    "    # get a random subset of the training data\n",
    "    indices = np.random.randint(low=0, high=len(inputs), size=batch_size)\n",
    "    input_batch = inputs[indices]\n",
    "    output_batch = outputs[indices]\n",
    "    \n",
    "    # run the optimizer and get the mse\n",
    "    _, mse_run = sess.run([opt, mse], feed_dict={input_ph: input_batch, output_ph: output_batch})\n",
    "    \n",
    "    # print the mse every so often\n",
    "    if training_step % 1000 == 0:\n",
    "        print('{0:04d} mse: {1:.3f}'.format(training_step, mse_run))\n",
    "        save_path = saver.save(sess, \"/tmp/model.ckpt\")\n",
    "        print(\"Model saved in path: %s\" % save_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the neural network is trained, we can use it to make predictions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Restoring parameters from /tmp/model.ckpt\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x13fed2cc0>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD8CAYAAACfF6SlAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzsnXl8VNXZ+L83k33fhossIWDABCeAkR/xFbBpwdKSvqKpiApWBI2SFl9QkSi+pgFTQdkqLRQ1KbaAdQlV34JVlKRWrEVEcQaCYZHduUwIsmVPzu+PuXcMaRCQJLOd7+dzP5l75mbuyeSc5z7nOc+iCCGQSCQSiX8R4O4OSCQSiaTrkcJfIpFI/BAp/CUSicQPkcJfIpFI/BAp/CUSicQPkcJfIpFI/BAp/CUSicQPkcJfIpFI/BAp/CUSicQPCXR3B85HYmKiSE5Odnc3JBKJxKv49NNPq4QQ5gtd57HCPzk5ma1bt7q7GxKJROJVKIpy4GKuk2YfiUQi8UOk8JdIJBI/RAp/iUQi8UOk8JdIJBI/RAp/iUQi8UOk8JdIJBI/RAp/iUQi8UOk8HcDmqa5uwsSicTPkcK/i9E0jby8PPkAkEgkbkUKfzewfPlyVFV1dzckEokfI4V/F2Jo/Q6Hw91dkUgkfo4U/p3A+Uw6qqpSWFhIQUEBNputi3sl8UXajjVpTpRcLFL4dzCapjFlypTzTkqLxeJ6AMiJKrkcWu8faZqGzWZrdz9JjjNJe0jh3wkoiuJ6bUzMSZMmnfMAaG33N65p/TsSyYVQVZXly5fjcDiYMmUK+fn5FBYWut43xpV0MJC0hxT+HYyqqsyfPx/ApYlt3rwZq9WKw+FwTUhVVbHZbK6VgvFwkJNVcik4HA7y8/OZNWsWxcXFmM1m8vLysNlsTJ06FYfDIR0MJO0ihX8HYQhrTdOYPXs2kyZNYubMmUycOJE1a9awfPlyqqqqGDduHFOnTqW8vJwxY8ZQUVHBggULCA4Odn1W21WBRNIeNpuN2bNnc+rUKRYuXIjD4aCiooLp06cDUFdXx8yZM93cS4nHIoTwyOPaa68V3oLdbhc5OTnCbre7jrKyMjF69GgxevRoUVpaKtLT00VMTIwICgoSxcXFwmq1irKyMjFq1ChhtVpdv2N8TtvPlUiEEMJqtQohnGNj7NixoqysTNjtdlFaWipGjBghgoKCRFRUlMjIyBDFxcVi7Nix54xLie8DbBUXIWPdLuTPd3iT8BdCuAR4Tk6OsFqtIicnRxQXF4sRI0aI9PR0AYiUlBSxePFikZ6eLrp37y5KS0tFdHS0GD16tCgrKxOqqoqysrJzPldOWImB1WoVqqq6xtrIkSNFdna2KC4uFqqqitLSUrF48WIREREhAgICRExMjEvRMBQROZ58n4sV/h1i9lEUpURRlGOKorTrv6g4eU5RlD2KonyhKEpGR9zXU9A0jYKCAhwOB9OnT8disTB9+nQeffRRampqAAgNDcVut7Nw4UJsNht33XUXBw4c4PTp04wdO5bq6mqSk5NJTEw8x9QjbbWS1gwYMACz2VmeVQjBqFGjmD17NtHR0SxdupTrH3uM/WfP0q9fPxoaGpgxYwZ33303p0+f5vTp027uvcSjuJgnxIUO4AYgA7Cd5/2xwNuAAlwH/PtCn+mNmv+oUaOE2WwWZWVloqysTMTHx4vU1FQRFRUlANcRGRkpEhISRGxsrJg8ebLIyMgQJpNJxMTEiMzMTDF69GiXdieRCOEcX2PHjhUjR44UVqtVlJaWisjISBEeHi6Sk5NFYGCg+AZEi36c0ceaoigiMjJSpKamioSEBJfZSOK7cJGav+K89vJRFCUZ+JsQwtLOeyuBciHEy/r5l0CWEOLr833e0KFDhbcVcF+3bh1FRUVEREQQHR3NsGHDeOaZZzCbzezfvx+AvcCVwNChQ9m6dSsBAQGEhoaiKAp5eXls27aNhoYGQkJCCAkJobi4WGr/forhFaZpGhMnTuSJJ56gqKiIw4cPc+jQIc6ePQtASEgIx+vrCW/z+zVAet++VFdXA3DllVeyYcMGQK4ofRlFUT4VQgy90HVd5e3TEzjU6vyw3ubVtPbwKS8v57777mPXrl2cPXuWnJwcFixYwNmzZ/lw/36qgTNAX/2n8WCLjo5GCEFTUxNLly7lhhtuYPny5axevZri4mJ3/WkSN6NpGlOnTkXTNBwOBzt27KC6upqxY8eya9cumpubCQoKAjhH8J8AJgMtQDhg/eorcnNzSUpKAmDz5s3ceuutLjdjiR9zMcuDizmAZM5v9vkbMKLV+fvA0HauywW2AluTkpI6Z03UQdjtdpGdnS3KysrEiBEjxMiRI0Xv3r0FIMLCwkTv3r3FVn353dLOcaaVGcg4QkJChMlkci3tjY3jtpvAEt+nPW8eVVVFRkaGCAsLc42Z1uNrf6uxNA5EU6ux1k03AYWHhwuTySTS09NFdna2NC36IHTlhu9FcATo3eq8l952DkKI54UQQ4UQQ41NLU+mrq6OJ598kh07drB//36OHj1KcHAwv62tpeLQITLApZHV4NTK3g0KQujtZ9p8XlxcHM3NzYwePZrZs2e74gQmTpxIeXl51/1hEo9g1qxZjB8/nkmTJjF8+HB+85vfABATE8MdPXpwhm/HVzlO7Ss4OBhVVVkfGMiPgEb9mkPAoIAAhBCEhIRw6NAhpkyZ0uV/k8SDuJgnxMUcfLfmn825G75bLvR53rDha2jnqampAhBms1n8sx0N/zEQoaGhom/fviI1NVXMNJlEczsrgICAABEWFib69esn8vLyRHZ2tiseQPr7+w/GqtJqtYrrrrvOtcEbEREhAPE8uMZPPYhf6U4EcO5KMjo6WnRrtTqoB/EDEMnJySIqKsrlKirHlW9BV/r5Ay8DX+NUNA4DU4EHgAf09xXg9zj3O620Y/Jpe3i68G+9LM/MzBTBwcHisTZCf1ObyRgeHu56/Ys2y/IJEyaIwsJCERIS4rpm8uTJrriB1oFfEt+m9dhSVVUUFha6PMY+bzPGurUaXz169BBhYWEiPDxcJCYmnjP2jAdAE4j/Fx4uwsPDRWlpqQz+8kG6VPh3xuHJwt+YMCNHjhTp6ekiJSVFdANR00abDw4Odk2+1NRUkZmZKXr37i1MJpNITk4Wd/ToIepb/c6VkZEiKSlJBAQEiDFjxgiz2SxKS0vPiR6WKwDfpvVD3m63i+LiYmEymURQUJCwtxH8qampIiUlRaiq6hpfMTExYvHixSI5OVmYTCYxbtw4AQhVVV0PAGN8ZmRkuPaV5JjyHS5W+MvcPpdI64IsDocDq9XK0aNH2QeE4rTtRwImk4nAwECSk5MJDAykqKiI3NxcNE3j/vvv5+TJk7xx4gRXR0VRg9Muu/PMGfodOsT48eP5+OOPaWhoID4+3pWp0cjiKN30fBNjbBm1HgybfHNzMwcaG+mmX1cDDElJoampCU3TSE5Oxmw2M2HCBJqamnjuuefYv38/gYGBfPDBB+Tl5XHy5Eky+dYL6CRQU1NDVVUVhYWFLpdSif8ghf8lYghggIMHDwJwrKbGtfH2UHAwY8aMobm5mdraWkJDQ4mIiCA+Pp7S0lIsFgtWqxVVVamrq+OWBx7gCv0BEAS8JwTmV17h5MmTXHHFFRQVFXHvvfe63P4kvotR7Cc/P5/NmzejKAqxsbF8HBBAd/2asziViyNHjrBnzx5qa2vJzc3l7NmzFBQUcPbsWaqqqsjLy2Pw4ME0NDTw7rvvUldXx9G4ODbonxMFvL5rF+PGjXM9cGQ2Wf9CCv9LxAi8qaysJCgoCAffelzsTEigpKWFnTt3AtC7d2/MZjOpqakkJiZSUlLCokWLAKcnR1RUFGPHjqVPnz5E4tToAoDn9KOpqYk77riD3bt3k5OTAyAnqI9jNptxOBzk5uYycOBA3rnjDoa1tADO8dEtLIy8vDyKiorIzMxk5cqV9OvXj6+++or+/fuTkpJCQEAAlZWV3H777dTV1QEQFhZGr169uDU4mOP6vQYCs4KDOXv2LGazWa4q/Y2LsQ254/BEm79hcy8tLRUJCQniUEKCywZ7EERERIQICwsTI0eOFLNmzRKlpaWitLRUjBw50pWyYcSIESIhIUFkZmaKkSNHurJ/ZmRkiAkTJpzjt/1nPRmckQ7C8PmW+C7GXpLh2dPaTh8YGOjyCjOZTGLx4sVi1KhRYuTIkSIuLs411hISElxJBSMjI0VUVJQwmUwiIyNDpKSkiICAgHM+12QyieLiYrn56yMgN3w7B6vVKjIzM8WDrYT0SRDjxo1zTcri4mKRmZkpTCaTiIuLOydfT3Z2tigtLXX9NIK4hg0bJhRF+Y/AnbeCg0V4eLhISUmRWRn9hLKyMpGRkSGq9THQAOJqXeiHhoaK5ORkERMTIzIyMs5RIFqnEzfyABnvl5aWirKyMhEXFyfCwsLEjcHBLnfRb3TFpbWSIvFepPDvJOx2u8jIyBDHW2lOqqq6InQzMjJcUb+GNma328/Jwy6EcPnuG+2Gy2hhYaFYvHjxuQ8AfeJnZGRI4e/j2O12kZ6eLtaGh5+zAgwICHB58MTGxorCwkJhNpvFyJEj29XYW+fwt9vtLqGekpIioqKihKIoYpOiuO6xFUReXp4YPXq09P33cqTw7wSMMPun9QnTDOIPY8aIuLg4MWHCBKEoili8eLEYO3asKwBMCPEf7nSt8/4b52PHjhWZmZkiMzNTBAYGiqioqHMeAJtauebJ5blvYrh2Ai7l4qgulEeMGOFSEIyVZGlp6UVp6UYdgMWLFwtFUcTkyZNdLshVrcbYL0AUFhbKseXlSOHfwRgCeuTIkWJrUpJoAfGxoohZs2a58vHMmjXLFZ15PkHf+vNaYxTcMCb44sWLRVhY2DkPgE/ANfGlduZbGOOruLhYvNdGIPfs2VOUlpYKIb6NKjeKs1zsODB+Lz09XWRmZoqAgABXHEpr+39cXJwrl5QcX96JFP4diDEJSktLRVxcnNivT5bXWkXuFhYWitDQUFFWVnZBQX++exhRnaNHjxaZmZlCURShKMo5D4C/9u4tN359ELvd7lr1nebbRG3x8fGuvaO2AX+XsgI06gEYK4bWCeKu5tto89P6ClOmFPFepPDvIFpr7iNGjBAvttokM0Lrg4KCXJPqcqJwDfNQ25xBwcHBrgdAdXi4qy6r0T+J92OYFI+20sKTk5NFRkaGK3rXyPfzfT679bgySj8mJCS4UkL8rZWC8TmI9PR0ufHrpUjh34EYibVSU1PFW602YceNGyd69OghALF48WLX9ZcjkFsLdYvF4rLNTmm1OZffvfs5NYPlA8C7sdvtYtSoUSK/1SbvJt2tMywszOXmeznCuPW4GjVqlMjOzhZ5eXkiKChIjBs3TiiKIhytHgC5wcFS+HspUvh3EHa7XYwYMUIEBgaKbrpbp+GBER0d7Sqa3RkYmRzDwsJE9+7dxSb93s/r/v9y49d3KC0tFVZ9L+mo7nvfo0cPl9DvyBw8xiojODj4nPxTbe3/Uvh7Jxcr/GWE7wVQVZUVK1aQm5vL73GGxR8E3p88mdraWp555hnef//9Do+61TSNkpIS5s2bR3BwMHa7nT36e3HA3r172bx5s4zI9AHWrVvH7bffTpWeLuRvwM9+9jMaGhoAZ9RvQUGBKwfP5aKqKjk5OcyZM4eIiAjCwsIICQkhMTHRFWkeDlgeeeSy7yXxYC7mCeGOw9M0f7PZLD43m11af2BgoMu7pzO0b2N5Pnr0aLF48WIBiKJWWpkFXLneJd6L4YP/eI8erpTL43QtPCUl5bx+/JeL1WoVPXr0EMXFxSI9Pd2l+ZvNZvG8Ps6OjRvXofeUdA1Izb9jqKiooLGxkTGhoVztcADOilwzZ85kzZo1OByOTtG+VVVl6dKlBAcHc8011xATE8Oi+Hi24dTKHg8Pp6Ghgfz8fJnrxwtp/T+rq6tj4IkTAHwQEMD25GQiIyOJjo5m165dnTLGLBYL77zzDlOmTGHt2rWkpqYSFhZGc3Oz65pT69fLseXDSOH/HdhsNm6++WZ27tzJxKoqTEAlMM9s5rPPPmPNmjVYLJZOu7/FYqGkpIS0tDSSkpJobGxkm/7emZoa9u7dy6hRo6Tpx8tom7rZbrcTXVsLwBctLRw/fpwBAwbw0ksvsWnTJjqrpGnrsRsTE0NtbS3V1dXkAgdDQ+nX1ETA/fd3yr0l7kcK/+9gy5YtnD59mrq6Osz65LQCDoeDffv2kZaW1qn3NzKIOhwOjh07xgMPPICiv5eJ0xacn58v6/t6GUbq5oKCAhYtWsTVe/aQrb8XHRlJ//79iY+Px2w2YzabOzWTq6ZpFBQUMGHCBFdbYmIi7+rZQD/4xz/QNE2uAHyRi7ENueNwt83farWKxMREERISIn7VygXuiehokZSU5IqC7CzaunGWlpaK+Ph4AYjtel9eDA8XmZmZ0uPHC2ntcWO4D1fq/vUjR448Zy+ns/+/VqtVjBo1ylVmNCUlxWX3/zwoSAZ8eRlIm//lYbFYWLlyJb179yZTb9sGPHXqFDExMSxcuLBTtaG2VbtycnJ49tlniYiIYIt+TUtNDTfccIM0+3gZhtln+PDh/M///A9X6+27gH379gGcY+rp7P+vxWJhzZo1rF+/nuLiYqKiongjIoJmYFBjIwfuu6/DPI0knoMU/udB0zSKiorYv3+/S/h/BSQnJ7N27VoeeeSRTp8MrT9f0zTWrl1LQECAy/STBqxcudJlO5ZLc+/AeLCvX7+eAc8+S1+gGfg0K4vQ0FDmzp3bpYJW0zQcDgfLli3j5MmTHD58mKvz8vh3bCwAP7/22k7bd5C4Dyn828GYDAcPHmSZopACNAErg4Lo3bs3VVVVTJw40SV0uwLD+6d///68ExSEAIYD9zc0UFVV5dIm5QPAO6ioqOD+++/nGv18G1D04Yf06dOHxMTELuuHpmlMmTKF/Px8srOzyc/PJzg4mEWLFnFQjzOo3bWLKVOmyLHlY0jh3wZN05g6dSpVVVX87Gc/I7GxEYDNwCfh4cyYMYOsrCzeeeedTvX0aQ9D+3o3PJwPgoIAGBIQwMKFC3E4HLIMn5fxq169XMJ/i6IwaNAg5syZQ0FBQZcJWlVVKSkpobi4mOzsbK666iqOHj1KS0sL/6qpASBh+3YGVlZ2SX8kXYcU/m1wOBx8/fXX3HTTTaxatYohevuZ6GiuvPJKSkpK0DStywW/0beDBw+SlJTEPt0fO6qmhsjISPLz87u8P5LvR3l5Obfeeiuj9+9HAfYCT8bGMmnSJEpKSrrcvq6qquvYuHEjmzZtYsKECSzDqfQowIA9e6ioqOiyPkk6Hyn8W2GYTg4cOEBCQgJ/Bq4EzgALe/QgPj6e+fPnu027tlgsvPbaazz33HMcNJkA+Anw9SuvsHfvXjk5vQBN05g9ezZmsxnDUfhgaCgzZszgqaee4tSpU26xrxsrDWNsv/322wAYIyqMbzejJb6BFP5tiIqK4vXXX2fRokUYU7Ac+GjvXqZNm+YWjd9A0zSWLVtGWloaMQsWsAUIBCYBhw4dIicnp0v3ISSXzubNm9m2bRsP7drl2uidW1fH4sWLEUJ0+WYvcM5+kaZpPPXUU65I3xr9mpsDAviwtFTa/X0IKfxbYdg/q6ureXHGDEbo7UlDh/Luu++Sk5Pj1v4BLF++HIC//vWvfKI4/X5qgLCwMACqqqrc1TXJBdA0jeLiYnr16sUwve1TIHTMGN544w0++OADsrKyurxfrd2KDceCq666iuDgYGYAXwARLS3M+uYbuafkQ0jh34aKigrGjx/PnYcOEQ5Ux8Ux5tAhqqur3dovQzsDp+1fCEFEgPPfl4VT6CcmJlJUVCS1Mw9m1qxZTKitZZB+vglnJHlRUZFb3SlbC3Wz2UxQUBDNzc2EhYWxTTcxfvLJJzKa3IeQwr8NiYmJJCUlkayf/6umhm7duvHLX/7SrSYVQzsDKCgoYMWKFYQ8+yzfAIOApcCNN95ISEiI2/ooOT+GS+XDDz/MSP3hvAd4OT0di8XCkiVLPEarVlWVN998k/fee4977rmHJt0ElBUby7Jly6Ry4StcTBiwOw53pHcwSieOA9Gsh7cX6amTOzudw6Vgt9uF1WoV8fHx4s+tCrwoekF5iWdSVlYmIiIixNZWBYHCw8M9amwZGOknTCaTuLpVkZcPxoxxd9ckFwCZ3uHS0DSNadOmsXv3bibhdG+rBMIWL+aNN95wiy32fKiqitls5oUXXiA0Ohpwav/BwcEsXbqUdevWSe3MA0lLS+OZmBgycCbP3xgd7Sqi4kkYq5QVK1bw5JNPcjopibf1977atEk6FfgIUvjrqKrKvHnzuP766xkUFQXAjrAw5s2bh0PP4+8pGJOzpKSEHvffjwCGAffV19O9e3eWLl3KpEmT5APAw6ioqCBe/5/8A7ClpGCxWDwudYKqqixYsID6+nqWLl3KsWPHOKG/d1VAgHQq8BGk8Ncx3Chnmkz0P30agO0hIZw5c8bjBnvrqMzoX/yC9/X2QcCRI0c4e/Ys9fX1HvfQ8mc0TePhhx/mKiM4b8AAFi1a5LFR2RaLheXLl9O3b18aGxup1Fcnw+rrKf3v/5Ybvz6AFP46qqoyePBgmvVBbVUUxv/zn7z77rseZfIxMNzyAA7pLp9xQK9evVi0aBFnzpyRVb48AOP737x5Mz/dto0hOE0+RZWV3HzzzcycOdNj/0cWi4VFixYRGxvLoVGjKNfbR0KnZ7WVdD5S+OOcoHPnzqWwsBAjhKtSCKqqqjxS8LemqqqKfUIA8FMg6uBBXnvtNXbu3MmUKVM8Uqv0Fwz33PLyclasWMEQPR/TP4BtSUkMGjTIo7x82iMrK4spU6bwyiuvsEdvCzpzxiNiXiSXycXsCrvj6CpvH7vdLq655hoBiKW6R0MDiAndu3tNcfTCwkLxcSuvn8jISFdxeYl7sVqtYvTo0SI5Odnl5bMaREJCgkd6+bTFbreLsWPHipSUFPG03v+zIEbGxbmKy0s8C7rS20dRlJ8oivKloih7FEX5jwxjiqJMVhTFoSjK5/pxb0fct6NISEigR48eJOjnm4Dbfv/7Ls2u+H3RNI1///vfHNZzrwOcOXOGRYsWMX78eI/vvz9QXV3N3YcPu7x83gwK4vnnn/f4VaVBSUkJeXl5PIYz9XQYkFtbK2NKvJ2LeUJ81wGYcCYm7AcEA9uBgW2umQz87lI+tyv9/O12uygsLBR7dc3mwNChrnZvwGq1itMTJogWEF+ACA0NFWlpaV6zcvFV7Ha7GDVqlIiKihKr9bH1T5PJa0pvGqVEy8rKxNixY4Wqqq7yjiUBAV6xcvFH6ELNfxiwRwixTwjRAPwFGNcBn9tlOBwO+v/61/QFTgMPh4W5iqd7OkYB7n/8+Md8A1iAZ+rqOHDggMd5KfkbqqqSl5dHU1MTqXrbNxERNDY2eoUnllFoftmyZfz85z8/Zzxd29Li9pQnksujI4R/T+BQq/PDeltbfq4oyheKoryuKErv9j5IUZRcRVG2KoqytTMnR1tTyJYtW4jWN03/AfS97jqvEPzw7QRdXlrKB+HhAIQCNTU1PPjggzIgx41omsbSpUt5tLHRZfJ5VQisVqvXVF0zXD6HDRtGeHg4RaGhnADSAfuECaxbt87dXZR8T7rK2+f/gGQhxCBgI/BSexcJIZ4XQgwVQgztrMCXtuUOy8vLeXzqVFcGz1NhYaxatcqrhKbFYqGkpIRR//3fAPTHWWs4NjZWunu6EYfDwdmzZ+nf1ATAh8Bf6up44okneO2117xKwTCbzaSlpaF0784GvT0uOJjc3FyvmiuSb+kI4X8EaK3J99LbXAghjgsh6vXTF4FrO+C+34vW6Ws1TWPfvn38DojGWaB92aBBpKamelzU5YVQVZXI224D4AacmyyTJ0+muLjYJWTkQ6Dr0DSNGTNmsHfvXgbqbQeBPn36sGXLFnd27XujKAqHDx9G0dOHJ9XU0KiXOZV4Hx0h/D8B+iuK0ldRlGDgduCt1hcoinJFq9Ob+LZAkFswBP/UqVM5efIkvfUgqe2KQnBwsMdGXV6I8vh43sWZlyitqopp06axfv164D9XPJLORVVVsrOz+U1EhCuw628hISxYsOCcB7K3oKoqL774Iunp6WxQFARwPXBvTc2FflXioVy28BdCNAG/At7BKdRfFULsUBRlrqIoN+mXPagoyg5FUbYDD+JUTN2Kw+Hg6NGj/OOhh/h/ur1/t6IwY8YMt1bruhyysrJIGDwYAHNzM0lJSTz66KPYbLZzVjySzmfdunU8/PDDROubpB8GB/Pj5cspKSlxc8++PxaLhT/96U980a8fG/W26/RVgMT76BCbvxBigxBigBDiSiFEkd72pBDiLf31Y0KIq4UQg4UQPxRC7OqI+35fDA+ZwYMHcytOTfkLYPXVV7sKtHsjNpuNd778EoARtbX0PXiQlpYW1/tS8Hcdw4cPp0+fPvRsaADAHhzMyy+/zKlTp7zC0+d8mM1mFEXhoH4+5IoryM/Pl3Z/L8Rv0ztkZ2ezatUq0vXzyMGD2bhxo1cuyQ0sFgtpa9awFWdt3/ENDfz4xz/22pWMN1NRUcGoQ4fI0s8/rq1l2rRpAF69Ce9wODhw4AD79fPkykp67d3r0TmKJO3jd8LfSIccGxtLUXi4q5xe9fDhgPdrxzk5OcS3ihx95ZVXmDt3rpyYXYhRBP2HwcEAbAF+p5fcjIqKYv78+V47zsxmM8OGDSOksJCv4uIIBO4OCKCurs7dXZNcIn4n/AFOnz7N5MmTGaBvVm1TFIa/8ILPpEMwBMv1+vmzzz4r8/t3MSdOnCCkthaAnQEBXHnllQwYMICSkhKvXompqsodd9xBYWEh759wZvmvqKhgx44dXm3O8kf8TvgbG5+FhYUuF7zDYWFkZGT4xIaozWZjwunT7DWZGAiUmEy0tLQwZ84cr//bvIkVv/wlP9VfN0RH43A4mDlzplv71BFomsYf//jHc/L6jAAGDBjgde7R/o7fCX/D/zp41izSgGagJCiIRx991Ks1Mvh2I3vWrFls1wX9qeZmampq+OCDD9zeJyolAAAgAElEQVTcO/9A0zQmTpzIgccfJxynyWdVaiqvv/46q1ev9okHsKIoBAYG8kh0NAdDQ+kvBLN375aav5fhd8LfyLfSQ/eC+ScQf8strFmzxuvNIsaqJisriyEDBgBwHRASEsLvfvc76ZHRBTgcDhRFIWXIEABEVhZPP/00aWlpPiH4jTE2ZMgQ4uPjeVe39ffr109u+noZfif8jc24DN23/zjw0ksvkZ2d7TOTU9M0Xq2pcdX2za2vZ/LkyTLRWyejaRqzZ8/m9OnTfP3uuwB8tnkzt956q0/tuRj5fuLj411tgTYb27dvZ/PmzW7smeSSuJjUn+44Oiuls9VqFesURbSA2A+isLBQFBcXd8q93InVahUfhoW5CrwoiiKCgoJkGt5OpqysTLyblyeaQTSDeMRsFmVlZV6RwvlSsNvtwmKxiBtANOppnjdmZYnuXlQEyVehK4u5eAuaplFZWUmIrvVvDQ3lhhtuYP369T6jlbVmn55Q7AqcOWUyMjJIS0tzb6d8GJvNxq233sq2FStQgHcUhbs3bfIZk09b+vTpgyUvj7f186/Ky4mNjZUbv16C3wh/YyNu/qRJ/FBvq66r47HHHqOwsNDnJqfFYmHUffcBMAZIPnCA3Nxc93bKxzGbzVxxxRWE68pFpRBs2bLFJ3MqqarK/Pnz+eKLL7DrbWlhYZw4cYKKCrem7pJcJH4j/FVV5YYbbuDR2lpCgT3A14WFvPHGG17v5XM+KsePd0X73ikES5cu9Snbs6fhcDjYu3cvA/Tz0NhYsrOzfcKFuD0sFgvz5s1jV2AgAP9VW8v8665j2bJlcox5AX4j/I1at0YGz68iI702te7FYGxs79AnJsCOHTs4fPiw631Jx1JVVcXU2lp+DDQC/4qNxeFw+KTgb81vheATk4kAINPh8NmHna/hN8IfILdbN1cGzx3BwTzyyCM+O0hVVWXNmjWM/fnPAcgAWlpaqKysZO3atT5pinA3WVlZ3DLQGTr4FrA5MNCr8/hcDNXV1TQ3N7O9uRmATz/6yJVGXOLZ+IXwN/L5xLz1liuD50PV1Tz88MM+PTFVVeX4Aw9QjVP4L8X5ACgoKGD69Ok+++BzFyUlJXy8cycAleC1ufsvlYCAAIys/mOBZx95hPLycjf2SHIx+IXwV1WVkpISUnTtpK5nTwoLC+nevbube9b5xKel8VF0NAARQGhoKIpu+pJ0HDabjdmzZzNK/257RUczfPhwnxb8mqaxZs0aFi5cyNLkZL4A4oAn6upYuHChTytWvoBfCH+AyrvvpufJkwAsP3KERYsWMWvWLJ+enODchGzQM0qm4tT8hRAUFRXJydmBmM1mnjSZGCYEVUBpWprPpztQVZXCwkLee+89oqKisOmFXWpqa8nJyfH5ueXt+Lzw1zSN8vJytE2bANgMvG02YzKZSExM9GkBaOT6ib7zTgQwHMhtaCAhIYFp06bJydkBtB4/yXqys/fCw7n70UcpKCjw6fEFTo+fqVOnsmvXLurrnWW6s+PiKC0t9fm/3dvxaeFv2PoXLlzIf+leL3bg7Nmz9O3bF8CnNz6NPCyDnnySD/Xc8oOAY8eOkZubK3P9XCZGXWSbzcaIESPYd9BZ3+pkYyMrVqzwyfiR9hg+fDgLFiygpF8/TgBXnDjBJB/2pPMVfFr4G7b+Rw8fpkdtLU3AywkJLFu2jO7du2M2m33eLU1VVVRVpe8PnaFtVwD19fXEx8fLSMzLxHi4ms1mwsLCuE5vD4mPZ8mSJT4bP9IaI3hy3rx5fLR3Lxv0dpOe8M1XFStfwKeFPzht3setVgA+BDY2NPDyyy+zYMECl2D0dTRN4/29ewFntG+WorBv3z6ZhKsDUFUVh8NBrs3GMJwpnMVvfuMXgh++dSlet24dw4YNQ9Ht/r3OnGHz5s0+vbL2dnxe+AOk6+mbj4eEMGTIEObMmeM3kxOcE/SnH37I4Z49CQRuamoiIiKCFStWyInZAWzcuJFTevzI7p49mTJlipt71LWoqkpaWhqPPvoobzY1IYD/AnZPn+43pi9vJPDCl3g3kfffTx/gFPDL+nru1sPPExMT/eoBAPBZfT29gDDg4YcfJicnx91d8nrKy8uZPXs2f9TPjx85Qnl5uc8mc2sPY2/t9OnTfB4aynuNjdwIDKirk6ZFD8anNX9N0/j6888BZ9GWYzgDcbKzs/3CE6M1DocDxWQCnDn+FyxYwL333itz/XQAo7t352bjJDGRxMREvzJ3GHtrRpEXJSkJgJbqamla9GB8Wvif3LyZIXqR9hPBwQQGBjJnzhzWr1/vV8tRw+UzfvlyaoKCGAT8pqaGY8eOudzzJJdOeXk5t9xyCyMOHSIc+ACYp5t/fN2RoC2qqmI2m6mvr+cr/Tv4CbB4yhTpVeah+LTwT1i2zJXB8xcNDQQHB3PjjTeyfPlyvzL5GME4i9es4ZvhwwFntO+BAwdoaGhwb+e8FE3TWLZsGffddx8RelvzwIGUl5djsVj8SvAbGFlNfxcbyxdBQYQDU0+fdne3JOfBZ4W/pmkc1n2N/42z6HRSUhJms9kvJ6bFYqGwsJDtlZUApOGM9p0wYYJffh+Xi6qqZGdn869//YtBetvenTup1L9ff8RsNpOSksL+/fvZ2tgIgHBznyTnx2eFv7p+PYN0k89OoFu3bhw/ftznQ+6/C4vFwnXz5gFwPTDTZGL16tUyCdf3YN26dUybNo2ML74gC2gBoiZMYM2aNX5j62+LUeAlICCAFr1tWEvLd/6OxH34rPA//uc/A7AvNJSngbi4OF599VW/Mve0x9Fhw3hXz/XTp7mZzz77jFtvvVXaZS8BTdNYsWIFqqpiOXUKgPeArzMz/c7W35asrCwKCgpYEh/PCeBqoM+iRe7ulqQdfFb4m3bvBuCzujqSkpI4fvw4iYmJbu6V+7FYLHTPyACgHxASEsLzzz/v9w/FS8HhcHDq1CnOnDmDMaIOARs2bPiuX/ML1q1bx2OPPYbo1s0V7Vt34oTfroY8GZ8U/icffZSYI0cQwGogJiaGxx57TAo4nKmH/6Zr+T/BGe0ruXg0TWPGjBmEhISQrij8WG+3ZGWxevVqv9b6jRTPy5cvJyYmhuDISAAO/f3vjB8/Xj4APAyfFP6BX3wBwEfAm8Dp06d57LHHpG0b56Zc+YgRaHq0789ra3nggQfkd3MJhIaGMnr0aCZVVxMObAMe1WtF+DNGrqN+/fqxe/du3mxpQQDX1Ndz3Sef+PV+28XQ+uHYFQ9KnxT+9bfdRguwRNdq//d//5e///3vZGVlubVfnoCqqsyZM4cNX38NOL0xJk+e7Cq+IbWzCzN//nw++OADwkJDARCDB/Paa6/5tdbfmmXLlvHCCy8QfNttbNTbhgYGymjf78DIEGvMwa4IEvRJ4d+Ync0vVZV1QjBmzBiys7PJysqSgq0VzboXxkjgiiuuQAhBRUWFX0WmXirGpKysrOTMmTOk6ZkrKyoqpFarY8SUFBUVsXr1ag7q7RFnzsjv6DswVk2GAtEVQag+KfwdDgd/1oNL3nnnHW6++WZsNpsUbDppaWm8OGwYu4GrgMiHHiIzM5OFCxf6VeTzpaKqKtOnTycvL485p0+TAVQDpvnz5X5SGw4ePEhCQgJf6ec/Bo6vW+fOLnk8qqqiaRrjxo3zHs1fUZSfKIrypaIoexRFyW/n/RBFUV7R3/+3oijJHXHf81FZWUmN7uM/efJk3njjDSwWi9+74RmoqkpxcTF7evRwta1atYqvv/5aLs0vQFpaGj179sSqF2p/Izycte+/L5WKVlgsFlauXElzczOre/dmC84Mkv0++sjdXfN4Kioq2LZtG1988UWnr5QuW/grimICfg/8FBgI3KEoysA2l00FTgghUoAlwILLve93MXz4cFJTU0lOTubhhx92CXwp+L/FYrGQkp4OQArw1VdfsX37dioqKtzbMQ9HVVUmTZqE8Yg8W1Mj69W2Q05ODmVlZfz617/mC73t7++8Ix0LLkBWVhYLFixg6NChna6IdYTmPwzYI4TYJ4RoAP4CjGtzzTjgJf3168AoRek8H0OHw8GxY8c4ceIEM2fOlFpZO9hsNn79wQcIIAv4FRARESFjIS6AzWZj4+rV3KKfh8TGsm7dOjnG2sFsNvP888+7zq93Y1+8AU3TsNlsPP300+Tl5XmFzb8nzhgXg8N6W7vXCCGagJNAQgfcu10sFgtPPPEE6enpLFmyRGpl7WA2m/kyLc3ljTEIp7YmbdfnR9M0pk2bxk+3bSMBZ9WuVampzJ8/X46xdnA4HFRUVPA/YWHsxGkWGPqHP7i7Wx6J4UywceNGmpqaeOaZZ7zD5t9RKIqSqyjKVkVRtl6OvcsosFFTUyNt2OdBVVVeeuklTkdHA9Adp91/7ty5Uov9Dk6ePOl6/TEghJBj7DxUVVURFBSEqqp8rLeVl5fLVCLtYCQKzM/PJyYmhsrKSs+3+QNHgN6tznvpbe1eoyhKIBADHG/7QUKI54UQQ4UQQy9nQiUmJhIaGkpwcPD3/gx/wGw2c0yPwvwJcANOFzMZjdk+69evZ+fOncTp5/E9evDiiy9Krb8djJTXDz74IPv373dl9+xdXS1Nse2gaRqlpaWuVDQtXZAQryOE/ydAf0VR+iqKEgzcDrzV5pq3gLv117cCm4QQnZrt1WQyERQU1Jm38HpUVSV03jyXN8bUoCACAgKYMWOGFGhtsNlsPP7449ymqkzQ20KSk6WZ7DwY/v5vvPEGAP8XE0MzMKixkdv27HFv5zwQVVWZOnUqDoeD5uZm0tLSPH/DV7fh/wp4B6gAXhVC7FAUZa6iKDfplxUDCYqi7AEeAv7DHbQjMZvNDB06VLp2XgBN0/jjH//o8saob2xEVVWG6wVfJN9iNpuZPXs2wzWNQMAWF8dLsbFSg/0OLBYLf/rTn8jIyCD+llt4W2+/orbWrf3yRDRNY8mSJdTX11NXV8esWbO8YsMXIcQGIcQAIcSVQogive1JIcRb+us6IcR4IUSKEGKYEGJfR9z3fKiqypIlS6RWdgFUVWXFihXEx8YCTreto0ePyrqrbdA0jYkTJzJv3jya9Bw+NpNJVkG7CCwWC5MmTWLVqlXY9bbob76R0b7tcPLkSer0qPGuwKM2fDsKo2at3Fi6MGazmRXXXsuXJhODgCVCkJ+fLzXaVqiqSl5eHvfee6+raldwcLD0JLtI7rzzTlJSUrDq58Pr63npllvkGGtDrb4iamlpYenSpf7l7dNRGPbGgoICOcAugKqqPPHEE/xLPw8H9u7dS0VFhXx46pSXl3PnnXcydPNmsoAaoNhkkl4+F4mqqnz44YdklZbydVISAcDIb75xd7c8BpvNRkVFBfv2OQ0iAQEBTJ482TvMPp6ITOdw8WRlZfGzn/0McNb2DQ0Npbq6mjFjxsgHAM6UDr1792aPnp5gLXCiZ9tQFsmFWLJkCW8fdKZ6i5blHQGn4L/xxhvZt28foXqWWIDy3/9eav7fF03TpOC/BEw33YQAhgP31tQQHx/PmjVr5L6JzgsvvECU/jokOpoXXnhBjq9LwOFwYLfbMcTZf1VXo0q7v6vo/bp165g9ezb9+/dn3S9+waqdOzv9+/FJ4d9V+bB9iaPDhrFVD/garvv+L1u2zO+/Q03TmDp1Kq+99hpX6W31Mjf9JaFpGrNnzyYsLIw5OIvfBAN7p093c888g3nz5jFs2DAKCgoI2b2bH69ahVJXB0891an39Unh3zY3tuS7MSankXs97MwZHnzwQSZOnOjWfnkCFRUV7Nixg4Dly7kRp73/j42N0lvlElBVlQULFtDY2Ag4hb/R7s8YXmQ33XQTBQUFACwKDiYUUMxm+MtfOvX+Pin8QQ6sS0FVVWbNmsWXuuviT4Aeu3eTm5vLpEmT/Fb7t9lsjB8/nuPHj5Oit23t2ZPfvPWWNId9D44cOTfw/0hpqd+OLfjW2aJ1NO+18fEoANd3fho8nxX+kksjKyuL4W+/zbagIGdtX2DOnDnMmTPHbx+kRl76Bx54gLN62+YjR3jsscf8Wmh9H8xmM0OGDKF///4sjInhBDCgqYm6++93d9fchqZpVFdXEx8fD8CTJhPxdrszFcYPftD5HRBCeORx7bXXCknXYrVaxUtBQaIFxPMgFEURMTExwmq1urtrbsFqtYrExESRkJAgKkC0gPhk8GBht9vd3TWvpLS0VERERAhA/Fn/PtfFx/vl92m328WIESOEyWQSgAgKChKl+nfyOVzWnAO2iouQsVLzl5yDUWRhGBASEkKzHtHqj5jNZlauXMlrMTFcBewEvnjwQb9dCV0OmqZRXFxMv379CAkJwUjw0E8PbPLHldTrr7/OXXfdBUBjYyND9fbe11/fJWZFKfwlLiwWC/F/+AMncOb3fyMpibq6OqqqqtzdtS7H2Iy76667+EIPvqm7/nqmTJni5p55J8amr6qqrF27lm+uvx4BDKqtpXHRIr/yzjO8EVeuXMmqVasA+DPOtMc1gFZU1CX9kMJfcg71sbFs0F8f++orwsLC/LK6l6qq3HnnnTQ1NXGt3hbu1h55P2azGSEEK1as4P9MJt7T27f98Y8UFhb6zYpKVVUmTpzIc889x+TJkwkPD8diMgFwfMAAnugiF+vATr+DxKsYMGAA1vBwqKmhb2Mjjfqg9DeMgkC/amlhOCAAdepUd3fLq1FVlTVr1lBRUcFnn33GgX/+E4DIM2f8Km5C0zSeeeYZvvnmG/7yl78wpq7OlTNqbWUlE59+uksehFLzl7jQNI0ZM2bwZkuLK9p3ntlMZWWlu7vWpWiaxrPPPss999yDRXfD2xodzdFhw9zcM+/H4XAwfvx45syZw1d628i6OnatXOnWfnU1kZGRqKpKXV0d04KCUIDquDiuKysjJyenS/oghb/EhaGZPfD22/wj0Lko7O5wcPvtt/tdkXK73c7SpUsx68I/ZMAAme20A7BYLDz++OM0NDSwJDTUVUjopB9FkxtZYquqqhg3bhwZesXB0MGDycrK6rJ+SOEvOQdVVUlMTORLxen3E1pXR2BgIEVFRX4V8NW9e3cWjB3LT/TzDw4d4tSpU27tky+gaRrvvfcegwcPZvr06a5CQsdPnPCrqOkBAwbQo0cPfvjmmySePUsT8OVNN13w9zoSKfwl/4HZbCZQz1o5DrhBCPbu3Ut1dbV7O9ZFqKrKz3/+cyLffBMT8FlAAGuSkwkJCXF317wew+snIiKCP/zhD672n0VH+03UtOHtExoaSqredqJfPzJmzuzSfkjhL2mX//74Y07060cgML6ujiuuuIIDBw74hXZmlNQzOBAbS1RUlCze0kFYLBbmzp1LcnIyj0RHsxtIPHEC7rnHL1aWa9euxWq1Un/wIEYSh4j09C7vhxT+knMwtBKHw8E/9IIbRuaRgQMH+oVXhsPh4MCBAxiVjFtaWmRZ0A5E0zSeeuopDh8+zE9/+lM+0Nu322w+7+9fUlLCww8/THR0NC+2tBAJ1MTEEKEXuu9KpPCXnIORERXguG7muR44fPgwc+fO9QvNv6qqikWnT5MG7AKeveoqv3jodRVGQrP6+npeeeUVV96kpp07fdrfX9M0SktLufLKKzl69Cixer3eU1df7Zb+SOEv+Q9UVaWqqor7AwLYCQwEFp85w4YNGxg9ejTl5eVu7mHnUl1djUnf8P4IUBTlu39BcsmkpaVx1VXOCgnWhAQEkFFTQ+yf/uTejnUiRvbcF154gXeys7lGbw8fPvw7f6+zkMJf0i5paWkMHTqUz8PCXG1LliyhW7duLFy40GeX5uvWrSM3N5fBQgDO4K6goCD3dsoHUVWVP/3pT2RmZnL1nDl8pJcwDPHhmBKbzUZOTg4333wzJ95809kG3PrZZ26ZT1L4S/4DowRmcXExURERgLO2b25uLs899xzFxcU+uTS32Wzcc889TDt+nAygCfg4JUUWBuokzGYzjY2NPPLII1ToJpC9//ynzyoWBjExMa4kbnGDB7N69Wq3jC8p/CXn0LoEptlsZkNiIs04o31Ny5dz8803+6zdv6qqijNnzpCkn38YFsaMv/5VbvR2Eg6Hg4iICEJCQlzRvkOrq1ErKtzar84kPDycooMHScKZxO1xN6ZPkcJfcg5tS2DuSEzk7/p76TjrP/gqaWlpDBw4kBQ9urla+vV3GpqmUVBQwNy5c+nZsydPA1sAE7C3sNDNvetYNE1D0zTuvPNOjhw5wgDdlHqwZ0+e3bDBbatKKfwl/4ExGFVVZd68eRzTNzxTwsJISUnx2RTPFRUVpH35JT9oagLgH9984/Ouh+7CUDISExM5ceIEEyZMcEX7+pKJrbXrdGxsLDfHxZGh1zDY6nC4dRUthb/kO6murmaPru2PqK0lwWbjtttuw2azublnHYvNZuOhhx7ixsZGFGAzsCoqihkzZviUMPIkWn+v+/fvJ1LfX6p99113danDMR5yZrOZ+vp67jpxAhOwD1hqsbjVhVgKf8l3Mnz4cP6Wns4nOBNwjW9oYPLkyT7l925kM92zZw/xetsuYOnSpaxZs0Zq/p2IxWLh9ddf5/jx49wHfIkz2nfrkCHu7lqHoaoqDoeDkJAQVzqHL4FgPaGbu5DCX/KdGJWX9kdGutref/99pk6d6lPa/w033EBaczPZ+rkD6Nevn/T06QL27dvHnj17OHv2LP/U2yoqKnxmfNlsNmbOnMmTJhNpetvG2FgiW80pdyCFv+SCmM1mmvXUxkYukilTplBQUOD1WrGmadx0000UFBRwb00NIcDngFpcTFZWlhT8XcCwYcMIDz+3TtromBg39aZjMVaVn332GaZ//xuA3WFh/CU0lDlz5rh1fEnhL7kgmzdvZlJdHQdDQxkIPL5vH0uXLmX69OleLxyNeIYePXq42uoGDpS1ersQi8XCU089RUBAAL83majBWUdi49ChXq/9GzUyHnzwQVex+i9qa8nJyenS3P3tIYW/5ILk5ORQUFDAm0YukqYmPv/8c4qKirxe8wfYsmULR48eJUs/vyI21uuFjjdh5Pjv3r0725ubKdNdbK+MivKJvSWHw8GQuXNJAs4AhWYz69atc/sYk8JfclHk5OQQq6c5GHjmDGfPnmXs2LFer/nbbDamT5/On4EUYC8wZMcOfvSjH7l9cvoLRs6bhoYGkpKS6K2nN86IjnZzzzqOXrrH3NaAALoPHszLL7/s9uBBKfwlF4XFYqFl7FgEMAyY3dJCUVGR1yd5q6yspE5f0QDsCA9n0KBBvPrqq26fnP5EVlYW06dP5+uvv+a3W7cC0GPfPlZlZXn1Q9hms7Fm/HiG6Htmn5pMzJkzx+0mH7hM4a8oSryiKBsVRdmt/4w7z3XNiqJ8rh9vXc49Je7jpytX8qHuntYXePDBB3nqqae81vRjs9koLi4mKSnJlWHxWE0NM2bM8IjJ6U/YbDZWrlzJj370I/4IlAMKcFN8vNc6FhibvaO++goFqAReTk8nMTHR3V0DLl/zzwfeF0L0B97Xz9ujVggxRD+6tlClpEMwJt9x3T0tCXj55ZfZtm0bFV6Yi8Vms5Gfn8+sWbN4LSaGgcA3wKlZs8jJyfFKYePNWCwWli1bxrZt24iPj2eP3t4tONirc/wfOXKEPvX1AFiBmpoaZs6c6RHj63KF/zjgJf31S8DNl/l5Eg/ECFGvqKhgrx6FORoYtG8fTU1NXrfxq2kas2fP5tSpU9x3333s2r4dcGova99/n/LycpnWwQ0MHz6cXr16cerUKc7om76h5eUs99L/hcPh4NkTJ0gBmoEVJhNFRUVuD+5yIYT43gfwTavXSuvzNtc1AVuBj4GbL+azr732WiHxHKxWq8jJyRHFxcXinyBaQJQEBIj09HRRXFzs7u5dMlarVZSWlorQ0FBh0/+eNSAyMzNFdna2sFqt7u6i32G320VZWZno0aOHAMRW/f9ybNw4d3ftkrFarWLkyJHig5AQ0QJiE4jIyEhhtVqF3W7v1HsDW8VFyNjACz0cFEV5D+jezltz2jxEhKIo50v52EcIcURRlH7AJkVRrEKIve3cKxfIBUhKSmr7tsSNWCwWli9fTkVFBRU4UzxHBQayf/9+HnjgAfr16+c1dnJD87fb7cyvq2MgcBroU1rKiwMGYDabvdbM4K1omsaUKVNoaGjgyiuvJDk5mW0ffUQGEKYXevEWDFt/3datXKubfI4AdXV1VFVVeYwjwQXNPkKI0UIISzvHm4CmKMoVAPrPY+f5jCP6z30493KuOc91zwshhgohhvqCf6+voaoqiYmJNOopj3/U0EBKVBSpqamkpaVd4Lc9B4fDwaxZs2hsbKSnbl74MDSURn1zUdL1qKpKSUkJS5YsoaGhgc8//9z1Xu2mTW7s2aWjqip5eXk8WltLGPAVcBfQq1cvj5onl2vzfwu4W399N/Bm2wsURYlTFCVEf52IU2nceZn3lbiRFampHIyIIA6YdvQo+/bt85oCLzabjR/+8IdMmzYNq9VKmq6ZVdfV8eSTT3r15qK3Y3zvQghqamp4AjgBJDocHLjZ+7YTjbLsO/Sf8fHx57vULVyu8J8P3Kgoym6ce4DzARRFGaooyov6NWnAVkVRtgNlwHwhhBT+Xoimaa6Ebu+ePetqr6mpodJLaq+azWaSkpIwm808HxbGQJwVlTakp7N8+XKPWZL7I5qmkZ+fT3FxMbNmzeJkSAgb9Pd2bdvmNZu+5eXlWO+4w5XBcyOQnp7OSy+95FGKxWUJfyHEcSHEKCFEf908VK23bxVC3Ku//kgIkS6EGKz/LO6Ijku6HlVVeeuttygsLCRCbxup/3zmmWe8YnJWVFRw9OhR5s6di6VvXwC2xMby+Nq1UvB7AEIINm7cyG9/+1t69+5Nrd5+TVCQRwnO86FpGg8++CBXNzQAsA1YBtxzzz0eN75khK/kktm+fdv3sucAABwuSURBVDunCwv5BrgKWCIEx44d83jTj6ZpLFu2jJdffpl9+/YRvtO5AO2ZkuJxE9MfMdI8LFiwAIvFwl133cXGoCAEYN63jy9/+UuPVzAqKirYv38/mfr51zidV+688053dqt9LsYlyB2HdPX0XAyXvDW6K97zINLT08V1113X6W5sl4vVahVWq1XsmzBBtIBoAPHp4sXu7pakFYYbblBQkADE3w3XYpNJjB492mPHmN1uF8OGDRN/1vt7CkQ3EAEBAaKsrKzL+sFFunpKzV/yvXjqqado0L1+BuHMkfPpp5+ydu1aj9XONE1j2rRp3HDDDXyyfj0Afwdm/PWvHttnf8RisZCTk8OCBQswmUwc13P994+KcnsO/O9CVVV+8IMfkKyf/ws4Ex5OQIBnilnP7JXEozFylJtnzqQZZ6K3h+rrGTt2LPn5+YwfP97jhKmmaTgcDhobG6mtreXKM2cAaAkPl3V6PZQ777yT3r17Y6upAeC/vvmGDbNne1yiN2Osl5SUsOXZZ7lOb6/AqSS98sorHuXi6eJilgfuOKTZx7Ox2+3CarWK9YoiWkCsDQ8XYWFhIjU11eOiY+12uxg7dqzIzMwU6enp4jF9Wd4MYkJIiDCbzR7XZ4kQZWVlwmQyCUB8rP/PXo+N9agIbLvdLnJyckRZWZmIj48Xf9P7WQkiPDxcZGRkiNGjR4vs7OwuM1chzT6SzsJms5GXlwdAfZwzkWtCTQ21tbXs37/f49w+jY3E/fv3c+211zJQb/8HsD4oiPnz58sNXw8kLS2NvrpHVoVuYuzTp49HlRBVVZXly5eTmJjI1VdfTX+9fTvQrVs3Dh06xJw5cyguLva41aUU/pJLQtM0CgoKKCwsBGCLXppuFM4sf/X19eTm5npcnv+srCzy8vJYtWqVq4j2UaBv3768/PLLHiFIJP+JERjV2NQEQNj27dx7770eV0I0Pz+fZ0wml/B/OyCA+Ph4Xn31VY+tBS2Fv+SSMDQdi8WCxWJhzIYNbDeZCAAmBQdzzTXX8Mwzz3hcnn9N03jjjTeYA2QAAuiWl0e3bt3c3DPJ+VBVldtvvx2AJ7p1oxIYCHwyeDDV1dVu7VtrHA4HQgj66vtInwGvRURw4MABj8nd3x5S+EsumdZaTGJiItv0KkWKHtjy7LPP8vnnn3uU3//69ev5/PPPXVGXR/v148bf/541a9awevVqj9TM/B1N01i/fj0pKSk4HA4+0fMwfbp1K+PHj2fdunVu7uG3SdxGjx5No16BbB/wwAMPUF5e7tHmRCn8JZeFxWJh4A9+AMBPgES7nV27/n97Zx8dVXkm8N+bmUxmJoQ4JMOkgWBgY5zQTN2daoYqcekhu7rQLi7Yg626YurxaHY5sqdmqbDak3rwQGmjrTVU28meHj8KSwf7pdW624nr2bOrFWsZaCxFFBCYGECQD8nns3/MvdcJnwmG3JnM+zvnnsy9c3Pvc5P3fe7zPu/zPO9blJeXc+DAAXuFM9i6dSvNzc2IiOXy6di5k02bNhEIBLTiz2Dcbje33norIsJHRh2mK/Pz+fa3v83ChQttlc2MIHvzzTepufdepgDvAV+bNo0nn3wyY9r/2dDKX/OJmb5+PXtLSvACi/fvJxgMcv3113PTTTfZHpbX1dVFbW0t99xzDyv42OVTtGQJ69atyyjXlGYogUCAaDRKbW0tXq+Xp0gtijL9gw84et99bNq0ybb/n1nnyu/309HRQdAIfHgVOHz4MKtWrWLt2rUZ3b608td8Isw67L9LLdrDgAgnT55k7dq1+Hw+7CzNbXbQTZs28Z3vfMey+l8Gbo3F2Lx5c0a5pjSn093dzdKlS1myZAn/DbxmKNkqh4M77riDW265xTYFK0abn/LSS1x68CAAyQkTyM/PB0ApZYtcw0Urf80nprm5maNHjwLwBeOYx+OxFH9XV5dtHfTkyZOsW7eOSy+9lM8Yx3wzZ3LFFVdQbSzcoslcamtrefrpp0kkErhcLrZ98AGQCi3u6emxLePXXH8AYO8PfgDAn4B/r65m5cqV3H///dx7770Z7VLUyl9zwZhr+9bU1OBcs4YTpJZ8+9m77/LRRx8xa9Ysuru7aWxs5Ktf/aotL4GmpiYefvhhlu7dS4jUeqI/GBzkwQcfzOhIDM3H1NTUcOzYMQYGBvijYU3/DXDdiRPs3LnTVtfKvHnzKDDyWv5Mqujhs88+y6OPPpqZWb3pDCcTzI5NZ/hmB8lkUpLJpMyePdvKwjwJ8mkQpZTMmjXLWrfUXAd4LDId4/G4hMNhcblcsnjxYnkmL08GQX5jrNNryq3JfJLJpNTX10swGBRADhjt7BiIz+eT+vp6W/6XiURC2vLzrQKBX5w4UYqLiyUWi415Vm866AxfzVhgRsusW7eOJcEghwEX8JzbjVKK3t5eDhw4QHd3t5UcdrGHwh0dHdx444288847TJw4kQ0bNvAZIxz1squvpqSkxJJdkx0UFRXR3NyMUoqFpCbtvcDbhhtorEgfZfj9fuqMkcirwC8//BCfz0d1dTUul4vVq1dndhsbzhvCjk1b/tlFMpmUuXPnStywhPpArgUpLy+XkpISaWhoGJN6LGYdn/r6emlubpaCggJZZViKvSD/FAqNaXldzehgWtDRaFSCwaD81vifDoLsnTZN4vH4RR/NmXV8rPvE4zKQVtZ88uTJVp0oO0eVDNPyt13Jn23Tyj/7SCQSEolE5KDRId4zinIFg8ExUbhmh4vH49LS0iJ5eXkCyC8MeX7rdEo8Hh8z15NmdDGV73XXXSeAHEp7ASxyOCQUCl1UV4up9E0Do3PmTKuIW35+vhQWFkqrsTZENih/7fbRjBp+vx+Xy8XLxn75wABLjM+rVq26qDH/5uRzR0cHixYtoqWlhfz8fD4NXGec02vUh2lra8vs4bjmNLq6uggEAsyePZsXX3wRgEmk1l8G+PHAAPv27btoETZm+4JU+OnBgwcRYyW4PwF9fX0cP36cb37zm3R0dNDU1JTRMf6Atvw1o0s8Hhe/3y8fGKWe3025ZyUUClmrMF1syywWi4nX6xWv1ytvGpbhAZBwOKwt/iwk3d2SSCSksrJSvF6vANJoTOSb/2PT/XMxMK8diUTke4GAdd87CwqkqqpKQqGQNfmsLX9NzjFnzhyWLFnCrySVADMNuA84cuQIR48epbOzk8bGxosW9tnY2MiuXbvo6ekh2N9vJXa9Dtxyyy3a4s9CzGKCAMuWLePIkSNMmzYNgPbBQd4wzpsEeL74xSGJX6PVxjo6Orj55pt55pln2Lx5M9ON674BdF1/PUePHuV73/seGzduzJ6SIcN5Q9ixacs/O4nH4xIIBKS8vFySpu/fsP69Xq9lHcXjcZk3b96oWkjJZFLC4bD4/X4pLy+XPxv3PwhSWFiYMQuAaC6cRCIhDQ0NEo/HpbKyUgBxuVxyJG2Bnn8xRnjpI4ZPQjKZlPnz51shnKFQSN417rcJxO/3SzQaHaUn/OSgLX/NWNPV1cWjjz5KW1sbZWVlrDeOlwOrgP7+frZt28bhw4d54IEH6DWqgF7ovU793NnZybvvvsuXvvQlrtq3jxnG9/8DPPjggxldYVEzPGpra3nqqacoLS3l2LFjlJaW0tvbSzEp/78CHnwjNRYwRwyjYYX39PRYawv8x8AA04BjwF2Ay+Wytc7QhaKVv2bUMDvbNddcQ2FhId+vqmKf8d1Xgd7eXgYHB+nr6wPg4YcfvqD7mJNvpuuoqamJrVu3cv/999PX18cvf/lLvktKERwCvlxYyLN6kfZxQyAQoLa2lhUrVgypnLnC+OkF8svKrEli+OQlRk6ePMkDDzzAoUOHOGhM9HYARwoKOHHiBI2Njdnh6klDK3/NqGJ2AJfLRUlJCU9OmADAZOBJ45z+/n4KjNrs54uKONN36Rad+Xn79u0opejr6+OGPXuYZpz7m7w8Lr/88tF6PE2G0NXVdVo9/y1z5vAn47MPcNx2m3VueokR89hIcLvdHD9+nFnJJJ8zju0EPvWpTzF16lTa29ttr2A7YobjG7Jj0z7/7CYej0tDQ4M0Nzdb/tH3Dd+/z+cbVjz0uXy2pk83kUhILBYTt9stCxYsEED2nRL9kUgkbEu111w8otGoFBcXS0VFhXi9XikuLpby8nI5llb+4VeGLz49Amc4cwHp35kRZD6fT15wOq3YfkAmTJggJSUlEovFMiZ/BJ3kpbELMwkmHA6L0+mUf0xLxtkIsmDBAikoKJBYLHZa8ld6Bz3TT5HUpN+8efMkEolISUmJ1NfXy5IlSwSQ76bdq72oSBKJhO0Zl5rRJ5FISCAQkFAoJD6fT5qbm8XtdgsglZWVQ14AZ5roP1t7OLUGldmWQ6GQFBQUSGdaO+aUBMZMaWNa+WtsxexEZrbtO0anOWJ0mmnTpkllZaXk5+cP6TwLFy48rQCcqexNJb5w4UKJx+NSX18v4XBYgsGgFfdtZhe/D1JSUiKRSETKysp0pM84JB6PWxa5x+OxIn+8Xq8sM/JMBkGOpUV6nSsG32xn9fX1VttNJpMSjUbF4XDII2kRRYscDqmqqrJyVzIJrfw1tpKupH0+n1wL0m90nhdAnE6nAOJ2uyUWi1kvi3R3TrrlFYlErI5mfldXVydVVVXidDrF7/dbZRwGQX69eLHVebXiH3+Y7WL+/PnS2toqGEaFuSmlZHdBwcejQIfDCi+eP3/+aaNBs+xHNBqVQCAgra2tkp+fL6FQyKomaravV4zrRyKRjKwTpZW/xnZM5V1XVyc+n0/2FhVZcfcYlrnH45Hi4mKJRCLi9/slFovJ3LlzpaGhYYiyr6+vl/r6eonFYlbMdVFRkTidTvH5fIIxqjCziouLizPSKtOMHumGQiQSkfLyckv5T5o0STDcPun+f9N/b5ZcjsVikkgkpLy8XFpbW2XevHnS2toqs2fPFo/HY7mSJqdd66cgLS0tY1qifCRo5a/JGMxJ2VejUekxi6ylWWler1disZjlxjGVvFkJ1EzsCYfDlisnEolIcXGxBAIBASxf7DGjowaDQW3x5wCmAjbbT0VFhWX5V1RUiNPpHPICMJMQI5GItLa2WiNP0+IPBoNSUlJiWfvmtjHNsFBKSUlJScbOJQ1X+etQT81Fpauri2XLlnHnnXfSsGwZfzaOfxYoLCzE7XZz8uRJdu3axaxZs9iyZQuHDx/mkUceoaenh9tuu4077riDu+++G6/Xy2WXXcbq1as5fvw4fr+f7u5uFjkcVBvXfROYFAyyceNGndQ1zunq6uIb3/gGS5cuJRqNcvz4cSvuX0S4+uqrERHWGWvqeoHLP/95HnroIfLz89mwYQOPPfYYbW1t/OQnP2HRokXs378fn89Hf38/Siny8/NxOp1WwuAWUolmTzzxBLW1tVkX2z+E4bwh7Ni05T9+SCaTEo/Hpa6uToLBoGWJvQlSVFRkDa0BKTCKZIXDYYlGo5KXlyd5eXkSDoet4nCLFy+2Jvc8Ho91vUOkCshlUqq95uKSHhQQiUSsiV+M+aSmpibxer1WuPEgyBOTJklVVZW4XC5rxJDuMsrLyxOllLWfHq320pw5kkgkZO7cuRlp9Ytoy1+TQQQCAWpqalBKsWPHDv5gHK8Cyh0Oenp6AHA6nfT09LBjxw62bNnCK6+8gtvtBuDDDz/krbfe4sSJE2zYsIHq6mp6e3t5/6OP8BrXex648cYbaWpqshKAdFbv+Ma0vP1+PyJCb28vZWVl5Ofnc/LkSdrb2zlx4gSVfFz++SuHDrFr1y5WrlxJdXU1e/bsYd++fRQVFTFx4kSuvfZaPB6PteLbjcbvdRUX882BAQ4cOMC2bdvo7u4e68cdXYbzhjjbBnwJ2AYMAlee47zrSZW93gF8fTjX1pb/+MP0/be2tsoHhiXVCVJWViYej0fy8vLE7XaL2+22fLcej0eqqqokGAwOGSEA1jUGQXaDNRcQDoetiI5MnJDTjD7m6LKwsFAAmT59ugQCAamsrJS8vDxxGAsLpfv/zTmCvLw8ufrqqy2rP72NORwOedv4nf9yu63RQibPJzEWE75ADXA5qTIXZ1T+gAN4G5hBannXPwAzz3dtrfzHF+khm8XFxfKi0aHM5R7N+vsFBQXidDot186UKVPE7XaL0+mUKVOmWB30WJriTxqhoy0tLRIIBIbUdNeKf/yTnrEbjUalqKjIegkAUlpaarmBXr7iCqvd7ExT8n6/XwKBgBQWForL5bJckFEjX+AYyBUOh7S0tGS8QTEmyt+6yLmV/+eAF9P27wPuO981tfIff5ghm8XFxRIOh+VDo2N1klrr1+ykfr9fHA7HEEvfDOd8Ic16M8NGnU6ntLa2WnkFmtwjvYSzmVhoth2PxyMul0vy8vLkuuuus8p/DILcx9D8AI/HY2WLu91uecU472XIiCUah0MmKf8bgR+l7d8KfP9819TKf3ySSCQkHA5LOByWXxurMA2ALGBogg4gHxqTuIcMJZ+u9M36KuYwPWrEcGtyGzP8MhKJnOYmTN/S3T+nunrMNjjX6ZQ+47zHnM6syRsZrvI/74SvUuo/lVJbz7AtON/vjhSl1J1KqdeVUq9n/WSK5qzs3r2bvr4+Vv7lX3KEVOnlp0mVXz4EHBLhGDABuMTYfGBN7J4A/q28nJ+0tADg8Xi45JJLxvgpNJmGGfrZ2dnJ22+/zWWXXUZFRQUVFRUAOBwO69wIMECqTR0ZHMTr9dLc3ExZWRkApaWl3NPfj4PUROW/FhSwcuXK7A7tPIXzKn8RaRCR2jNsPx/mPfYCFWn7U41jZ7rXEyJypYhc6ff7h3l5TTZRW1vL448/zv79+wH4tTel0r18rOiLjf0+4IixHTZ+bgEme70843IxdepUioqKqKqq4q677hqyfJ8m9zDLe8+ZM4fHH38cEWHPnj0cPHiQxYsXMzAwYJ27PT+f54y2VwhsPnGCtrY2uru7CQQCHDx4kL8wzj0+YwazZs2ipqbm9JtmM8MZHpxv49xuHyep0tfT+XjC99Pnu6Z2+4xv4vG4zJ49W4qKiuSJNPdO+jbNmPwlbTheUVFhDee9Xq9EIhFduVNzWglms6qs2W7S4/gxXD2n+v//xeEQj8cjHo9HVqUd/04olFVzSYxRtM8/AO8BPUAXxsQuqZX7nk87bx6wnVTUz8rhXFsr//FNetXPUCgkEyZMkMWLF4vT6Rziqy0sLJSqqiqZPHmyVbM9FApJU1OTVf5Bk9ucqT6/OQEci8UkHA5LIpGwEr7cRsgmxqRuuv/f4/GI0+mU38+YIYMgf/R6raJv2WJcjInyv5ibVv7jF9MyMyfQzExepZS0tLRIa2ureDwecTgcEgwGJRQKicPhkKKiIgmFQlZ9H634NSbnqs9vZuMmk0lrjQlzJOD1euWGqiprYvcoqSxxc0RwbM6cc14/Exmu8ndeNH+SRnMWAoEAa9asYfny5XR3d/Pcc8/xwx/+EIC6ujpmz55Nb28v06dPp7i4GICZM2dy++2385WvfIVnnnmGb33rW0PWb9XkNueaiDWzxAF+/OMf89prr7Fp0yZuuOEGPvvZz7JixQreqa6mavt2CoH/7ezECxxUioH16yk8z/WzFa38NbZQW1tLe3s7AC0tLSxfvhylFDNmzEBEmDlzJoC12Pvu3btZvnw5AGvWrOGhhx5i7dq11NTUjMuOqRkdAoEA9957LwCNjY0cPXqUiRMnsnDhQpqamkgkErS1tbEsGuWp7du5BPD29wOg/vqvh7St9AXhxwO6to/GVpqamvD7/bS3txONRiktLaWgoIDbb7/dCtkrLCykvb2dcDjM888/T3V1NXV1dSil7BZfk+Fs3bqVm2++me7ubtasWcPEiRNZvXo1jY2NvPDCC2zcuJFrrrmG9vZ29iYSVv0fgJKrrrI+d3V10dTUNL6iyYbjG7Jj0z7/3OBMvlSzPEN9ff2QUg3pq3ud7Xc1mlNJnxs6dZ7IrP9kTuomEgmRoiKR5ubTrpMt7Q094avJRtIjN06N3simiAtN5nFqGzIV/qk/T21j2dbmhqv8VerczOPKK6+U119/3W4xNDZwNt/qePO5asYesw1t3bqVhoYG1q9fbyVvBQKB09qY6e5pa2vLmranlNosIlee7zzt89dkHGfrZNnS+TSZS3r9/1AoRGlpKU1NTad9n76fTYp/JGjLX6PR5CSmlT/eRpTa8tdoNJpzYCr88aT4R4JW/hqNRpODaOWv0Wg0OYhW/hqNRpODaOWv0Wg0OYhW/hqNRpODaOWv0Wg0OYhW/hqNRpODZGySl1KqG9h1kS5fCmRzMXgtv/1k+zNku/yQ/c9wseS/VETOuwh6xir/i4lS6vXhZMBlKlp++8n2Z8h2+SH7n8Fu+bXbR6PRaHIQrfw1Go0mB8lV5f+E3QJ8QrT89pPtz5Dt8kP2P4Ot8uekz1+j0WhynVy1/DUajSanyVnlr5RaqpR6Sym1TSn1LbvluVCUUl9TSolSqtRuWUaCUmqt8fffopR6Vil1id0yDQel1PVKqT8ppXYopb5utzwjRSlVoZSKK6X+aLT9e+yW6UJQSjmUUr9XSv3KblkuBKXUJUqpnxp9oFMp9bmxliEnlb9S6vPAAuAKEfk08G2bRboglFIVwN8Cu+2W5QJ4CagVkc8A24H7bJbnvCilHMBjwN8BM4EvK6Vm2ivViOkHviYiM4FZwD9l4TMA3AN02i3EJ+C7wAsiEgSuwIZnyUnlD9wNrBaRHgARed9meS6Uh4F/BbJu4kZEfiMi/cbu/wFT7ZRnmNQBO0Rkp4j0AutJGRFZg4jsF5E3jM9HSSmdKfZKNTKUUlOB+cCP7JblQlBKFQPXAlEAEekVkcNjLUeuKv9qoF4p9apS6mWl1FV2CzRSlFILgL0i8ge7ZRkFGoFf2y3EMJgC7Enbf48sU5zpKKUqgb8CXrVXkhHzCCmjZ9BuQS6Q6UA38O+G6+pHSqnCsRbCOdY3HCuUUv8JlJ3hq5WknnsSqWHvVcB/KKVmSIaFPp3nGVaQcvlkLOeSX0R+bpyzkpQr4umxlC3XUUpNAGLAMhH50G55hotS6gvA+yKyWSk1x255LhAnEAaWisirSqnvAl8H7h9rIcYlItJwtu+UUncDmwxl/5pSapBUnY3usZJvOJztGZRSIVLWwx+UUpBymbyhlKoTkeQYinhOzvU/AFBKLQG+AMzNtBfvWdgLVKTtTzWOZRVKqXxSiv9pEdlktzwj5Brg75VS8wA3MFEp9ZSI3GKzXCPhPeA9ETFHXD8lpfzHlFx1+/wM+DyAUqoacJFFBaJEJCEik0WkUkQqSTWmcCYp/vOhlLqe1ND970XkhN3yDJPfAZcppaYrpVzATcAvbJZpRKiUtRAFOkWk1W55RoqI3CciU412fxPw2yxT/Bj9dI9S6nLj0Fzgj2Mtx7i1/M9DO9CulNoK9AK3ZYnlOZ74PlAAvGSMXv5PRO6yV6RzIyL9Sql/Bl4EHEC7iGyzWayRcg1wK5BQSr1pHFshIs/bKFMushR42jAidgK3j7UAOsNXo9FocpBcdftoNBpNTqOVv0aj0eQgWvlrNBpNDqKVv0aj0eQgWvlrNBpNDqKVv0aj0eQgWvlrNBpNDqKVv0aj0eQg/w9gEnvlCK3UnwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "sess = tf_reset()\n",
    "\n",
    "# create the model\n",
    "input_ph, output_ph, output_pred = create_model()\n",
    "\n",
    "# restore the saved model\n",
    "saver = tf.train.Saver()\n",
    "saver.restore(sess, \"/tmp/model.ckpt\")\n",
    "\n",
    "output_pred_run = sess.run(output_pred, feed_dict={input_ph: inputs})\n",
    "\n",
    "plt.scatter(inputs[:, 0], outputs[:, 0], c='k', marker='o', s=0.1)\n",
    "plt.scatter(inputs[:, 0], output_pred_run[:, 0], c='r', marker='o', s=0.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not so hard after all! There is much more functionality to Tensorflow besides what we've covered, but you now know the basics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5. Tips and tricks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### (a) Check your dimensions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "# example of \"surprising\" resulting dimensions due to broadcasting\n",
    "a = tf.constant(np.random.random((4, 1)))\n",
    "b = tf.constant(np.random.random((1, 4)))\n",
    "c = a * b\n",
    "assert c.get_shape() == (4, 4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### (b) Check what variables have been created"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "I_am_a_variable:0\n",
      "I_am_a_variable_too:0\n"
     ]
    }
   ],
   "source": [
    "sess = tf_reset()\n",
    "a = tf.get_variable('I_am_a_variable', shape=[4, 6])\n",
    "b = tf.get_variable('I_am_a_variable_too', shape=[2, 7])\n",
    "for var in tf.global_variables():\n",
    "    print(var.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### (c) Look at the [tensorflow API](https://www.tensorflow.org/api_docs/python/), or open up a python terminal and investigate!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function reduce_mean in module tensorflow.python.ops.math_ops:\n",
      "\n",
      "reduce_mean(input_tensor, axis=None, keepdims=None, name=None, reduction_indices=None, keep_dims=None)\n",
      "    Computes the mean of elements across dimensions of a tensor. (deprecated arguments)\n",
      "    \n",
      "    SOME ARGUMENTS ARE DEPRECATED. They will be removed in a future version.\n",
      "    Instructions for updating:\n",
      "    keep_dims is deprecated, use keepdims instead\n",
      "    \n",
      "    Reduces `input_tensor` along the dimensions given in `axis`.\n",
      "    Unless `keepdims` is true, the rank of the tensor is reduced by 1 for each\n",
      "    entry in `axis`. If `keepdims` is true, the reduced dimensions\n",
      "    are retained with length 1.\n",
      "    \n",
      "    If `axis` is None, all dimensions are reduced, and a\n",
      "    tensor with a single element is returned.\n",
      "    \n",
      "    For example:\n",
      "    \n",
      "    ```python\n",
      "    x = tf.constant([[1., 1.], [2., 2.]])\n",
      "    tf.reduce_mean(x)  # 1.5\n",
      "    tf.reduce_mean(x, 0)  # [1.5, 1.5]\n",
      "    tf.reduce_mean(x, 1)  # [1.,  2.]\n",
      "    ```\n",
      "    \n",
      "    Args:\n",
      "      input_tensor: The tensor to reduce. Should have numeric type.\n",
      "      axis: The dimensions to reduce. If `None` (the default),\n",
      "        reduces all dimensions. Must be in the range\n",
      "        `[-rank(input_tensor), rank(input_tensor))`.\n",
      "      keepdims: If true, retains reduced dimensions with length 1.\n",
      "      name: A name for the operation (optional).\n",
      "      reduction_indices: The old (deprecated) name for axis.\n",
      "      keep_dims: Deprecated alias for `keepdims`.\n",
      "    \n",
      "    Returns:\n",
      "      The reduced tensor.\n",
      "    \n",
      "    @compatibility(numpy)\n",
      "    Equivalent to np.mean\n",
      "    \n",
      "    Please note that `np.mean` has a `dtype` parameter that could be used to\n",
      "    specify the output type. By default this is `dtype=float64`. On the other\n",
      "    hand, `tf.reduce_mean` has an aggressive type inference from `input_tensor`,\n",
      "    for example:\n",
      "    \n",
      "    ```python\n",
      "    x = tf.constant([1, 0, 1, 0])\n",
      "    tf.reduce_mean(x)  # 0\n",
      "    y = tf.constant([1., 0., 1., 0.])\n",
      "    tf.reduce_mean(y)  # 0.5\n",
      "    ```\n",
      "    \n",
      "    @end_compatibility\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(tf.reduce_mean)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### (d) Tensorflow has some built-in layers to simplify your code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Help on function fully_connected in module tensorflow.contrib.layers.python.layers.layers:\n",
      "\n",
      "fully_connected(inputs, num_outputs, activation_fn=<function relu at 0x1280cc2f0>, normalizer_fn=None, normalizer_params=None, weights_initializer=<function variance_scaling_initializer.<locals>._initializer at 0x13c4dc048>, weights_regularizer=None, biases_initializer=<tensorflow.python.ops.init_ops.Zeros object at 0x13c4d4eb8>, biases_regularizer=None, reuse=None, variables_collections=None, outputs_collections=None, trainable=True, scope=None)\n",
      "    Adds a fully connected layer.\n",
      "    \n",
      "    `fully_connected` creates a variable called `weights`, representing a fully\n",
      "    connected weight matrix, which is multiplied by the `inputs` to produce a\n",
      "    `Tensor` of hidden units. If a `normalizer_fn` is provided (such as\n",
      "    `batch_norm`), it is then applied. Otherwise, if `normalizer_fn` is\n",
      "    None and a `biases_initializer` is provided then a `biases` variable would be\n",
      "    created and added the hidden units. Finally, if `activation_fn` is not `None`,\n",
      "    it is applied to the hidden units as well.\n",
      "    \n",
      "    Note: that if `inputs` have a rank greater than 2, then `inputs` is flattened\n",
      "    prior to the initial matrix multiply by `weights`.\n",
      "    \n",
      "    Args:\n",
      "      inputs: A tensor of at least rank 2 and static value for the last dimension;\n",
      "        i.e. `[batch_size, depth]`, `[None, None, None, channels]`.\n",
      "      num_outputs: Integer or long, the number of output units in the layer.\n",
      "      activation_fn: Activation function. The default value is a ReLU function.\n",
      "        Explicitly set it to None to skip it and maintain a linear activation.\n",
      "      normalizer_fn: Normalization function to use instead of `biases`. If\n",
      "        `normalizer_fn` is provided then `biases_initializer` and\n",
      "        `biases_regularizer` are ignored and `biases` are not created nor added.\n",
      "        default set to None for no normalizer function\n",
      "      normalizer_params: Normalization function parameters.\n",
      "      weights_initializer: An initializer for the weights.\n",
      "      weights_regularizer: Optional regularizer for the weights.\n",
      "      biases_initializer: An initializer for the biases. If None skip biases.\n",
      "      biases_regularizer: Optional regularizer for the biases.\n",
      "      reuse: Whether or not the layer and its variables should be reused. To be\n",
      "        able to reuse the layer scope must be given.\n",
      "      variables_collections: Optional list of collections for all the variables or\n",
      "        a dictionary containing a different list of collections per variable.\n",
      "      outputs_collections: Collection to add the outputs.\n",
      "      trainable: If `True` also add variables to the graph collection\n",
      "        `GraphKeys.TRAINABLE_VARIABLES` (see tf.Variable).\n",
      "      scope: Optional scope for variable_scope.\n",
      "    \n",
      "    Returns:\n",
      "       The tensor variable representing the result of the series of operations.\n",
      "    \n",
      "    Raises:\n",
      "      ValueError: If x has rank less than 2 or if its last dimension is not set.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "help(tf.contrib.layers.fully_connected)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### (e) Use [variable scope](https://www.tensorflow.org/guide/variables#sharing_variables) to keep your variables organized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "layer_0/W0:0\n",
      "layer_0/b0:0\n",
      "layer_1/W1:0\n",
      "layer_1/b1:0\n",
      "layer_2/W2:0\n",
      "layer_2/b2:0\n"
     ]
    }
   ],
   "source": [
    "sess = tf_reset()\n",
    "\n",
    "# create variables\n",
    "with tf.variable_scope('layer_0'):\n",
    "    W0 = tf.get_variable(name='W0', shape=[1, 20], initializer=tf.contrib.layers.xavier_initializer())\n",
    "    b0 = tf.get_variable(name='b0', shape=[20], initializer=tf.constant_initializer(0.))\n",
    "\n",
    "with tf.variable_scope('layer_1'):\n",
    "    W1 = tf.get_variable(name='W1', shape=[20, 20], initializer=tf.contrib.layers.xavier_initializer())\n",
    "    b1 = tf.get_variable(name='b1', shape=[20], initializer=tf.constant_initializer(0.))\n",
    "    \n",
    "with tf.variable_scope('layer_2'):\n",
    "    W2 = tf.get_variable(name='W2', shape=[20, 1], initializer=tf.contrib.layers.xavier_initializer())\n",
    "    b2 = tf.get_variable(name='b2', shape=[1], initializer=tf.constant_initializer(0.))\n",
    "\n",
    "# print the variables\n",
    "var_names = sorted([v.name for v in tf.global_variables()])\n",
    "print('\\n'.join(var_names))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### (f) You can specify which GPU you want to use and how much memory you want to use"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "gpu_device = 0\n",
    "gpu_frac = 0.5\n",
    "\n",
    "# make only one of the GPUs visible\n",
    "import os\n",
    "os.environ[\"CUDA_VISIBLE_DEVICES\"] = str(gpu_device)\n",
    "\n",
    "# only use part of the GPU memory\n",
    "gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=gpu_frac)\n",
    "config = tf.ConfigProto(gpu_options=gpu_options)\n",
    "\n",
    "# create the session\n",
    "tf_sess = tf.Session(graph=tf.Graph(), config=config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### (g) You can use [tensorboard](https://www.tensorflow.org/guide/summaries_and_tensorboard) to visualize and monitor the training process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.test.is_gpu_available()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}