Introduction.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Chapter 1. Introduction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* 1.1 What can we learn from the brain\n",
    "    * energy efficiency\n",
    "    * data efficiency\n",
    "    * world models and mental simulation\n",
    "    * compositionality and learning to learn\n",
    "    \n",
    "    \n",
    "* 1.2 History of the brain\n",
    "    * worms, flies, fish, birds, mice, humans\n",
    "    * spinal cord, brain stem, cerebellum, basal ganglia, thalamus, cortex\n",
    "    \n",
    "\n",
    "* 1.3 Paradigms of adaptation\n",
    "    * time scale: evolution, development, learning, imitation, planning, meta-learning\n",
    "    * objective: memory/supervised/reinforcement/unsupervised learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 What can we learn from the brain"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The progress of artificial intelligence (AI) in the last several years is phenomenal. Playing the master level in the game of Go was supposed to be decades away, but AlphaGo beat human champions. \n",
    "\n",
    "Today's success of AI is based on the combination of *deep neural networks* and *reinforcement learning* together with high-performance computing and massive data from the internet. \n",
    "\n",
    "Can this *deep learning revolution* solve all the problems in AI? Certainly not. Deep learning realized human-level vision and hearing, which is enabling applications of AI to areas that were challenging, such as autonomous driving and speech-capable agents. However, there are more to human intelligence than vision and hearing.\n",
    "\n",
    "Then what should we futhre learn from the brain and human cognition to bring AI to the next level? Here are some domains in which today's AI is still limited and learning from the computational architecture of the brain can be beneficial:\n",
    "\n",
    "* Energy efficiency\n",
    "\n",
    "* Data efficiency\n",
    "\n",
    "    * Modeling and simulating the world\n",
    "\n",
    "    * Modularity and compositionality\n",
    "\n",
    "    * Learning to learn\n",
    "    \n",
    "* Autonomy and sociality"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To understand how our brain realizes such energy-efficient operation, understanding the biophysics of neurons and the dynamics of their network is imperative. We will start at this level first, although briefly.\n",
    "\n",
    "To understand the mechanisms of learning in the brain, we first have to know different frameworks of learning and then understand how they can be computationaly realized efficiently and how they can possibly implemented in the brain.\n",
    "\n",
    "How we can imagine the future based on our experience and make a predictive judgement is a big open question. We will consider their possible mechanisms toward the end of this book."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 History of the brain"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Human intelligence, including the use of language and doing science, is made possible by the human brain. Our brain, however, is a product of millions of years of animal evolution. To understand how it really works, it is imperative to understand how it evolved, by also looking into brains of other animals.\n",
    "\n",
    "For example, the mammals from mice to humans share the same six-layer architecture of the neocortex.\n",
    "Vertebrates from fish and birds to humans share basically the same design of the forebrain, midbrain, hindbrain, and spinal cord.\n",
    "Even invertebrates from worms and insects share almost the same set of molecules with humans for building the brain. It goes without saying that analysis can be easier and more rigorous by using simpler animals.\n",
    "\n",
    "Major difference: degree of freedom in learning and planning."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Organization of the mammalian brain\n",
    "\n",
    "The spinal cord is often thought as the connection line between the brain and the body, but it constitutes a quite complex neural network for movement control and perception.\n",
    "\n",
    "The brain stem consists of dedicated neural circuits for a variety of physiological regulation and motor control.\n",
    "\n",
    ":::{figure-md} Fig_MBrain\n",
    "<img src=\"figures/MammalianBrain.jpg\" alt=\"Mammalian Brain\" width=\"500px\">\n",
    "\n",
    "The whole architecture of the mammalian brain.\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.3 Paradigms of adaptation\n",
    "\n",
    "Adaptation can happen in different time scales and for different objectives."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adaptation in various time scales\n",
    "\n",
    "* __Learning__: Learning can happen very quickly. An infant learns new objects and words everyday. There are some type of learning that takes longer, such as sophisticated motor skills.\n",
    "\n",
    "* __Meta-learning__: After we learn one skill, such as skiing, it helps learning a new skill, such as snowboarding, even though the required muscle outputs are quite different. This suggests that the brain has the capability for \"learning to learn\" or meta-learning.\n",
    "\n",
    "* __Imitation and cultural evolution__: Mimicking what others do seems like a simple job, but it actually involves sophisticated computation for mapping the vision of other's movement to own body movements. Even more importantly, imitation is the basis for cultural evolution, by which skills and knowledge can continue to progress beyond inidvidual's lifetime.\n",
    "\n",
    "* __Development__: The brain's neral circuit follows a standard course of development, from embryo to newborn and child to adult. Although the number of neurons in the brain is almost constant after birth, there appears to be genetic and epigenetic mechanisms to regulated how learning goes on in different parts of the brain at different stages of maturation.\n",
    "\n",
    "* __Evolution__: All those mechanims for adaptation of the brain is defined in the genome, which is a product of millions of years of evolution. Simple animals tend to have small brains speciallized for particular behaviors in particular environments, but can adapt rapidly to new environments by producing many offsprings in relatively short life cycles. Vertabrates, especially mammals, can adapt to a wid range of environments within a lifetime using higher capability of learning. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning objectives\n",
    "\n",
    "* Memory\n",
    "\n",
    "* Supervised learning\n",
    "\n",
    "* Reinforcement learning\n",
    "\n",
    "* Unsupervised learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Brain and learning algorithms\n",
    "\n",
    ":::{figure-md} Fig_SRU\n",
    "<img src=\"figures/SRU.jpg\" alt=\"Cerebelum-BG-Cortex\" width=\"500px\">\n",
    "\n",
    "The hypthesis that the cerebellum, the basal ganglia, and the cerebral cortex are specialized in supervised, reinforcement, and unsupervised learning, respectively (Doya, 1999).\n",
    ":::"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Hassabis D, Kumaran D, Summerfield C, Botvinick M (2017) Neuroscience-Inspired Artificial Intelligence. Neuron 95:245-258.\n",
    "* Lake BM, Ullman TD, Tenenbaum JB, Gershman SJ (2017) Building machines that learn and think like people. Behav Brain Sci 40:e253.\n",
    "* Minsky M (1986) Society of Mind. New York: Simon and Schuster.\n",
    "* Doya K (1999) What are the computations of the cerebellum, the basal ganglia, and the cerebral cortex. Neural Networks 12:961-974.\n",
    "* Doya K (2002) Metalearning and Neuromodulation. Neural Networks 15:495–506.\n",
    "* Thrun S, Pratt L, eds (1998) Learning to Learn: Springer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}