Finalized 03_nbdev

Author: nicole-brewer

03/01.py

@@ -0,0 +1,2 @@
+
+import dashboard

03_nbdev.ipynb

@@ -84,7 +84,7 @@
     "\n",
     "from IPython.core.magic import register_line_magic, register_cell_magic, needs_local_scope\n",
     "from IPython import get_ipython\n",
-    "import re, sys"
+    "import re, sys, os"
    ]
   },
   {
@@ -178,147 +178,149 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9c7ecc7a-0058-4d2f-be99-0d50a6fe0519",
+   "id": "ca63138c-13ff-445e-8a55-7176def3366f",
    "metadata": {},
    "source": [
-    "## nb_export\n",
-    "\n",
-    "Next we can use the nb_export function to compile the exported cells into a python file. The second parameter tells nbdev which package to export the module to. Let's go ahead and export this notebook to the `dashboard` package. \n",
+    "### %answer line magic\n",
     "\n",
-    "> **IMPORTANT**: do not forget to save before exporting the notebook, or your most recent changes will not make it into the py file. It is also a good habit to \"Restart Kernel and Clear All Outputs\" and then save your notebook at the same time as you export. That way, when you get around to committing your code, your git diff isn't filled up with a bunch of serial numbers and other unintelligible things."
+    "The code below allows us to import code from the nbdev-generated python files like the one we loaded above. You can think of this function working like the `%load` line magic, which pulls in code from an external file. But instead of loading the entire file, this function loads only the python code that came from an exported notebook cell. "
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "0b8b5029-0636-4dda-ae11-d5641450ec35",
+   "id": "bf826708-05c7-48a6-b8ce-8423fe897faf",
    "metadata": {},
    "outputs": [],
    "source": [
-    "from nbdev.export import nb_export\n",
+    "#| export\n",
+    "@register_line_magic('answer')\n",
+    "def answer(inputs):\n",
+    "    '''\n",
+    "    This is a cell magic called answer that allows tutorial goers to import the correct answer from the key. \n",
+    "    '''\n",
+    "    words = []\n",
+    "    for word in inputs.split(' '):\n",
+    "        if not word.startswith('#') and len(word) != 0:\n",
+    "            words.append(word)\n",
+    "        else:\n",
+    "            break\n",
+    "    \n",
+    "    flag = False\n",
+    "    if len(words) == 2:\n",
+    "        if words[1] == '-e':\n",
+    "            flag = True\n",
+    "        else:\n",
+    "            filepath = words[0]\n",
+    "            cell_number = int(words[1])\n",
     "\n",
-    "nb_export('03_nbdev.ipynb', 'dashboard')\n",
-    "nb_export('03_nbdev.ipynb', 'key/dashboard')"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "dcd23a2c-ec98-4ab5-a362-1ff9e1e5f37d",
-   "metadata": {},
-   "source": [
-    "Usually, we will put `nb_export` at the end of our file for convenience, but we will continue on from here."
+    "            with open(filepath, 'r') as file:\n",
+    "                lines = file.readlines()\n",
+    "\n",
+    "            pattern = r'# %%\\s+(.+)\\s+(\\d+)'\n",
+    "            start_line = None\n",
+    "            end_line = None\n",
+    "\n",
+    "            for i, line in enumerate(lines):\n",
+    "                if re.match(pattern, line):\n",
+    "                    match = re.search(pattern, line)\n",
+    "                    if match and int(match.group(2)) == cell_number:\n",
+    "                        start_line = i + 1\n",
+    "                        break\n",
+    "            if start_line is not None:\n",
+    "                for i in range(start_line, len(lines)):\n",
+    "                    if re.match(pattern, lines[i]):\n",
+    "                        end_line = i\n",
+    "                        break\n",
+    "                else:\n",
+    "                    end_line = len(lines)\n",
+    "\n",
+    "            if start_line is not None and end_line is not None:\n",
+    "                code_chunk = f\"#| export\\n# %answer {inputs}\\n\\n\" + ''.join(lines[start_line:end_line])\n",
+    "                code_chunk = code_chunk.rstrip(\"\\n\")\n",
+    "                get_ipython().set_next_input(code_chunk, replace=True)\n",
+    "            else:\n",
+    "                raise Exception(f\"Cell number {cell_number} not found in the Python file.\")\n",
+    "        \n",
+    "    if len(words) == 1 or words[1] == '-e':\n",
+    "        filepath = words[0]\n",
+    "        with open(filepath, 'r') as file:\n",
+    "            lines = file.readlines()\n",
+    "        code_chunk = ''.join(lines[:])\n",
+    "        if flag:\n",
+    "            code_chunk = f\"# %%export {filepath}\\n\\n\" + code_chunk\n",
+    "        else: \n",
+    "            code_chunk = f\"# %answer {filepath}\\n\\n\" + code_chunk\n",
+    "        get_ipython().set_next_input(code_chunk, replace=True)\n",
+    "    \n",
+    "    with open(filepath, 'r') as file:\n",
+    "        lines = file.readlines()"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "72d3d366-c995-4e87-9db0-5eb02ba04142",
+   "id": "8550a7c2-5176-4a9e-b66d-db1ccd44fdf3",
    "metadata": {},
    "source": [
-    "Given this information, what file do you think the code will be exported to? Remember that our default export was set to `__init__`."
+    "If at any point in this tutorial you have trouble coming up with an answer on your own, you can always import it from the answer key.\n",
+    "\n",
+    "### Pull from exported module\n",
+    "\n",
+    "This will be the case when we are pulling in an answer to a cell that starts with `#| export`. The path to the answer key files is the same as the default, but is preceeded by the `key` directory. For example, the next line will load the code from the 6th cell of this notebook, which is the import statements. "
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "3a56bb17-3345-402e-9f99-4d295048ed85",
+   "id": "d2cb3849-b5f0-4313-928e-1cf3bcc425d2",
    "metadata": {},
    "outputs": [],
    "source": [
-    "%load dashboard/__init__.py"
+    "# %answer key/dashboard/__init__.py 6"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "4ef6b07d-1023-48ec-bdb6-879d1c7d216d",
+   "id": "7c9ff319-d009-4d66-8eea-e663b3f41d62",
    "metadata": {},
    "source": [
-    "There are a few thing to notice here. One is that we get a warning about this file being autogenerated. That is, if we make changes to `dashboard/__init__.py`, those changes will be overwritten the next time we run `nb_export`.\n",
+    "### Pull from file\n",
     "\n",
-    "The second thing to notice is that there is a handy comment to tell us where the code came from. `# %% ../_nbdev.ipynb 9` tells us that the code came from the 9th code cell. We will use this information to write our next magic function."
+    "Sometimes we will want to check an answer from a cell that isn't a module exported by nbdev. In this case, we will provide the file path only.\n",
+    "\n",
+    "**In both cases, these paths will be provided for you in comments.**"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "ca63138c-13ff-445e-8a55-7176def3366f",
+   "id": "9c7ecc7a-0058-4d2f-be99-0d50a6fe0519",
    "metadata": {},
    "source": [
-    "### %answer line magic\n",
+    "## nb_export\n",
     "\n",
-    "The code below allows us to import code from the nbdev-generated python files like the one we loaded above. You can think of this function working like the `%load` line magic, which pulls in code from an external file. But instead of loading the entire file, this function loads only the python code that came from an exported notebook cell. "
+    "Next we can use the nb_export function to compile the exported cells into a python file. The second parameter tells nbdev which package to export the module to. Let's go ahead and export this notebook to the `dashboard` package. \n",
+    "\n",
+    "> **IMPORTANT**: do not forget to save before exporting the notebook, or your most recent changes will not make it into the py file. It is also a good habit to \"Restart Kernel and Clear All Outputs\" and then save your notebook at the same time as you export. That way, when you get around to committing your code, your git diff isn't filled up with a bunch of serial numbers and other unintelligible things."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "fd6d6913-b174-4214-86fe-6985939ed041",
+   "id": "0b8b5029-0636-4dda-ae11-d5641450ec35",
    "metadata": {},
    "outputs": [],
    "source": [
-    "#| export \n",
+    "from nbdev.export import nb_export\n",
     "\n",
-    "@register_line_magic('answer')\n",
-    "def answer(inputs):\n",
-    "    '''\n",
-    "    This is a cell magic called answer that allows tutorial goers to import the correct answer from the key. \n",
-    "    '''\n",
-    "    words = []\n",
-    "    for word in inputs.split(' '):\n",
-    "        if not word.startswith('#'):\n",
-    "            words.append(word)\n",
-    "        else:\n",
-    "            break\n",
-    "    \n",
-    "    assert len(words) == 2,  \"%answer takes a filepath and a cell number\"\n",
-    "    filepath = words[0]\n",
-    "    cell_number = int(words[1])\n",
-    "    \n",
-    "    with open(filepath, 'r') as file:\n",
-    "        lines = file.readlines()\n",
-    "\n",
-    "    pattern = r'# %% .+ \\d+'\n",
-    "    start_line = None\n",
-    "    end_line = None\n",
-    "\n",
-    "    for i, line in enumerate(lines):\n",
-    "        if re.match(pattern, line):\n",
-    "            match = re.search(r'\\d+', line)\n",
-    "            if match and int(match.group()) == cell_number:\n",
-    "                start_line = i + 1\n",
-    "                break\n",
-    "    if start_line is not None:\n",
-    "        for i in range(start_line, len(lines)):\n",
-    "            if re.match(pattern, lines[i]):\n",
-    "                end_line = i\n",
-    "                break\n",
-    "        else:\n",
-    "            end_line = len(lines)\n",
-    "            \n",
-    "    if start_line is not None and end_line is not None:\n",
-    "        code_chunk = f\"# %answer {inputs}\\n\" + ''.join(lines[start_line:end_line])\n",
-    "        code_chunk = code_chunk.rstrip(\"\\n\")\n",
-    "        get_ipython().set_next_input(code_chunk, replace=True)\n",
-    "    else:\n",
-    "        print(f\"Cell number {cell_number} not found in the Python file.\")"
+    "nb_export('03_nbdev.ipynb', 'dashboard')"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "8550a7c2-5176-4a9e-b66d-db1ccd44fdf3",
-   "metadata": {},
-   "source": [
-    "If at any point in this tutorial you have trouble coming up with an answer on your own, you can always import it from the answer key. The path to the answer key files is the same as the default, but is preceeded by the `key` directory. These paths will be provided for you in comments.\n",
-    "\n",
-    "What do you think the directory is for the answer key to this notebook? Try to load the code from the 9th cell of this notebook."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "bfcce99c-1a35-4e32-81e0-38ac16ae14c2",
+   "id": "dcd23a2c-ec98-4ab5-a362-1ff9e1e5f37d",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "%answer key/dashboard/__init__.py 9 # %answer <path>/<to>/<file>.py 9"
+    "Usually, we will put `nb_export` at the end of our file for convenience, but we will continue on from here."
    ]
   },
   {
@@ -336,7 +338,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import dashboard # import ..."
+    "# %answer 03/01.py\n",
+    "\n",
+    "# import ..."
    ]
   },
   {
@@ -346,7 +350,7 @@
    "source": [
     "So for most modules, the import statement would look like `dashboard.__init__` but the `__init__` file is special, and will be imported any time we make an import from `dashboard` package.\n",
     "\n",
-    "Great job! Now we know the basic idea of what nbdev does."
+    "Great job! Now we know the basic idea of what nbdev does. Go ahead and use the `%load` line magic inspect the exported file below."
    ]
   },
   {
@@ -356,7 +360,19 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "%load dashboard/__init__.py # %load pkg/module.py"
+    "# %answer key/03/02.py\n",
+    "\n",
+    "# %load pkg/module.py"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4ef6b07d-1023-48ec-bdb6-879d1c7d216d",
+   "metadata": {},
+   "source": [
+    "There are a few thing to notice here. One is that we get a warning about this file being autogenerated. That is, if we make changes to `dashboard/__init__.py`, those changes will be overwritten the next time we run `nb_export`.\n",
+    "\n",
+    "The second thing to notice is that there is a handy comment to tell us where the code came from. `# %% ../_nbdev.ipynb 9` tells us that the code came from the 9th code cell. We will use this information to write our next magic function."
    ]
   },
   {
@@ -369,18 +385,55 @@
     "Of course, nbdev has a lot of other nice features, but the notebook export is the core of literate programming. You can do your work incrementally, leave in a lot of markdown, and read your code like a book. At the same time, you can keep the code in clean, importable python files."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "4fa634ee-53f9-41d1-8ad4-8d55f5e154be",
+   "metadata": {},
+   "source": [
+    "### The End\n",
+    "\n",
+    "We will not go over the code below. Proceed to the next notebook.\n",
+    "___"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "aea02e66-ce09-44f5-ac85-b5539ae1a08d",
+   "id": "97fb40d0-242e-4aea-aa50-d054d38f8b55",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "#| export\n",
+    "@register_cell_magic('export')\n",
+    "def export(line, cell=None):\n",
+    "    line_args = line.split()\n",
+    "    export_filepath = None\n",
+    "\n",
+    "    if len(line_args):\n",
+    "        export_filepath = line_args[0]\n",
+    "        directory = os.path.dirname(export_filepath)\n",
+    "        os.makedirs(directory, exist_ok=True)\n",
+    "        with open(export_filepath, 'w') as file:\n",
+    "            file.write(cell)\n",
+    "        print(f\"exported to {export_filepath}\")\n",
+    "            \n",
+    "    processed_lines = []\n",
+    "    for line in cell.split('\\n'):\n",
+    "        comment_match = re.search(r'#', line)\n",
+    "        if comment_match:\n",
+    "            line = line[comment_match.start():]\n",
+    "            processed_lines.append(line)\n",
+    "\n",
+    "    processed_cell = '\\n'.join(processed_lines)\n",
+    "    if len(line_args):\n",
+    "        processed_cell = '# %answer ' + line_args[0] + '\\n\\n' + processed_cell\n",
+    "    get_ipython().set_next_input(processed_cell, replace=True)"
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "1c88008a-4058-4128-b3ca-0c2a1925da0d",
+   "id": "f38ff439-b7c6-4f66-bcea-ca87a079d7cd",
    "metadata": {},
    "outputs": [],
    "source": []