fix AlexNet

Author: AnirudhDagar

Ch09_Modern_Convolutional_Networks/AlexNet.ipynb

@@ -641,7 +641,7 @@
    ],
    "source": [
     "from IPython.display import SVG\n",
-    "SVG(filename=\"./img/alexnet-all.svg\")"
+    "SVG(filename=\"../img/alexnet-all.svg\")"
    ]
   },
   {
@@ -698,7 +698,6 @@
     "import d2l\n",
     "import torch\n",
     "import torch.nn as nn\n",
-    "import os\n",
     "import torch.optim as optim"
    ]
   },
@@ -712,7 +711,7 @@
     "    def forward(self, input):\n",
     "        return input.view(input.size(0), -1)\n",
     "\n",
-    "net=nn.Sequential(\n",
+    "net = nn.Sequential(\n",
     "            nn.Conv2d(1, 96, kernel_size=11, stride=4, padding=1),\n",
     "            nn.ReLU(inplace=True),\n",
     "            nn.MaxPool2d(kernel_size=3, stride=2),\n",
@@ -779,7 +778,7 @@
     }
    ],
    "source": [
-    "X=torch.randn(size=(1,1,224,224))\n",
+    "X = torch.randn(size=(1,1,224,224))\n",
     "\n",
     "for layer in net:\n",
     "    X=layer(X)\n",
@@ -816,31 +815,34 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "batch_size=128\n",
-    "train_iter,test_iter=d2l.load_data_fashion_mnist(batch_size,resize=224)"
+    "batch_size = 128\n",
+    "train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, resize=224)"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 7,
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "training on cpu\n",
-      "epoch 1, loss 0.0125, train acc 0.390, test acc 0.667, time 7254.1 sec\n",
-      "epoch 2, loss 0.0057, train acc 0.726, test acc 0.777, time 7257.3 sec\n",
-      "epoch 3, loss 0.0046, train acc 0.782, test acc 0.805, time 34438.2 sec\n"
+      "training on cuda:0\n",
+      "epoch 1, loss 0.0127, train acc 0.393, test acc 0.712, time 31.3 sec\n",
+      "epoch 2, loss 0.0056, train acc 0.733, test acc 0.776, time 30.8 sec\n",
+      "epoch 3, loss 0.0045, train acc 0.782, test acc 0.806, time 32.1 sec\n",
+      "epoch 4, loss 0.0040, train acc 0.809, test acc 0.831, time 32.0 sec\n",
+      "epoch 5, loss 0.0037, train acc 0.826, test acc 0.844, time 32.2 sec\n"
      ]
     }
    ],
    "source": [
-    "lr,num_epochs,device=0.01,5,d2l.try_gpu()\n",
+    "lr, num_epochs, device = 0.01, 5, d2l.try_gpu()\n",
     "def init_weights(m):\n",
     "    if type(m) == nn.Linear or type(m) == nn.Conv2d:\n",
     "        torch.nn.init.xavier_uniform_(m.weight)\n",
+    "\n",
     "net.apply(init_weights)\n",
     "optimizer = optim.Adam(net.parameters(), lr=0.001, weight_decay=0.0005)\n",
     "criterion = nn.CrossEntropyLoss()\n",
@@ -851,45 +853,29 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Summary\n",
-    "• AlexNet has a similar structure to that of LeNet, but uses more convolutional layers and a larger\n",
-    "parameter space to fit the large-scale data set ImageNet.\n",
-    "\n",
-    "• Today AlexNet has been surpassed by much more effective architectures but it is a key step from\n",
-    "shallow to deep networks that are used nowadays.\n",
+    "## Summary\n",
     "\n",
-    "• Although it seems that there are only a few more lines in AlexNet’s implementation than in LeNet, it\n",
-    "took the academic community many years to embrace this conceptual change and take advantage of\n",
-    "its excellent experimental results. This was also due to the lack of efficient computational tools.\n",
-    "\n",
-    "• Dropout, ReLU and preprocessing were the other key steps in achieving excellent performance in\n",
-    "computer vision tasks"
+    "* AlexNet has a similar structure to that of LeNet, but uses more convolutional layers and a larger parameter space to fit the large-scale data set ImageNet.\n",
+    "* Today AlexNet has been surpassed by much more effective architectures but it is a key step from shallow to deep networks that are used nowadays.\n",
+    "* Although it seems that there are only a few more lines in AlexNet's implementation than in LeNet, it took the academic community many years to embrace this conceptual change and take advantage of its excellent experimental results. This was also due to the lack of efficient computational tools.\n",
+    "* Dropout, ReLU and preprocessing were the other key steps in achieving excellent performance in computer vision tasks."
    ]
   },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "### Exercises\n",
-    "1. Try increasing the number of epochs. Compared with LeNet, how are the results different? Why?\n",
+    "## Exercises\n",
     "\n",
+    "1. Try increasing the number of epochs. Compared with LeNet, how are the results different? Why?\n",
     "2. AlexNet may be too complex for the Fashion-MNIST data set.\n",
-    "\n",
-    "• Try to simplify the model to make the training faster, while ensuring that the accuracy does not\n",
-    "drop significantly.\n",
-    "\n",
-    "• Can you design a better model that works directly on 28 x 28 images.\n",
-    "\n",
+    "    * Try to simplify the model to make the training faster, while ensuring that the accuracy does not drop significantly.\n",
+    "    * Can you design a better model that works directly on $28 \\times 28$ images.\n",
     "3. Modify the batch size, and observe the changes in accuracy and GPU memory.\n",
-    "\n",
     "4. Rooflines\n",
-    "\n",
-    "• What is the dominant part for the memory footprint of AlexNet?\n",
-    "\n",
-    "• What is the dominant part for computation in AlexNet?\n",
-    "\n",
-    "• How about memory bandwidth when computing the results?\n",
-    "\n",
+    "    * What is the dominant part for the memory footprint of AlexNet?\n",
+    "    * What is the dominant part for computation in AlexNet?\n",
+    "    * How about memory bandwidth when computing the results?\n",
     "5. Apply dropout and ReLU to LeNet5. Does it improve? How about preprocessing?"
    ]
   }
@@ -910,7 +896,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.6.5"
+   "version": "3.6.8"
   }
  },
  "nbformat": 4,