{
"cells": [
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"%matplotlib inline"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\nDefining a Neural Network in PyTorch\n====================================\nDeep learning uses artificial neural networks (models), which are\ncomputing systems that are composed of many layers of interconnected\nunits. By passing data through these interconnected units, a neural\nnetwork is able to learn how to approximate the computations required to\ntransform inputs into outputs. In PyTorch, neural networks can be\nconstructed using the ``torch.nn`` package.\n\nIntroduction\n------------\nPyTorch provides the elegantly designed modules and classes, including\n``torch.nn``, to help you create and train neural networks. An\n``nn.Module`` contains layers, and a method ``forward(input)`` that\nreturns the ``output``.\n\nIn this recipe, we will use ``torch.nn`` to define a neural network\nintended for the `MNIST\ndataset `__.\n\nSetup\n-----\nBefore we begin, we need to install ``torch`` if it isn\u2019t already\navailable.\n\n::\n\n pip install torch\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Steps\n-----\n\n1. Import all necessary libraries for loading our data\n2. Define and initialize the neural network\n3. Specify how data will pass through your model\n4. [Optional] Pass data through your model to test\n\n1. Import necessary libraries for loading our data\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nFor this recipe, we will use ``torch`` and its subsidiaries ``torch.nn``\nand ``torch.nn.functional``.\n\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import torch\nimport torch.nn as nn\nimport torch.nn.functional as F"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"2. Define and intialize the neural network\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nOur network will recognize images. We will use a process built into\nPyTorch called convolution. Convolution adds each element of an image to\nits local neighbors, weighted by a kernel, or a small matrix, that\nhelps us extract certain features (like edge detection, sharpness,\nblurriness, etc.) from the input image.\n\nThere are two requirements for defining the ``Net`` class of your model.\nThe first is writing an ``__init__`` function that references\n``nn.Module``. This function is where you define the fully connected\nlayers in your neural network.\n\nUsing convolution, we will define our model to take 1 input image\nchannel, and output match our target of 10 labels representing numbers 0\nthrough 9. This algorithm is yours to create, we will follow a standard\nMNIST algorithm.\n\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"class Net(nn.Module):\n def __init__(self):\n super(Net, self).__init__()\n\n # First 2D convolutional layer, taking in 1 input channel (image),\n # outputting 32 convolutional features, with a square kernel size of 3\n self.conv1 = nn.Conv2d(1, 32, 3, 1)\n # Second 2D convolutional layer, taking in the 32 input layers,\n # outputting 64 convolutional features, with a square kernel size of 3\n self.conv2 = nn.Conv2d(32, 64, 3, 1)\n\n # Designed to ensure that adjacent pixels are either all 0s or all active\n # with an input probability\n self.dropout1 = nn.Dropout2d(0.25)\n self.dropout2 = nn.Dropout2d(0.5)\n\n # First fully connected layer\n self.fc1 = nn.Linear(9216, 128)\n # Second fully connected layer that outputs our 10 labels\n self.fc2 = nn.Linear(128, 10)\n\nmy_nn = Net()\nprint(my_nn)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We have finished defining our neural network, now we have to define how\nour data will pass through it.\n\n3. Specify how data will pass through your model\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nWhen you use PyTorch to build a model, you just have to define the\n``forward`` function, that will pass the data into the computation graph\n(i.e. our neural network). This will represent our feed-forward\nalgorithm.\n\nYou can use any of the Tensor operations in the ``forward`` function.\n\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"class Net(nn.Module):\n def __init__(self):\n super(Net, self).__init__()\n self.conv1 = nn.Conv2d(1, 32, 3, 1)\n self.conv2 = nn.Conv2d(32, 64, 3, 1)\n self.dropout1 = nn.Dropout2d(0.25)\n self.dropout2 = nn.Dropout2d(0.5)\n self.fc1 = nn.Linear(9216, 128)\n self.fc2 = nn.Linear(128, 10)\n\n # x represents our data\n def forward(self, x):\n # Pass data through conv1\n x = self.conv1(x)\n # Use the rectified-linear activation function over x\n x = F.relu(x)\n\n x = self.conv2(x)\n x = F.relu(x)\n\n # Run max pooling over x\n x = F.max_pool2d(x, 2)\n # Pass data through dropout1\n x = self.dropout1(x)\n # Flatten x with start_dim=1\n x = torch.flatten(x, 1)\n # Pass data through fc1\n x = self.fc1(x)\n x = F.relu(x)\n x = self.dropout2(x)\n x = self.fc2(x)\n\n # Apply softmax to x \n output = F.log_softmax(x, dim=1)\n return output"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"4. [Optional] Pass data through your model to test\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nTo ensure we receive our desired output, let\u2019s test our model by passing\nsome random data through it.\n\n\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Equates to one random 28x28 image\nrandom_data = torch.rand((1, 1, 28, 28))\n\nmy_nn = Net()\nresult = my_nn(random_data)\nprint (result)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Each number in this resulting tensor equates to the prediction of the\nlabel the random tensor is associated to.\n\nCongratulations! You have successfully defined a neural network in\nPyTorch.\n\nLearn More\n----------\n\nTake a look at these other recipes to continue your learning:\n\n- `What is a state_dict in PyTorch `__\n- `Saving and loading models for inference in PyTorch `__\n\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.13"
}
},
"nbformat": 4,
"nbformat_minor": 0
}