torch-use-gpu.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "46cbaad3-86be-4024-8486-ba1ae99c885b",
   "metadata": {},
   "source": [
    "# Using GPU resources with PyTorch\n",
    "\n",
    "This notebook walks you through how to use a GPU, using the PyTorch machine learning framework. You can run each cell one by one as you work through the content, or run all cells and read through all output at the same time.\n",
    "\n",
    "In this notebook you will learn how to:\n",
    "\n",
    "1. Import Torch libraries\n",
    "1. List available GPUs.\n",
    "1. Check that GPUs are enabled.\n",
    "1. Assign a GPU device and retrieve the device name.\n",
    "1. Load vectors, matrices, and data onto a GPU.\n",
    "1. Load a neural network model onto a GPU.\n",
    "1. Train the neural network model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "133c19f2",
   "metadata": {},
   "source": [
    "First, use the following command to get some basic information about the GPU resources available to your notebook server:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "707a70d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "!nvidia-smi"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d6e404d",
   "metadata": {},
   "source": [
    "## Check that you can see your GPU\n",
    "\n",
    "Import the torch and torchvision libraries you need in order to work with PyTorch, and ensure that your GPU resources are visible in your notebook server."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8c1148a",
   "metadata": {},
   "source": [
    "The following commands import the torch and torchvision utilities, as well as some plotting and tqdm helpers:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b85e7327-fc5b-4efd-92e9-9502a329b7a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install torchvision==0.9.1\n",
    "!pip install tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "52215703-b42f-4986-b8aa-7b69b1fc2a10",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.nn.functional as F\n",
    "from torch.utils.data import TensorDataset\n",
    "import torch.optim as optim\n",
    "import torchvision\n",
    "from torchvision import datasets\n",
    "import torchvision.transforms as transforms\n",
    "import matplotlib.pyplot as plt\n",
    "from tqdm import tqdm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "886428f5-927e-4ee8-acd7-b7171cba60aa",
   "metadata": {},
   "source": [
    "Now that these libraries are imported, you can use the following commands to check whether a GPU is available, and how many GPUs this notebook server has access to:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b6ae7609-a3ac-4c42-a723-5942ba35cbef",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.cuda.is_available()  # Do we have a GPU? Should return True."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1bd42b00-318f-4a7f-991b-6aea8162469c",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.cuda.device_count()  # How many GPUs do we have access to?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6632626-9385-46b7-b2c9-37fe6fc12a24",
   "metadata": {},
   "source": [
    "Make sure that you can see at least 1 GPU available before continuing to the next section.\n",
    "\n",
    "If you see 0 GPUs available, click *File* -> *Hub Control Panel* to go back to the notebook server control panel.\n",
    "\n",
    "Stop your notebook server and start your notebook server again, making sure to use a GPU compatible notebook image and add at least 1 GPU. If you selected 1 or more GPUs and you still cannot see a GPU when you run the previous commands, contact your administrator.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b018d342-1674-4800-8bd1-e8a18f2d632c",
   "metadata": {},
   "source": [
    "## Assign your GPU as a device\n",
    "\n",
    "Assign the first GPU device to the `device` variable, and get the device name for your GPU."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1dcee4ed-1a94-4ca6-9b9a-d708a0e40d52",
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "print(device)  # Check which device we got"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6686219-275c-4ec0-909b-07d9934be4cb",
   "metadata": {},
   "source": [
    "If the output of the `print` command is `cuda:0`, continue to the next step.\n",
    "\n",
    "If not, your environment is not set up correctly. Make sure that you use a GPU compatible notebook image and add at least 1 GPU when you start your notebook server."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b25685a6",
   "metadata": {},
   "source": [
    "Next, check the device name of the GPU:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86a4537e-0719-4335-b051-321e03d13268",
   "metadata": {},
   "outputs": [],
   "source": [
    "torch.cuda.get_device_name(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d851d797-c6e3-4b70-92a4-30115307ac1d",
   "metadata": {},
   "source": [
    "## Loading vectors, matrices, and other data onto the GPU\n",
    "\n",
    "Run the following commands to create a data structure and load it onto the GPU device, or create your data structure on the GPU device directly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b73245d8-27e9-48fb-939f-ce952f4fe2cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train = torch.IntTensor([0, 30, 50, 75, 70])  # Initialize a Tensor of Integers with no device specified\n",
    "print(X_train.is_cuda, \",\", X_train.device)  # Check which device Tensor is created on"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dde4baa6-84a6-4c32-ab20-5be6d529a22a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Move the Tensor to the device we want to use\n",
    "X_train = X_train.cuda()\n",
    "# Alternative method: specify the device using the variable\n",
    "# X_train = X_train.to(device)\n",
    "# Confirm that the Tensor is on the GPU now\n",
    "print(X_train.is_cuda, \",\", X_train.device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "947393c8-a9ac-42ff-a790-7a3b21d87763",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Alternative method: Initialize the Tensor directly on a specific device.\n",
    "X_test = torch.cuda.IntTensor([30, 40, 50], device=device)\n",
    "print(X_test.is_cuda, \",\", X_test.device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd36b890-b52d-4968-838d-f749aa4d456f",
   "metadata": {},
   "source": [
    "## Loading a Neural Network Model onto the GPU\n",
    "\n",
    "Run the following commands to create or load a model onto your GPU device."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd51a2fb",
   "metadata": {},
   "source": [
    "The following code is a basic, fully connected neural network built in Torch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23ac26d2-8019-40ad-a659-985977c7fdfa",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Here is a basic fully connected neural network built in Torch.\n",
    "# If we want to load it / train it on our GPU, we must first put it on the GPU\n",
    "# Otherwise it will remain on CPU by default.\n",
    "\n",
    "batch_size = 100\n",
    "\n",
    "\n",
    "class SimpleNet(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(SimpleNet, self).__init__()\n",
    "        self.fc1 = nn.Linear(784, 784)\n",
    "        self.fc2 = nn.Linear(784, 10)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = x.view(batch_size, -1)\n",
    "        x = self.fc1(x)\n",
    "        x = F.relu(x)\n",
    "        x = self.fc2(x)\n",
    "        output = F.softmax(x, dim=1)\n",
    "        return output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffcf46a9",
   "metadata": {},
   "source": [
    "Running the above code starts the neural network running on the CPU by default.\n",
    "\n",
    "The following code moves the model onto the GPU, so that it can be trained with a large data set more quickly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0545d521-be18-4623-9cbc-f6836e2126c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = SimpleNet().to(device)  # Load the neural network model onto the GPU"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "843ada70-fb3d-4439-a643-ad7a1ac6da27",
   "metadata": {},
   "source": [
    "## Training the Neural Network Model\n",
    "\n",
    "The examples in this section show you how to train your neural network model using the [FashionMNIST data set](https://github.com/zalandoresearch/fashion-mnist)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "634989ee",
   "metadata": {},
   "source": [
    "The following code uses the PyTorch data loader to download the data set, and set up training and testing data sets to work with."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57e786b0-ec30-4c9d-995b-8000912e6189",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "    Data loading, train and test set via the PyTorch dataloader.\n",
    "\"\"\"\n",
    "# Transform our data into Tensors to normalize the data\n",
    "train_transform=transforms.Compose([\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Normalize((0.1307,), (0.3081,))\n",
    "        ])\n",
    "\n",
    "test_transform=transforms.Compose([\n",
    "        transforms.ToTensor(),\n",
    "        transforms.Normalize((0.1307,), (0.3081,)),\n",
    "        ])\n",
    "\n",
    "# Set up a training data set\n",
    "trainset = datasets.FashionMNIST('./data', train=True, download=True,\n",
    "                   transform=train_transform)\n",
    "train_loader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,\n",
    "                                          shuffle=False, num_workers=2)\n",
    "\n",
    "# Set up a test data set\n",
    "testset = datasets.FashionMNIST('./data', train=False,\n",
    "                   transform=test_transform)\n",
    "test_loader = torch.utils.data.DataLoader(testset, batch_size=batch_size,\n",
    "                                         shuffle=False, num_workers=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f77af8df",
   "metadata": {},
   "source": [
    "Place the labels from the FashionMNIST data set into dictionary format, and plot a selection of the data to verify:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2fed4ee2-5900-452a-875d-973ae27206db",
   "metadata": {},
   "outputs": [],
   "source": [
    "# A dictionary to map our class numbers to their items.\n",
    "labels_map = {\n",
    "    0: \"T-Shirt\",\n",
    "    1: \"Trouser\",\n",
    "    2: \"Pullover\",\n",
    "    3: \"Dress\",\n",
    "    4: \"Coat\",\n",
    "    5: \"Sandal\",\n",
    "    6: \"Shirt\",\n",
    "    7: \"Sneaker\",\n",
    "    8: \"Bag\",\n",
    "    9: \"Ankle Boot\",\n",
    "}\n",
    "\n",
    "# Plotting 9 random different items from the training data set, trainset.\n",
    "figure = plt.figure(figsize=(8, 8))\n",
    "for i in range(1, 3 * 3 + 1):\n",
    "    sample_idx = torch.randint(len(trainset), size=(1,)).item()\n",
    "    img, label = trainset[sample_idx]\n",
    "    figure.add_subplot(3, 3, i)\n",
    "    plt.title(labels_map[label])\n",
    "    plt.axis(\"off\")\n",
    "    plt.imshow(img.view(28,28), cmap=\"gray\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88822b5b-565c-4f9b-9dbb-9647a2fc6c01",
   "metadata": {},
   "source": [
    "Run the following code to train the model and see how well it can classify fashion items into the 10 classes in the dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1024390a-3e87-42a6-b406-b1067402b568",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(model, device, train_loader, optimizer, epoch):\n",
    "    \"\"\"Model training function\"\"\"\n",
    "    model.train()\n",
    "    print(device)\n",
    "    for batch_idx, (data, target) in tqdm(enumerate(train_loader)):\n",
    "        data, target = data.to(device), target.to(device)\n",
    "        optimizer.zero_grad()\n",
    "        output = model(data)\n",
    "        loss = F.nll_loss(output, target)\n",
    "        loss.backward()\n",
    "        optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "049398c5-00d7-4706-9206-aa291cb0ad54",
   "metadata": {},
   "outputs": [],
   "source": [
    "def test(model, device, test_loader):\n",
    "    model.eval()\n",
    "    test_loss = 0\n",
    "    correct = 0\n",
    "    # Use the no_grad method to increase computation speed\n",
    "    # since computing the gradient is not necessary in this step.\n",
    "    with torch.no_grad():\n",
    "        for data, target in test_loader:\n",
    "            data, target = data.to(device), target.to(device)\n",
    "            output = model(data)\n",
    "            test_loss += F.nll_loss(output, target, reduction='sum').item()  # sum up batch loss\n",
    "            pred = output.argmax(dim=1, keepdim=True)  # get the index of the max log-probability\n",
    "            correct += pred.eq(target.view_as(pred)).sum().item()\n",
    "\n",
    "    test_loss /= len(test_loader.dataset)\n",
    "\n",
    "    print('\\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
    "        test_loss, correct, len(test_loader.dataset),\n",
    "        100. * correct / len(test_loader.dataset)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9dbf0b2f-1e88-4e99-aa25-b56eae471a04",
   "metadata": {},
   "outputs": [],
   "source": [
    "# number of  training 'epochs'\n",
    "EPOCHS = 5\n",
    "# our optimization strategy used in training.\n",
    "optimizer = optim.Adadelta(model.parameters(), lr=0.01)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33e4b14b-4a77-4af1-bc88-4b659c908e3f",
   "metadata": {},
   "outputs": [],
   "source": [
    "for epoch in range(1, EPOCHS + 1):\n",
    "        print( f\"EPOCH: {epoch}\")\n",
    "        train(model, device, train_loader, optimizer, epoch)\n",
    "        test(model, device, test_loader)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b82007ad",
   "metadata": {},
   "source": [
    "The accuracy of the model increases over a number of epochs, from about 63% in the first epoch to about 72% in the fifth. (The exact numbers here might vary, depending on random weight initialization on your notebook server.)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73bcd335-5960-40dc-972e-7d76dc3e2674",
   "metadata": {},
   "source": [
    "## Saving the model state\n",
    "\n",
    "Now that the model is trained, save it so that it can be used in the next notebook, *Loading and Running a PyTorch Model*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50684945-fee9-4a13-89c2-511b27bc817c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Saving the model's weights!\n",
    "torch.save(model.state_dict(), \"mnist_fashion_SimpleNet.pt\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "e7370f93d1d0cde622a1f8e1c04877d8463912d04d973331ad4851f04de6915a"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}