Apply torch.cuda.amp.GradScaler to torch.amp.GradScaler replacements in README, docs, and notebooks

Co-authored-by: vfdev-5 <[email protected]>

Author: Copilot

README.md

@@ -397,7 +397,7 @@ Few pointers to get you started:
 - [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pytorch/ignite/blob/master/examples/notebooks/FastaiLRFinder_MNIST.ipynb) [Basic example of LR finder on
   MNIST](https://github.com/pytorch/ignite/blob/master/examples/notebooks/FastaiLRFinder_MNIST.ipynb)
 - [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pytorch/ignite/blob/master/examples/notebooks/Cifar100_bench_amp.ipynb) [Benchmark mixed precision training on Cifar100:
-  torch.cuda.amp vs nvidia/apex](https://github.com/pytorch/ignite/blob/master/examples/notebooks/Cifar100_bench_amp.ipynb)
+  torch.amp vs nvidia/apex](https://github.com/pytorch/ignite/blob/master/examples/notebooks/Cifar100_bench_amp.ipynb)
 - [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/pytorch/ignite/blob/master/examples/notebooks/MNIST_on_TPU.ipynb) [MNIST training on a single
   TPU](https://github.com/pytorch/ignite/blob/master/examples/notebooks/MNIST_on_TPU.ipynb)
 - [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1E9zJrptnLJ_PKhmaP5Vhb6DTVRvyrKHx) [CIFAR10 Training on multiple TPUs](https://github.com/pytorch/ignite/tree/master/examples/cifar10)

docs/source/conf.py

@@ -354,7 +354,7 @@ def run(self):
     ("py:class", "torch.optim.optimizer.Optimizer"),
     ("py:class", "torch.utils.data.dataset.Dataset"),
     ("py:class", "torch.utils.data.sampler.BatchSampler"),
-    ("py:class", "torch.cuda.amp.grad_scaler.GradScaler"),
+    ("py:class", "torch.amp.grad_scaler.GradScaler"),
     ("py:class", "torch.optim.lr_scheduler._LRScheduler"),
     ("py:class", "torch.optim.lr_scheduler.LRScheduler"),
     ("py:class", "torch.utils.data.dataloader.DataLoader"),

examples/notebooks/CycleGAN_with_torch_cuda_amp.ipynb

@@ -875,10 +875,10 @@
     "As suggested, we divide the objective by 2 while optimizing D, which slows down the rate at which D learns, relative to the rate of G. \n",
     "\n",
     "According to the paper:\n",
-    "- generator A is trained  minimize $\\text{mean}_{x \\in A}[(D_B(G(x)) − 1)^2]$ and cycle loss $\\text{mean}_{x \\in A}\\left[ |F(G(x)) - x|_1 \\right]$\n",
-    "- generator B is trained  minimize $\\text{mean}_{y \\in B}[(D_A(F(y)) − 1)^2]$ and cycle loss $\\text{mean}_{y \\in B}\\left[ |G(F(y)) - y|_1 \\right]$\n",
-    "- discriminators A is trained to minimize $\\text{mean}_{x \\in A}[(D_A(x) − 1)^2] + \\text{mean}_{y \\in B}[D_A(F(y))^2]$.\n",
-    "- discriminator B is trained to minimize $\\text{mean}_{y \\in B}[(D_B(y) − 1)^2] + \\text{mean}_{x \\in A}[D_B(G(x))^2]$."
+    "- generator A is trained  minimize $\\text{mean}_{x \\in A}[(D_B(G(x)) \u2212 1)^2]$ and cycle loss $\\text{mean}_{x \\in A}\\left[ |F(G(x)) - x|_1 \\right]$\n",
+    "- generator B is trained  minimize $\\text{mean}_{y \\in B}[(D_A(F(y)) \u2212 1)^2]$ and cycle loss $\\text{mean}_{y \\in B}\\left[ |G(F(y)) - y|_1 \\right]$\n",
+    "- discriminators A is trained to minimize $\\text{mean}_{x \\in A}[(D_A(x) \u2212 1)^2] + \\text{mean}_{y \\in B}[D_A(F(y))^2]$.\n",
+    "- discriminator B is trained to minimize $\\text{mean}_{y \\in B}[(D_B(y) \u2212 1)^2] + \\text{mean}_{x \\in A}[D_B(G(x))^2]$."
    ]
   },
   {
@@ -887,7 +887,7 @@
     "id": "JE8dLeEfIl_Z"
    },
    "source": [
-    "We will use [`torch.amp.autocast`](https://pytorch.org/docs/master/amp.html#torch.amp.autocast) and [`torch.cuda.amp.GradScaler`](https://pytorch.org/docs/master/amp.html#torch.cuda.amp.GradScaler) to perform automatic mixed precision training. Our code follows a [typical mixed precision training example](https://pytorch.org/docs/master/notes/amp_examples.html#typical-mixed-precision-training)."
+    "We will use [`torch.amp.autocast`](https://pytorch.org/docs/master/amp.html#torch.amp.autocast) and [`torch.amp.GradScaler`](https://pytorch.org/docs/master/amp.html#torch.amp.GradScaler) to perform automatic mixed precision training. Our code follows a [typical mixed precision training example](https://pytorch.org/docs/master/notes/amp_examples.html#typical-mixed-precision-training)."
    ]
   },
   {
@@ -896,8 +896,7 @@
     "id": "vrJls4p-FRcA"
    },
    "source": [
-    "from torch.cuda.amp import GradScaler\n",
-    "from torch.amp import autocast\n",
+    "from torch.amp import autocast, GradScaler\n",
     "\n",
     "from ignite.utils import convert_tensor\n",
     "import torch.nn.functional as F\n",
@@ -924,7 +923,7 @@
     "\n",
     "\n",
     "def compute_loss_discriminator(decision_real, decision_fake):\n",
-    "    # loss = mean (D_b(y) − 1)^2 + mean D_b(G(x))^2    \n",
+    "    # loss = mean (D_b(y) \u2212 1)^2 + mean D_b(G(x))^2    \n",
     "    loss = F.mse_loss(decision_fake, torch.zeros_like(decision_fake))\n",
     "    loss += F.mse_loss(decision_real, torch.ones_like(decision_real))\n",
     "    return loss\n",
@@ -954,10 +953,10 @@
     "        decision_fake_b = discriminator_B(fake_b)\n",
     "\n",
     "        # Compute loss for generators and update generators\n",
-    "        # loss_a2b = GAN loss: mean (D_b(G(x)) − 1)^2 + Forward cycle loss: || F(G(x)) - x ||_1    \n",
+    "        # loss_a2b = GAN loss: mean (D_b(G(x)) \u2212 1)^2 + Forward cycle loss: || F(G(x)) - x ||_1    \n",
     "        loss_a2b = compute_loss_generator(decision_fake_b, real_a, rec_a, lambda_value)    \n",
     "\n",
-    "        # loss_b2a = GAN loss: mean (D_a(F(x)) − 1)^2 + Backward cycle loss: || G(F(y)) - y ||_1\n",
+    "        # loss_b2a = GAN loss: mean (D_a(F(x)) \u2212 1)^2 + Backward cycle loss: || G(F(y)) - y ||_1\n",
     "        loss_b2a = compute_loss_generator(decision_fake_a, real_b, rec_b, lambda_value)\n",
     "\n",
     "        # total generators loss:\n",
@@ -977,10 +976,10 @@
     "        decision_real_a, decision_fake_a = discriminator_forward_pass(discriminator_A, real_a, fake_a.detach(), fake_a_buffer)    \n",
     "        decision_real_b, decision_fake_b = discriminator_forward_pass(discriminator_B, real_b, fake_b.detach(), fake_b_buffer)    \n",
     "        # Compute loss for discriminators and update discriminators\n",
-    "        # loss_a = mean (D_a(y) − 1)^2 + mean D_a(F(x))^2\n",
+    "        # loss_a = mean (D_a(y) \u2212 1)^2 + mean D_a(F(x))^2\n",
     "        loss_a = compute_loss_discriminator(decision_real_a, decision_fake_a)\n",
     "\n",
-    "        # loss_b = mean (D_b(y) − 1)^2 + mean D_b(G(x))^2\n",
+    "        # loss_b = mean (D_b(y) \u2212 1)^2 + mean D_b(G(x))^2\n",
     "        loss_b = compute_loss_discriminator(decision_real_b, decision_fake_b)\n",
     "      \n",
     "        # total discriminators loss:\n",
@@ -1578,4 +1577,4 @@
    "outputs": []
   }
  ]
-}
+}