00_tensorflow_fundamentals.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "T4",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/fyemane/TensorFlow-ZTM-/blob/main/00_tensorflow_fundamentals.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# TensorFlow Fundamentals\n",
        "\n"
      ],
      "metadata": {
        "id": "M8LW7iKxLfoT"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Creating your first tensors with Tensorflow and `tf.constant()`\n",
        "\n",
        "In this notebook, we're going to cover some of the fundamental concepts of tensors using TensorFlow.\n",
        "\n",
        "More specifically, we're going to cover:\n",
        "* Introduction to tensors\n",
        "* Getting information from tensors\n",
        "* Manipulating tensors\n",
        "* Tensors and NumPy\n",
        "* Using @tf.function (a way to speed up your regular Python functions)\n",
        "* Using GPUs with TensorFlow (or TPUs)\n",
        "* Exercises to try for yourself!!"
      ],
      "metadata": {
        "id": "hjjKSNaDK36Z"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Introduction to tensors"
      ],
      "metadata": {
        "id": "0Jo2AVWmLv1L"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Import TensorFlow\n",
        "import tensorflow as tf\n",
        "print(tf.__version__)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XBaW59ZGMNRv",
        "outputId": "5fcbfe64-7b90-47b7-e45c-3d10683e8e15"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2.15.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Create tensors with tf.constant()\n",
        "# https://www.tensorflow.org/api_docs/python/tf/constant\n",
        "scalar = tf.constant(7)\n",
        "scalar"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KXiZLh9VMb-G",
        "outputId": "eadb0276-b6b4-417a-bcea-143d0149337a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int32, numpy=7>"
            ]
          },
          "metadata": {},
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Check the number of dimensions of a tensor (ndim stands for number of dimensions)\n",
        "scalar.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KzTIAWWHMvtY",
        "outputId": "36ca7a34-a961-4528-d4e8-952c9f349133"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a vector by inputting list\n",
        "vector = tf.constant([10, 10])\n",
        "vector"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rjcc_FuwNOzF",
        "outputId": "e3ba7917-e58c-41cf-a0df-a1b0a7640b1c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([10, 10], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Check the dimension of our vector\n",
        "vector.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "G0Pix3bXNjAn",
        "outputId": "4d11121e-0f32-49a3-9d2d-b9e7dcad72a2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a matrix (has more than 1 dimension)\n",
        "matrix = tf.constant([[10, 7],\n",
        "                      [7, 10]])\n",
        "matrix"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oC46ocKaORfn",
        "outputId": "4743590c-ce60-4b79-bd01-e54c4070e773"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 7, 10]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Check the dimension of our matrix\n",
        "matrix.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8Olz2jSYOeM8",
        "outputId": "d766dbaf-6777-4678-ae72-d8ec117513e6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Create another matrix\n",
        "another_matrix = tf.constant([[10., 7.],\n",
        "                              [3., 2.],\n",
        "                              [8., 9.]], dtype=tf.float16) # Specify the data type with dtype parameter\n",
        "# tensor with tf.constant() has int32 dtype by default, aka int32 precision\n",
        "# higher the number of precision, more digits stored on computer's memory\n",
        "another_matrix"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WgtYcURROhP4",
        "outputId": "9ed1ff11-9777-40e4-9879-0332b52250e9"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=float16, numpy=\n",
              "array([[10.,  7.],\n",
              "       [ 3.,  2.],\n",
              "       [ 8.,  9.]], dtype=float16)>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# What's the number of dimensions of another_matrix?\n",
        "another_matrix.ndim # should be 2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "byqtJMcRPYe3",
        "outputId": "61c42c3c-fd63-4144-dc61-27ae01cf1247"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "2"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Let's create a tensor\n",
        "tensor = tf.constant([[[1, 2, 3],\n",
        "                       [4, 5, 6]],\n",
        "                      [[7, 8, 9],\n",
        "                       [10, 11, 12]],\n",
        "                      [[13, 14, 15],\n",
        "                       [16, 17, 18]]])\n",
        "tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OSN9qAR4PlUT",
        "outputId": "1f607958-d81f-4007-9193-a584577f1ca5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2, 3), dtype=int32, numpy=\n",
              "array([[[ 1,  2,  3],\n",
              "        [ 4,  5,  6]],\n",
              "\n",
              "       [[ 7,  8,  9],\n",
              "        [10, 11, 12]],\n",
              "\n",
              "       [[13, 14, 15],\n",
              "        [16, 17, 18]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tensor.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JRkpEDDmQELm",
        "outputId": "30680dd3-f044-4785-8b69-9ba0012ac725"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "What we've created so far:\n",
        "* Scalar: a single number\n",
        "* Vector: a number with direction (e.g., wind speed and direction)\n",
        "* matrix: a 2-dimensional array of numbers\n",
        "* Tensor: an n-dimensional array of numbers (where n can be any number, a 0-dimensional tensor is a scalar, a 1-dimensional tensor is a vector)"
      ],
      "metadata": {
        "id": "4rgzLmm4Qcb6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Creating tensors with TensorFlow and `tf.Variable()`\n",
        "\n",
        "https://www.tensorflow.org/api_docs/python/tf/Variable"
      ],
      "metadata": {
        "id": "7CLvAScrSmGW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create the same tensor with tf.Variable as above\n",
        "changeable_tensor = tf.Variable([10, 7])\n",
        "unchangeable_tensor = tf.constant([10, 7])\n",
        "changeable_tensor, unchangeable_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "597M9ABmS4xF",
        "outputId": "2f7cae4c-1954-47bc-e6a4-ec3262c3072f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([10,  7], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2,), dtype=int32, numpy=array([10,  7], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# # Let's try to change one of the elements in our changeable tensor\n",
        "# changeable_tensor[0] = 7 # gives error\n",
        "# changeable_tensor"
      ],
      "metadata": {
        "id": "WMEBxgrNUACS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# How about we try .assign()\n",
        "changeable_tensor[0].assign(7)\n",
        "changeable_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ttKW9L5sUPxa",
        "outputId": "e64529cf-04d0-444e-afe7-7928a0644cdc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Variable 'Variable:0' shape=(2,) dtype=int32, numpy=array([7, 7], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# # Let's try changing our unchangeable tensor\n",
        "# unchangeable_tensor[0].assign(7) # gives error, has no attribute .assign()\n",
        "# unchangeable_tensor"
      ],
      "metadata": {
        "id": "hcsvdkuuUgVi"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Note:** Rarely in practice will you need to decide whether to use `tf.constant` or `tf.Variable` to create tensors, as TensorFlow does this for you. However, if in doubt, use `tf.constant` and change it later if needed."
      ],
      "metadata": {
        "id": "cFq4ztDAVNsi"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Creating random tensors with TensorFlow\n",
        "\n",
        "Random tensors are tensors of some arbitrary size which contains random numbers.\n",
        "\n",
        "NN will initialize itself with random weights (only at beginning). As it sees more data, it will update its representation outputs and tweak weights to better suit the data."
      ],
      "metadata": {
        "id": "yPayw6AgUrTz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create two random (but the same) tensors\n",
        "# https://www.tensorflow.org/api_docs/python/tf/random/uniform\n",
        "# https://www.tensorflow.org/api_docs/python/tf/random/normal\n",
        "random_1 = tf.random.Generator.from_seed(7) # set seed for reproducibility\n",
        "random_1 = random_1.normal(shape=(3, 2))\n",
        "random_2 = tf.random.Generator.from_seed(7)\n",
        "random_2 = random_2.normal(shape=(3, 2))\n",
        "\n",
        "# Are they equal?\n",
        "random_1, random_2, random_1 == random_2"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9dja7WA_VGLy",
        "outputId": "a1e11248-e9d6-4fae-a260-3e7a3d5c30a0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-1.3240396 ,  0.28785667],\n",
              "        [-0.8757901 , -0.08857018],\n",
              "        [ 0.69211644,  0.84215707]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=float32, numpy=\n",
              " array([[-1.3240396 ,  0.28785667],\n",
              "        [-0.8757901 , -0.08857018],\n",
              "        [ 0.69211644,  0.84215707]], dtype=float32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=bool, numpy=\n",
              " array([[ True,  True],\n",
              "        [ True,  True],\n",
              "        [ True,  True]])>)"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Shuffling the order of elements in tensors\n",
        "\n",
        "NN can be affected by the order of tensors it has to find patterns in."
      ],
      "metadata": {
        "id": "c9hNnMlshMgU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Shuffle a tensor (valuable for when you want to shuffle your data so the inherent order doesn't affect learning)\n",
        "not_shuffled = tf.constant([[10, 7],\n",
        "                            [3, 4],\n",
        "                            [2, 5]])\n",
        "# https://www.tensorflow.org/api_docs/python/tf/random/shuffle\n",
        "# Shuffle our non-shuffled tensor in its first dimension\n",
        "tf.random.shuffle(not_shuffled)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_ErgBX0Qlgbc",
        "outputId": "d4c8e0f3-b8f6-490f-9ff2-e5c4fea306db"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[ 3,  4],\n",
              "       [ 2,  5],\n",
              "       [10,  7]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Shuffle our non-shuffled tensor in its first dimension\n",
        "# https://www.tensorflow.org/api_docs/python/tf/random/set_seed\n",
        "tf.random.set_seed(42)\n",
        "tf.random.shuffle(not_shuffled, seed=42)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U2whXuApmQRu",
        "outputId": "201b6035-8e61-4ead-d53f-a22e488d1545"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4],\n",
              "       [ 2,  5]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Exercise:** Read through TensorFlow documentation on random seed generation: https://www.tensorflow.org/api_docs/python/tf/random/set_seed and practice writing 5 random tensors and shuffle them.\n",
        "\n",
        "It looks like if we want our shuffled tensors to be in the same order, we've got to use the global level random seed as well as the operation level random seed."
      ],
      "metadata": {
        "id": "k0cSFr6PqTx0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tf.random.set_seed(42) # global level random seed\n",
        "tf.random.shuffle(not_shuffled, seed=42) # operation level random seed"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cnv9arE7xbu8",
        "outputId": "12a3c3dd-b57d-40ee-b671-ed7e85a8c7b8"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4],\n",
              "       [ 2,  5]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "It's important to shuffle the labeled data so that the pattens the NN learns are tuned to different labels throughout the entire training cycle.\n",
        "\n",
        "As the NN initializes itself with random patterns, it will gen different numbers every time it is run. To make reproducible experiments, shuffle in similar order, initialize similar random pattern, and run the experiment."
      ],
      "metadata": {
        "id": "PXW7kkOK2DIj"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Create tensors with NumPy arrays"
      ],
      "metadata": {
        "id": "bT-eniFwxzdh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# https://www.tensorflow.org/api_docs/python/tf/ones\n",
        "tf.ones([10, 7])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TEbdtjYk2w1b",
        "outputId": "b64ed682-69c9-4c4e-bc7b-a3d66a174820"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(10, 7), dtype=float32, numpy=\n",
              "array([[1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.],\n",
              "       [1., 1., 1., 1., 1., 1., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# https://www.tensorflow.org/api_docs/python/tf/zeros\n",
        "tf.zeros(shape=(3, 4))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "prj5srSq2zAQ",
        "outputId": "22b78a78-3b57-40c3-d550-684f02111db1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 4), dtype=float32, numpy=\n",
              "array([[0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.],\n",
              "       [0., 0., 0., 0.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Turn NumPy arrays into tensors\n",
        "\n",
        "The main difference between NumPy arrays and TensorFlow tensors is that tensors can be run on a GPU (much faster for numerical computing)."
      ],
      "metadata": {
        "id": "irXM7YEf3JpV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# You can also turn NumPy arrays into tensors\n",
        "import numpy as np\n",
        "numpy_A = np.arange(1, 25, dtype=np.int32) # create a NumPy array between 1 and 25\n",
        "numpy_A\n",
        "\n",
        "# X = tf.constant(some_matrix) # capital for matrix\n",
        "# y = tf.constant(vector) # non-capital for vector"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5OYPNSGK2665",
        "outputId": "9e0be9dd-f17b-4b2e-bb64-b095abcca3b6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "       18, 19, 20, 21, 22, 23, 24], dtype=int32)"
            ]
          },
          "metadata": {},
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# to readjust tensor/array shape, new elements must add to same # of tensor elements\n",
        "A = tf.constant(numpy_A, shape=(3, 8))\n",
        "B = tf.constant(numpy_A)\n",
        "A, B"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LwkcpXTe3m8D",
        "outputId": "efc6bc9b-0db7-4829-adfd-65ea8dc13a28"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 8), dtype=int32, numpy=\n",
              " array([[ 1,  2,  3,  4,  5,  6,  7,  8],\n",
              "        [ 9, 10, 11, 12, 13, 14, 15, 16],\n",
              "        [17, 18, 19, 20, 21, 22, 23, 24]], dtype=int32)>,\n",
              " <tf.Tensor: shape=(24,), dtype=int32, numpy=\n",
              " array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,\n",
              "        18, 19, 20, 21, 22, 23, 24], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Getting information from your tensors (tensor attributes)\n",
        "\n",
        "When dealing with tensors you probably want to be aware of the following attributes:\n",
        "* Shape: The length (number of elements) of each of the dimensions of a tensor - `tensor.shape`\n",
        "* Rank: The number of tensor dimensions. A scalar has rank 0, a vector has rank 1, a matrix has rank 2, a tensor has rank n. - `tensor.ndim`\n",
        "* Axis or dimension: A particular dimension of a tensor. - `tensor[0]`, `tensor[:, 1]`...\n",
        "* Size: The total number of items in a tensor. - `tf.size(tensor)`"
      ],
      "metadata": {
        "id": "SnG32I9w3wsN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a rank 4 tensor (4 dimensions)\n",
        "rank_4_tensor = tf.zeros(shape=[2,3,4,5])\n",
        "rank_4_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZNFNckLebqkO",
        "outputId": "9c72f7ac-8879-42e9-d2f6-5e09890436db"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3, 4, 5), dtype=float32, numpy=\n",
              "array([[[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]],\n",
              "\n",
              "        [[0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.],\n",
              "         [0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "rank_4_tensor.shape, rank_4_tensor.ndim, tf.size(rank_4_tensor)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4-44SJxVcttr",
        "outputId": "b7d10a09-1ab7-40ee-9444-48e85ff4d10a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([2, 3, 4, 5]), 4, <tf.Tensor: shape=(), dtype=int32, numpy=120>)"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "2 * 3 * 4 * 5"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RtmSI806gSLd",
        "outputId": "aa04ab96-dc39-4f54-9b90-5bd3c99cf815"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "120"
            ]
          },
          "metadata": {},
          "execution_count": 27
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get various attributes of our tensor\n",
        "print(\"Datatype of every element: \", rank_4_tensor.dtype)\n",
        "print(\"Number of dimensions (rank): \", rank_4_tensor.ndim)\n",
        "print(\"Shape of tensor: \", rank_4_tensor.shape)\n",
        "print(\"Elements along the 0 axis: \", rank_4_tensor.shape[0])\n",
        "print(\"Elements along the last axis: \", rank_4_tensor.shape[-1]) # -1 can also be 3\n",
        "print(\"Total number of elements in our tensor: \", tf.size(rank_4_tensor))\n",
        "# convert output into NumPy integer\n",
        "\n",
        "print(\"Total number of elements in our tensor: \", tf.size(rank_4_tensor).numpy())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "YoFFWTrblXQX",
        "outputId": "cbfdf911-d6a3-49b3-f096-8a27fa0a73fe"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Datatype of every element:  <dtype: 'float32'>\n",
            "Number of dimensions (rank):  4\n",
            "Shape of tensor:  (2, 3, 4, 5)\n",
            "Elements along the 0 axis:  2\n",
            "Elements along the last axis:  5\n",
            "Total number of elements in our tensor:  tf.Tensor(120, shape=(), dtype=int32)\n",
            "Total number of elements in our tensor:  120\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Indexing and expanding tensors\n",
        "\n",
        "Tensors can be indexed just like Python lists."
      ],
      "metadata": {
        "id": "R8OekYXvl47K"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "some_list = [1, 2, 3, 4]\n",
        "some_list[:2]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1IfBwk_en2lr",
        "outputId": "8cf840bf-8548-445f-f685-6b909df3c52b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[1, 2]"
            ]
          },
          "metadata": {},
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the first 2 elements of each dimension\n",
        "rank_4_tensor[:2, :2, :2, :2]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BWXCIGLWmZmT",
        "outputId": "8787957b-883b-492b-d053-e31e01b324b3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 2, 2), dtype=float32, numpy=\n",
              "array([[[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]],\n",
              "\n",
              "\n",
              "       [[[0., 0.],\n",
              "         [0., 0.]],\n",
              "\n",
              "        [[0., 0.],\n",
              "         [0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the first element from each dimension from each index except for the final one\n",
        "rank_4_tensor[:1, :1,:1]\n"
      ],
      "metadata": {
        "id": "zDG51xGNntD9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0f8fc9c4-b11f-43db-d8a5-c9d56e1ea2c0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 5), dtype=float32, numpy=array([[[[0., 0., 0., 0., 0.]]]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a rank 2 tensor (2 dimensions)\n",
        "rank_2_tensor = tf.constant([[10, 7],\n",
        "                             [3, 4]])\n",
        "rank_2_tensor.shape, rank_2_tensor.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1_ylfBhCp878",
        "outputId": "1194aa7b-f47e-4a7b-bc4d-064f11d35641"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(TensorShape([2, 2]), 2)"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the last item of each row of our rank 2 tensor\n",
        "rank_2_tensor[:, -1]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6aFS9dJ9q71Z",
        "outputId": "8dab324c-b890-47d7-b056-a44fddbec35f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([7, 4], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Add in extra dimension to our rank 2 tensor\n",
        "# https://www.tensorflow.org/api_docs/python/tf/Tensor\n",
        "rank_3_tensor = rank_2_tensor[..., tf.newaxis]\n",
        "rank_3_tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GHzfnmHRrGw7",
        "outputId": "68bc87b5-f5f1-4c20-eb7f-ed2585e04562"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Alternative to tf.newaxis\n",
        "# https://www.tensorflow.org/api_docs/python/tf/expand_dims\n",
        "tf.expand_dims(rank_2_tensor, axis=-1) # \"-1\" mean expand the final axis"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IlchiL92reZX",
        "outputId": "69f7a09d-ee2a-47bc-ba3a-18dbfdc2bb4a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2, 1), dtype=int32, numpy=\n",
              "array([[[10],\n",
              "        [ 7]],\n",
              "\n",
              "       [[ 3],\n",
              "        [ 4]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Expand the 0-axis\n",
        "tf.expand_dims(rank_2_tensor, axis=0)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pSfftBTzsFp-",
        "outputId": "21374067-8741-4c97-da39-5710f639eb96"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 2, 2), dtype=int32, numpy=\n",
              "array([[[10,  7],\n",
              "        [ 3,  4]]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Manipulating tensors with basic operations\n",
        "\n",
        "**Basic operations**\n",
        "\n",
        "`+`, `-`, `*`, `/`"
      ],
      "metadata": {
        "id": "_r_EsoewsSie"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# You can add values to a tensor using the addition operator\n",
        "tensor = tf.constant([[10, 7], [3, 4]])\n",
        "tensor + 10"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "QlmJ0yjDsqtX",
        "outputId": "24ffc9c5-31b1-48e3-f9bd-8433236fa734"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[20, 17],\n",
              "       [13, 14]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Original tensor is unchanged\n",
        "tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qUWMbq_os64j",
        "outputId": "3e8d976d-b381-414e-8049-2d46ec125b5f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[10,  7],\n",
              "       [ 3,  4]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 38
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Multiplication also works\n",
        "tensor * 10"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aE5Vx-y-unLg",
        "outputId": "de62128c-d159-4c14-ded9-1a5fe5152ca7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[100,  70],\n",
              "       [ 30,  40]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Subtraction if you want\n",
        "tensor - 10"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1X5tCT6cuwRU",
        "outputId": "4e9de118-5a5a-436e-cc50-1590b760faa5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 0, -3],\n",
              "       [-7, -6]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 40
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# We can use the tensorflow built-in function too\n",
        "# tensorflow functions can run faster using a GPU/TPU\n",
        "# https://www.tensorflow.org/api_docs/python/tf/math/multiply\n",
        "tf.multiply(tensor, 10)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aOyWB41ruzb3",
        "outputId": "c4e2cc48-d98d-46fd-cf76-b90dc27dbef1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[100,  70],\n",
              "       [ 30,  40]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 41
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Matrix multiplication with tensors\n",
        "\n",
        "In machine learning, matrix multiplication is one of the most common tensor operations. http://matrixmultiplication.xyz/\n",
        "\n",
        "There are two rules our tensors (or matrices) need to fulfill if we're giong to matrix multiply them:\n",
        "1. The inner dimensions must match\n",
        "2. The resulting matrix has the shape of the outer dimensions\n",
        "\n",
        "**Resource:** Info and example of matrix multiplication: https://www.mathsisfun.com/algebra/matrix-multiplying.html\n",
        "\n"
      ],
      "metadata": {
        "id": "_PwTdupZvI9x"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Matrix multiplication in tensorflow\n",
        "# https://www.tensorflow.org/api_docs/python/tf/linalg/matmul\n",
        "print(tensor)\n",
        "tf.matmul(tensor, tensor)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Td3W0RCHvi7K",
        "outputId": "10d806a7-7247-4ad6-bb85-1e4891150f59"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "tf.Tensor(\n",
            "[[10  7]\n",
            " [ 3  4]], shape=(2, 2), dtype=int32)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[121,  98],\n",
              "       [ 42,  37]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# element-wise multiplication\n",
        "tensor * tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Tcb_JQ55xWm5",
        "outputId": "ff5da92d-b490-4c92-da36-c948b3bfc086"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[100,  49],\n",
              "       [  9,  16]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 43
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Matrix multiplication with Python operator \"@\"\n",
        "tensor @ tensor"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h7EwCT1jxN4J",
        "outputId": "e2a7b172-d416-4953-b303-1be8915f1446"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[121,  98],\n",
              "       [ 42,  37]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 44
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tensor.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zLmx9P8KxfF-",
        "outputId": "bdd0c5ec-1f75-44e9-fb0d-6673a4e3a46c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([2, 2])"
            ]
          },
          "metadata": {},
          "execution_count": 45
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a (3, 2) tensor\n",
        "X = tf.constant([[1, 2],\n",
        "                 [3, 4],\n",
        "                 [5, 6]])\n",
        "# Create another (3, 2) tensor\n",
        "Y = tf.constant([[7, 8],\n",
        "                 [9, 10],\n",
        "                 [11, 12]])\n",
        "X, Y"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ech64BS9xwpL",
        "outputId": "727087b7-3595-458c-adc5-0e25404a2456"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[ 7,  8],\n",
              "        [ 9, 10],\n",
              "        [11, 12]], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Try to matrix multiply tensors of same shape\n",
        "# tf.matmul(X, Y)"
      ],
      "metadata": {
        "id": "IwBgtEhN37ne"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "DzbTSqi-4AHI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Let's change the shape of Y\n",
        "# https://www.tensorflow.org/api_docs/python/tf/reshape\n",
        "tf.reshape(Y, shape=(2, 3))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MwOIZn6p4pqj",
        "outputId": "ff99a439-1ad6-44fa-9275-12184eb91020"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              "array([[ 7,  8,  9],\n",
              "       [10, 11, 12]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 48
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Try to multiply X by reshaped Y\n",
        "X @ tf.reshape(Y, shape=(2, 3))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "C6S17xuQ69N4",
        "outputId": "48533a8d-e527-4a68-859c-4d024d050a62"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tf.matmul(X, tf.reshape(Y, shape=(2, 3)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "y50rpCPA7JKr",
        "outputId": "8b022d90-6ab8-4ad7-88b4-33ec53b31187"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Try to change the shape of X instead of Y\n",
        "tf.matmul(tf.reshape(X, shape=(2, 3)), Y)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zlQ577q-7dud",
        "outputId": "77a1aa29-53b8-4e5f-e3bb-7594ddd9a0f5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 58,  64],\n",
              "       [139, 154]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Transpose also reshapes the tensor but flips the axes\n",
        "# https://www.tensorflow.org/api_docs/python/tf/transpose\n",
        "X, tf.transpose(X), tf.reshape(X, shape=(2, 3))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "uaBzuuER7l1p",
        "outputId": "13e15b31-4523-4a0b-a3af-46f62ac738d2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 3, 5],\n",
              "        [2, 4, 6]], dtype=int32)>,\n",
              " <tf.Tensor: shape=(2, 3), dtype=int32, numpy=\n",
              " array([[1, 2, 3],\n",
              "        [4, 5, 6]], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 52
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Try matrix multiplication with tranpose rather than reshape\n",
        "tf.matmul(tf.transpose(X), Y)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9N5p6dwq8Laz",
        "outputId": "7a8daaea-81e0-4fa5-bacf-bc3a3e47785d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 89,  98],\n",
              "       [116, 128]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 53
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**The dot product**\n",
        "\n",
        "Matrix multiplication is also referred to as the dot product.\n",
        "\n",
        "You can perform matrix multiplication using operator `@` or the tensorflow functions:\n",
        "* `tf.matmul()` - https://www.tensorflow.org/api_docs/python/tf/linalg/matmul\n",
        "* `tf.tensordot()` - https://www.tensorflow.org/api_docs/python/tf/tensordot"
      ],
      "metadata": {
        "id": "04Ct-EJi81Dp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X, Y"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6LVqCCdK9dC1",
        "outputId": "1e76be9b-5c7e-4140-cb5a-2e8f6ad554c3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[1, 2],\n",
              "        [3, 4],\n",
              "        [5, 6]], dtype=int32)>,\n",
              " <tf.Tensor: shape=(3, 2), dtype=int32, numpy=\n",
              " array([[ 7,  8],\n",
              "        [ 9, 10],\n",
              "        [11, 12]], dtype=int32)>)"
            ]
          },
          "metadata": {},
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Performs the dot product on X and Y (requires X or Y to be transposed)\n",
        "tf.tensordot(tf.transpose(X), Y, axes=1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LcAtQF2E932E",
        "outputId": "72b7a714-a964-415f-d880-bb365c277dca"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2, 2), dtype=int32, numpy=\n",
              "array([[ 89,  98],\n",
              "       [116, 128]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Perform matrix multiplication between X and Y (transposed)\n",
        "tf.matmul(X, tf.transpose(Y))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3-MTfJBb975a",
        "outputId": "3f0bc9da-8bc1-4e0f-8990-ff348c7b4d99"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 23,  29,  35],\n",
              "       [ 53,  67,  81],\n",
              "       [ 83, 105, 127]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 56
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Perform matrix multiplication between X and Y (reshaped)\n",
        "tf.matmul(X, tf.reshape(Y, shape=(2, 3)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wMbMXjVr-tQM",
        "outputId": "ca7bd252-ce3c-460e-b9ce-8894c3495db4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 27,  30,  33],\n",
              "       [ 61,  68,  75],\n",
              "       [ 95, 106, 117]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Check the values of Y, reshape Y and transposed Y\n",
        "print(\"Normaal Y:\")\n",
        "print(Y, \"\\n\") # \"\\n\" is for newline\n",
        "\n",
        "print(\"Y reshaped to (2, 3):\")\n",
        "print(tf.reshape(Y, (2, 3)), \"\\n\")\n",
        "\n",
        "print(\"Y transposed:\")\n",
        "print(tf.transpose(Y))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LO7hKj4N-y8T",
        "outputId": "b174a22e-df17-459f-ce9f-537bf5396792"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Normaal Y:\n",
            "tf.Tensor(\n",
            "[[ 7  8]\n",
            " [ 9 10]\n",
            " [11 12]], shape=(3, 2), dtype=int32) \n",
            "\n",
            "Y reshaped to (2, 3):\n",
            "tf.Tensor(\n",
            "[[ 7  8  9]\n",
            " [10 11 12]], shape=(2, 3), dtype=int32) \n",
            "\n",
            "Y transposed:\n",
            "tf.Tensor(\n",
            "[[ 7  9 11]\n",
            " [ 8 10 12]], shape=(2, 3), dtype=int32)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tf.matmul(X, tf.transpose(Y))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JUueeHY9_XTF",
        "outputId": "e6e7192d-ff6a-4bab-9605-629e803b9f43"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3, 3), dtype=int32, numpy=\n",
              "array([[ 23,  29,  35],\n",
              "       [ 53,  67,  81],\n",
              "       [ 83, 105, 127]], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 59
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Generally, when performing matrix multiplication on two tensors and one of the axes doesn't line up, you will transpose (rather than reshape) one of the tensors to satisfy the matrix multiplication rules."
      ],
      "metadata": {
        "id": "Kr5KCXl__6xS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Changing the datatype of tensors"
      ],
      "metadata": {
        "id": "If8PESuvApZe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a new tensor with default datatype (float32)\n",
        "B = tf.constant([1.7, 7.4])\n",
        "B, B.dtype"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0bZNQChxCFQi",
        "outputId": "57f8437e-f3c5-472d-df5d-b6eb09303067"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float32, numpy=array([1.7, 7.4], dtype=float32)>,\n",
              " tf.float32)"
            ]
          },
          "metadata": {},
          "execution_count": 60
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "C = tf.constant([7, 10])\n",
        "C"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xWKTllOnCOiP",
        "outputId": "7c3dcfed-386f-4fba-9400-a505cd7875f2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ 7, 10], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 61
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Change from float32 to float16 (reduced precision)\n",
        "# https://www.tensorflow.org/guide/mixed_precision\n",
        "# https://www.tensorflow.org/api_docs/python/tf/cast\n",
        "D = tf.cast(B, dtype=tf.float16)\n",
        "D, D.dtype"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "x89IVZtKCQxF",
        "outputId": "1ccfbdb5-4177-459c-b9f5-5ac26721e55f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(2,), dtype=float16, numpy=array([1.7, 7.4], dtype=float16)>,\n",
              " tf.float16)"
            ]
          },
          "metadata": {},
          "execution_count": 62
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Change from int32 to float32\n",
        "E = tf.cast(C, dtype=tf.float32)\n",
        "E"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cC4FpwObETzb",
        "outputId": "999129f3-bcf8-4df7-cdf3-d0037b08b295"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=float32, numpy=array([ 7., 10.], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "F = tf.cast(E, dtype=tf.float16)\n",
        "F"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l3W701saFYq2",
        "outputId": "f29381e1-b118-43f8-bb60-cf151f160736"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=float16, numpy=array([ 7., 10.], dtype=float16)>"
            ]
          },
          "metadata": {},
          "execution_count": 64
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Tensor aggregation (finding the min, max, mean & more)\n",
        "\n",
        "Aggregating tensors = condensing them from multiple vlaues down to a smaller amount of values."
      ],
      "metadata": {
        "id": "nTd1qyL9FkuR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the absolute values\n",
        "D = tf.constant([-7, -10])\n",
        "D"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VxGeomqfHzzf",
        "outputId": "e893ed84-1f0d-4ea0-e694-fc6d89492ae6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ -7, -10], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 65
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Get the absolute values\n",
        "# https://www.tensorflow.org/api_docs/python/tf/math/abs\n",
        "tf.abs(D)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5akcGvWZIuAw",
        "outputId": "8ff3d114-4393-440e-b187-7840051b9f99"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(2,), dtype=int32, numpy=array([ 7, 10], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 66
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's go through the following forms of aggregation:\n",
        "* Get the minimum\n",
        "* Get the maximum\n",
        "* Get the mean of a tensor\n",
        "* Get the sum of a tensor"
      ],
      "metadata": {
        "id": "5GiiG_PwI1g0"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a random tensor with values between 0 and 100 of size 50\n",
        "E = tf.constant(np.random.randint(0, 100, size=50))\n",
        "E"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NzVoVVmrJOru",
        "outputId": "a39fd2bb-80e7-4293-fae3-4bac99c67bf6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=int64, numpy=\n",
              "array([81, 44, 67, 48,  1, 60, 41, 13, 20, 31, 16, 64, 10, 82, 16, 87, 18,\n",
              "       97, 13,  2, 87, 27, 96, 40, 52, 54, 33, 74, 58, 99, 56, 95, 63, 33,\n",
              "       34, 87,  5, 43, 35, 67, 29, 44, 21, 41, 17, 44, 90, 70, 79, 77])>"
            ]
          },
          "metadata": {},
          "execution_count": 67
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tf.size(E), E.shape, E.ndim"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "t6KVUfLaJa_A",
        "outputId": "0007c6a5-2927-4a19-92a9-2d65d6a245bb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=int32, numpy=50>, TensorShape([50]), 1)"
            ]
          },
          "metadata": {},
          "execution_count": 68
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the mimimum\n",
        "# https://www.tensorflow.org/api_docs/python/tf/math/reduce_min\n",
        "tf.reduce_min(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2D7Nma-YJhFf",
        "outputId": "006a85d4-42b7-4cda-b0f9-0c718b124807"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=1>"
            ]
          },
          "metadata": {},
          "execution_count": 69
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the maximum\n",
        "# https://www.tensorflow.org/api_docs/python/tf/math/reduce_max\n",
        "tf.reduce_max(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TAoo7edHJoKA",
        "outputId": "cd9e8584-63af-4471-9679-eb7f9119ed5b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=99>"
            ]
          },
          "metadata": {},
          "execution_count": 70
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the mean\n",
        "# https://www.tensorflow.org/api_docs/python/tf/math/reduce_mean\n",
        "tf.reduce_mean(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mDdqi9ntJxS5",
        "outputId": "e6e711a3-9f93-47b5-e3a8-581923e30458"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=49>"
            ]
          },
          "metadata": {},
          "execution_count": 71
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the sum\n",
        "# https://www.tensorflow.org/api_docs/python/tf/math/reduce_sum\n",
        "tf.reduce_sum(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qFXpdLkgJ244",
        "outputId": "92969018-5c02-46b5-aea3-75aa376e15c3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=2461>"
            ]
          },
          "metadata": {},
          "execution_count": 72
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Exercise:** With what we've just learned, find the variance and standard deviation of our tensor using TensorFlow methods."
      ],
      "metadata": {
        "id": "UTg20EaxJ8Vj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the variance of our tensor\n",
        "# https://www.tensorflow.org/api_docs/python/tf/math/reduce_variance\n",
        "# tf.reduce_variance(E) # won't work"
      ],
      "metadata": {
        "id": "AwcNf8-WNNyo"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# To find the variance of our tensor, we need access to tensorflow_probability\n",
        "import tensorflow_probability as tfp\n",
        "tfp.stats.variance(E)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "etFW2iCzNEBr",
        "outputId": "787948e6-63a4-449b-f1ec-61fb4ec25770"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=803>"
            ]
          },
          "metadata": {},
          "execution_count": 74
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the standard deviation of recast tensor\n",
        "tf.math.reduce_std(tf.cast(E, dtype=tf.float32))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P5Uhk24tKMi0",
        "outputId": "fd2f7c17-0f90-4b08-8477-2eda191cd881"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=28.343811>"
            ]
          },
          "metadata": {},
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the variance of our E tensor\n",
        "tf.math.reduce_variance(tf.cast(E, dtype=tf.float32))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f6by7pAQOYde",
        "outputId": "85eb96da-49e3-4448-cba9-7e3c22cee4f5"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=803.3716>"
            ]
          },
          "metadata": {},
          "execution_count": 76
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Finding the positional maximum and minimum of a tensor (argmax and argmin)"
      ],
      "metadata": {
        "id": "yiuGlosCM1-6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a new tensor for finding positional minimum and maximium\n",
        "tf.random.set_seed(42)\n",
        "F = tf.random.uniform(shape=[50])\n",
        "F"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "zHR1y3wsOO29",
        "outputId": "a2591da7-3e37-4075-dbb8-07f5f0e72090"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              "array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "       0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "       0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "       0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "       0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "       0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "       0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "       0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "       0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "       0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 77
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the positional maximum\n",
        "tf.argmax(F), np.argmax(F)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "G3bE9YJ6SE14",
        "outputId": "4fb423cb-a63c-4630-c52b-9457474e5c00"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(), dtype=int64, numpy=42>, 42)"
            ]
          },
          "metadata": {},
          "execution_count": 78
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Index on our largest value position\n",
        "F[tf.argmax(F)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s-abIA_fSSRk",
        "outputId": "b6b12e38-8836-4b1f-b888-03df20bea7d7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
            ]
          },
          "metadata": {},
          "execution_count": 79
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the max value of F\n",
        "tf.reduce_max(F)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SkHMzrrSSUp4",
        "outputId": "d45f485a-4234-4d7f-fa5b-8948ee7249d0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.9671384>"
            ]
          },
          "metadata": {},
          "execution_count": 80
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Check for equality\n",
        "F[tf.argmax(F)] == tf.reduce_max(F)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CASX0cUmSgc5",
        "outputId": "a1116469-6572-41e1-b785-929ca9ab6f89"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=bool, numpy=True>"
            ]
          },
          "metadata": {},
          "execution_count": 81
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the positional minimum\n",
        "tf.argmin(F)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i7jdwA_ISmOK",
        "outputId": "da7d45f5-c04d-4c34-f45b-723cfac670b0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=int64, numpy=16>"
            ]
          },
          "metadata": {},
          "execution_count": 82
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the minimum using the positional minimum index\n",
        "F[tf.argmin(F)]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MOBPa4nxTAuU",
        "outputId": "c78d7e8c-63bf-4f9b-e367-e0d0a1738354"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(), dtype=float32, numpy=0.009463668>"
            ]
          },
          "metadata": {},
          "execution_count": 83
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Squeezing a tensor (removing all 1-dimension axes)"
      ],
      "metadata": {
        "id": "Z0dISMXdTD7o"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a tensor to get started\n",
        "tf.random.set_seed(42)\n",
        "G = tf.constant(tf.random.uniform(shape=[50]),shape=(1, 1, 1, 1, 50))\n",
        "G"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GoT5VDY4V-UQ",
        "outputId": "9f505fcd-b583-4ccf-825b-e6eb4cf71a7f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1, 1, 1, 1, 50), dtype=float32, numpy=\n",
              "array([[[[[0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "           0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "           0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "           0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "           0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "           0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "           0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "           0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "           0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "           0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043]]]]],\n",
              "      dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 84
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "G.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6zpWfg2DWGW0",
        "outputId": "a3000fa8-883b-4e2d-f660-d66b56488249"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TensorShape([1, 1, 1, 1, 50])"
            ]
          },
          "metadata": {},
          "execution_count": 85
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "G_squeezed = tf.squeeze(G)\n",
        "G_squeezed, G_squeezed.shape"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fU1_IoXtWSfa",
        "outputId": "7b59ac15-a534-4ba2-fc78-01e1f01e9d2f"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(<tf.Tensor: shape=(50,), dtype=float32, numpy=\n",
              " array([0.6645621 , 0.44100678, 0.3528825 , 0.46448255, 0.03366041,\n",
              "        0.68467236, 0.74011743, 0.8724445 , 0.22632635, 0.22319686,\n",
              "        0.3103881 , 0.7223358 , 0.13318717, 0.5480639 , 0.5746088 ,\n",
              "        0.8996835 , 0.00946367, 0.5212307 , 0.6345445 , 0.1993283 ,\n",
              "        0.72942245, 0.54583454, 0.10756552, 0.6767061 , 0.6602763 ,\n",
              "        0.33695042, 0.60141766, 0.21062577, 0.8527372 , 0.44062173,\n",
              "        0.9485276 , 0.23752594, 0.81179297, 0.5263394 , 0.494308  ,\n",
              "        0.21612847, 0.8457197 , 0.8718841 , 0.3083862 , 0.6868038 ,\n",
              "        0.23764038, 0.7817228 , 0.9671384 , 0.06870162, 0.79873943,\n",
              "        0.66028714, 0.5871513 , 0.16461694, 0.7381023 , 0.32054043],\n",
              "       dtype=float32)>,\n",
              " TensorShape([50]))"
            ]
          },
          "metadata": {},
          "execution_count": 86
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## One-hot encoding tensors"
      ],
      "metadata": {
        "id": "OtoF2YEeWalD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a list of indices\n",
        "some_list = [0, 1, 2, 3] # could be red, green, blue, purple\n",
        "\n",
        "# One hot encode our list of indices\n",
        "tf.one_hot(some_list, depth=4)\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "37Ris6l9W2Wy",
        "outputId": "7f0b9b86-65d1-4009-e644-ee388086fd0a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=float32, numpy=\n",
              "array([[1., 0., 0., 0.],\n",
              "       [0., 1., 0., 0.],\n",
              "       [0., 0., 1., 0.],\n",
              "       [0., 0., 0., 1.]], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 87
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Specify custom values for one hot encoding\n",
        "tf.one_hot(some_list, depth=4, on_value=\"yo i love deep learning\", off_value=\"i also like to dance\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SWTqz03KkAIo",
        "outputId": "03e87028-a205-4c92-c0eb-f623a4facb73"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(4, 4), dtype=string, numpy=\n",
              "array([[b'yo i love deep learning', b'i also like to dance',\n",
              "        b'i also like to dance', b'i also like to dance'],\n",
              "       [b'i also like to dance', b'yo i love deep learning',\n",
              "        b'i also like to dance', b'i also like to dance'],\n",
              "       [b'i also like to dance', b'i also like to dance',\n",
              "        b'yo i love deep learning', b'i also like to dance'],\n",
              "       [b'i also like to dance', b'i also like to dance',\n",
              "        b'i also like to dance', b'yo i love deep learning']],\n",
              "      dtype=object)>"
            ]
          },
          "metadata": {},
          "execution_count": 88
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Trying out more tensor math operations"
      ],
      "metadata": {
        "id": "eMN2BYHJkdeo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "H = tf.range(1, 10)\n",
        "H"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T8nPAU4dk7Jr",
        "outputId": "e16e2813-583f-40fe-b353-717ef796105c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([1, 2, 3, 4, 5, 6, 7, 8, 9], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 89
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Square it\n",
        "tf.square(H)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f0cMLU0Vk-ZR",
        "outputId": "ffc4b13d-6a0c-41ea-9b2f-42ea4e903472"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=int32, numpy=array([ 1,  4,  9, 16, 25, 36, 49, 64, 81], dtype=int32)>"
            ]
          },
          "metadata": {},
          "execution_count": 90
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the square root (will error, method requires non-int type)\n",
        "# tf.sqrt(tf.cast(H)"
      ],
      "metadata": {
        "id": "J2-PsAzalZyZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the square root\n",
        "tf.sqrt(tf.cast(H, dtype=tf.float32))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7CRJ3boQlAMV",
        "outputId": "6f6059ee-a58f-47f4-fb07-c4b0b622f04b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
              "array([1.       , 1.4142135, 1.7320508, 2.       , 2.236068 , 2.4494898,\n",
              "       2.6457512, 2.828427 , 3.       ], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 92
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Find the log\n",
        "tf.math.log(tf.cast(H, dtype=tf.float32))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7MOwpJvQlM0d",
        "outputId": "fa83729f-3e3a-40ae-e76c-ed85343a3f02"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(9,), dtype=float32, numpy=\n",
              "array([0.       , 0.6931472, 1.0986123, 1.3862944, 1.609438 , 1.7917595,\n",
              "       1.9459102, 2.0794415, 2.1972246], dtype=float32)>"
            ]
          },
          "metadata": {},
          "execution_count": 93
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Exploring TensorFlow and NumPy's compatibility\n",
        "\n",
        "TensorFlow interacts beautifully with NumPy arrays.\n",
        "\n",
        "**Note:** One of the main differences between a TensorFlow tensor and a NumPy array is that a TensorFlow tensor can be run on a GPU or TPU for faster numerical processing."
      ],
      "metadata": {
        "id": "bdfXvKjZln2t"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Create a tensor directly from a NumPy array\n",
        "J = tf.constant(np.array([3., 7., 10.]))\n",
        "J"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "s1vPNmGLmmyd",
        "outputId": "3c08929b-db22-4d21-d0e7-1e35774ea7d7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(3,), dtype=float64, numpy=array([ 3.,  7., 10.])>"
            ]
          },
          "metadata": {},
          "execution_count": 94
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert our tensor back to a NumPy array\n",
        "np.array(J), type(np.array(J))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0-iRlzJ9m5y3",
        "outputId": "d5cd2ec2-869c-4adf-ced8-b308e7204ab7"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 3.,  7., 10.]), numpy.ndarray)"
            ]
          },
          "metadata": {},
          "execution_count": 95
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert tensor J to a NumPy array\n",
        "J.numpy(), type(J.numpy())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "85xChK0Cm-Fm",
        "outputId": "ac5febb6-d275-41b0-f7fc-d089524668ba"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([ 3.,  7., 10.]), numpy.ndarray)"
            ]
          },
          "metadata": {},
          "execution_count": 96
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "J = tf.constant([3.])\n",
        "J.numpy()[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "LWNKvgu_nGnq",
        "outputId": "cfd2c1e6-be5b-48b3-d266-efe18c6bddc3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "3.0"
            ]
          },
          "metadata": {},
          "execution_count": 97
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# The default types of each are slightly different\n",
        "numpy_J = tf.constant([np.array([3., 7., 10.])])\n",
        "tensor_J = tf.constant([3., 7., 10.])\n",
        "# Check the data types of each\n",
        "numpy_J.dtype, tensor_J.dtype"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qxN654uCnLfL",
        "outputId": "2a729214-454e-413d-87a3-71239e8b2132"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(tf.float64, tf.float32)"
            ]
          },
          "metadata": {},
          "execution_count": 98
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Making sure our tensor operations run really fast on GPUs"
      ],
      "metadata": {
        "id": "nDNB2IGeoI-U"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# https://www.tensorflow.org/api_docs/python/tf/config/\n",
        "import tensorflow as tp\n",
        "tf.config.list_physical_devices()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BfzoeEBMphzB",
        "outputId": "9a569d08-ae91-4229-c4b5-f162c65bec03"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU')]"
            ]
          },
          "metadata": {},
          "execution_count": 99
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# ! - run on command line, command checks what GPU you are using\n",
        "!nvidia-smi"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "wqSfbOEGpvnF",
        "outputId": "c43ffc64-56db-41b3-d52f-255616dcf4a6"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/bin/bash: line 1: nvidia-smi: command not found\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Note:** If you have access to a CUDA-enabled GPU, TensorFlow will automatically use it whenever possible."
      ],
      "metadata": {
        "id": "mGU73lPVqxoQ"
      }
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "cGVjovUarUjV"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}