SMUMSE_ProgWorkshop_Lesson1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SMU Master of Science (Economics) Programming Workshop in Python\n",
    "\n",
    "\n",
    "## Introduction\n",
    "This is an introductory course/workshop in Programming with Python, aimed at achieving the following objectives:\n",
    "1. Give a brief introduction/overview of Python\n",
    "2. Equip students with foundational programming knowledge for the MSc Programme in Economics\n",
    "\n",
    "Throughout the duration of the course, we will learn programming using an application-based approach. For most of the course, students will be required to write code (in the form of in-class assignments) - this is probably the best way to learn coding/programming. \n",
    "\n",
    "More often than not, we are not interested in whether you got the answer right or wrong - we are more interested in the way you think about the question, and your approach towards solving it.\n",
    "\n",
    "#### Course Outline\n",
    "The course will be split into the following modules:\n",
    "\n",
    "1. Introduction, Syntax, Operations and Data Structures\n",
    "2. Loops, Conditional Statements, Functions and Classes\n",
    "3. NumPy and Pandas - Data Cleaning, Reading and Manipulation\n",
    "4. Matplotlib and Seaborn - Data Plotting\n",
    "5. Algorithms\n",
    "\n",
    "In today's class, we will be covering the first module on syntax, operations and data structures."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Basic Syntax and Operations\n",
    "\n",
    "In the earlier days of programming, humans have to key in specific code (machine-readable code) that the machines can interpret in order for them to execute the action specified. For example, Fortran and C are 2 such \"low-level\" languages - they are languages that can be easily interpreted by the computer, but not necessarily easy for humans to understand:\n",
    "\n",
    "<img src=\"images/fortran_code.jpg\">\n",
    "\n",
    "However, in recent times, for sake of simplicity and convenience, newer languages, such as R and Python, are \"higher-level\" languages - they have a build-in interpreter that translates what we provide as code/inputs for the machine to interpret/execute\n",
    "\n",
    "Then, one benefit of newer languages such as Python and R stems from its readability and simplicity, but comes at the cost of longer computation. This wouldn't be an issue as long as we are not dealing with *Big Data*.\n",
    "\n",
    "In this chapter, we will start off with a quick introduction to the syntax (or grammar) and basic operations of the Python language - this will give you a taste of what Python can and cannot do.\n",
    "\n",
    "<img src=\"images/python_code.jpeg\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Variable Assignment\n",
    "\n",
    "In the following example, we assign the value 200 * 17 to the variable \"x\", and then we use the print function to print its value. Note that we can also assign strings, which are also known as text, but we will have to use quotation marks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "175\n",
      "Hello World.\n"
     ]
    }
   ],
   "source": [
    "# Problem 1 (Note that to make comments in Python, we can include a #)\n",
    "x = 200 - 25\n",
    "print(x) \n",
    "\n",
    "y = \"Hello World.\"\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10.0\n"
     ]
    }
   ],
   "source": [
    "z = 200/20\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Type \n",
    "\n",
    "Note that in the above code block, there are 2 lines of output. The function, `print`, takes an argument, and prints it out. Below, we can use the `print` function along with the `type` function, to learn more about the type of the variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "int"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "str"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "float"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "str"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = 'ABC'\n",
    "type(z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "float"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = 10.0/3\n",
    "type(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Not surprisingly, '3400' was assigned to the type, `int` while 'Hello World' was assigned to the type `str`. There are many different data types in Python but the main ones are as follows:\n",
    "\n",
    "1. int - integers\n",
    "2. float - floating point (decimals)\n",
    "3. str - string variables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Concatenating Strings and Integers\n",
    "\n",
    "After seeing those examples, you may have the following questions in mind:\n",
    "1. Can we add a string to an integer?\n",
    "2. Can we add a string to a string?\n",
    "3. Can we do multiplication with a string?\n",
    "\n",
    "As it turns out, one of the best ways to understand more about programming is to **actively** code. Another way to learn more about programming is to learn to ask the right questions (most programmers use StackOverflow). \n",
    "\n",
    "Below, we answer the 3 questions in sequential order."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "unsupported operand type(s) for +: 'int' and 'str'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-8-a7a75c93ae58>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Question 1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0;36m123\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;34m'ABC'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: unsupported operand type(s) for +: 'int' and 'str'"
     ]
    }
   ],
   "source": [
    "# Question 1\n",
    "123 + 'ABC'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As it turns out, we cannot do so. This is also our first example of an error; in this case, a TypeError. To learn more about the error, we can look at the text that comes after the error: essentially, the error says that we cannot add integers and strings together. However, there is a way to circumvent this, but we digress from this question, and move on to the second question: can we add a string to a string?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "y = 'Hello World.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It turns out that we can add a string to a string. However, note that since we did not assign another variable to store the value returned by the addition of the variables, we cannot retrieve the values. This implies the following: whenever you're interested in a result, or think that you may need it again, save it! This reduces the complexity of your program, since you do not have to run the same function over and over again. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "##### Naming Conventions and Comments\n",
    "It is pretty important to know how to name your variables. \n",
    "\n",
    "Suppose you're naming a variable which contains the mean of the GDP variable, one sensible name will be to name it **gdp_mean**, instead of **x**. While this distinction seems pretty trivial right now, when you begin to write longer chunks of code, naming conventions make it easy for other readers to know what you're doing, and for you to follow what you're actually doing. In addition, having variables with uninformative names makes it difficult to carry on writing code if you need to call these variables in the future.\n",
    "\n",
    "Writing comments also help your readers understand what each block of code (be it a loop, function or conditional statements) are doing. As you begin to write more and more code, and begin to call previously defined functions, it becomes very difficult for your readers (and you) to follow what's going on. Writing comments can often help to reduce the reading burden of your readers.\n",
    "\n",
    "---\n",
    "\n",
    "And back to the main course. Adding 2 strings together essentially implies appending them to one another. Because of this, we can also do multiplication with a string."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello World.Hello World.Hello World.Hello World.Hello World.Hello World.Hello World.Hello World.Hello World.Hello World.'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the following code block multiplies y by 10 times\n",
    "y * 10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Hello World.Hello World.Hello World.'"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "3 * y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mathematical Operations in Python\n",
    "\n",
    "In what follows, I will show basic mathematical operations for integers in Python, including:\n",
    "\n",
    "1. Addition\n",
    "2. Subtraction\n",
    "3. Multiplication\n",
    "4. Division\n",
    "5. Exponential"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10.0"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "3.2 + 6.8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-3.3"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "4 - 7.3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "18.0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "5 * 3.6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = 2.5\n",
    "type(x)\n",
    "int(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'float'> 1.6666666666666667\n"
     ]
    }
   ],
   "source": [
    "j = 5 / 3\n",
    "print(type(j), j)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is our first instance of a floating point variable. Floating point work the same way as integers; we can carry out multiplication, addition, division etc. on them. However, when we try to convert a floating point to an integer, Python rounds it down. We will learn more about type conversion in the next section."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "int(j)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Double asterisks implies \"to the power of\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "64"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "4 ** 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Type Conversion\n",
    "\n",
    "In practice, one can convert the integer type variable to a string type. One way to do so is to use the `str` function. The `str` function takes a variable (integer, string, float) and converts it to a string variable. On the contrary, you cannot use the `int` function on strings, unless the string contains only numbers. \n",
    "\n",
    "Below, we show some examples of what can and cannot be done."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3400 <class 'int'>\n"
     ]
    }
   ],
   "source": [
    "x = 200 * 17\n",
    "print(x, type(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3400 <class 'str'>\n"
     ]
    }
   ],
   "source": [
    "k = str(x)\n",
    "print(k, type(k))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3400 <class 'int'>\n"
     ]
    }
   ],
   "source": [
    "z = int(k)\n",
    "print(z, type(z))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "invalid literal for int() with base 10: 'Hello World.'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-22-0d918221cf0c>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0my\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"Hello World.\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 2\u001b[0;31m \u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m: invalid literal for int() with base 10: 'Hello World.'"
     ]
    }
   ],
   "source": [
    "y = \"Hello World.\"\n",
    "int(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, we have the second error: a ValueError. The text that follows after the error says that Python was unable to convert a text, in this case, \"Hello World\", to an integer. No surprises here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Structures\n",
    "\n",
    "So far, we have seen quite a number of data types. How do we store these data types in a structure that is easily accessible, yet manipulable? \n",
    "\n",
    "It turns out that there are many different types of data structures that provides us with a schema for doing so. For example, we have ordered lists, tuples and more. Listed below are typical data structures used:\n",
    "\n",
    "1. List (ordered list)\n",
    "2. Sets\n",
    "3. Dictionaries (hash-tables; key-value pairs)\n",
    "4. Tuples\n",
    "\n",
    "In this subsection, we learn more about the first 3 types of data structures."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Lists\n",
    "In what follows, we give a somewhat brief summary of what lists are. Essentially, a list is a collection of variables that has the following properties: \n",
    "\n",
    "1. They are **ordered**\n",
    "2. They are **mutable**\n",
    "3. They allow for duplicate members (taken from [here](https://www.w3schools.com/python/python_lists.asp)). \n",
    "\n",
    "Finally, lists can be constructed in Python using either the square brackets, [ ] or the function, `list()`.\n",
    "\n",
    "We can also change the elements in a list - add new members to a list, remove members from a list and switch elements within the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3400\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_list = []\n",
    "type(new_list)\n",
    "\n",
    "print(x)\n",
    "len(new_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[25, 36, 42, 17, 19, 19, 51]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lst = [25, 36, 42, 17, 19, 19, 51]\n",
    "print(lst)\n",
    "len(lst)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[25, 36]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lst[:-5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[25, 36, 42]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lst[0:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One interesting aspect of Python is that it uses zero-indexing i.e. the first element in the list is denoted by the index 0 instead of 1.\n",
    "\n",
    "In the next code block, we demonstrate how to extract an element of the list. We can use the square brackets, [0], to query the 0-th element of the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "42"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Extracting third element from the list, lst.\n",
    "lst[2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "51"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lst[-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[25, 36, 42, 17, 19]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Extract first 5 elements, does not include the \"5\" index\n",
    "lst[0:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[42, 17, 19, 19, 51]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Extract last 5 elements\n",
    "lst[-5:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The first element in the list is Alex.\n"
     ]
    }
   ],
   "source": [
    "names = [\"Alex\", \"Bob\", \"Charlie\"] # assign the variable x to a list containing 3 names\n",
    "print(\"The first element in the list is \" + names[0] + \".\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Bob'"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "names[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Charlie'"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "names[-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to strings, when we concatenate 2 lists together, Python appends the list, `y`, to the back of list `x`.\n",
    "\n",
    "That is, it creates a new list `[[x[0], x[1], ...], [y[0], y[1], ...]`. \n",
    "\n",
    "Thus, the order of the operation matters: x + y yields a different output compared to y + x. Similar to strings as well, we can also multiply lists. Multiplying lists concatenates the lists together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lst1 = [10, 12, 15]\n",
    "lst2 = [15, 23, 40]\n",
    "lst1[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Alex', 'Bob', 'Charlie', 'Diane', 'Elaine']\n",
      "5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['Alex', 'Bob', 'Charlie', 'Diane', 'Elaine']"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = ['Alex', 'Bob', 'Charlie']\n",
    "y = [\"Diane\", \"Elaine\"] # assigns the variable to a list containing 2 names\n",
    "print(x+y)\n",
    "print(len(x+y))\n",
    "\n",
    "z = x + y\n",
    "z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lists are a simple way of storing and extracting information. One can easily verify whether an element is in a list, using the list method, `in`. \n",
    "\n",
    "Python checks whether the element is in the list, and returns True if the element is in the list and False otherwise. However, note that this method is **sensitive to capital letters**, as evident from the following examples.\n",
    "\n",
    "Note: Methods and functions are really similar, although there are some fundamental differences between methods and functions. For example, methods are unique to functions. This will be discussed in a future lesson."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Alex', 'Bob', 'Charlie']\n"
     ]
    }
   ],
   "source": [
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"alex\" in x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"Bobby\" in x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'alex'"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[0].lower()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'alex'"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"ALEX\".lower()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['alex', 'bob', 'charlie']"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "small_letters = [i.lower() for i in x]\n",
    "small_letters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"ALEX\".lower() in small_letters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lists can perform a variety of operations, which are referred to as methods (you can check them out [here](https://www.programiz.com/python-programming/methods/list)). We will go through some of them in this lesson:\n",
    "\n",
    "1. append\n",
    "2. extend\n",
    "3. insert\n",
    "4. remove\n",
    "5. reverse\n",
    "6. sort\n",
    "\n",
    "Note that these methods are non-reversible. After applying these methods, the list gets altered. Once you perform an operation (method), the list will be different from what it used to be."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "lst1 = ['a', 'b', 'c']\n",
    "lst2 = ['d', 'e']\n",
    "lst3 = lst1 + lst2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Alex', 'Bob', 'Charlie']\n",
      "['Alex', 'Bob', 'Charlie', 'Daniel', 'Edgar']\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The list method, append, add an element to the list (note than an element of a list can be a list itself)\n",
    "x = ['Alex', 'Bob', 'Charlie']\n",
    "print(x)\n",
    "x.append(\"Daniel\")\n",
    "x.append(\"Edgar\")\n",
    "print(x)\n",
    "len(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Alex', 'Bob', 'Charlie', 'Daniel', 'Edgar']"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Alex', 'Bob', 'Kai', 'Charlie', 'Daniel', 'Edgar']\n"
     ]
    }
   ],
   "source": [
    "# The list method, insert, inserts an element to the list \n",
    "# (requires 2 arguments - 1st argument is the position, and 2nd argument is the element)\n",
    "x.insert(2, \"Kai\")\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['A', 'B', 'C+', 'C', 'C-', 'D']"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "alp = ['A', 'B', 'D']\n",
    "alp.insert(2, 'C+')\n",
    "alp.insert(3, 'C')\n",
    "alp.insert(4, 'C-')\n",
    "alp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['A', 'C', 'D']\n"
     ]
    }
   ],
   "source": [
    "# The list method, remove, removes an element from the list, if it is present\n",
    "x = ['A', 'B', 'C', 'D']\n",
    "x.remove('B') # only remove first \"B\"\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "ename": "ValueError",
     "evalue": "list.remove(x): x not in list",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-49-f972367ea1b1>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'E'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m: list.remove(x): x not in list"
     ]
    }
   ],
   "source": [
    "x.remove('E')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['A', 'B', 'C', 'D']"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.insert(1, 'B')\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['D', 'C', 'B', 'A']\n"
     ]
    }
   ],
   "source": [
    "# The list method, reverse, reverses the entire list index\n",
    "x.reverse()\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The list method, sort, sorts the list by numerical or alphabetical order\n",
    "x.sort()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = ['A', '3', 'B', \"a\"]\n",
    "x.sort()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['3', 'A', 'B', 'a']"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = ['Apple', '100', '50', '9', '0100000', 'Sea']\n",
    "z.sort()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['0100000', '100', '50', '9', 'Apple', 'Sea']"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = ['A', ['B', 'C']]\n",
    "z.reverse()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[['B', 'C'], 'A']"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['A']"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z.remove(['B', 'C'])\n",
    "z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['A', 'B', 'C']"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "['A'] + ['B', 'C']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As it turns out, you cannot sort a list that contains both lists and strings. \n",
    "\n",
    "In-class assignment:\n",
    "\n",
    "Can you sort a list if it has both integers and strings? Use an example to show this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'<' not supported between instances of 'int' and 'str'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-61-9ead0988cd54>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0;31m# Your code here\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0mx\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m'230'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m10\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m\"Abc\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0mx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msort\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: '<' not supported between instances of 'int' and 'str'"
     ]
    }
   ],
   "source": [
    "# Your code here\n",
    "x = ['230', 10, \"Abc\"]\n",
    "x.sort()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have some experience with lists, here's a quick in-class assignment:\n",
    "\n",
    "1. Create 2 lists: one that contains the numbers, 3, 5, 7 and another that contains strings \"Helen\", \"Jake\", \"Betty\".\n",
    "2. Add the integer 4 to the first list so that it is the second element, and the string \"Sarah\" to the second one such that it is the first element in the second list.\n",
    "3. Sort both lists in reverse order (i.e. first string should return [7, 5, 4, 3])\n",
    "4. Append the second list to the first list.\n",
    "5. Check if you can sort them.\n",
    "\n",
    "Note: there are many ways to do this problem."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[7, 5, 4, 3]"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Your code here\n",
    "# Starting point\n",
    "[3, 5, 7]\n",
    "[\"Helen\", \"Jake\", \"Betty\"]\n",
    "\n",
    "\n",
    "# Operations\n",
    "[3, 4, 5, 7]\n",
    "[\"Sarah\", \"Helen\", \"Jake\", \"Betty\"]\n",
    "[7, 5, 4, 3]\n",
    "\n",
    "\n",
    "# Ending point\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sets\n",
    "\n",
    "A set is a collection of variables and has the following properties: \n",
    "1. They are **unordered** \n",
    "2. They are **unindexed**\n",
    "3. They do not not allow for duplicate members. \n",
    "\n",
    "That is, sets contain only **unique** values. To create a set, we can use the curly brackets, { }, or the function, set()."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'A', 'B', 'C'}"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = [\"A\", \"B\", \"C\", \"A\"]\n",
    "set_x = set(x)\n",
    "set_x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'set' object is not subscriptable",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-64-e65081aa0771>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mset_x\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mTypeError\u001b[0m: 'set' object is not subscriptable"
     ]
    }
   ],
   "source": [
    "set_x[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "set"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_a = {\"Apples\", \"Bananas\", \"Oranges\", \"Pears\", \"Apples\"}\n",
    "set_a\n",
    "\n",
    "type(set_a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous code block, we created a set with 5 elements, but the element \"Apples\" appears twice in the set. \n",
    "\n",
    "Noting that sets contain only unique values, we should expect `set_a` to have 4 elements"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Apples', 'Bananas', 'Oranges', 'Pears'}"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to lists, one can check whether an element is in the set using the \"in\" function, but note that it is case-sensitive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "False\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(\"Bananas\" in set_a)\n",
    "print(\"bananas\" in set_a)\n",
    "\n",
    "# The following code is robust to errors resulting from different cases\n",
    "\"BANANAS\".lower() in [fruit.lower() for fruit in list(set_a)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\"bananas\" not in set_a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To add an element to the set, we can use the set method, \"add\". To add more than one element, we can use the set method, \"update\", to update a set with **another set**. To check how many elements the set contains, we can use the function, `len()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Apples', 'Bananas', 'Oranges', 'Pears', 'Watermelons'}"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_a.add(\"Watermelons\")\n",
    "set_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Apples', 'Bananas', 'Oranges', 'Pears', 'Watermelons'}"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_a.add(\"Pears\")\n",
    "set_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Apples', 'Bananas', 'Durians', 'Mangoes', 'Oranges', 'Pears', 'Watermelons'}"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Note that the elements in the set may not retain their index\n",
    "set_b = {\"Mangoes\", \"Durians\"}\n",
    "set_b.update(set_a)\n",
    "set_b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(set_a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What if we were to add an element that is already present in the set?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Apples', 'Bananas', 'Durians', 'Oranges', 'Pears', 'Watermelons'}"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_a.add(\"Durians\")\n",
    "set_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(set_a)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As it turns out, adding the string, \"Durians\" does not change the set in any way, since the string **already** appears in the set. This is one of the key differences between sets and lists. Another difference (as we have previously discussed) is that sets may not retain their structure as they are constructed, but lists do.\n",
    "\n",
    "---\n",
    "\n",
    "To remove an element from the set, we can use the set method, \"remove\". However, if an element does not exist in the set, this will raise an error. \n",
    "\n",
    "For that reason, some prefer to use the set method, \"discard\". Below, we show both methods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Apples', 'Bananas', 'Durians', 'Oranges', 'Pears', 'Watermelons'}"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'Jackfruits'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-76-d9a586ebeed9>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mset_a\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Jackfruits\"\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# One can think of this as a variable assignment itself\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m: 'Jackfruits'"
     ]
    }
   ],
   "source": [
    "set_a.remove(\"Jackfruits\") # One can think of this as a variable assignment itself"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the element \"Jackfruits\" does not appear in the set, this raises a KeyError. What does the \"KeyError\" here refer to? In Python, \"keys\" and \"elements\" in a set are used interchangeably - there is an error with regard to the removing the element from the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'Mangoes'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-77-359d036d1b8d>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mset_a\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mremove\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Mangoes\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m: 'Mangoes'"
     ]
    }
   ],
   "source": [
    "set_a.remove(\"Mangoes\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Apples', 'Bananas', 'Durians', 'Oranges', 'Pears', 'Watermelons'}"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Apples', 'Bananas', 'Durians', 'Oranges', 'Pears', 'Watermelons'}"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_a.discard(\"Jackfruits\")\n",
    "set_a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Apples', 'Bananas', 'Oranges', 'Pears', 'Watermelons'}"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set_a.discard(\"Durians\")\n",
    "set_a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To learn more about sets, you can refer to the link [here](https://www.w3schools.com/python/python_sets.asp)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Dictionaries\n",
    "\n",
    "A dictionary is a collection of variables that have the following properties:\n",
    "1. They are **unordered**\n",
    "2. They are **changeable**\n",
    "3. They are **indexed** \n",
    "\n",
    "In Python dictionaries are also written with curly brackets _{ }_, but they contain 2 types of elements: keys and values. \n",
    "\n",
    "Dictionaries are perhaps the most useful of all data structures, since we are able store 2 types of information with one data structure. \n",
    "\n",
    "For example, a teacher may be keen to store the test results of students. Then, the keys are the names of the students, while the values are their test results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'John': 185,\n",
       " 'Marie': 165,\n",
       " 'Jack': 190,\n",
       " 'Stacy': 177,\n",
       " 'Jackson': 168,\n",
       " 'Dana': 180}"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We begin by defining a dictionary: \"height\", that contains 6 key-value pairs \n",
    "# where keys are names, and values are height\n",
    "\n",
    "height = {\n",
    "    \"John\": 185,\n",
    "    \"Marie\": 165,\n",
    "    \"Jack\": 190,\n",
    "    \"Stacy\": 177,\n",
    "    \"Jackson\": 168,\n",
    "    'Dana': 180\n",
    "}\n",
    "\n",
    "height"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "185"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# To access the height (value) of a certain individual, we can use the key (name of the individual)\n",
    "height['John']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# To check if a name is in the dictionary, we can use the \"in\" function\n",
    "'Stacy' in height.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "175 in height.values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['John', 'Marie', 'Jack', 'Stacy', 'Jackson', 'Dana']"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# To check the names of the dictionary, we can use the \"keys\" method\n",
    "list(height.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[185, 165, 190, 177, 168, 180]"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Similarly, we can do the same for values using the \"values\" method\n",
    "list(height.values())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous 2 code blocks, they returned an object `dict_values`. One can coerce these objects to be lists, by using the function, `list()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Dana', 'Jack', 'Jackson', 'John', 'Marie', 'Stacy'}"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "set(height.keys())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "180"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can also change the values of a specific key. For example, suppose Dana grew taller by 5cm this year.\n",
    "height['Dana']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "155"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "height['Dana'] = 150\n",
    "height['Dana'] += 5 # Adds 5 to her height\n",
    "height['Dana']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'John': 185,\n",
       " 'Marie': 165,\n",
       " 'Jack': 190,\n",
       " 'Stacy': 177,\n",
       " 'Jackson': 168,\n",
       " 'Dana': 155,\n",
       " 'Luke': 180}"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# In addition, we can add new keys to the dictionary\n",
    "height[\"Luke\"] = 180\n",
    "height"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(height)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_values([185, 165, 190, 177, 168, 155, 180])"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "height.pop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'John': 185,\n",
       " 'Marie': 165,\n",
       " 'Jack': 190,\n",
       " 'Stacy': 177,\n",
       " 'Dana': 155,\n",
       " 'Luke': 180}"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# To remove a key-value pair from the dictionary, we can use the \"pop\" method\n",
    "height.pop(\"Jackson\")\n",
    "height"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'Lucy'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-180-6e9370734d09>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mheight\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpop\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Lucy\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m: 'Lucy'"
     ]
    }
   ],
   "source": [
    "height.pop(\"Lucy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'John': 185, 'Marie': 165, 'Stacy': 177, 'Dana': 155, 'Luke': 180}"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Alternatively, one can use the del (which stands for delete) function\n",
    "del height['Jack']\n",
    "height"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'Lucy'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-99-1980f5cd6f00>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;32mdel\u001b[0m \u001b[0mheight\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"Lucy\"\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m: 'Lucy'"
     ]
    }
   ],
   "source": [
    "del height[\"Lucy\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In-class assignment:\n",
    "\n",
    "1. Create a dictionary, called \"weight\", using the follow dataset:\n",
    "\n",
    "|Name  | Weight| Gender |\n",
    "|------|-------|--------|\n",
    "|Jane  | 60.0  | Female |\n",
    "|John  | 81.2  |  Male  |\n",
    "|Tina  | 59.2  | Female |\n",
    "|Lena  | 48.5  | Female |\n",
    "|Kane  | 59.2  |  Male  |\n",
    "|Ryan  | 38.7  |  Male  |\n",
    "\n",
    "2. Suppose data for a new person (Kate) is available, and she weighs 43.0 kg. Add this information in.\n",
    "3. What is the average weight for the males in the group? (Hint: the mean function will be useful: mean( ))\n",
    "4. What about the average weight for the females?\n",
    "\n",
    "Suppose information on the height of the individuals is given:\n",
    "\n",
    "|Name  | Height | Gender |\n",
    "|------|------- |--------|\n",
    "|Jane  | 149.0  | Female |\n",
    "|John  | 139.2  |  Male  |\n",
    "|Tina  | 128.6  | Female |\n",
    "|Lena  | 159.3  | Female |\n",
    "|Kane  | 116.8  |  Male  |\n",
    "|Ryan  | 138.3  |  Male  |\n",
    "|Kate  | 128.8  | Female |\n",
    "\n",
    "1. Create another dictionary, \"height\" using the above dataset. Note that with the inclusion of this dataset, we have information on the height and weight of the 7 individuals.\n",
    "2. Create a new list, \"data\" which contains 2 dictionaries (height and weight). For each dictionary, keys are the names and values are the corresponding heights and weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Please do your in-class assignment here"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Application of Data Structures (Prelude to the next module)\n",
    "In this section of the class, I will show an example of why lists and dictionaries are so important and powerful in computing. Consider the following problem, where we have a specific text, and we need to find the number of times each word appears in the text (suppose we have 10,000 texts with 10,000 words each). This may be because we expect words that appear more frequently to be more important or provide more information about the text.\n",
    "\n",
    "For the sake of simplicity, we focus on only 1 text, but it is entirely plausible to have 10,000 texts to analyse. Our approach here is to break the problem down into smaller problems which are easier to solve, and then combine the solutions to the smaller problems and check if we can find a \"global\" solution.\n",
    "\n",
    "Here, you will also see a glimpse of function definition and for-loops (both to be studied in the second module)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's proceed to work on converting our letters to lower-case, and remove punctuations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Why should you learn to write programs?\n",
      "\n",
      "Writing programs (or programming) is a very creative \n",
      "and rewarding activity.  You can write programs for \n",
      "many reasons, ranging from making your living to solving\n",
      "a difficult data analysis problem to having fun to helping\n",
      "someone else solve a problem.  This book assumes that \n",
      "everyone needs to know how to program, and that once \n",
      "you know how to program you will figure out what you want \n",
      "to do with your newfound skills.  \n",
      "\n",
      "We are surrounded in our daily lives with computers ranging \n",
      "from laptops to cell phones.  We can think of these computers\n",
      "as our personal assistants who can take care of many things\n",
      "on our behalf.  The hardware in our current-day computers \n",
      "is essentially built to continuously ask us the question, \n",
      "What would you like me to do next?\n",
      "\n",
      "Programmers add an operating system and a set of applications\n",
      "to the hardware and we end up with a Personal Digital\n",
      "Assistant that is quite helpful and capable of helping\n",
      "us do many different things.\n",
      "\n",
      "Our computers are fast and have vast amounts of memory and \n",
      "could be very helpful to us if we only knew the language to\n",
      "speak to explain to the computer what we would like it to \n",
      "do next.  If we knew this language, we could tell the \n",
      "computer to do tasks on our behalf that were repetitive.  \n",
      "Interestingly, the kinds of things computers can do best\n",
      "are often the kinds of things that we humans find boring\n",
      "and mind-numbing.\n",
      "\n",
      "For example, look at the first three paragraphs of this\n",
      "chapter and tell me the most commonly used word and how\n",
      "many times the word is used.  While you were able to read\n",
      "and understand the words in a few seconds, counting them\n",
      "is almost painful because it is not the kind of problem \n",
      "that human minds are designed to solve.  For a computer\n",
      "the opposite is true, reading and understanding text \n",
      "from a piece of paper is hard for a computer to do \n",
      "but counting the words and telling you how many times\n",
      "the most used word was used is very easy for the\n",
      "computer:\n",
      "\n",
      "Our personal information analysis assistant quickly \n",
      "told us that the word to was used sixteen times in the\n",
      "first three paragraphs of this chapter.\n",
      "\n",
      "This very fact that computers are good at things \n",
      "that humans are not is why you need to become\n",
      "skilled at talking computer language.  Once you learn\n",
      "this new language, you can delegate mundane tasks\n",
      "to your partner (the computer), leaving more time \n",
      "for you to do the \n",
      "things that you are uniquely suited for.  You bring \n",
      "creativity, intuition, and inventiveness to this\n",
      "partnership.  \n",
      "\n",
      "Creativity and motivation\n",
      "\n",
      "While this book is not intended for professional programmers, professional\n",
      "programming can be a very rewarding job both financially and personally.\n",
      "Building useful, elegant, and clever programs for others to use is a very\n",
      "creative activity.  Your computer or Personal Digital Assistant (PDA) \n",
      "usually contains many different programs from many different groups of \n",
      "programmers, each competing for your attention and interest.  They try \n",
      "their best to meet your needs and give you a great user experience in the\n",
      "process.   In some situations, when you choose a piece of software, the \n",
      "programmers are directly compensated because of your choice.\n",
      "\n",
      "If we think of programs as the creative output of groups of programmers,\n",
      "perhaps the following figure is a more sensible version of our PDA:\n",
      "\n",
      "For now, our primary motivation is not to make money or please end users, but\n",
      "instead for us to be more productive in handling the data and \n",
      "information that we will encounter in our lives.\n",
      "When you first start, you will be both the programmer and the end user of\n",
      "your programs.  As you gain skill as a programmer and\n",
      "programming feels more creative to you, your thoughts may turn\n",
      "toward developing programs for others.\n",
      "\n",
      "Computer hardware architecture\n",
      "\n",
      "Before we start learning the language we \n",
      "speak to give instructions to computers to \n",
      "develop software, we need to learn a small amount about \n",
      "how computers are built.  \n",
      "\n",
      "Central Processing Unit (or CPU) is \n",
      "the part of the computer that is built to be obsessed \n",
      "with what is next?  If your computer is rated\n",
      "at three Gigahertz, it means that the CPU will ask What next?\n",
      "three billion times per second.  You are going to have to \n",
      "learn how to talk fast to keep up with the CPU.\n",
      "\n",
      "Main Memory is used to store information\n",
      "that the CPU needs in a hurry.  The main memory is nearly as \n",
      "fast as the CPU.  But the information stored in the main\n",
      "memory vanishes when the computer is turned off.\n",
      "\n",
      "Secondary Memory is also used to store\n",
      "information, but it is much slower than the main memory.\n",
      "The advantage of the secondary memory is that it can\n",
      "store information even when there is no power to the\n",
      "computer.  Examples of secondary memory are disk drives\n",
      "or flash memory (typically found in USB sticks and portable\n",
      "music players).\n",
      "\n",
      "Input and Output Devices are simply our\n",
      "screen, keyboard, mouse, microphone, speaker, touchpad, etc.  \n",
      "They are all of the ways we interact with the computer.\n",
      "\n",
      "These days, most computers also have a\n",
      "Network Connection to retrieve information over a network.\n",
      "We can think of the network as a very slow place to store and\n",
      "retrieve data that might not always be up.  So in a sense,\n",
      "the network is a slower and at times unreliable form of\n",
      "Secondary Memory.\n",
      "\n",
      "While most of the detail of how these components work is best left \n",
      "to computer builders, it helps to have some terminology\n",
      "so we can talk about these different parts as we write our programs.\n",
      "\n",
      "As a programmer, your job is to use and orchestrate \n",
      "each of these resources to solve the problem that you need to solve\n",
      "and analyze the data you get from the solution.  As a programmer you will \n",
      "mostly be talking to the CPU and telling it what to \n",
      "do next.  Sometimes you will tell the CPU to use the main memory,\n",
      "secondary memory, network, or the input/output devices.\n",
      "\n",
      "You need to be the person who answers the CPU's What next? \n",
      "question.  But it would be very uncomfortable to shrink you \n",
      "down to five mm  tall and insert you into the computer just so you \n",
      "could issue a command three billion times per second.  So instead,\n",
      "you must write down your instructions in advance.\n",
      "We call these stored instructions a program and the act \n",
      "of writing these instructions down and getting the instructions to \n",
      "be correct programming.\n",
      "\n",
      "Understanding programming\n",
      "\n",
      "In the rest of this book, we will try to turn you into a person\n",
      "who is skilled in the art of programming.  In the end you will be a \n",
      "programmer --- perhaps not a professional programmer, but \n",
      "at least you will have the skills to look at a data/information\n",
      "analysis problem and develop a program to solve the problem.\n",
      "\n",
      "problem solving\n",
      "\n",
      "In a sense, you need two skills to be a programmer:\n",
      "\n",
      "First, you need to know the programming language (Python) -\n",
      "you need to know the vocabulary and the grammar.  You need to be able \n",
      "to spell the words in this new language properly and know how to construct \n",
      "well-formed sentences in this new language.\n",
      "\n",
      "Second, you need to tell a story.  In writing a story,\n",
      "you combine words and sentences to convey an idea to the reader. \n",
      "There is a skill and art in constructing the story, and skill in\n",
      "story writing is improved by doing some writing and getting some\n",
      "feedback.  In programming, our program is the story and the \n",
      "problem you are trying to solve is the idea.\n",
      "\n",
      "itemize\n",
      "\n",
      "Once you learn one programming language such as Python, you will \n",
      "find it much easier to learn a second programming language such\n",
      "as JavaScript or C++.  The new programming language has very different \n",
      "vocabulary and grammar but the problem-solving skills \n",
      "will be the same across all programming languages.\n",
      "\n",
      "You will learn the vocabulary and sentences of Python pretty quickly.\n",
      "It will take longer for you to be able to write a coherent program\n",
      "to solve a brand-new problem.  We teach programming much like we teach\n",
      "writing.  We start reading and explaining programs, then we write \n",
      "simple programs, and then we write increasingly complex programs over time.\n",
      "At some point you get your muse and see the patterns on your own\n",
      "and can see more naturally how to take a problem and \n",
      "write a program that solves that problem.  And once you get \n",
      "to that point, programming becomes a very pleasant and creative process.  \n",
      "\n",
      "We start with the vocabulary and structure of Python programs.  Be patient\n",
      "as the simple examples remind you of when you started reading for the first\n",
      "time. \n",
      "\n",
      "Words and sentences\n",
      "\n",
      "Unlike human languages, the Python vocabulary is actually pretty small.\n",
      "We call this vocabulary the reserved words.  These are words that\n",
      "have very special meaning to Python.  When Python sees these words in \n",
      "a Python program, they have one and only one meaning to Python.  Later\n",
      "as you write programs you will make up your own words that have meaning to \n",
      "you called variables.   You will have great latitude in choosing\n",
      "your names for your variables, but you cannot use any of Python's \n",
      "reserved words as a name for a variable.\n",
      "\n",
      "When we train a dog, we use special words like\n",
      "sit, stay, and fetch.  When you talk to a dog and\n",
      "don't use any of the reserved words, they just look at you with a \n",
      "quizzical look on their face until you say a reserved word.  \n",
      "For example, if you say, \n",
      "I wish more people would walk to improve their overall health, \n",
      "what most dogs likely hear is,\n",
      "blah blah blah walk blah blah blah blah.\n",
      "That is because walk is a reserved word in dog language.  \n",
      "\n",
      "The reserved words in the language where humans talk to \n",
      "Python include the following:\n",
      "\n",
      "and       del       from      not       while    \n",
      "as        elif      global    or        with     \n",
      "assert    else      if        pass      yield    \n",
      "break     except    import    print              \n",
      "class     exec      in        raise              \n",
      "continue  finally   is        return             \n",
      "def       for       lambda    try\n",
      "\n",
      "That is it, and unlike a dog, Python is already completely trained.\n",
      "When you say try, Python will try every time you say it without\n",
      "fail.\n",
      "\n",
      "We will learn these reserved words and how they are used in good time,\n",
      "but for now we will focus on the Python equivalent of speak (in \n",
      "human-to-dog language).  The nice thing about telling Python to speak\n",
      "is that we can even tell it what to say by giving it a message in quotes:\n",
      "\n",
      "And we have even written our first syntactically correct Python sentence.\n",
      "Our sentence starts with the reserved word print followed\n",
      "by a string of text of our choosing enclosed in single quotes.\n",
      "\n",
      "Conversing with Python\n",
      "\n",
      "Now that we have a word and a simple sentence that we know in Python,\n",
      "we need to know how to start a conversation with Python to test \n",
      "our new language skills.\n",
      "\n",
      "Before you can converse with Python, you must first install the Python\n",
      "software on your computer and learn how to start Python on your \n",
      "computer.  That is too much detail for this chapter so I suggest\n",
      "that you consult www.py4e.com where I have detailed\n",
      "instructions and screencasts of setting up and starting Python \n",
      "on Macintosh and Windows systems.  At some point, you will be in \n",
      "a terminal or command window and you will type python and \n",
      "the Python interpreter will start executing in interactive mode\n",
      "and appear somewhat as follows:\n",
      "interactive mode\n",
      "\n",
      "The >>> prompt is the Python interpreter's way of asking you, What\n",
      "do you want me to do next?  Python is ready to have a conversation with\n",
      "you.  All you have to know is how to speak the Python language.\n",
      "\n",
      "Let's say for example that you did not know even the simplest Python language\n",
      "words or sentences. You might want to use the standard line that astronauts \n",
      "use when they land on a faraway planet and try to speak with the inhabitants\n",
      "of the planet:\n",
      "\n",
      "This is not going so well.  Unless you think of something quickly,\n",
      "the inhabitants of the planet are likely to stab you with their spears, \n",
      "put you on a spit, roast you over a fire, and eat you for dinner.\n",
      "\n",
      "At this point, you should also realize that while Python \n",
      "is amazingly complex and powerful and very picky about \n",
      "the syntax you use to communicate with it, Python is \n",
      "not intelligent.  You are really just having a conversation\n",
      "with yourself, but using proper syntax.\n",
      "\n",
      "In a sense, when you use a program written by someone else\n",
      "the conversation is between you and those other\n",
      "programmers with Python acting as an intermediary.  Python\n",
      "is a way for the creators of programs to express how the \n",
      "conversation is supposed to proceed.  And\n",
      "in just a few more chapters, you will be one of those\n",
      "programmers using Python to talk to the users of your program.\n",
      "\n",
      "Before we leave our first conversation with the Python \n",
      "interpreter, you should probably know the proper way\n",
      "to say good-bye when interacting with the inhabitants\n",
      "of Planet Python:\n",
      "\n",
      "You will notice that the error is different for the first two\n",
      "incorrect attempts.   The second error is different because \n",
      "if is a reserved word and Python saw the reserved word\n",
      "and thought we were trying to say something but got the syntax\n",
      "of the sentence wrong.\n",
      "\n",
      "Terminology: interpreter and compiler\n",
      "\n",
      "Python is a high-level language intended to be relatively\n",
      "straightforward for humans to read and write and for computers\n",
      "to read and process.  Other high-level languages include Java, C++,\n",
      "PHP, Ruby, Basic, Perl, JavaScript, and many more.  The actual hardware\n",
      "inside the Central Processing Unit (CPU) does not understand any\n",
      "of these high-level languages.\n",
      "\n",
      "The CPU understands a language we call machine language.  Machine\n",
      "language is very simple and frankly very tiresome to write because it \n",
      "is represented all in zeros and ones.\n",
      "\n",
      "Machine language seems quite simple on the surface, given that there \n",
      "are only zeros and ones, but its syntax is even more complex\n",
      "and far more intricate than Python.  So very few programmers ever write\n",
      "machine language.  Instead we build various translators to allow\n",
      "programmers to write in high-level languages like Python or JavaScript\n",
      "and these translators convert the programs to machine language for actual\n",
      "execution by the CPU.\n",
      "\n",
      "Since machine language is tied to the computer hardware, machine language\n",
      "is not portable across different types of hardware.  Programs written in \n",
      "high-level languages can be moved between different computers by using a \n",
      "different interpreter on the new machine or recompiling the code to create\n",
      "a machine language version of the program for the new machine.\n",
      "\n",
      "These programming language translators fall into two general categories:\n",
      "(one) interpreters and (two) compilers.\n",
      "\n",
      "An interpreter reads the source code of the program as written by the\n",
      "programmer, parses the source code, and interprets the instructions on the fly.\n",
      "Python is an interpreter and when we are running Python interactively, \n",
      "we can type a line of Python (a sentence) and Python processes it immediately\n",
      "and is ready for us to type another line of Python.   \n",
      "\n",
      "Some of the lines of Python tell Python that you want it to remember some \n",
      "value for later.   We need to pick a name for that value to be remembered and\n",
      "we can use that symbolic name to retrieve the value later.  We use the \n",
      "term variable to refer to the labels we use to refer to this stored data.\n",
      "\n",
      "In this example, we ask Python to remember the value six and use the label x\n",
      "so we can retrieve the value later.   We verify that Python has actually remembered\n",
      "the value using x and multiply\n",
      "it by seven and put the newly computed value in y.  Then we ask Python to print out\n",
      "the value currently in y.\n",
      "\n",
      "Even though we are typing these commands into Python one line at a time, Python\n",
      "is treating them as an ordered sequence of statements with later statements able\n",
      "to retrieve data created in earlier statements.   We are writing our first \n",
      "simple paragraph with four sentences in a logical and meaningful order.\n",
      "\n",
      "It is the nature of an interpreter to be able to have an interactive conversation\n",
      "as shown above.  A compiler needs to be handed the entire program in a file, and then \n",
      "it runs a process to translate the high-level source code into machine language\n",
      "and then the compiler puts the resulting machine language into a file for later\n",
      "execution.\n",
      "\n",
      "If you have a Windows system, often these executable machine language programs have a\n",
      "suffix of .exe or .dll which stand for executable and dynamic link\n",
      "library respectively.  In Linux and Macintosh, there is no suffix that uniquely marks\n",
      "a file as executable.\n",
      "\n",
      "If you were to open an executable file in a text editor, it would look \n",
      "completely crazy and be unreadable:\n",
      "\n",
      "It is not easy to read or write machine language, so it is nice that we have\n",
      "compilers that allow us to write in high-level\n",
      "languages like Python or C.\n",
      "\n",
      "Now at this point in our discussion of compilers and interpreters, you should \n",
      "be wondering a bit about the Python interpreter itself.  What language is \n",
      "it written in?  Is it written in a compiled language?  When we type\n",
      "python, what exactly is happening?\n",
      "\n",
      "The Python interpreter is written in a high-level language called C.  \n",
      "You can look at the actual source code for the Python interpreter by\n",
      "going to www.python.org and working your way to their source code.\n",
      "So Python is a program itself and it is compiled into machine code.\n",
      "When you installed Python on your computer (or the vendor installed it),\n",
      "you copied a machine-code copy of the translated Python program onto your\n",
      "system.   In Windows, the executable machine code for Python itself is likely\n",
      "in a file.\n",
      "\n",
      "That is more than you really need to know to be a Python programmer, but\n",
      "sometimes it pays to answer those little nagging questions right at \n",
      "the beginning.\n",
      "\n",
      "Writing a program\n",
      "\n",
      "Typing commands into the Python interpreter is a great way to experiment \n",
      "with Python's features, but it is not recommended for solving more complex problems.\n",
      "\n",
      "When we want to write a program, \n",
      "we use a text editor to write the Python instructions into a file,\n",
      "which is called a script.  By\n",
      "convention, Python scripts have names that end with .py.\n",
      "\n",
      "script\n",
      "\n",
      "To execute the script, you have to tell the Python interpreter \n",
      "the name of the file.  In a Unix or Windows command window, \n",
      "you would type python hello.py as follows:\n",
      "\n",
      "We call the Python interpreter and tell it to read its source code from\n",
      "the file hello.py instead of prompting us for lines of Python code\n",
      "interactively.\n",
      "\n",
      "You will notice that there was no need to have quit() at the end of\n",
      "the Python program in the file.   When Python is reading your source code\n",
      "from a file, it knows to stop when it reaches the end of the file.\n",
      "\n",
      "What is a program?\n",
      "\n",
      "The definition of a program at its most basic is a sequence\n",
      "of Python statements that have been crafted to do something.\n",
      "Even our simple hello.py script is a program.  It is a one-line\n",
      "program and is not particularly useful, but in the strictest definition,\n",
      "it is a Python program.\n",
      "\n",
      "It might be easiest to understand what a program is by thinking about a problem \n",
      "that a program might be built to solve, and then looking at a program\n",
      "that would solve that problem.\n",
      "\n",
      "Lets say you are doing Social Computing research on Facebook posts and \n",
      "you are interested in the most frequently used word in a series of posts.\n",
      "You could print out the stream of Facebook posts and pore over the text\n",
      "looking for the most common word, but that would take a long time and be very \n",
      "mistake prone.  You would be smart to write a Python program to handle the\n",
      "task quickly and accurately so you can spend the weekend doing something \n",
      "fun.\n",
      "\n",
      "For example, look at the following text about a clown and a car.  Look at the \n",
      "text and figure out the most common word and how many times it occurs.\n",
      "\n",
      "Then imagine that you are doing this task looking at millions of lines of \n",
      "text.  Frankly it would be quicker for you to learn Python and write a \n",
      "Python program to count the words than it would be to manually \n",
      "scan the words.\n",
      "\n",
      "The even better news is that I already came up with a simple program to \n",
      "find the most common word in a text file.  I wrote it,\n",
      "tested it, and now I am giving it to you to use so you can save some time.\n",
      "\n",
      "You don't even need to know Python to use this program.  You will need to get through \n",
      "Chapter ten of this book to fully understand the awesome Python techniques that were\n",
      "used to make the program.  You are the end user, you simply use the program and marvel\n",
      "at its cleverness and how it saved you so much manual effort.\n",
      "You simply type the code \n",
      "into a file called words.py and run it or you download the source \n",
      "code from http://www.py4e.com/code/ and run it.\n",
      "\n",
      "This is a good example of how Python and the Python language are acting as an intermediary\n",
      "between you (the end user) and me (the programmer).  Python is a way for us to exchange useful\n",
      "instruction sequences (i.e., programs) in a common language that can be used by anyone who \n",
      "installs Python on their computer.  So neither of us are talking to Python,\n",
      "instead we are communicating with each other through Python.\n",
      "\n",
      "The building blocks of programs\n",
      "\n",
      "In the next few chapters, we will learn more about the vocabulary, sentence structure,\n",
      "paragraph structure, and story structure of Python.  We will learn about the powerful\n",
      "capabilities of Python and how to compose those capabilities together to create useful\n",
      "programs.\n",
      "\n",
      "There are some low-level conceptual patterns that we use to construct programs.  These\n",
      "constructs are not just for Python programs, they are part of every programming language\n",
      "from machine language up to the high-level languages.\n",
      "\n",
      "description\n",
      "\n",
      "Get data from the outside world.  This might be \n",
      "reading data from a file, or even some kind of sensor like \n",
      "a microphone or GPS.  In our initial programs, our input will come from the user\n",
      "typing data on the keyboard.\n",
      "\n",
      "Display the results of the program on a screen\n",
      "or store them in a file or perhaps write them to a device like a\n",
      "speaker to play music or speak text.\n",
      "\n",
      "Perform statements one after\n",
      "another in the order they are encountered in the script.\n",
      "\n",
      "Check for certain conditions and\n",
      "then execute or skip a sequence of statements.\n",
      "\n",
      "Perform some set of statements \n",
      "repeatedly, usually with\n",
      "some variation.\n",
      "\n",
      "Write a set of instructions once and give them a name\n",
      "and then reuse those instructions as needed throughout your program.\n",
      "\n",
      "description\n",
      "\n",
      "It sounds almost too simple to be true, and of course it is never\n",
      "so simple.  It is like saying that walking is simply\n",
      "putting one foot in front of the other.  The art \n",
      "of writing a program is composing and weaving these\n",
      "basic elements together many times over to produce something\n",
      "that is useful to its users.\n",
      "\n",
      "The word counting program above directly uses all of \n",
      "these patterns except for one.\n",
      "\n",
      "What could possibly go wrong?\n",
      "\n",
      "As we saw in our earliest conversations with Python, we must\n",
      "communicate very precisely when we write Python code.  The smallest\n",
      "deviation or mistake will cause Python to give up looking at your\n",
      "program.\n",
      "\n",
      "Beginning programmers often take the fact that Python leaves no\n",
      "room for errors as evidence that Python is mean, hateful, and cruel.\n",
      "While Python seems to like everyone else, Python knows them \n",
      "personally and holds a grudge against them.  Because of this grudge,\n",
      "Python takes our perfectly written programs and rejects them as \n",
      "unfit just to torment us.\n",
      "\n",
      "There is little to be gained by arguing with Python.  It is just a tool.\n",
      "It has no emotions and it is happy and ready to serve you whenever you\n",
      "need it.  Its error messages sound harsh, but they are just Python's\n",
      "call for help.  It has looked at what you typed, and it simply cannot\n",
      "understand what you have entered.\n",
      "\n",
      "Python is much more like a dog, loving you unconditionally, having a few\n",
      "key words that it understands, looking you with a sweet look on its\n",
      "face (>>>), and waiting for you to say something it understands.\n",
      "When Python says SyntaxError: invalid syntax, it is simply wagging\n",
      "its tail and saying, You seemed to say something but I just don't\n",
      "understand what you meant, but please keep talking to me (>>>).\n",
      "\n",
      "As your programs become increasingly sophisticated, you will encounter three \n",
      "general types of errors:\n",
      "\n",
      "description\n",
      "\n",
      "These are the first errors you will make and the easiest\n",
      "to fix.  A syntax error means that you have violated the grammar rules of Python.\n",
      "Python does its best to point right at the line and character where \n",
      "it noticed it was confused.  The only tricky bit of syntax errors is that sometimes\n",
      "the mistake that needs fixing is actually earlier in the program than where Python\n",
      "noticed it was confused.  So the line and character that Python indicates in \n",
      "a syntax error may just be a starting point for your investigation.\n",
      "\n",
      "A logic error is when your program has good syntax but there is a mistake \n",
      "in the order of the statements or perhaps a mistake in how the statements relate to one another.\n",
      "A good example of a logic error might be, take a drink from your water bottle, put it \n",
      "in your backpack, walk to the library, and then put the top back on the bottle.\n",
      "\n",
      "A semantic error is when your description of the steps to take \n",
      "is syntactically perfect and in the right order, but there is simply a mistake in \n",
      "the program.  The program is perfectly correct but it does not do what\n",
      "you intended for it to do. A simple example would\n",
      "be if you were giving a person directions to a restaurant and said, ...when you reach\n",
      "the intersection with the gas station, turn left and go one mile and the restaurant\n",
      "is a red building on your left.  Your friend is very late and calls you to tell you that\n",
      "they are on a farm and walking around behind a barn, with no sign of a restaurant.  \n",
      "Then you say did you turn left or right at the gas station? and \n",
      "they say, I followed your directions perfectly, I have \n",
      "them written down, it says turn left and go one mile at the gas station.  Then you say,\n",
      "I am very sorry, because while my instructions were syntactically correct, they \n",
      "sadly contained a small but undetected semantic error.. \n",
      "\n",
      "description\n",
      "\n",
      "Again in all three types of errors, Python is merely trying its hardest to \n",
      "do exactly what you have asked.\n",
      "\n",
      "The learning journey\n",
      "\n",
      "As you progress through the rest of the book, don't be afraid if the concepts \n",
      "don't seem to fit together well the first time.  When you were learning to speak, \n",
      "it was not a problem  for your first few years that you just made cute gurgling noises.\n",
      "And it was OK if it took six months for you to move from simple vocabulary to \n",
      "simple sentences and took five or six more years to move from sentences to paragraphs, and a few more years to be able to write an interesting complete short story on your own.\n",
      "\n",
      "We want you to learn Python much more rapidly, so we teach it all at the same time\n",
      "over the next few chapters.  \n",
      "But it is like learning a new language that takes time to absorb and understand\n",
      "before it feels natural.\n",
      "That leads to some confusion as we visit and revisit\n",
      "topics to try to get you to see the big picture while we are defining the tiny\n",
      "fragments that make up that big picture.  While the book is written linearly, and\n",
      "if you are taking a course it will progress in a linear fashion, don't hesitate\n",
      "to be very nonlinear in how you approach the material.  Look forwards and backwards\n",
      "and read with a light touch.  By skimming more advanced material without \n",
      "fully understanding the details, you can get a better understanding of the why? \n",
      "of programming.  By reviewing previous material and even redoing earlier \n",
      "exercises, you will realize that you actually learned a lot of material even \n",
      "if the material you are currently staring at seems a bit impenetrable.\n",
      "\n",
      "Usually when you are learning your first programming language, there are a few\n",
      "wonderful Ah Hah! moments where you can look up from pounding away at some rock\n",
      "with a hammer and chisel and step away and see that you are indeed building \n",
      "a beautiful sculpture.\n",
      "\n",
      "If something seems particularly hard, there is usually no value in staying up all \n",
      "night and staring at it.   Take a break, take a nap, have a snack, explain what you \n",
      "are having a problem with to someone (or perhaps your dog), and then come back to it with\n",
      "fresh eyes.  I assure you that once you learn the programming concepts in the book\n",
      "you will look back and see that it was all really easy and elegant and it simply \n",
      "took you a bit of time to absorb it.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Read and save text data to variable (text)\n",
    "f = open(\"text/intro.txt\", \"r\")\n",
    "text = f.read()\n",
    "print(text)\n",
    "f.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "# Data cleaning using a function\n",
    "def data_cleaning(text):\n",
    "    '''\n",
    "    This function strips input of whitespaces, converts it into lower case and removes all punctuation from input.\n",
    "    In addition, it returns the text as a list by splitting on \"spaces\".\n",
    "    \n",
    "    Input: str\n",
    "    Output: list\n",
    "    '''\n",
    "    text = text.strip() # Remove whitespaces from the text (white spaces are \\n, \\t)\n",
    "    text = text.lower() # Converts the text to lower-case\n",
    "\n",
    "    # Remove punctuations from text and save to the newtext variable\n",
    "    punctuations = '''!()-[]{};:'\"\\,<>./?@#$%^&*_~''' # provide the punctuations\n",
    "    newtext = \"\"\n",
    "    for char in text: # iterate over each character in the text and if it is a punctuation, remove it.\n",
    "        if char in punctuations: continue\n",
    "        else: newtext += char\n",
    "    \n",
    "    text_list = newtext.split() # converts the text to a list by splitting on the space between 2 words\n",
    "    return text_list\n",
    "\n",
    "text_list = data_cleaning(text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['why',\n",
       " 'should',\n",
       " 'you',\n",
       " 'learn',\n",
       " 'to',\n",
       " 'write',\n",
       " 'programs',\n",
       " 'writing',\n",
       " 'programs',\n",
       " 'or']"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The output from the previous function returns a list of strings that are all in lower-case\n",
    "text_list[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a dictionary with words as key and values as the number of times the word appears in the list\n",
    "word_count = {}\n",
    "\n",
    "for word in text_list: # Loop through the words in the list, and keep count of each word by creating a dictionary\n",
    "    # If appear before, add 1\n",
    "    if word in word_count.keys(): word_count[word] += 1\n",
    "    # Else initiate with value 1\n",
    "    else: word_count[word] = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('the', 249),\n",
       " ('to', 205),\n",
       " ('a', 172),\n",
       " ('and', 164),\n",
       " ('you', 152),\n",
       " ('is', 114),\n",
       " ('of', 103),\n",
       " ('python', 103),\n",
       " ('in', 81),\n",
       " ('it', 76),\n",
       " ('that', 73),\n",
       " ('we', 67),\n",
       " ('for', 52),\n",
       " ('are', 43),\n",
       " ('be', 42),\n",
       " ('program', 41),\n",
       " ('language', 41),\n",
       " ('your', 39),\n",
       " ('with', 37),\n",
       " ('will', 34),\n",
       " ('as', 31),\n",
       " ('at', 31),\n",
       " ('or', 29),\n",
       " ('have', 27),\n",
       " ('programs', 26),\n",
       " ('our', 25),\n",
       " ('when', 25),\n",
       " ('this', 24),\n",
       " ('but', 24),\n",
       " ('write', 23),\n",
       " ('on', 23),\n",
       " ('can', 22),\n",
       " ('what', 21),\n",
       " ('computer', 21),\n",
       " ('very', 20),\n",
       " ('how', 20),\n",
       " ('programming', 19),\n",
       " ('these', 19),\n",
       " ('use', 19),\n",
       " ('words', 18),\n",
       " ('not', 18),\n",
       " ('more', 18),\n",
       " ('from', 17),\n",
       " ('so', 17),\n",
       " ('machine', 17),\n",
       " ('need', 16),\n",
       " ('problem', 15),\n",
       " ('do', 15),\n",
       " ('if', 15),\n",
       " ('word', 15)]"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get top 50 words that appears in the text\n",
    "sorted_word_count[:50]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "import wordcloud\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "wc = wordcloud.WordCloud(text)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "At the end of this lesson, we have discussed the following:\n",
    "\n",
    "1. Different types of data types\n",
    "2. Different types of data structures\n",
    "3. How to use these data types and structures to store data we are interested in"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}