inference exercise update

Author: erodner

notebooks/08/exercise_gpt_inference.ipynb

@@ -6,11 +6,11 @@
    "source": [
     "<img style=\"float: right;\" src=\"../../assets/htwlogo.svg\">\n",
     "\n",
-    "# Exercise: Sentinment analysis with LLMs\n",
+    "# Exercise: GPT inference and XMAS-Fun with LLMs\n",
     "\n",
     "**Author**: _Erik Rodner_ <br>\n",
     "\n",
-    "In this exercise, we will use LLMs for text categorization and in particular for sentiment analysis, i.e. trying to classify text \n",
+    "In this exercise, we will use LLMs for text categorization such as sentiment analysis, i.e. trying to classify text \n",
     "into certain emotional categories: positive, negative, neutral.\n",
     "\n",
     "**What kind of applications can be built with this technology? (good or bad)?**\n"
@@ -65,11 +65,103 @@
     "            {\"role\": \"system\", \"content\": system_prompt},\n",
     "            {\"role\": \"user\", \"content\": user_prompt}\n",
     "        ],\n",
+    "        max_tokens=512,\n",
     "        temperature=temperature)\n",
     "    pure_text = response.choices[0].message.content.strip()\n",
     "    return pure_text\n",
     "        "
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Temperature and first steps\n",
+    "\n",
+    "The function ``prompt_gpt`` is super-simple. Why is it reasonable to split user and system prompt?\n",
+    "What is the effect of the temperature parameter?"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "prompt_gpt(\"Be a helpful assistant and super friendly\", \"Hi\", temperature=1.0)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Text categorization"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "(Inspired by Nick Diakopoulos - https://generative-ai-newsroom.com/using-gpt-3-for-news-tasks-82ac9cd3cc20)\n",
+    "\n",
+    "Generative models like GPT3 can be used for analytic tasks like classifying a text into different categories.\n",
+    "Assume you're an engaged journalist doing a follow up story on recent coverage around the need to replace gas cooktops, such as https://www.washingtonpost.com/climate-solutions/2023/02/04/how-to-use-gas-stove-safely/. You want to run a quick survey with people who have switched to induction stoves and ask them why they switched. To find these people you decide to scan the more than 2,700 comments made on the last story.\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# For purposes of the demo we just scan 6 comments, but this could easily be scaled up.\n",
+    "sample_comments = [\"We were considering converting to gas when replacing our stove. We went with induction instead. It’s amazing. Cooks food/boils water faster. Is 90% efficient vs. gas 30% efficacy. Equipped with an air fryer function. We had to purchase new cook wear that is required because of the magnification properties on the burners. You need pots that are made to adhere so to speak to burners while heating. But it’s worth it! Love our new pots and get dinner ready faster. Now if it would only clean -up.\",\n",
+    "     \"When we first moved into our house (we weren't the first owner) it had a conventional electric cooktop. Conventional electric stoves just plain suck for a variety of reasons. But we were lucky, because the builder of our house included both a gas pipe and a 220 volt outlet in the island where the cooktop sat. We considered replacing the electric cooktop with an induction one but didn't want to spend the time, effort and money to replace the majority of our cookware with pots and pans that were magnetic. So in our case it was easy work to yank the electric cooktop and replace it with a gas one. It took the appliance store guys about 30 minutes to make the switch. A couple of years ago we remodeled our kitchen, and in the big scheme of cost and inconvenience, replacing our cookware was pretty trivial. So we finally got our induction cooktop. It combines the best features of electric (more efficient than gas, no air pollution) with the best features of gas (easy and quick to regulate temperature), and includes advantages neither one has (doesn't heat up the kitchen, cooktop doesn't stay burning hot for long when cooking is done), and it heats things faster than either electric or gas. Without a doubt induction cooking is the way of the future. And I'm sure prices will come down as more home owners who can't or don't want to spend on high-end appliances start to demand them.\",\n",
+    "     \"We have a gas stove, and like it. Our range hood is vented to the outside. We will be getting an induction range within 2 to 4 years. We are older, and it is a safer option since there will not be an open flame. We took this decision before the \\\"controversy\\\" arose. Gas stoves are better than standard electric in some areas (where gas is cheaper), and work very well. It is also true that the products of combustion are a problem. Houseplants help, opening windows helps, range hoods help. Nothing works perfectly.\",\n",
+    "     \"Unless you have a small home with little ventilation, the risk from gas stove emissions is quite small. After cooking on a induction stove top for a year in a rental we opted for gas when we put a new stove in our own house. It is well ventilated and this old house provides a lot of unplanned ventilation too.\",\n",
+    "     \"And just so we're all clear: the suitability of a pan for induction cooking is not \\\"adhesion\\\". Testing with a magnet is a quick way to determine whether the pan has enough ferrous metal to respond to the magnetic field generate by the element.\",\n",
+    "     \"The gas stove hullabaloo is absurd. The matter of the plastic straws is infinitely more serious because they stay out there essentially forever and affect other creatures. I have no problem banning them.\"\n",
+    "    ]\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Write code that analyzes above comments and classifies them into being relevant or not relevant for the use case."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "## YOUR CODE"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Your creativity\n",
+    "\n",
+    "You have the whole lecture time to:\n",
+    "1. Write an algorithm that generates personalized christmas cards for the whole family.\n",
+    "2. Writes a list of new years resolutions.\n",
+    "3. A small bot that answers emails for you.\n",
+    "4. A small bot that asks you ML questions and evaluates your answers.\n",
+    "5. Summarizes lecture material at a given URL \n",
+    "6. ... (whatever you like)\n",
+    "\n",
+    "Only constraint: do not waste my OpenAI budget ;-P"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": []
   }
  ],
  "metadata": {