intro_to_ai_ethics/03-identifying-bias-in-ai.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7bb787cb",
   "metadata": {
    "papermill": {
     "duration": 0.020088,
     "end_time": "2022-03-06T21:19:36.856151",
     "exception": false,
     "start_time": "2022-03-06T21:19:36.836063",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "**This notebook is an exercise in the [AI Ethics](https://www.kaggle.com/learn/ai-ethics) course.  You can reference the tutorial at [this link](https://www.kaggle.com/alexisbcook/identifying-bias-in-ai).**\n",
    "\n",
    "---\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a0a0457",
   "metadata": {
    "papermill": {
     "duration": 0.017665,
     "end_time": "2022-03-06T21:19:36.892907",
     "exception": false,
     "start_time": "2022-03-06T21:19:36.875242",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "In the tutorial, you learned about six different types of bias.  In this exercise, you'll train a model with **real data** and get practice with identifying bias.  Don't worry if you're new to coding: you'll still be able to complete the exercise!\n",
    "\n",
    "# Introduction\n",
    "\n",
    "At the end of 2017, the [Civil Comments](https://medium.com/@aja_15265/saying-goodbye-to-civil-comments-41859d3a2b1d) platform shut down and released their ~2 million public comments in a lasting open archive. Jigsaw sponsored this effort and helped to comprehensively annotate the data.  In 2019, Kaggle held the [Jigsaw Unintended Bias in Toxicity Classification](https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification/overview) competition so that data scientists worldwide could work together to investigate ways to mitigate bias.\n",
    "\n",
    "The code cell below loads some of the data from the competition.  We'll work with thousands of comments, where each comment is labeled as either \"toxic\" or \"not toxic\".\n",
    "\n",
    "Begin by running the next code cell.  \n",
    "- Clicking inside the code cell.\n",
    "- Click on the triangle (in the shape of a \"Play button\") that appears to the left of the code cell.\n",
    "\n",
    "The code will run for approximately 30 seconds.  When it finishes, you should see as output a message saying that the data was successfully loaded, along with two examples of comments: one is toxic, and the other is not."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7460cd34",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-06T21:19:36.938375Z",
     "iopub.status.busy": "2022-03-06T21:19:36.937694Z",
     "iopub.status.idle": "2022-03-06T21:19:47.512723Z",
     "shell.execute_reply": "2022-03-06T21:19:47.511818Z",
     "shell.execute_reply.started": "2022-03-06T20:55:06.009124Z"
    },
    "papermill": {
     "duration": 10.602003,
     "end_time": "2022-03-06T21:19:47.512973",
     "exception": false,
     "start_time": "2022-03-06T21:19:36.910970",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data successfully loaded!\n",
      "\n",
      "Sample toxic comment: Too dumb to even answer.\n",
      "Sample not-toxic comment: No they aren't.\n"
     ]
    }
   ],
   "source": [
    "# Set up feedback system\n",
    "from learntools.core import binder\n",
    "binder.bind(globals())\n",
    "from learntools.ethics.ex3 import *\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "# Get the same results each time\n",
    "np.random.seed(0)\n",
    "\n",
    "# Load the training data\n",
    "data = pd.read_csv(\"../input/jigsaw-snapshot/data.csv\")\n",
    "comments = data[\"comment_text\"]\n",
    "target = (data[\"target\"]>0.7).astype(int)\n",
    "\n",
    "# Break into training and test sets\n",
    "comments_train, comments_test, y_train, y_test = train_test_split(comments, target, test_size=0.30, stratify=target)\n",
    "\n",
    "# Get vocabulary from training data\n",
    "vectorizer = CountVectorizer()\n",
    "vectorizer.fit(comments_train)\n",
    "\n",
    "# Get word counts for training and test sets\n",
    "X_train = vectorizer.transform(comments_train)\n",
    "X_test = vectorizer.transform(comments_test)\n",
    "\n",
    "# Preview the dataset\n",
    "print(\"Data successfully loaded!\\n\")\n",
    "print(\"Sample toxic comment:\", comments_train.iloc[22])\n",
    "print(\"Sample not-toxic comment:\", comments_train.iloc[17])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61dc0432",
   "metadata": {
    "papermill": {
     "duration": 0.018462,
     "end_time": "2022-03-06T21:19:47.551338",
     "exception": false,
     "start_time": "2022-03-06T21:19:47.532876",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Run the next code cell without changes to use the data to train a simple model.  The output shows the accuracy of the model on some test data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "24860762",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-06T21:19:47.596877Z",
     "iopub.status.busy": "2022-03-06T21:19:47.596185Z",
     "iopub.status.idle": "2022-03-06T21:20:14.104731Z",
     "shell.execute_reply": "2022-03-06T21:20:14.105825Z",
     "shell.execute_reply.started": "2022-03-06T20:59:16.067140Z"
    },
    "papermill": {
     "duration": 26.535897,
     "end_time": "2022-03-06T21:20:14.106133",
     "exception": false,
     "start_time": "2022-03-06T21:19:47.570236",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.9304755967877966\n"
     ]
    }
   ],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "# Train a model and evaluate performance on test dataset\n",
    "classifier = LogisticRegression(max_iter=2000)\n",
    "classifier.fit(X_train, y_train)\n",
    "score = classifier.score(X_test, y_test)\n",
    "print(\"Accuracy:\", score)\n",
    "\n",
    "# Function to classify any string\n",
    "def classify_string(string, investigate=False):\n",
    "    prediction = classifier.predict(vectorizer.transform([string]))[0]\n",
    "    if prediction == 0:\n",
    "        print(\"NOT TOXIC:\", string)\n",
    "    else:\n",
    "        print(\"TOXIC:\", string)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7877d062",
   "metadata": {
    "papermill": {
     "duration": 0.024243,
     "end_time": "2022-03-06T21:20:14.193031",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.168788",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Roughly 93% of the comments in the test data are classified correctly!\n",
    "\n",
    "# 1) Try out the model\n",
    "\n",
    "You'll use the next code cell to write your own comments and supply them to the model: does the model classify them as toxic?  \n",
    "\n",
    "1. Begin by running the code cell as-is to classify the comment `\"I love apples\"`.  You should see that was classified as \"NOT TOXIC\".\n",
    "\n",
    "2. Then, try out another comment: `\"Apples are stupid\"`.  To do this, change only `\"I love apples\"` and leaving the rest of the code as-is.  Make sure that your comment is enclosed in quotes, as below.\n",
    "```python\n",
    "my_comment = \"Apples are stupid\"\n",
    "```\n",
    "3. Try out several comments (not necessarily about apples!), to see how the model performs: does it perform as suspected?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "b4dee9a3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-06T21:20:14.236204Z",
     "iopub.status.busy": "2022-03-06T21:20:14.233770Z",
     "iopub.status.idle": "2022-03-06T21:20:14.244554Z",
     "shell.execute_reply": "2022-03-06T21:20:14.245130Z",
     "shell.execute_reply.started": "2022-03-06T21:03:54.045837Z"
    },
    "papermill": {
     "duration": 0.033359,
     "end_time": "2022-03-06T21:20:14.245321",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.211962",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "NOT TOXIC: I love apples\n"
     ]
    },
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.14285714285714285, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"1_TryOut\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\"></span>"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Comment to pass through the model\n",
    "my_comment = \"I love apples\"\n",
    "\n",
    "# Do not change the code below\n",
    "classify_string(my_comment)\n",
    "q_1.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "528b8ed9",
   "metadata": {
    "papermill": {
     "duration": 0.019959,
     "end_time": "2022-03-06T21:20:14.285762",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.265803",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "Once you're done with testing comments, we'll move on to understand how the model makes decisions.  Run the next code cell without changes.\n",
    "\n",
    "The model assigns each of roughly 58,000 words a coefficient, where higher coefficients denote words that the model thinks are more toxic.  The code cell outputs the ten words that are considered most toxic, along with their coefficients.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d81372a7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-06T21:20:14.331168Z",
     "iopub.status.busy": "2022-03-06T21:20:14.330559Z",
     "iopub.status.idle": "2022-03-06T21:20:14.398177Z",
     "shell.execute_reply": "2022-03-06T21:20:14.397649Z",
     "shell.execute_reply.started": "2022-03-06T21:05:55.397364Z"
    },
    "papermill": {
     "duration": 0.092278,
     "end_time": "2022-03-06T21:20:14.398318",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.306040",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>word</th>\n",
       "      <th>coeff</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>20745</th>\n",
       "      <td>fools</td>\n",
       "      <td>6.278428</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>34211</th>\n",
       "      <td>moron</td>\n",
       "      <td>6.332332</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16844</th>\n",
       "      <td>dumb</td>\n",
       "      <td>6.359258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12907</th>\n",
       "      <td>crap</td>\n",
       "      <td>6.489638</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>38317</th>\n",
       "      <td>pathetic</td>\n",
       "      <td>6.554183</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25850</th>\n",
       "      <td>idiotic</td>\n",
       "      <td>7.004782</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49802</th>\n",
       "      <td>stupidity</td>\n",
       "      <td>7.552952</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25858</th>\n",
       "      <td>idiots</td>\n",
       "      <td>8.601077</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25847</th>\n",
       "      <td>idiot</td>\n",
       "      <td>8.604506</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>49789</th>\n",
       "      <td>stupid</td>\n",
       "      <td>9.277706</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            word     coeff\n",
       "20745      fools  6.278428\n",
       "34211      moron  6.332332\n",
       "16844       dumb  6.359258\n",
       "12907       crap  6.489638\n",
       "38317   pathetic  6.554183\n",
       "25850    idiotic  7.004782\n",
       "49802  stupidity  7.552952\n",
       "25858     idiots  8.601077\n",
       "25847      idiot  8.604506\n",
       "49789     stupid  9.277706"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "coefficients = pd.DataFrame({\"word\": sorted(list(vectorizer.vocabulary_.keys())), \"coeff\": classifier.coef_[0]})\n",
    "coefficients.sort_values(by=['coeff']).tail(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7259ae3",
   "metadata": {
    "papermill": {
     "duration": 0.020216,
     "end_time": "2022-03-06T21:20:14.439282",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.419066",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 2) Most toxic words\n",
    "\n",
    "Take a look at the most toxic words from the code cell above.  Are you surprised to see any of them?  Are there any words that seem like they should not be in the list?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "246692a3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-06T21:20:14.485473Z",
     "iopub.status.busy": "2022-03-06T21:20:14.484864Z",
     "iopub.status.idle": "2022-03-06T21:20:14.489453Z",
     "shell.execute_reply": "2022-03-06T21:20:14.488833Z",
     "shell.execute_reply.started": "2022-03-06T21:06:27.596694Z"
    },
    "papermill": {
     "duration": 0.02997,
     "end_time": "2022-03-06T21:20:14.489581",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.459611",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.14285714285714285, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"2_MostToxic\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Solution:</span> None of the words are surprising.  They are all clearly toxic."
      ],
      "text/plain": [
       "Solution: None of the words are surprising.  They are all clearly toxic."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Check your answer (Run this code cell to get credit!)\n",
    "q_2.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4725475",
   "metadata": {
    "papermill": {
     "duration": 0.021144,
     "end_time": "2022-03-06T21:20:14.532229",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.511085",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 3) A closer investigation\n",
    "\n",
    "We'll take a closer look at how the model classifies comments.\n",
    "1. Begin by running the code cell as-is to classify the comment `\"I have a christian friend\"`.  You should see that was classified as \"NOT TOXIC\".  In addition, you can see what scores were assigned to some of the individual words.  Note that all words in the comment likely won't appear.\n",
    "2. Next, try out another comment: `\"I have a muslim friend\"`.  To do this, change only `\"I have a christian friend\"` and leave the rest of the code as-is. Make sure that your comment is enclosed in quotes, as below.\n",
    "```python\n",
    "new_comment = \"I have a muslim friend\"\n",
    "```\n",
    "3. Try out two more comments: `\"I have a white friend\"` and `\"I have a black friend\"` (in each case, do not add punctuation to the comment).\n",
    "4. Feel free to try out more comments, to see how the model classifies them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "4da35b2e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-06T21:20:14.591778Z",
     "iopub.status.busy": "2022-03-06T21:20:14.591165Z",
     "iopub.status.idle": "2022-03-06T21:20:14.598675Z",
     "shell.execute_reply": "2022-03-06T21:20:14.598162Z",
     "shell.execute_reply.started": "2022-03-06T21:10:52.094584Z"
    },
    "papermill": {
     "duration": 0.045141,
     "end_time": "2022-03-06T21:20:14.598813",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.553672",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TOXIC: I have a black friend\n"
     ]
    },
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.14285714285714285, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"3_CloserInvestigation\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\"></span>"
      ],
      "text/plain": []
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Set the value of new_comment\n",
    "new_comment = \"I have a black friend\"\n",
    "\n",
    "# Do not change the code below\n",
    "classify_string(new_comment)\n",
    "coefficients[coefficients.word.isin(new_comment.split())]\n",
    "q_3.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0269f526",
   "metadata": {
    "papermill": {
     "duration": 0.023165,
     "end_time": "2022-03-06T21:20:14.645401",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.622236",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 4) Identify bias\n",
    "\n",
    "Do you see any signs of potential bias in the model?  In the code cell above,\n",
    "- How did the model classify `\"I have a christian friend\"` and `\"I have a muslim friend\"`?  \n",
    "- How did it classify `\"I have a white friend\"` and `\"I have a black friend\"`?    \n",
    "\n",
    "Once you have an answer, run the next code cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a9d738dd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-06T21:20:14.696983Z",
     "iopub.status.busy": "2022-03-06T21:20:14.696314Z",
     "iopub.status.idle": "2022-03-06T21:20:14.701269Z",
     "shell.execute_reply": "2022-03-06T21:20:14.700615Z",
     "shell.execute_reply.started": "2022-03-06T21:11:19.322176Z"
    },
    "papermill": {
     "duration": 0.032685,
     "end_time": "2022-03-06T21:20:14.701398",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.668713",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.14285714285714285, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"4_IdentifyBias\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Solution:</span> The comment `I have a muslim friend` was marked as toxic, whereas `I have a christian friend` was not.  Likewise, `I have a black friend` was marked as toxic, whereas `I have a white friend` was not.  None of these comments should be marked as toxic, but the model seems to erroneously associate some identities as toxic.  This is a sign of bias: the model seems biased in favor of `christian` and against `muslim`, and it seems biased in favor of `white` and against `black`."
      ],
      "text/plain": [
       "Solution: The comment `I have a muslim friend` was marked as toxic, whereas `I have a christian friend` was not.  Likewise, `I have a black friend` was marked as toxic, whereas `I have a white friend` was not.  None of these comments should be marked as toxic, but the model seems to erroneously associate some identities as toxic.  This is a sign of bias: the model seems biased in favor of `christian` and against `muslim`, and it seems biased in favor of `white` and against `black`."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Check your answer (Run this code cell to get credit!)\n",
    "q_4.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f96be4d5",
   "metadata": {
    "papermill": {
     "duration": 0.024204,
     "end_time": "2022-03-06T21:20:14.750177",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.725973",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 5) Test your understanding\n",
    "\n",
    "We'll step away from the Jigsaw competition data and consider a similar (but hypothetical!) scenario where you're working with a dataset of online comments to train a model to classify comments as toxic.\n",
    "\n",
    "You notice that comments that refer to Islam are more likely to be toxic than comments that refer to other religions, because the online community is islamophobic.  What type of bias can this introduce to your model?\n",
    "\n",
    "Once you have answered the question, run the next code cell to see the official answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "35a3ef1a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-06T21:20:14.803893Z",
     "iopub.status.busy": "2022-03-06T21:20:14.803294Z",
     "iopub.status.idle": "2022-03-06T21:20:14.807824Z",
     "shell.execute_reply": "2022-03-06T21:20:14.807222Z",
     "shell.execute_reply.started": "2022-03-06T21:13:38.632707Z"
    },
    "papermill": {
     "duration": 0.033242,
     "end_time": "2022-03-06T21:20:14.807973",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.774731",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.14285714285714285, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"5_TestUnderstanding\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Solution:</span> Comments that refer to Islam are more likely to be classified as toxic, because of a flawed state of the online community where the data was collected.  This can introduce **historical bias**."
      ],
      "text/plain": [
       "Solution: Comments that refer to Islam are more likely to be classified as toxic, because of a flawed state of the online community where the data was collected.  This can introduce **historical bias**."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Check your answer (Run this code cell to get credit!)\n",
    "q_5.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bfda111",
   "metadata": {
    "papermill": {
     "duration": 0.025284,
     "end_time": "2022-03-06T21:20:14.859090",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.833806",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 6) Test your understanding, part 2\n",
    "\n",
    "We'll continue with the same hypothetical scenario, where you're trying to train a model to classify online comments as toxic.\n",
    "\n",
    "You take any comments that are not already in English and translate them to English with a separate tool.  Then, you treat all posts as if they were originally expressed in English.  What type of bias will your model suffer from?\n",
    "\n",
    "Once you have answered the question, run the next code cell to see the official answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "c36e2641",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-06T21:20:14.915534Z",
     "iopub.status.busy": "2022-03-06T21:20:14.914871Z",
     "iopub.status.idle": "2022-03-06T21:20:14.918566Z",
     "shell.execute_reply": "2022-03-06T21:20:14.919140Z",
     "shell.execute_reply.started": "2022-03-06T21:15:29.781037Z"
    },
    "papermill": {
     "duration": 0.034408,
     "end_time": "2022-03-06T21:20:14.919301",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.884893",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.14285714285714285, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"6_TestUnderstandingTwo\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Solution:</span> By translating comments to English, we introduce additional error when classifying non-English comments.  This can introduce **measurement bias**, since non-English comments will often not be translated perfectly.  It could also introduce **aggregation bias**: the model would likely perform better for comments expressed in all languages, if the comments from different languages were treated differently."
      ],
      "text/plain": [
       "Solution: By translating comments to English, we introduce additional error when classifying non-English comments.  This can introduce **measurement bias**, since non-English comments will often not be translated perfectly.  It could also introduce **aggregation bias**: the model would likely perform better for comments expressed in all languages, if the comments from different languages were treated differently."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Check your answer (Run this code cell to get credit!)\n",
    "q_6.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "571e4cf0",
   "metadata": {
    "papermill": {
     "duration": 0.026751,
     "end_time": "2022-03-06T21:20:14.973346",
     "exception": false,
     "start_time": "2022-03-06T21:20:14.946595",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# 7) Test your understanding, part 3\n",
    "\n",
    "We'll continue with the same hypothetical scenario, where you're trying to train a model to classify online comments as toxic.\n",
    "\n",
    "The dataset you're using to train the model contains comments primarily from users based in the United Kingdom.  \n",
    "\n",
    "After training a model, you evaluate its performance with another dataset of comments, also primarily from users based in the United Kingdom -- and it gets great performance!  You deploy it for a company based in Australia, and it does not perform well, because of differences between British and Australian English.  What types of bias does the model suffer from?\n",
    "\n",
    "Once you have answered the question, run the next code cell to see the official answer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "45280366",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-03-06T21:20:15.030674Z",
     "iopub.status.busy": "2022-03-06T21:20:15.030104Z",
     "iopub.status.idle": "2022-03-06T21:20:15.035877Z",
     "shell.execute_reply": "2022-03-06T21:20:15.036450Z",
     "shell.execute_reply.started": "2022-03-06T21:17:41.711570Z"
    },
    "papermill": {
     "duration": 0.036196,
     "end_time": "2022-03-06T21:20:15.036602",
     "exception": false,
     "start_time": "2022-03-06T21:20:15.000406",
     "status": "completed"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "parent.postMessage({\"jupyterEvent\": \"custom.exercise_interaction\", \"data\": {\"outcomeType\": 1, \"valueTowardsCompletion\": 0.14285714285714285, \"interactionType\": 1, \"questionType\": 4, \"questionId\": \"7_TestUnderstandingThree\", \"learnToolsVersion\": \"0.3.4\", \"failureMessage\": \"\", \"exceptionClass\": \"\", \"trace\": \"\"}}, \"*\")"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "<span style=\"color:#33cc33\">Solution:</span> If the model is evaluated based on comments from users in the United Kingdom and deployed to users in Australia, this will lead to **evaluation bias** and **deployment bias**.  The model will also have **representation bias**, because it was built to serve users in Australia, but was trained with data from users based in the United Kingdom."
      ],
      "text/plain": [
       "Solution: If the model is evaluated based on comments from users in the United Kingdom and deployed to users in Australia, this will lead to **evaluation bias** and **deployment bias**.  The model will also have **representation bias**, because it was built to serve users in Australia, but was trained with data from users based in the United Kingdom."
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Check your answer (Run this code cell to get credit!)\n",
    "q_7.check()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e812df4",
   "metadata": {
    "papermill": {
     "duration": 0.027865,
     "end_time": "2022-03-06T21:20:15.092401",
     "exception": false,
     "start_time": "2022-03-06T21:20:15.064536",
     "status": "completed"
    },
    "tags": []
   },
   "source": [
    "# Learn more\n",
    "\n",
    "To continue learning about bias, check out the [Jigsaw Unintended Bias in Toxicity Classification](https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification/overview) competition that was introduced in this exercise.  \n",
    "- Kaggler [Dieter](https://www.kaggle.com/christofhenkel) has written a helpful two-part series that teaches you how to preprocess the data and train a neural network to make a competition submission.  [Get started here](https://www.kaggle.com/christofhenkel/how-to-preprocessing-for-glove-part1-eda).\n",
    "- Many Kagglers have written helpful notebooks that you can use to get started.  [Check them out](https://www.kaggle.com/c/jigsaw-unintended-bias-in-toxicity-classification/notebooks?sortBy=voteCount&group=everyone&pageSize=20&competitionId=12500) on the competition page.\n",
    "\n",
    "Another Kaggle competition that you can use to learn about bias is the [Inclusive Images Challenge](https://www.kaggle.com/c/inclusive-images-challenge), which you can read more about in [this blog post](https://ai.googleblog.com/2018/09/introducing-inclusive-images-competition.html).  The competition focuses on **evaluation bias** in computer vision.\n",
    "\n",
    "# Keep going\n",
    "\n",
    "How can you quantify bias in machine learning applications?  Continue to **[learn how to measure fairness](https://www.kaggle.com/alexisbcook/ai-fairness)**."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  },
  "papermill": {
   "default_parameters": {},
   "duration": 48.256881,
   "end_time": "2022-03-06T21:20:15.831106",
   "environment_variables": {},
   "exception": null,
   "input_path": "__notebook__.ipynb",
   "output_path": "__notebook__.ipynb",
   "parameters": {},
   "start_time": "2022-03-06T21:19:27.574225",
   "version": "2.3.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}