Root_Diary.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# **Root Directory Problem / Solution Outline**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/wikipedia_network_graph_small.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **EXECUTIVE SUMMARY**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### _PROBLEM_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The Rood Directory is solving Wikipedia's organization problem. While the problem is large and complex, the Root Directory project was started to answer a seemingly benign question: how many pages on Wikipedia are related to mathematics? The question was harder to answer than it first appeared.\n",
    "\n",
    "One possible answer came from the Mathematics Portal, where an answer was posted as **about 31,444 pages** on Wikipedia being related to the field of mathematics. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src='images/31444.png'/>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After asking the Mathematics Portal editors to describe how they found this number, it was discovered that the number was 3 years old and documentation for its origin of the number did not exist. Additionally, other statistics from around that same time were found and they did not agree with one another. One statistic stated that the answer was ~17,000 and another as ~24,000. The statistic of ~31,444 was deleted from the article. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### _THE BIGGER PICTURE_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the question is simple, it exposes a bigger problem Wikipedia faces: organization of its content. A solution to Wikipedia's organization problem can have a major impact on how Wikipedia editors and readers interact with the website as a whole, providing better navigation of its content to improve user experience."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### _PROPOSED SOLUTION_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The proposed solution is to use Natural Language Processing techniques to train machine learning models to find content within broad, high-level categories such as Mathematics, Physics, Biology, Arts, Engineering, and etc.\n",
    "\n",
    "This approach has many benefits of other tools available, such the graph analysis methods of community detection. These models are explored below. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### _RESULTS_\n",
    "\n",
    "The "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **The Problem**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wikipedia's system of categorizing content is all over the place. \n",
    "\n",
    "#### _Wikipedia Methods of Categorizing:_\n",
    "\n",
    "- [Portal page](https://en.wikipedia.org/wiki/Portal:Contents/Portals#Mathematics_and_logic)\n",
    "- [Contents page](https://en.wikipedia.org/wiki/Portal:Contents/Mathematics_and_logic)\n",
    "- [Category page](https://en.wikipedia.org/wiki/Category:Mathematics)\n",
    "- [Outline page](https://en.wikipedia.org/wiki/Outline_of_mathematics)\n",
    "- [Areas of Mathematics page](https://en.wikipedia.org/wiki/Areas_of_mathematics)\n",
    "- [Indices](https://en.wikipedia.org/wiki/Category:Mathematics-related_lists)\n",
    "- [Overviews](https://en.wikipedia.org/wiki/Category:Mathematics-related_lists)\n",
    "- [Glossaries](https://en.wikipedia.org/wiki/Category:Mathematics-related_lists)\n",
    "\n",
    "Of these, Mathematics is organized using subset of these structures:\n",
    "\n",
    "- [Mathematics Portal page](https://en.wikipedia.org/wiki/Portal:Contents/Portals#Mathematics_and_logic)\n",
    "- [Mathematics Contents page](https://en.wikipedia.org/wiki/Portal:Contents/Mathematics_and_logic)\n",
    "- [Mathematics Category page](https://en.wikipedia.org/wiki/Category:Mathematics)\n",
    "- [Mathematics Outline page](https://en.wikipedia.org/wiki/Outline_of_mathematics)\n",
    "- [Areas of Mathematics page](https://en.wikipedia.org/wiki/Areas_of_mathematics)\n",
    "- [Mathematics Lists](https://en.wikipedia.org/wiki/Category:Mathematics-related_lists)\n",
    "- [lists of lists of lists](https://en.wikipedia.org/wiki/List_of_lists_of_lists#Mathematics_and_logic)\n",
    "\n",
    "This also includes the [list of mathematics topics page](https://en.wikipedia.org/wiki/Lists_of_mathematics_topics)\n",
    "\n",
    "Contents Overview Outlines Lists Portals Glossaries Categories Indices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### _Wikidata Methods of Categorizing:_\n",
    "\n",
    "Outside of the Wikipedia project, the WikiMedia Foundation also includes the the Wikidata project. This project is based on a graph database (using the Sparql language) that is community edited organize data based on their interconnections.\n",
    "\n",
    "```sparql\n",
    " SELECT distinct ?item ?article ?sitelink ?linkTo WHERE {\n",
    "       { ?item wdt:P361* wd:Q395 .}\n",
    "       union\n",
    "       { ?item wdt:P361/wdt:P279* wd:Q395 .}\n",
    "       union\n",
    "       { ?item wdt:P31/wdt:P279* wd:Q1936384 .}\n",
    "       union\n",
    "       { ?item wdt:P921/wdt:P279* wd:Q395 .}\n",
    "       optional \n",
    "       { ?sitelink ^schema:name ?article .\n",
    "         ?article schema:about ?item ;\n",
    "         schema:isPartOf <https://en.wikipedia.org/> .\n",
    "       }\n",
    "       OPTIONAL { ?item wdt:P361 ?linkTo. }\n",
    "       SERVICE wikibase:label { bd:serviceParam wikibase:language \"en\". }\n",
    "       }\n",
    "```\n",
    "While graph databases are dope, when humans are used to create nodes and relationships the data is not entirely reliable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### _Wikipedia Vital Articles Page_\n",
    "Addictionally, some Wikipedians a have been hand-selecting Wikipedia articles to be given the designation of a \"Vital article.\" The page [\"Wikipedia:Vital_articles\"](https://en.wikipedia.org/wiki/Wikipedia:Vital_articles) includes a subcategory of [vital mathematics articles](https://en.wikipedia.org/wiki/Wikipedia:Vital_articles/Level/5/Mathematics). With stated goal of compiling 1,100 articles, its current count of only 489 articles reflects the difficulty faced with manually sorting articles into categories."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **Data Wrangling**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### _Wikidata Search_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While the results from the Wikidata database search did not answer our original question, we can use the results to create a seed dataset.\n",
    "\n",
    "While ~800 articles may be enough to train a Natural Language Processing model, more data is always better.\n",
    "\n",
    "Even though the Wikidata query is complex and not likely something that an average wikimedia user is capable of writing, we will use this number as our evaluation metric. This metric represents the ability of wikimedia contributors to structure content around a major category.\n",
    "\n",
    "### _Webscraping_\n",
    "\n",
    "To accomplish this, we can turn to scraping Wikipedia category strucutures. For each category structure page, we can find all the internal wiki links, store them in a container. This represents a depth search of one. Next, scraping the pages in the container for more internal wiki links results in a depth search of two. The number of iterations we perform on the articles in our container will determine the depth of our search. This can result in a large set of links, however it is a bit crude since links can be arbitrarily added to a page.\n",
    "\n",
    "While web scraping is fun, the Wikimedia Foundation has already developed a program for us that does this for us. It is called the [PetScan](https://petscan.wmflabs.org/).\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "https://en.wikipedia.org/wiki/Special:CategoryTree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### _Node Weighting_\n",
    "The deeper we traverse each tree, we are more likely to find pages within the tree that are not highly correlated with our target category, or to provide noise into the machine learning model during training. Since we have tree depth data, we can give nodes close to the root a higher weighting than nodes farther away from the root. During the scraping process, we are keeping track of the depth while saving the node to a file.\n",
    "\n",
    "ADD AN IMAGE OF TREES WITH A GRADIENT AWAY FROM ROOT"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### _PetScan searching_\n",
    "\n",
    "Contributors to the Wikimedia Foundation have built a tool for finding articles based on categories. It is a powerful tool, but it is too crude to answer our original question. Instead, we will use it to find our seed dataset. \n",
    "\n",
    "PetScan takes in a category parameter and a depth of search parameter. This roughly translates to the same process that we would have used if we were scarping Wikipedia ourselves.  \n",
    "\n",
    "Searching the Mathematics category with a depth of one yields 1,216 results. Kinda disappointing. A depth search of two yields 11,393 results, and a depth search of three yiedls 34,013. Well, that escalates quickly. However, with larger depths, we are also getting more noise in the results. We'll pick a depth search of one, and supplement this set by also searching mathematics subtopics using Petscan. After searching the Algebra, Arithmetic, Calculus, Combinatorics, Game Theory, Geometry, Linear Algebra, Probability, Statistics, and Topology categories with a depth of one, respectively, we have a total of about 10,000 results to add to our Wikidata results. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### _Putting it All Together_\n",
    "\n",
    "After combining all of the articles we gathered, we can delete any duplicates and make a count of our bounty.\n",
    "\n",
    "_Seed Dataset Count: 10,338_\n",
    "\n",
    "If that was all we had to do, we'd be golden. But we want to be thorough and check that our seed dataset is purged of bad articles.\n",
    "\n",
    "### _Manual Cleaning_\n",
    "\n",
    "There is no way around it: each item in our seed dataset needs to be checked and purged if it is not related to mathematics, or is only tangentially related to mathematics. Common articles that were purged included people related to mathematics, articles about academic journals, and universities.\n",
    "\n",
    "_Seed Dataset Count after Cleaning: 8,687_\n",
    "\n",
    "_Hours of Manual Cleaning: 4.25_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Noise Dataset\n",
    "\n",
    "In order to train a model to teach it how to recognize mathematics articles, we need a dataset of non-mathematics articles. For this we will use Physics, Biology, Chemistry, and Computer Science articles. We will again use Petscan with a depth of one search.\n",
    "\n",
    "Physics:\n",
    "\n",
    "Biology:\n",
    "\n",
    "Chemistry:\n",
    "\n",
    "Computer Science:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **Evaluation Metric**\n",
    "\n",
    "As described above, the evaluation metric we will be using is 800 mathematics related articles. This metric was derived from a search of the Wikidata graph database for mathematics-related nodes with associated English wikipedia articles. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **Lets See What We Can Do with Our Seed Dataset!**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Traversing Wikipedia"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "['Wikipedia Data Science: Working with the World’s Largest Encyclopedia'](https://towardsdatascience.com/wikipedia-data-science-working-with-the-worlds-largest-encyclopedia-c08efbac5f5c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Traversing XML tree iteratively:\n",
    "xml.sax\n",
    "\n",
    "parsing wikipedia content pages:\n",
    "mwparserfromhell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "http://wiki.knoesis.org/index.php/File:HPCO-treemap.png\n",
    "\n",
    "https://tools.wmflabs.org/vcat/render?wiki=enwiki&category=Biology&rel=subcategory\n",
    "\n",
    "https://en.wikipedia.org/wiki/Special:CategoryTree?target=mathematics&mode=all&namespaces=&title=Special%3ACategoryTree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interesting results from deep searches of wikipedia trees:\n",
    "Depth of 6 in mathematics:\n",
    "- 'Category:Pokémon',\n",
    "- 'Category:Pokémon_anime',\n",
    "- 'Category:Pokémon_characters',\n",
    "- 'Category:Pokémon_lists',\n",
    "- 'Category:Pokémon_manga',\n",
    "- 'Category:Pokémon_media',\n",
    "- 'Category:Pokémon_World_Championships',\n",
    "- 'Category:Pokémon_Trading_Card_Game',\n",
    "- 'Category:Pokémon_video_games',\n",
    "- 'Category:Pokémon_RPGs',\n",
    "- 'Category:People_with_Crohn's_Disease'\n",
    "- 'Category:Women's_Prisons_in_Australia'\n",
    "- 'Category:Sex_differences_in_psychology',\n",
    "- 'Category:Sex_hormones',\n",
    "- 'Category:Sex-determination_systems',\n",
    "- 'Category:Sexual_anatomy',\n",
    "- 'Category:Sexual_dimorphism',\n",
    "- 'Category:Sexual_selection',\n",
    "- 'Category:Testosterone',\n",
    "- 'Category:Comparative_education',\n",
    "- 'Category:Language_comparison',\n",
    "- 'Category:Comparative_law',\n",
    "- 'Category:Comparative_literature',\n",
    "- 'Category:Power_(physics)',\n",
    "- 'Category:Radioactivity',\n",
    "- 'Category:Velocity',"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Wikipedia Category Cluster Visualization "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Wikipedia Subset Category of Mathematics Cluster Graph:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/mathematics_parents_children_fdp_small.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mathematics Tree Graph:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/mathematics_children.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4.3 Category hierarchy as isa graph\n",
    "\n",
    "In many of the knowledge discovery tasks, Wikipedia category hierarchy is treated as an isa graph. Though it makes sense (in most of the cases) to treat the concept represented by a child category as a specific type of a broader concept represented by a parent category (e.g., Computer_science is a type of Applied_sciences), it is often the case that long distance isa relationship in category hierarchy does not make sense. For example, we can trace a path from the category “Optical_storage” to “Biology” as follows:\n",
    "\n",
    "Optical_computer_storage is a descendant of Biology as per the following relation hierarchy: \n",
    "\n",
    "Optical_computer_storage → \n",
    "Computer_storage_devices → \n",
    "Recorders → \n",
    "Equipment → \n",
    "Technology → \n",
    "Intellectual_works → \n",
    "Creativity → \n",
    "Intelligence →\n",
    "Neuroscience → \n",
    "Biology\n",
    "\n",
    "Similarly, from “Automotive_testing_agencies” to “Algebra” as follows:\n",
    "Automotive_testing_agencies is a descendant of Algebra as per the following relation hierarchy:\n",
    "\n",
    "Automotive_testing_agencies → \n",
    "Automobiles → \n",
    "Private_transport → \n",
    "Transport → \n",
    "Industries → \n",
    "Economic_systems → \n",
    "Economics → \n",
    "Society → \n",
    "Structure → \n",
    "Dimension → \n",
    "Manifolds → \n",
    "Geometric_topology → \n",
    "Structures_on_manifolds → \n",
    "Algebraic_geometry → \n",
    "Abstract_algebra → \n",
    "Algebra\n",
    "\n",
    "As explained in Section 3.1, our manual inspection of a few (39) randomly sampled documents show that, is-a relation does not make sense beyond 7 levels. We found that, between 3 to 5 levels, we can get a reasonably good isa relation.\n",
    "\n",
    "\n",
    "Bairi R. B. & Carman M. & Ramakrishnan G. (2015). On the Evolution of Wikipedia: Dynamics of Categories and Articles. Papers from the 2015 ICWSM Workshop, pp. 9."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Steps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/graph_decomposition_small.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add to the above image: extra edges leading away from the visible trees"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/primary_edge_tree_small.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Model Creation\n",
    "\n",
    "Vectorizer: \n",
    "Use count vectorizer. Use TF\\*IDF. Use Word2Vec. Use Doc2Vec.\n",
    "\n",
    "Models:\n",
    "Naive Bayes, Logistic Regression, RNN."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unsupervised Topic Searching\n",
    "[Concept Search Project](http://mccormickml.com/2017/02/22/concept-search-on-wikipedia/)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Logistic Regression 1000 most important words, least to most important:\n",
    "\n",
    "['scienc',\n",
    " 'econom',\n",
    " 'john',\n",
    " 'edit',\n",
    " 'actuari',\n",
    " 'puzzl',\n",
    " 'cipher',\n",
    " 'publish',\n",
    " '(mathematics)',\n",
    " 'polit',\n",
    " 'b',\n",
    " 'book',\n",
    " '2',\n",
    " 'space',\n",
    " 'z',\n",
    " 'born',\n",
    " 'physic',\n",
    " 'category:theorem',\n",
    " 'order',\n",
    " 'cube',\n",
    " 'manifold',\n",
    " 'prime',\n",
    " 'prefix',\n",
    " 'theorist',\n",
    " 'univers',\n",
    " 'random',\n",
    " 'categori theori',\n",
    " 'curv',\n",
    " 'prize',\n",
    " 'census',\n",
    " 'p',\n",
    " 'category:mathematician',\n",
    " 'fractal',\n",
    " 'signatur',\n",
    " 'system engin',\n",
    " 'languag',\n",
    " 'integ',\n",
    " 'olympiad',\n",
    " 'n',\n",
    " 'school',\n",
    " 'notat',\n",
    " 'symbol',\n",
    " 'arithmet',\n",
    " 'tile',\n",
    " 'differenti',\n",
    " 'theori',\n",
    " 'coordin',\n",
    " 'x',\n",
    " 'field',\n",
    " 'finit',\n",
    " 'conjectur',\n",
    " 'category:mathemat',\n",
    " 'paradox',\n",
    " 'variabl',\n",
    " 'mathemat journal',\n",
    " 'american mathemat',\n",
    " 'calculus',\n",
    " 'he',\n",
    " 'mathemat educ',\n",
    " 'lemma',\n",
    " 'cryptographi',\n",
    " 'numer',\n",
    " 'equat',\n",
    " 'distribut',\n",
    " 'dynam',\n",
    " 'complex',\n",
    " 'neural network',\n",
    " 'encrypt',\n",
    " 'inequ',\n",
    " 'probabl',\n",
    " 'neural',\n",
    " 'model',\n",
    " 'game',\n",
    " 'statist',\n",
    " 'graph',\n",
    " 'valid',\n",
    " 'logic',\n",
    " 'group',\n",
    " 'calcul',\n",
    " 'philosophi',\n",
    " 'cambridg',\n",
    " 'cryptograph',\n",
    " 'topolog',\n",
    " 'formal',\n",
    " 'proof',\n",
    " 'function',\n",
    " 'problem',\n",
    " 'appli mathemat',\n",
    " 'algorithm',\n",
    " 'mathematics.',\n",
    " 'mathemat societi',\n",
    " 'set',\n",
    " 'uml',\n",
    " 'math',\n",
    " 'algebra',\n",
    " 'geometri',\n",
    " 'number',\n",
    " 'theorem',\n",
    " 'in mathemat',\n",
    " 'mathematician']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 100 words most likely to indicate not mathematics, least to most important:\n",
    "\n",
    "['light',\n",
    " 'truck',\n",
    " 'robot',\n",
    " 'tool',\n",
    " 'typic',\n",
    " 'interact',\n",
    " 'product',\n",
    " 'dam',\n",
    " 'organis',\n",
    " 'air',\n",
    " 'instrument',\n",
    " 'liquid',\n",
    " 'qualiti',\n",
    " 'reliabl',\n",
    " 'countri',\n",
    " 'compani',\n",
    " 'nois',\n",
    " 'protect',\n",
    " '-',\n",
    " 'control',\n",
    " 'drill',\n",
    " 'mine',\n",
    " 'citi',\n",
    " 'busi',\n",
    " '3d',\n",
    " 'qualif',\n",
    " 'aeronaut',\n",
    " 'drum',\n",
    " 'flow',\n",
    " 'genet',\n",
    " 'actor',\n",
    " 'video',\n",
    " 'mainten',\n",
    " 'test',\n",
    " 'chemic',\n",
    " 'optic',\n",
    " 'ship',\n",
    " 'develop',\n",
    " 'built',\n",
    " 'combust',\n",
    " 'failur',\n",
    " 'load',\n",
    " 'pipe',\n",
    " 'beam',\n",
    " 'sound',\n",
    " 'this categori',\n",
    " 'biotechnolog',\n",
    " 'manag',\n",
    " 'activ',\n",
    " 'measur',\n",
    " 'temperatur',\n",
    " 'river',\n",
    " 'speed',\n",
    " 'interfac',\n",
    " 'glass',\n",
    " 'wall',\n",
    " 'bridg',\n",
    " 'user',\n",
    " 'redirect',\n",
    " 'cut',\n",
    " 'land',\n",
    " 'voltag',\n",
    " 'architectur',\n",
    " 'equip',\n",
    " 'requir',\n",
    " 'circuit',\n",
    " 'station',\n",
    " 'flight',\n",
    " 'audio',\n",
    " 'steel',\n",
    " 'project',\n",
    " 'water',\n",
    " 'aircraft',\n",
    " 'frequenc',\n",
    " 'organ',\n",
    " 'industri',\n",
    " 'survey',\n",
    " 'electr',\n",
    " 'electron',\n",
    " 'devic',\n",
    " 'unit',\n",
    " 'broadcast',\n",
    " 'comic',\n",
    " 'pressur',\n",
    " 'metal',\n",
    " 'genom',\n",
    " 'safeti',\n",
    " 'technolog',\n",
    " 'vehicl',\n",
    " 'telecommun',\n",
    " 'tensor',\n",
    " 'technic',\n",
    " 'servic',\n",
    " 'locat',\n",
    " 'materi',\n",
    " 'design',\n",
    " 'manufactur',\n",
    " 'use',\n",
    " 'build',\n",
    " 'standard',\n",
    " 'engin']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example words that were reduced to zero by Lasso Regression:\n",
    "\n",
    "['0 1',\n",
    " '0 1 real',\n",
    " '1 real',\n",
    " 'a',\n",
    " 'a similar',\n",
    " 'abelian',\n",
    " 'abelian von',\n",
    " 'abelian von neumann',\n",
    " 'algebra bound',\n",
    " 'algebra i.e.',\n",
    " 'algebra isomorph',\n",
    " 'algebras.',\n",
    " 'atom',\n",
    " 'banach',\n",
    " 'banach space',\n",
    " 'banach spaces.',\n",
    " 'borel',\n",
    " 'borel set',\n",
    " 'bound function',\n",
    " 'brief',\n",
    " 'classic',\n",
    " 'complet',\n",
    " 'complet measur',\n",
    " 'consid',\n",
    " 'copi',\n",
    " 'count',\n",
    " 'count measur',\n",
    " 'decompos',\n",
    " 'decomposit',\n",
    " 'deep',\n",
    " 'deep result',\n",
    " 'discret',\n",
    " 'discret set.',\n",
    " 'discret space.',\n",
    " 'dorothi',\n",
    " 'everi',\n",
    " 'extend',\n",
    " 'finit set.',\n",
    " 'function discret',\n",
    " 'function general',\n",
    " 'general',\n",
    " 'general measur',\n",
    " 'given',\n",
    " 'i.e.',\n",
    " 'import',\n",
    " 'import classic',\n",
    " 'import role',\n",
    " 'import role theori',\n",
    " 'in brief',\n",
    " 'integ finit',\n",
    " 'interv',\n",
    " 'interv 0',\n",
    " 'interv 0 1',\n",
    " 'irv',\n",
    " 'irv segal',\n",
    " 'isomorph',\n",
    " 'it',\n",
    " 'it extend',\n",
    " 'kuratowski',\n",
    " 'lp',\n",
    " 'lp space',\n",
    " 'maharam',\n",
    " 'mathemat measur',\n",
    " 'mathemat measur space',\n",
    " 'measur discret',\n",
    " 'measur function',\n",
    " 'measur space',\n",
    " 'neumann',\n",
    " 'neumann algebra',\n",
    " 'neumann algebras.',\n",
    " 'non-atom',\n",
    " 'part',\n",
    " 'part use',\n",
    " 'parts.',\n",
    " 'play',\n",
    " 'play import',\n",
    " 'play import role',\n",
    " 'polish',\n",
    " 'polish space',\n",
    " 'polish space borel',\n",
    " 'product unit',\n",
    " 'pure',\n",
    " 'real',\n",
    " 'real integ',\n",
    " 'refer mathemat',\n",
    " 'result',\n",
    " 'result import',\n",
    " 'role',\n",
    " 'role theori',\n",
    " 'segal',\n",
    " 'set real',\n",
    " 'set refer',\n",
    " 'set.',\n",
    " 'set. a',\n",
    " 'set. refer',\n",
    " 'similar',\n",
    " 'similar theorem',\n",
    " 'space abelian',\n",
    " 'space banach',\n",
    " 'space banach space',\n",
    " 'space borel',\n",
    " 'space borel set',\n",
    " 'space complet',\n",
    " 'space given',\n",
    " 'space measur',\n",
    " 'space measur function',\n",
    " 'space play',\n",
    " 'space state',\n",
    " 'space suffici',\n",
    " 'space theori',\n",
    " 'space unit',\n",
    " 'space unit interv',\n",
    " 'space.',\n",
    " 'space. the',\n",
    " 'spaces.',\n",
    " 'spaces. in',\n",
    " 'spatial',\n",
    " 'state',\n",
    " 'state complet',\n",
    " 'suffici',\n",
    " 'tensor product',\n",
    " 'term',\n",
    " 'the result',\n",
    " 'the theorem',\n",
    " 'theorem given',\n",
    " 'theorem in',\n",
    " 'theorem in mathemat',\n",
    " 'theori banach',\n",
    " 'theori consid',\n",
    " 'translat languag',\n",
    " 'understand',\n",
    " 'unit interv',\n",
    " 'unit interv 0',\n",
    " 'use count',\n",
    " 'von',\n",
    " 'von neumann',\n",
    " 'von neumann algebra',\n",
    " 'von neumann algebras.',\n",
    " 'σ-finit',\n",
    " '. this',\n",
    " '1 6',\n",
    " '10',\n",
    " '10 number',\n",
    " '157',\n",
    " '19',\n",
    " '1968',\n",
    " '1968.',\n",
    " '1971',\n",
    " '1973',\n",
    " '1973 pp.',\n",
    " '1975.',\n",
    " '1976',\n",
    " '1977',\n",
    " '1980',\n",
    " '1984',\n",
    " '1989',\n",
    " '1989 .',\n",
    " '1994.',\n",
    " '1997',\n",
    " '2 6',\n",
    " '2008',\n",
    " '3',\n",
    " '38',\n",
    " '39',\n",
    " '3–66',\n",
    " '542',\n",
    " '59',\n",
    " '6',\n",
    " '62',\n",
    " '70',\n",
    " '72',\n",
    " '73',\n",
    " '79',\n",
    " '97',\n",
    " 'a.',\n",
    " 'actual',\n",
    " 'advanc',\n",
    " 'algebra method',\n",
    " 'algebra topology.',\n",
    " 'appli',\n",
    " 'applic',\n",
    " 'applic geometr',\n",
    " 'approxim',\n",
    " 'approxim mathemat',\n",
    " 'arbitrari',\n",
    " 'arc.',\n",
    " 'associ',\n",
    " 'asymptot',\n",
    " 'background',\n",
    " 'birkhäus',\n",
    " 'borsuk',\n",
    " 'branch',\n",
    " 'branch topolog',\n",
    " 'c*-algebra',\n",
    " 'categor',\n",
    " 'characterist',\n",
    " 'circle.',\n",
    " 'circle. it',\n",
    " 'circle. this',\n",
    " 'circumst',\n",
    " 'close',\n",
    " 'coher',\n",
    " 'coin',\n",
    " 'coincid',\n",
    " 'compact',\n",
    " 'compact space',\n",
    " 'compact subset',\n",
    " 'compacta',\n",
    " 'complex.',\n",
    " 'concern',\n",
    " 'contents.',\n",
    " 'continu',\n",
    " 'cw',\n",
    " 'cw complex.',\n",
    " 'd.a.',\n",
    " 'deform',\n",
    " 'develop refer',\n",
    " 'develop strong',\n",
    " 'différentiell',\n",
    " 'dimension',\n",
    " 'doe',\n",
    " 'doe appli',\n",
    " 'domin',\n",
    " 'dover',\n",
    " 'due',\n",
    " 'duke',\n",
    " 'duke math.',\n",
    " 'duke math. j.',\n",
    " 'dynam system',\n",
    " 'e.g.',\n",
    " 'edward',\n",
    " 'edward h.',\n",
    " 'eilenberg',\n",
    " 'elli',\n",
    " 'equival',\n",
    " 'exampl',\n",
    " 'exampl area',\n",
    " 'for',\n",
    " 'for purpos',\n",
    " 'found.',\n",
    " 'foundat',\n",
    " 'foundat algebra',\n",
    " 'functor',\n",
    " 'fund',\n",
    " 'fund.',\n",
    " 'fundament',\n",
    " 'fundament exampl',\n",
    " 'further',\n",
    " 'further develop',\n",
    " 'futur',\n",
    " 'gain',\n",
    " 'general arbitrari',\n",
    " 'general categori',\n",
    " 'geometr topolog',\n",
    " 'geometri oper',\n",
    " 'global',\n",
    " 'great',\n",
    " 'great popular',\n",
    " 'grothendieck',\n",
    " 'group group',\n",
    " 'group group isomorph',\n",
    " 'group isomorph',\n",
    " 'h.',\n",
    " 'h. m.',\n",
    " 'hard',\n",
    " 'hast',\n",
    " 'hilton',\n",
    " 'homolog',\n",
    " 'homolog theori',\n",
    " 'homolog theory.',\n",
    " 'homotop',\n",
    " 'homotopi',\n",
    " 'homotopi equival',\n",
    " 'homotopi group',\n",
    " 'homotopi theori',\n",
    " 'homotopi theory.',\n",
    " 'homotopi theory. the',\n",
    " 'i',\n",
    " 'i ii',\n",
    " 'ii',\n",
    " 'in algebra',\n",
    " 'instead',\n",
    " 'invari',\n",
    " 'j.',\n",
    " 'jack',\n",
    " 'jean-marc',\n",
    " 'journal',\n",
    " 'journal mathemat',\n",
    " 'k.',\n",
    " 'karol',\n",
    " 'karol borsuk',\n",
    " 'later',\n",
    " 'lectur note',\n",
    " 'like',\n",
    " 'littl',\n",
    " 'live',\n",
    " 'live work',\n",
    " 'london',\n",
    " 'london math.',\n",
    " 'london math. soc.',\n",
    " 'lore',\n",
    " 'm.',\n",
    " 'marius',\n",
    " 'math.',\n",
    " 'math. j.',\n",
    " 'math. soc.',\n",
    " 'mathemat applic',\n",
    " 'mathemat volum',\n",
    " 'maths.',\n",
    " 'method',\n",
    " 'method approxim',\n",
    " 'method oper',\n",
    " 'michael',\n",
    " 'monograph',\n",
    " 'morphism',\n",
    " 'mountain',\n",
    " 'mountain journal',\n",
    " 'mountain journal mathemat',\n",
    " 'naiv',\n",
    " 'no.',\n",
    " 'no. 1',\n",
    " 'noncommut',\n",
    " 'noncommut geometri',\n",
    " 'norman',\n",
    " 'norman steenrod',\n",
    " 'note',\n",
    " 'notic',\n",
    " 'number 3',\n",
    " 'on',\n",
    " 'oper',\n",
    " 'oper algebra',\n",
    " 'oper theori',\n",
    " 'p.',\n",
    " 'p.j.',\n",
    " 'paper',\n",
    " 'past',\n",
    " 'past present',\n",
    " 'past present futur',\n",
    " 'pdf',\n",
    " 'phil.',\n",
    " 'plane',\n",
    " 'point',\n",
    " 'point homotopi',\n",
    " 'polish mathematician',\n",
    " 'polyhedra.',\n",
    " 'popular',\n",
    " 'porter',\n",
    " 'pp.',\n",
    " 'predict',\n",
    " 'present',\n",
    " 'present futur',\n",
    " 'proc.',\n",
    " 'promot',\n",
    " 'properti',\n",
    " 'provid',\n",
    " 'publish littl',\n",
    " 'purpos',\n",
    " 'reflect',\n",
    " 'reinvent',\n",
    " 'render',\n",
    " 'reprint',\n",
    " 'reprint dover',\n",
    " 'rocki',\n",
    " 'rocki mountain',\n",
    " 'rocki mountain journal',\n",
    " 's.',\n",
    " 'samuel',\n",
    " 'samuel eilenberg',\n",
    " 'shape',\n",
    " 'shape.',\n",
    " 'sine',\n",
    " 'sine curv',\n",
    " 'singular',\n",
    " 'singular homolog',\n",
    " 'soc.',\n",
    " 'sophist',\n",
    " 'space general',\n",
    " 'space homotopi',\n",
    " 'sphere',\n",
    " 'sphere .',\n",
    " 'springer-verlag.',\n",
    " 'steenrod',\n",
    " 'strong',\n",
    " 'style',\n",
    " 'subset',\n",
    " 'subset plane',\n",
    " 'summer',\n",
    " 'system',\n",
    " 'terri',\n",
    " 'theorem doe',\n",
    " 'theorem doe appli',\n",
    " 'theori applic',\n",
    " 'theori associ',\n",
    " 'theori asymptot',\n",
    " 'theori branch',\n",
    " 'theori categor',\n",
    " 'theori general',\n",
    " 'theori homotopi',\n",
    " 'theori homotopi theori',\n",
    " 'theori oper',\n",
    " 'theori oper algebra',\n",
    " 'theori p.',\n",
    " 'theori shape',\n",
    " 'theory.',\n",
    " 'theory. the',\n",
    " 'this',\n",
    " 'this compact',\n",
    " 'this continu',\n",
    " 'tim',\n",
    " 'tom',\n",
    " 'topolog homotopi',\n",
    " 'topolog space',\n",
    " 'topolog space homotopi',\n",
    " 'topologi',\n",
    " 'topologist',\n",
    " 'topology.',\n",
    " 'view',\n",
    " 'volum',\n",
    " 'volum 10',\n",
    " 'warsaw',\n",
    " 'warszawa',\n",
    " 'whitehead',\n",
    " 'work',\n",
    " 'włodzimierz',\n",
    " 'year',\n",
    " 'čech',\n",
    " 'absolut',\n",
    " 'data',\n",
    " 'data set',\n",
    " 'impli',\n",
    " 'in statist',\n",
    " 'indic',\n",
    " 'inher',\n",
    " 'larson',\n",
    " 'level',\n",
    " 'level level',\n",
    " 'level measur',\n",
    " 'magnitud',\n",
    " 'mathematician refer',\n",
    " 'measur ratio',\n",
    " 'nature.',\n",
    " 'point use',\n",
    " 'ratio',\n",
    " 'ratio measur',\n",
    " 'refer point',\n",
    " 'ron',\n",
    " 'ron larson',\n",
    " 'set indic',\n",
    " 'use data',\n",
    " 'use ratio',\n",
    " 'zero',\n",
    " 'zero in',\n",
    " 'zero use',\n",
    " '. there',\n",
    " '. there general',\n",
    " '...',\n",
    " '... k',\n",
    " '....',\n",
    " '1 ...',\n",
    " '1 integ',\n",
    " '1.',\n",
    " '1. from',\n",
    " '1972',\n",
    " '1987',\n",
    " '19th',\n",
    " '19th centuri',\n",
    " '2 z',\n",
    " '2.',\n",
    " '3-space',\n",
    " '3.',\n",
    " '3. if',\n",
    " '3. we',\n",
    " '8',\n",
    " '8.2',\n",
    " '= 1',\n",
    " '= 1 ...',\n",
    " '= n',\n",
    " 'a b',\n",
    " 'a similar argument',\n",
    " \"a'i\",\n",
    " \"a'k\",\n",
    " 'a1',\n",
    " 'a1 ...',\n",
    " 'accordingly.',\n",
    " 'act',\n",
    " 'act set',\n",
    " 'action',\n",
    " 'action homeomorph',\n",
    " 'action linear',\n",
    " 'action set',\n",
    " 'alexand',\n",
    " 'alexand lubotzki',\n",
    " 'altern',\n",
    " 'altern form',\n",
    " 'alternative.',\n",
    " 'alternative. the',\n",
    " 'amalgam',\n",
    " 'american',\n",
    " 'american mathemat societi',\n",
    " 'and',\n",
    " 'appear',\n",
    " 'argument',\n",
    " 'argument goe',\n",
    " 'as',\n",
    " 'assum',\n",
    " 'assumpt',\n",
    " 'attract',\n",
    " 'attract fix',\n",
    " 'attract fix point',\n",
    " 'attribut',\n",
    " 'automorph',\n",
    " 'automorph group',\n",
    " 'avail',\n",
    " 'b element',\n",
    " 'basi',\n",
    " 'basic',\n",
    " 'basic fact',\n",
    " 'bernard',\n",
    " 'bestvina',\n",
    " 'boundari',\n",
    " 'by',\n",
    " 'by definit',\n",
    " 'case',\n",
    " 'case if',\n",
    " 'centuri',\n",
    " 'ch.',\n",
    " 'chain',\n",
    " 'check',\n",
    " 'check given',\n",
    " 'check use',\n",
    " 'claim.',\n",
    " 'claim. the',\n",
    " 'class',\n",
    " 'class group',\n",
    " 'combin',\n",
    " 'common',\n",
    " 'commut',\n",
    " 'conclud',\n",
    " 'consid case',\n",
    " 'consid standard',\n",
    " 'contain',\n",
    " 'contain free',\n",
    " 'contain proof',\n",
    " 'context',\n",
    " 'context group',\n",
    " 'corollari',\n",
    " 'cyclic',\n",
    " 'cyclic group',\n",
    " 'cyclic group subgroup',\n",
    " 'cyclic subgroup',\n",
    " 'definit',\n",
    " 'descript',\n",
    " 'discontinu',\n",
    " 'discret group',\n",
    " 'disjoint',\n",
    " 'distinct',\n",
    " 'distinct point',\n",
    " 'element',\n",
    " 'element exact',\n",
    " 'element finit',\n",
    " 'element g',\n",
    " 'element group',\n",
    " 'element infinit',\n",
    " 'element set',\n",
    " 'element.',\n",
    " 'element. to',\n",
    " 'ensur',\n",
    " 'essay',\n",
    " 'exact',\n",
    " 'exampl let',\n",
    " 'exampl one',\n",
    " 'exampl proof',\n",
    " 'exampl special',\n",
    " 'exist',\n",
    " 'exist m',\n",
    " 'explan',\n",
    " 'explicit',\n",
    " 'explicit descript',\n",
    " 'extens',\n",
    " 'extensions.',\n",
    " 'fact',\n",
    " 'famous',\n",
    " 'famous result',\n",
    " 'felix',\n",
    " 'felix klein',\n",
    " 'final',\n",
    " 'finish',\n",
    " 'finit generat',\n",
    " 'finit generat group',\n",
    " 'finit group',\n",
    " 'finite.',\n",
    " 'fix',\n",
    " 'fix point',\n",
    " 'fix point mathemat',\n",
    " 'fix point.',\n",
    " 'follow',\n",
    " 'follow chain',\n",
    " 'follow corollari',\n",
    " 'follow hold',\n",
    " 'follow properti',\n",
    " 'follow statement',\n",
    " 'form',\n",
    " 'formal statement',\n",
    " 'free',\n",
    " 'free group',\n",
    " 'free group.',\n",
    " 'free product',\n",
    " 'freeli',\n",
    " 'from',\n",
    " 'from.',\n",
    " 'fulli',\n",
    " 'function analysi',\n",
    " 'g',\n",
    " 'g act',\n",
    " 'g generat',\n",
    " 'g group',\n",
    " 'g h',\n",
    " 'g x1',\n",
    " 'g ∈',\n",
    " \"g'n\",\n",
    " 'general assum',\n",
    " 'general context',\n",
    " 'general group',\n",
    " 'general use',\n",
    " 'generat a',\n",
    " 'generat free',\n",
    " 'generat group',\n",
    " 'generat linear',\n",
    " 'generat set',\n",
    " 'generat set group',\n",
    " 'geometr finit',\n",
    " 'geometr function',\n",
    " 'geometr function analysi',\n",
    " 'geometr group',\n",
    " 'geometr group theori',\n",
    " 'geometr group theory.',\n",
    " 'geometri topolog',\n",
    " 'gh',\n",
    " 'goe',\n",
    " 'greater',\n",
    " 'greater 2.',\n",
    " 'gromov.',\n",
    " 'group act',\n",
    " 'group act set',\n",
    " 'group action',\n",
    " 'group action homeomorph',\n",
    " 'group action set',\n",
    " 'group automorph',\n",
    " 'group automorph group',\n",
    " 'group context',\n",
    " 'group cyclic',\n",
    " 'group cyclic group',\n",
    " 'group discret',\n",
    " 'group exampl',\n",
    " 'group free',\n",
    " 'group free group',\n",
    " 'group g',\n",
    " 'group g act',\n",
    " 'group generat',\n",
    " 'group geometr',\n",
    " 'group geometr group',\n",
    " 'group group action',\n",
    " 'group histori',\n",
    " 'group hyperbol',\n",
    " 'group impli',\n",
    " 'group invention',\n",
    " 'group invention mathematica',\n",
    " 'group isometri',\n",
    " 'group linear',\n",
    " 'group mathemat',\n",
    " 'group mathemat group',\n",
    " 'group order',\n",
    " 'group order group',\n",
    " 'group proper',\n",
    " 'group rank',\n",
    " 'group riemann',\n",
    " 'group riemann surfac',\n",
    " 'group studi',\n",
    " 'group subgroup',\n",
    " 'group theori',\n",
    " 'group theory.',\n",
    " 'group tit',\n",
    " 'group virtual',\n",
    " 'group.',\n",
    " 'groups.',\n",
    " 'guarante',\n",
    " 'h',\n",
    " 'h =',\n",
    " 'h free',\n",
    " 'h x',\n",
    " 'h ∈',\n",
    " \"h'i\",\n",
    " \"h'k\",\n",
    " 'h1',\n",
    " 'h2',\n",
    " 'harp',\n",
    " 'hi',\n",
    " 'histori',\n",
    " 'histori formal',\n",
    " 'histori the',\n",
    " 'hold',\n",
    " 'hold for',\n",
    " 'homeomorph',\n",
    " 'homeomorphisms.',\n",
    " 'hyperbol 3-space',\n",
    " 'h∞',\n",
    " 'i.',\n",
    " 'idea',\n",
    " 'ident element.',\n",
    " 'if',\n",
    " 'if let',\n",
    " 'impli exist',\n",
    " 'in case',\n",
    " 'includ',\n",
    " 'includ use',\n",
    " 'inde',\n",
    " 'inde let',\n",
    " 'infinit',\n",
    " 'infinit cyclic',\n",
    " 'infinit cyclic group',\n",
    " 'institut public',\n",
    " 'integ n',\n",
    " 'integ n ≥',\n",
    " 'invention',\n",
    " 'invention mathematica',\n",
    " 'involv',\n",
    " 'isometri',\n",
    " 'isometri hyperbol',\n",
    " 'it hard',\n",
    " 'it known',\n",
    " 'jacqu',\n",
    " 'jacqu tit',\n",
    " 'journal algebra',\n",
    " 'just',\n",
    " 'k',\n",
    " 'k ≥',\n",
    " 'k ≥ 2.',\n",
    " 'k. then',\n",
    " 'key',\n",
    " 'key tool',\n",
    " 'klein',\n",
    " 'kleinian',\n",
    " 'kleinian group',\n",
    " 'kleinian groups.',\n",
    " 'known',\n",
    " 'la',\n",
    " 'late',\n",
    " 'late 19th',\n",
    " 'late 19th centuri',\n",
    " 'lemma .',\n",
    " 'lemma guarante',\n",
    " 'lemma impli',\n",
    " 'lemma in',\n",
    " 'lemma in mathemat',\n",
    " 'lemma mathemat',\n",
    " 'lemma proof',\n",
    " 'lemma prove',\n",
    " 'lemma refer',\n",
    " 'lemma the',\n",
    " 'lemma use',\n",
    " 'let',\n",
    " 'let a1',\n",
    " 'let g',\n",
    " 'let g group',\n",
    " 'let w',\n",
    " 'linear',\n",
    " 'linear group',\n",
    " 'linear group group',\n",
    " 'linear groups.',\n",
    " 'linear transform',\n",
    " 'linear transformations.',\n",
    " 'loss',\n",
    " 'loss general',\n",
    " 'loss general assum',\n",
    " 'lubotzki',\n",
    " 'lyndon',\n",
    " 'm',\n",
    " 'm m',\n",
    " 'm ≥',\n",
    " 'make',\n",
    " 'make assumpt',\n",
    " 'map class',\n",
    " 'map class group',\n",
    " 'mathemat free',\n",
    " 'mathemat group',\n",
    " 'mathemat group action',\n",
    " 'mathemat scienc research',\n",
    " 'mathemat statement',\n",
    " 'mathematica',\n",
    " 'matric',\n",
    " 'mladen',\n",
    " 'mladen bestvina',\n",
    " 'modern',\n",
    " 'modern version',\n",
    " 'möbius',\n",
    " 'möbius transform',\n",
    " 'n ≥',\n",
    " 'neighborhood',\n",
    " 'neighbourhood',\n",
    " 'neighbourhood mathemat',\n",
    " 'new',\n",
    " 'new york',\n",
    " 'non-commut',\n",
    " 'nonempti',\n",
    " 'nonempti subset',\n",
    " 'nontrivi',\n",
    " 'nontrivial.',\n",
    " 'one',\n",
    " 'one famous',\n",
    " 'one use',\n",
    " 'order 3',\n",
    " 'order 3.',\n",
    " 'order greater',\n",
    " 'order group',\n",
    " 'order group theori',\n",
    " 'order k',\n",
    " 'order.',\n",
    " 'others.',\n",
    " 'others. refer',\n",
    " 'outer',\n",
    " 'outer automorph',\n",
    " 'outer automorph group',\n",
    " 'overview',\n",
    " 'pairwis',\n",
    " 'pairwis disjoint',\n",
    " 'paper contain',\n",
    " 'particular',\n",
    " 'particular group',\n",
    " 'particular proof',\n",
    " 'point fix',\n",
    " 'point fix point',\n",
    " 'point g',\n",
    " 'point mathemat',\n",
    " 'point.',\n",
    " 'product order',\n",
    " 'product.',\n",
    " 'product. the',\n",
    " 'proof by',\n",
    " 'proof consid',\n",
    " 'proof exampl',\n",
    " 'proof the',\n",
    " 'proof this',\n",
    " 'proper',\n",
    " 'proper discontinu',\n",
    " 'properti fix',\n",
    " 'prove',\n",
    " 'prove claim.',\n",
    " 'public',\n",
    " 'put',\n",
    " 'r2',\n",
    " 'rank',\n",
    " 'reduc',\n",
    " 'reduc word',\n",
    " 'refer see',\n",
    " 'repel',\n",
    " 'research institut',\n",
    " 'respectively.',\n",
    " 'respectively. then',\n",
    " 'result known',\n",
    " 'result state',\n",
    " 'riemann',\n",
    " 'riemann sphere',\n",
    " 'riemann sphere.',\n",
    " 'riemann surfac',\n",
    " 'roger',\n",
    " 'roger lyndon',\n",
    " 'schottki',\n",
    " 'schottki group',\n",
    " 'scienc research',\n",
    " 'scienc research institut',\n",
    " 'see',\n",
    " 'see free',\n",
    " 'semigroup',\n",
    " 'semigroup group',\n",
    " 'semigroup.',\n",
    " 'set generat',\n",
    " 'set group',\n",
    " 'set map',\n",
    " 'set specif',\n",
    " 'set x',\n",
    " 'set x.',\n",
    " 'similar argument',\n",
    " 'sinc',\n",
    " 'sinc g',\n",
    " 'sinc group',\n",
    " 'sketch',\n",
    " 'sketch proof',\n",
    " 'sketch proof the',\n",
    " 'sl',\n",
    " 'sl 2',\n",
    " 'sl 2 z',\n",
    " 'so-cal',\n",
    " 'societi',\n",
    " 'solvabl',\n",
    " 'solvabl group',\n",
    " 'space. a',\n",
    " 'special',\n",
    " 'special linear',\n",
    " 'special linear group',\n",
    " 'specif type',\n",
    " 'sphere.',\n",
    " 'sphere. the',\n",
    " 'springer',\n",
    " 'springer new',\n",
    " 'springer new york',\n",
    " 'standard action',\n",
    " 'state finit',\n",
    " 'statement',\n",
    " 'statement follow',\n",
    " 'studi',\n",
    " 'subgroup',\n",
    " 'subgroup free',\n",
    " 'subgroup g',\n",
    " 'subgroup given',\n",
    " 'subgroup group',\n",
    " 'subgroup h',\n",
    " 'subgroup order',\n",
    " 'subgroups.',\n",
    " 'subgroups. in',\n",
    " 'such',\n",
    " 'suffic',\n",
    " 'suffic check',\n",
    " 'suppos',\n",
    " 'suppos exist',\n",
    " 'surfac',\n",
    " 'surfac set',\n",
    " 'take',\n",
    " 'teichmüller',\n",
    " 'teichmüller space',\n",
    " 'the follow',\n",
    " 'the follow statement',\n",
    " 'the group',\n",
    " 'the group g',\n",
    " 'then',\n",
    " 'then exist',\n",
    " 'then proof',\n",
    " 'theori american',\n",
    " 'theori american mathemat',\n",
    " 'theori free',\n",
    " 'theori free group',\n",
    " 'theori particular',\n",
    " 'theori roger',\n",
    " 'there',\n",
    " 'there general',\n",
    " 'there version',\n",
    " 'these',\n",
    " 'these general',\n",
    " 'this prove',\n",
    " 'this statement',\n",
    " 'this version',\n",
    " 'thurston',\n",
    " 'tit',\n",
    " 'to',\n",
    " 'tool use',\n",
    " 'topolog free',\n",
    " 'topolog geometr',\n",
    " 'topolog geometr group',\n",
    " 'torsion-fre',\n",
    " 'transform',\n",
    " 'transformations.',\n",
    " 'tree',\n",
    " 'trees.',\n",
    " 'two.',\n",
    " 'two. the',\n",
    " 'type',\n",
    " 'type action',\n",
    " 'use altern',\n",
    " 'use explicit',\n",
    " 'use geometr',\n",
    " 'use particular',\n",
    " 'use studi',\n",
    " 'util',\n",
    " 'variat',\n",
    " 'version',\n",
    " 'version avail',\n",
    " 'version general',\n",
    " 'virtual',\n",
    " 'w',\n",
    " 'w.',\n",
    " 'we',\n",
    " 'we follow',\n",
    " 'we let',\n",
    " 'we make',\n",
    " 'where',\n",
    " 'wide',\n",
    " 'wide use',\n",
    " 'word',\n",
    " 'word-hyperbol',\n",
    " 'word-hyperbol group',\n",
    " 'x follow',\n",
    " 'x let',\n",
    " 'x −',\n",
    " \"x'i.\",\n",
    " \"x'k\",\n",
    " 'x.',\n",
    " 'x. let',\n",
    " 'x1',\n",
    " 'x1 x2',\n",
    " 'x2',\n",
    " 'x2.'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **Next Steps**\n",
    "\n",
    "During this project, I played around with the gensim python package for creating corpuses from Wikipedia data. This package handles many if not most of the tasks we performed using xml.sax, mwparserfromhell and sklearn. Not only is a powerful tool for creating a natural language proecessing pipeline, but it is also otimized for speed with built-in multiprocessing options."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### STEPS FOR FUTURE DEVELOPMENT:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Explore generalizability of model on to arXiv.org papers.\n",
    "* Train only on page metadata to explore tradeoff associated with classification speed increase and reduction in precision/recall"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Graph Analysis Approach / Community Detection\n",
    "\n",
    "Community detection tools consist of unsupervised, heuristic learning models that "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}