example_searches.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import weaviate\n",
    "from typing import List\n",
    "\n",
    "client = weaviate.Client(\"http://localhost:8080\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def run_query(text, limit=5) -> List:\n",
    "    response = client.query.get(\"Newsletter\", [\"header\", \"text_content\", \"links\", \"received_date\"]).with_near_text({\"concepts\": [text]}).with_limit(limit).do()\n",
    "    return response['data']['Get']['Newsletter']\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'header': 'MODEL-TUNING VIA PROMPTS: IMPROVING ADVERSARIAL ROBUSTNESS IN NLP (15 MINUTE READ)',\n",
       "  'links': ['https://arxiv.org/abs/2303.07320?utm_source=tldrai'],\n",
       "  'received_date': '2023-03-15T13:13:19Z',\n",
       "  'text_content': 'The MVP method demonstrates surprising gains in adversarial robustness, improving performance against adversarial word-level synonym substitutions by an average of 8% over standard methods, and even outperforming state-of-art defenses by 3.5%. By modifying the input with a prompt template instead of modifying the model by appending an MLP head, MVP achieves better results in downstream tasks while maintaining clean accuracy.'},\n",
       " {'header': 'AN EXAMPLE OF LLM PROMPTING FOR PROGRAMMING (12 MINUTE READ)',\n",
       "  'links': ['https://martinfowler.com/articles/2023-chatgpt-xu-hao.html?utm_source=tldrnewsletter'],\n",
       "  'received_date': '2023-04-19T10:59:59Z',\n",
       "  'text_content': 'This article gives examples of how to use ChatGPT to produce useful self-tested code. The prompt starts by setting the context for the application and how the code should be structured. Most of the prompt is setting out the design guidelines the code should follow. The prompt ensures that ChatGPT generates useful information about the problem first before it generates any code. Using this structure, ChatGPT can be more informed of its task and generate better responses.'},\n",
       " {'header': 'MLC LLM (GITHUB REPO)',\n",
       "  'links': ['https://github.com/mlc-ai/mlc-llm?utm_source=tldrnewsletter'],\n",
       "  'received_date': '2023-05-01T11:01:29Z',\n",
       "  'text_content': 'MLC LLM can run any language model natively on a diverse set of hardware backends, enabling anyone to develop, optimize, and deploy AI models natively on many devices.'},\n",
       " {'header': 'EMERGENT ABILITIES OF LARGE LANGUAGE MODELS (12 MINUTE READ)',\n",
       "  'links': ['https://www.assemblyai.com/blog/emergent-abilities-of-large-language-models/?utm_source=tldrai'],\n",
       "  'received_date': '2023-03-09T14:13:00Z',\n",
       "  'text_content': 'Language models (LMs) are a class of probabilistic models that learn patterns in natural language. LMs can be utilized for generative purposes to generate, say, the next event in a story by exploiting their knowledge of these patterns. In recent years, significant efforts have been put into scaling LMs into Large Language Models (LLMs). As LLMs are scaled they hit a series of critical scales at which new abilities are suddenly “unlocked”. LLMs are not directly trained to have these abilities. These emergent abilities include performing arithmetic, answering questions, summarizing passages, and more.'},\n",
       " {'header': 'LANGCHAIN (GITHUB REPO)',\n",
       "  'links': ['https://github.com/hwchase17/langchain?utm_source=tldrnewsletter'],\n",
       "  'received_date': '2022-12-13T12:57:39Z',\n",
       "  'text_content': 'LangChain is a library that assists with the development of applications that combine large language models with other sources of computation or knowledge. It is designed to help with LLM and prompts, chaining LLMs, agents, and memory. Several open and closed source demos of applications built with LangChain are available in the full documentation.'}]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"framework for dealing with llm prompts\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'header': 'LLMS ARE NOT GREAT UNCERTAIN AGENT SIMULATORS (17 MINUTE READ)',\n",
       "  'links': ['https://www.danieldjohnson.com/2023/03/27/uncertain_simulators/?utm_source=tldrai'],\n",
       "  'received_date': '2023-03-28T13:11:38Z',\n",
       "  'text_content': '“The hallucination problem” is really a problem of uncertainty estimation and calibration. Language models are great at simulating agents, but often those agents (people who wrote stuff online) have access to privileged information that the LLM does not. This means the language model is simulating an agent without full information, which leads to uncertainty. There are ways around this, but it is potentially the next big open problem in language modeling.'},\n",
       " {'header': 'LANGUAGE MODELS CAN CRITIQUE THEMSELVES? (9 MINUTE READ)',\n",
       "  'links': ['https://evjang.com/2023/03/26/self-reflection.html?utm_source=tldrai'],\n",
       "  'received_date': '2023-04-04T13:12:46Z',\n",
       "  'text_content': 'In the spirit of solutions being easier to evaluate than generate, language models at scale possess the amazing ability to properly critique their output according to some questions proposed. Doing this recursively and by generating their own questions, language models can increase the quality of their generations dramatically.'},\n",
       " {'header': 'AGENTS ARE THE NEW PARADIGM (26 MINUTE READ)',\n",
       "  'links': ['https://arxiv.org/abs/2304.03442?utm_source=tldrai'],\n",
       "  'received_date': '2023-04-11T13:13:44Z',\n",
       "  'text_content': \"What happens if you use language models to simulate an entire town? They ask each other out on Valentine's dates. This paper explores what happens if you give language models open-ended tasks and shows that you can observe remarkable human-like social capabilities from these agents, such as keeping schedules, planning activities, and reminiscing about previous interactions. This is one step in a broader trend of language models moving more towards agent-like paradigms, which dramatically changes the safety considerations for true model alignment.\"},\n",
       " {'header': 'AGENTS FOR SCIENCE (15 MINUTE READ)',\n",
       "  'links': ['https://arxiv.org/abs/2304.05332?utm_source=tldrai'],\n",
       "  'received_date': '2023-04-13T13:13:30Z',\n",
       "  'text_content': 'The idea of language models acting as planners for tools (which may be other language models) is increasing in popularity in the community. Essentially a central, powerful, language model such as GPT-4 is given access to a set of tools. These tools could be web search, documentation, python engine, and in this case an experimentation API. Powered by these additional capabilities, the model was able to synthesize chemicals, reason about reactions, and even refused to synthesize heroin - which is an interesting example of potentially out of distribution alignment.'},\n",
       " {'header': 'LANGUAGE IS NOT ALL YOU NEED: ALIGNING PERCEPTION WITH LANGUAGE MODELS (20 MINUTE READ)',\n",
       "  'links': ['https://arxiv.org/abs/2302.14045?utm_source=tldrai'],\n",
       "  'received_date': '2023-02-28T14:12:11Z',\n",
       "  'text_content': 'KOSMOS-12 is a Multimodal Large Language Model that can learn in context, follow instructions, and perceive general modalities. It achieves impressive performance on a range of tasks, including language understanding, perception-language, and vision tasks. The model was trained on web-scale multimodal corpora and benefits from cross-modal transfer. The authors also introduce a dataset for diagnosing the nonverbal reasoning capability of MLLMs.'}]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"New framework for evaluating agency in large language models\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'header': 'ROMANIA UNVEILS WORLD’S FIRST AI GOVERNMENT “ADVISOR” (2 MINUTE READ)',\n",
       "  'links': ['https://interestingengineering.com/innovation/romania-unveils-worlds-first-ai-advisor?utm_source=tldrai'],\n",
       "  'received_date': '2023-03-03T14:14:03Z',\n",
       "  'text_content': 'The Romanian government has unveiled \"Ion,\" an artificial intelligence (AI) based platform built to record Romanians\\' voices and opinions and use them to guide state policy decisions.'},\n",
       " {'header': 'INSIDER Q&A: OPENAI CTO MIRA MURATI ON SHEPHERDING CHATGPT (6 MINUTE READ)',\n",
       "  'links': ['https://apnews.com/article/openai-cto-mira-murati-chatgpt-gpt4-dalle-0e701dd406b4a1a2d625e779de0c5164?utm_source=tldrnewsletter'],\n",
       "  'received_date': '2023-04-26T11:00:40Z',\n",
       "  'text_content': \"Mira Murati is OpenAI's chief technology officer. She leads OpenAI's research, product, and safety teams. This article features an interview with Murati where she talks about AI safeguards and OpenAI's vision for artificial general intelligence. The interview also covers her views on AI regulation, OpenAI as a company, and ChatGPT's launch.\"},\n",
       " {'header': 'THE UK IS SPENDING £100 MILLION TO DEVELOP ITS OWN SOVEREIGN AI (1 MINUTE READ)',\n",
       "  'links': ['https://www.engadget.com/the-uk-is-creating-a-100-million-ai-taskforce-143507868.html?src=rss?utm_source=tldrai'],\n",
       "  'received_date': '2023-04-25T13:15:11Z',\n",
       "  'text_content': 'Prime Minister Rishi Sunak and Technology Secretary Michelle Donelan have pledged an initial £100 million (about $124.5 million) to establish a Foundation Model Taskforce. The team will develop AI that ideally makes the country \"globally competitive,\" and will work with the industry to make these systems safer and more reliable.'},\n",
       " {'header': \"A PRIVACY HERO'S FINAL WISH: AN INSTITUTE TO REDIRECT AI'S FUTURE (4 MINUTE READ)\",\n",
       "  'links': ['https://archive.ph/lViGi?utm_source=tldrnewsletter'],\n",
       "  'received_date': '2023-03-07T12:12:58Z',\n",
       "  'text_content': \"Peter Eckersley was a web encryption pioneer. The AI Objectives Institute (AOI), has taken on Eckersley's work to redirect the future of artificial intelligence away from destructive forces. Eckersley envisioned AOI as a hub for major AI labs to come together and incubate ideas that actively contribute to human flourishing. AI has the potential to amplify existing global issues such as environmental destruction, exploitation of the poor, and rampant nationalism. Rather than attempting to restrict AI's progress, AOI aims to steer AI towards a better future by channeling its efforts to help alleviate current global issues.\"},\n",
       " {'header': 'HYPE GROWS OVER “AUTONOMOUS” AI AGENTS (4 MINUTE READ)',\n",
       "  'links': ['https://arstechnica.com/information-technology/2023/04/hype-grows-over-autonomous-ai-agents-that-loop-gpt-4-outputs/?utm_source=tldrai'],\n",
       "  'received_date': '2023-04-18T13:14:11Z',\n",
       "  'text_content': 'Enthusiasm is building around autonomous AI agents that incorporate GPT-4 outputs in a continuous loop, showcasing the potential of these advanced models to drive innovation and create new applications across various industries and fields.'}]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"AI agency\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'header': 'DESIGN NOTES ON THE 2023 WIKIPEDIA REDESIGN (25 MINUTE READ)',\n",
       "  'links': ['https://alexhollender.com/wikipedia-2023-redesign?utm_source=tldrnewsletter'],\n",
       "  'received_date': '2023-03-16T11:13:28Z',\n",
       "  'text_content': 'The Wikipedia desktop interface has been redesigned. The last significant change to the interface was made in 2004. Since then, usability for new editors has been improved and the editing experience was updated. However, the general layout and reading experience remained largely unchanged until the redesign began in 2019. Devices have changed significantly since 2004, resulting in the need for a modernized interface. This article contains notes on the redesign of the Wikipedia desktop interface by Alex Hollender, who led the project.'},\n",
       " {'header': 'OPENPLAYGROUND (GITHUB REPO)',\n",
       "  'links': ['https://github.com/nat/openplayground?utm_source=tldrnewsletter'],\n",
       "  'received_date': '2023-04-05T10:59:01Z',\n",
       "  'text_content': 'openplayground is a large language model playground that can run on a laptop. It allows users to leverage a variety of language models on their local machine. openplayground provides a full suite of UI features, including keyboard shortcuts, logprops, and parameter tuning. Multiple models can be run side-by-side for comparison purposes.'},\n",
       " {'header': 'WEB LLM (GITHUB REPO)',\n",
       "  'links': ['https://github.com/mlc-ai/web-llm?utm_source=tldrnewsletter'],\n",
       "  'received_date': '2023-04-17T10:58:43Z',\n",
       "  'text_content': 'Web LLM is a project that runs large language models directly in the browser. It runs everything within the browser with no server support. Web LLM aims to provide AI assistants with privacy for everyone. A web demo is available.'},\n",
       " {'header': 'INVOKEAI (GITHUB REPO)',\n",
       "  'links': ['https://github.com/invoke-ai/InvokeAI?utm_source=tldrnewsletter'],\n",
       "  'received_date': '2022-12-05T12:55:55Z',\n",
       "  'text_content': 'InvokeAI is a Stable Diffusion toolkit. It features a web interface and an easy-to-use command-line interface. InvokeAI runs on Windows, macOS, and Linux machines with as little as 4GB of GPU RAM.'},\n",
       " {'header': 'WHAT THE HECK IS OAUTH? (SPONSOR)',\n",
       "  'links': ['https://fusionauth.io/ebooks/modern-guide-to-oauth?utm_medium=email&utm_source=tldr&utm_campaign=oauth_ebook'],\n",
       "  'received_date': '2023-01-25T12:05:41Z',\n",
       "  'text_content': 'When you click on the \"Login with Google\" button on a website form, that is OAuth in action. It\\'s the standard that lets users grant websites access to their information on other sites. But how does it work? We put together an expert guide based on hundreds of conversations and implementations of an OAuth server that has been downloaded millions of times. No fluff, just a real-world perspective covering the modern use cases of OAuth.'}]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"User interface\")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "vector-db-env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.5"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}