move llm notebooks and folders from root to new LLM folder (#164)

* fixed dependencies

* fixed pip install using sys.executable

* creating data directory for RAG data files

* added beautifulsoup4

* updated setup instructions

* moved llm content from root to LLM folder

* renamed 09 notebook

* updated README

* minor update

* minor update notebook

Author: krisrak

1_native_gpu.ipynb LLM/01_native_gpu.ipynb1_native_gpu.ipynb renamed to LLM/01_native_gpu.ipynb

@@ -2,9 +2,7 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "metadata": {
-    "jp-MarkdownHeadingCollapsed": true
-   },
+   "metadata": {},
    "source": [
     "# Building a Retrieval-Augmented Generation (RAG) System Workshop\n",
     "\n",
@@ -52,7 +50,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# What is RAG (Retrieval-augmented generation)\n",
+    "## What is RAG (Retrieval-augmented generation)\n",
     "\n",
     "**Retrieval-augmented generation (RAG)** is a technique for augmenting LLM knowledge with additional, often private or real-time, data. LLMs can reason about wide-ranging topics, but their knowledge is limited to the public data up to a specific point in time that they were trained on. If you want to build AI applications that can reason about private data or data introduced after a model’s cutoff date, you need to augment the knowledge of the model with the specific information it needs. The process of bringing the appropriate information and inserting it into the model prompt is known as Retrieval Augmented Generation (RAG)."
    ]
@@ -61,52 +59,90 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Step 1: Install Llamacpp python for SYCL\n",
-    "The llama.cpp SYCL backend is designed to support Intel GPU firstly. Based on the cross-platform feature of SYCL.\n",
+    "## Setup llama.cpp python for Intel CPUs and GPUs\n",
+    "The llama.cpp SYCL backend is designed to support Intel GPU. Based on the cross-platform feature of SYCL.\n",
+    "\n",
+    "We will setup Python environment and corresponding custom kernel for Jupyter Notebook, and we will install/build llama.cpp that will be used for the RAG Application.\n",
+    "\n",
+    "### Step 1: Create and activate Python environment:\n",
+    "\n",
+    "Open Terminal, make sure mini-forge is install and create new virtual environment\n",
+    "\n",
+    "```\n",
+    "    conda create -n llm-sycl python=3.11\n",
+    "\n",
+    "    conda activate llm-sycl\n",
     "\n",
-    "### After installation of conda-forge, open the Miniforge Prompt, and create a new python environment:\n",
-    "  ```\n",
-    "  conda create -n llm-sycl python=3.11\n",
+    "```\n",
+    "_Note: In case you want to remove the virtual environment, run the following command:_\n",
+    "```\n",
+    "    [conda remove -n llm-sycl --all]\n",
+    "```\n",
     "\n",
-    "  ```\n",
+    "### Step 2: Setup a custom kernel for Jupyter notebook:\n",
+    "\n",
+    "Run the following commands in the terminal to setup custom kernel for the Jupyter Notebook.\n",
     "\n",
-    "### Activate the new environment\n",
     "```\n",
-    "conda activate llm-sycl\n",
+    "    conda install -c conda-forge ipykernel\n",
     "\n",
+    "    python -m ipykernel install --user --name=llm-sycl\n",
+    "```\n",
+    "_Note: In case you want to remove the custom kernel from Jupyter, run the following command:_\n",
+    "```\n",
+    "    [python -m jupyter kernelspec uninstall llm-sycl]\n",
     "```\n",
     "\n",
     "<img src=\"Assets/llm4.png\">\n",
     "\n",
-    "## For Windows\n",
-    "### With the llm-sycl environment active, enable oneAPI environment. \n",
-    "Type oneapi in the windows search and then open the Intel oneAPI command prompt for Intel 64 for Visual Studio 2022 App.\n",
+    "### Step 3: Install and Build llama.cpp\n",
+    "\n",
+    "### For Linux\n",
     "\n",
+    "#### 1. Enable oneAPI environment\n",
     "\n",
-    "#### Run the below command in the VS command prompt and you should see the below sycl devices displayed in the console\n",
-    "There should be one or more level-zero GPU devices displayed as ext_oneapi_level_zero:gpu.\n",
+    "Make sure oneAPI Base Toolkit is installed to use the SYCL compiler for building llama.cpp\n",
     "\n",
+    "Run the following commands in terminal to initialize oneAPI environment and check available devices:\n",
+    "\n",
+    "```\n",
+    "    source /opt/intel/oneapi/setvars.sh\n",
+    "    sycl-ls\n",
     "```\n",
-    "sycl-ls\n",
+    "\n",
+    "#### 2. Install and build llama.cpp Python\n",
+    "\n",
+    "Run the following commands in terminal to install and build llama.cpp\n",
+    "\n",
+    "```\n",
+    "    CMAKE_ARGS=\"-DGGML_SYCL=on -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx\" pip install llama-cpp-python==0.3.1\n",
+    "```\n",
+    "\n",
+    "### For Windows\n",
+    "\n",
+    "#### 1. Enable oneAPI environment\n",
+    "\n",
+    "Make sure oneAPI Base Toolkit is installed to use the SYCL compiler for building llama.cpp\n",
+    "\n",
+    "Type oneapi in the windows search and then open the Intel oneAPI command prompt for Intel 64 for Visual Studio 2022 App.\n",
+    "\n",
+    "Run the following commands to initialize oneAPI environment and check available devices:\n",
     "\n",
     "```\n",
-    "### Install build tools\n",
+    "    @call \"C:\\Program Files (x86)\\Intel\\oneAPI\\setvars.bat\" intel64 --force\n",
+    "    sycl-ls\n",
+    "```\n",
+    "\n",
+    "#### 2. Install build tools\n",
     "\n",
     "* Download & install [cmake for Windows](https://cmake.org/download/):\n",
     "* The new Visual Studio will install Ninja as default. (If not, please install it manually: https://ninja-build.org/)\n",
     "\n",
-    "### Install llama.cpp Python\n",
+    "#### 3. Install and build llama.cpp Python\n",
     "\n",
-    "  \n",
     "* On the oneAPI command line window, step into the llama.cpp main directory and run the following:\n",
     "  \n",
-    "  ```\n",
-    "  @call \"C:\\Program Files (x86)\\Intel\\oneAPI\\setvars.bat\" intel64 --force\n",
-    "\n",
-    "    Open a new terminal and perform the following steps:\n",
-    "\n",
-    "\n",
-    "# Set the environment variables\n",
+    "```\n",
     "    set CMAKE_GENERATOR=Ninja\n",
     "    set CMAKE_C_COMPILER=cl\n",
     "    set CMAKE_CXX_COMPILER=icx\n",
@@ -115,40 +151,19 @@
     "    set CMAKE_ARGS=\"-DGGML_SYCL=ON -DGGML_SYCL_F16=ON -DCMAKE_CXX_COMPILER=icx -DCMAKE_C_COMPILER=cl\"\n",
     "    \n",
     "    pip install llama-cpp-python==0.3.1 -U --force --no-cache-dir --verbose\n",
-    "\n",
-    "## For Linux\n",
-    "    export GGML_SYCL=on\n",
-    "\n",
-    "    source /opt/intel/oneapi/setvars.sh   \n",
-    "    CMAKE_ARGS=\"-DGGML_SYCL=on -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx\" pip install llama-cpp-python==0.3.1\n",
-    "\n",
-    "### Setup a custom kernel for Jupyter notebook:\n",
-    "    conda install -c conda-forge ipykernel\n",
-    "\n",
-    "    python -m ipykernel install --user --name=llm-sycl"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "!pip install -r rag/requirements.txt"
+    "```\n"
    ]
   },
   {
    "attachments": {},
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## Run QA over Document\n",
-    "\n",
-    "Now, when model created, we can setup Chatbot\n",
+    "## RAG Application Details\n",
     "\n",
     "A typical RAG application has two main components:\n",
     "\n",
-    "- **Indexing**: a pipeline for ingesting data from a source and indexing it. This usually happen offline.\n",
+    "- **Indexing**: a pipeline for ingesting data from a source and indexing it. This usually happens offline.\n",
     "\n",
     "- **Retrieval and generation**: the actual RAG chain, which takes the user query at run time and retrieves the relevant data from the index, then passes that to the model.\n",
     "\n",
@@ -170,6 +185,28 @@
     "![Retrieval and generation pipeline](https://github.com/openvinotoolkit/openvino_notebooks/assets/91237924/f0545ddc-c0cd-4569-8c86-9879fdab105a)\n"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Install Python Modules\n",
+    "In Jupyter Lab, select `llm-sycl` as the kernel.\n",
+    "\n",
+    "You can now proceed installing modules and running python code in Jupyter Notebook"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "scrolled": true
+   },
+   "outputs": [],
+   "source": [
+    "import sys\n",
+    "!{sys.executable} -m pip install -r rag/requirements.txt"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -609,7 +646,9 @@
     "    \"\"\"\n",
     "    model_name_or_path = \"TheBloke/Llama-2-7B-Chat-GGUF\"\n",
     "    model_basename = \"llama-2-7b-chat.Q4_K_M.gguf\"\n",
+    "    print(\"Downloading Model...\\n   \" + model_name_or_path + \"/\" + model_basename)\n",
     "    MODEL_PATH = hf_hub_download(repo_id=model_name_or_path, filename=model_basename)\n",
+    "    print(\"Download Complete.\")\n",
     "    \n",
     "    # Initialize components that don't need to be recreated\n",
     "    embedding_fn = initialize_embedding_fn()\n",
@@ -718,9 +757,9 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "llm-test2",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
-   "name": "llm-test2"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -732,7 +771,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.11.10"
+   "version": "3.11.5"
   }
  },
  "nbformat": 4,