Merge pull request #13 from mlrun/main

merge main

Author: daniels290813

01_churn_ml_model.ipynb

@@ -5,25 +5,37 @@
    "id": "326b3005-9a8e-4b07-9d4c-bb2e2ae5ffd9",
    "metadata": {},
    "source": [
-    "# Churn Model Training + Deployment Pipeline\n",
+    "# Churn Model: training and deployment pipeline\n",
     "\n",
-    "The first part of this demo is to train a tabular ML model for predicting churn. This will be used later in our application pipeline to provide context to the LLM to customize the response.\n",
+    "The first part of this demo is to train a tabular ML model for predicting churn. This will be used later in the application pipeline to provide context to the LLM to customize the response.\n",
     "\n",
-    "This training pipeline is a standard ML training and deployment pipeline. The only \"unusual\" step is an additional pre-processing step that generates a sentiment analysis score using historical conversational data from a chat log. This sentiment score is used as a model input - we will see later that it is actually the most important feature when predicting churn.\n",
-    "\n",
-    "In addition to model training, this notebook demonstrates how to log datasets, run ML pipelines remotely, deploy models as serving functions, and enable model monitoring for production use. The workflow is fully automated using MLRun, making it easy to track experiments, manage artifacts, and monitor deployed models in real time.\n",
-    "\n",
-    "# Churn Model Training + Deployment Pipeline\n",
-    "\n",
-    "The first part of this demo is to train a tabular ML model for predicting churn. This will be used later in our application pipeline to provide context to the LLM to customize the response.\n",
-    "\n",
-    "This training pipeline is a standard ML training and deployment pipeline. The only \"unusual\" step is an additional pre-processing step that generates a sentiment analysis score using historical conversational data from a chat log. This sentiment score is used as a model input - we will see later that it is actually the most important feature when predicting churn.\n",
+    "This training pipeline is a standard ML training and deployment pipeline. The only \"unusual\" step is an additional pre-processing step that generates a sentiment analysis score using historical conversational data from a chat log. This sentiment score is used as a model input - you'll see later that it is actually the most important feature when predicting churn.\n",
     "\n",
     "In addition to model training, this notebook demonstrates how to log datasets, run ML pipelines remotely, deploy models as serving functions, and enable model monitoring for production use. The workflow is fully automated using MLRun, making it easy to track experiments, manage artifacts, and monitor deployed models in real time.\n",
     "\n",
     "![](images/01_churn_ml_model_architecture.png)"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "6fdd7bba",
+   "metadata": {},
+   "source": [
+    "## Table of contents\n",
+    "    1. [Install MLRun](#install-mlrun)\n",
+    "    2. [Set up the project](#set-up-the-project)\n",
+    "    2. [Log the dataset](#log-the-dataset)\n",
+    "    2. [Run the pipeline](#run-the-pipeline)\n",
+    "    2. [View the output training artifacts](#view-the-output-training-artifacts)\n",
+    "    2. [Test the model endpoint](#test-the-model-endpoint)\n",
+    "    2. [Model monitoring](#model-monitoring)\n",
+    " \n",
+    " (install-mlrun)=\n",
+    " ## Install MLRun and set up the environment\n",
+    "\n",
+    " First import mlrun and other required packages:"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -51,13 +63,13 @@
    "id": "3a1ca344-b7fd-47a7-86b7-575d352851a4",
    "metadata": {},
    "source": [
-    "### Setup Project\n",
+    "### Set the up project\n",
     "\n",
-    "First, create and populate the MLRun project. Under the hood, this will execute [project_setup.py](project_setup.py) to add our MLRun functions, workflows, and build the project image - see the [documentation](https://docs.mlrun.org/en/stable/projects/project-setup.html) for more information.\n",
+    "Now, create and populate the MLRun project. Under the hood, this executes [project_setup.py](project_setup.py) to add our MLRun functions, workflows, and build the project image. See the [documentation](https://docs.mlrun.org/en/stable/projects/project-setup.html) for more information.\n",
     "\n",
     "Make sure to enable `force_build` on the first run to build the project image. After the initial setup, you can disable `force_build` to speed up subsequent runs. This setup ensures all dependencies and source code are packaged for reproducible and scalable ML workflows.\n",
     "\n",
-    "Set the OpenAI credentials in the project and local environment - **be sure to update [.env.example](.env.example) as described in the [README](README.md)**."
+    "Set the OpenAI credentials in the project and local environment - **be sure to update [.env.example](.env.example) as described in the [README](README.md#prerequisites)**."
    ]
   },
   {
@@ -103,7 +115,7 @@
    "id": "43af04dc",
    "metadata": {},
    "source": [
-    "*Please note that you will need to set the `\"build_image\": True` once to build the default image for the project. After you successfully built the image, set the `\"build_image\": False\"` to continue.*"
+    "**Set the `\\\"build_image\\\": True` once to build the default image for the project. After you successfully built the image, set the `\\\"build_image\\\": False\\\"` to continue.**"
    ]
   },
   {
@@ -133,7 +145,7 @@
    "id": "6c1e6f8a-b144-4311-bdf4-7a5fee3e6bd9",
    "metadata": {},
    "source": [
-    "Enables project model monitoring and deploys required infrastructure - see [documentation](https://docs.mlrun.org/en/stable/tutorials/05-model-monitoring.html#realtime-monitor-drift-tutor) for more information. Only needs to be run once."
+    "Enable project model monitoring on the project and deploy required infrastructure. See [documentation](https://docs.mlrun.org/en/stable/tutorials/05-model-monitoring.html#realtime-monitor-drift-tutor) for more information. This only needs to be run once."
    ]
   },
   {
@@ -154,7 +166,7 @@
    "id": "7e50bd0d-096f-4322-b509-31b3ac358782",
    "metadata": {},
    "source": [
-    "### Log Dataset"
+    "### Log  the dataset"
    ]
   },
   {
@@ -186,15 +198,17 @@
    "id": "a813c814-e7e1-45b8-b991-72579efe8eda",
    "metadata": {},
    "source": [
-    "### Run Pipeline"
+    "### Run the pipeline"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "cab7d535-244d-4f21-9e1e-61310afed118",
    "metadata": {},
    "source": [
-    "Submits [train_and_deploy_workflow.py](src/workflows/train_and_deploy_workflow.py) via Kubeflow Pipelines. ***Note**: Requires minimum 6 CPU to run (see `sentiment_fn` step).*"
+    "This step submits the [train_and_deploy_workflow.py](src/workflows/train_and_deploy_workflow.py) via Kubeflow Pipelines. \n",
+    "\n",
+    "This requires a minimum of 6 CPUs to run (see `sentiment_fn` step in train_and_deploy_workflow.py)."
    ]
   },
   {
@@ -517,13 +531,13 @@
    "id": "8e4207f4-4fe7-474d-b5d0-761da5a3226e",
    "metadata": {},
    "source": [
-    "### View Output Training Artifacts\n",
+    "### View the output training artifacts\n",
     "\n",
     "View the training artifacts in the MLRun UI like so:\n",
     "\n",
     "![](images/churn_pipeline.png)\n",
     "\n",
-    "Or pull the artifacts directly into your notebook for additional exploration like below - see nore information about using data & artifacts in the [documentation](https://docs.mlrun.org/en/stable/concepts/data.html)."
+    "Alternatively, you can pull the artifacts directly into your notebook for additional exploration like below. See more information about using data and artifacts in the [documentation](https://docs.mlrun.org/en/stable/concepts/data.html)."
    ]
   },
   {
@@ -926,15 +940,15 @@
    "id": "0dfeee46-4f82-4277-aad1-3cc5583ca8f3",
    "metadata": {},
    "source": [
-    "### Test Model Endpoint"
+    "### Test  the model endpoint"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "9c4d263a-297a-4d2d-b8f4-03e5790e0789",
    "metadata": {},
    "source": [
-    "Test newly deployed real-time endpoint using test data."
+    "Test the newly deployed real-time endpoint using test data."
    ]
   },
   {
@@ -998,9 +1012,9 @@
    "id": "e536babe-e987-4b33-a93c-243fd8abeb43",
    "metadata": {},
    "source": [
-    "### Model Monitoring\n",
+    "### Model monitoring\n",
     "\n",
-    "Once the churn model is deployed and invoked, you will be able to view the model monitoring results in the MLRun UI:\n",
+    "Once the churn model is deployed and invoked, you can view the model monitoring results in the MLRun UI:</br>\n",
     "![](images/tabular_model_monitoring.png)"
    ]
   }

02_guardrail_deployment.ipynb

@@ -5,9 +5,9 @@
    "id": "7bd0dfc6-4ed5-40f1-ab93-9fcb72fdb3b6",
    "metadata": {},
    "source": [
-    "# Guardrail Deployment\n",
+    "# Guardrail deployment\n",
     "\n",
-    "The second part of the demo is to deploy guardrails to be used later in our application pipeline to filter user inputs. This notebook will also deploy an LLM as a Judge monitoring application to monitor our generative input guardrail for banking topic adherence.\n",
+    "The second part of the demo is to deploy guardrails to be used later in the application pipeline to filter user inputs. This notebook will also deploy an LLM as a Judge monitoring application to monitor our generative input guardrail for banking topic adherence.\n",
     "\n",
     "In this notebook, you will:\n",
     "- Set up and configure project secrets and environment variables.\n",
@@ -51,7 +51,7 @@
    "id": "a763105c-c7b4-4fa4-b719-a371fd897f51",
    "metadata": {},
    "source": [
-    "### Setup Project\n",
+    "### Setup the project\n",
     "\n",
     "Load the previously created project in the first notebook."
    ]
@@ -71,15 +71,15 @@
    "id": "0e8481d6-7361-4bfa-b705-0a5765399f48",
    "metadata": {},
    "source": [
-    "### LLM as a Judge Monitoring Application"
+    "### LLM as a judge monitoring application"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "d88f46ba",
    "metadata": {},
    "source": [
-    "The \"LLM as a Judge\" monitoring application leverages a large language model (LLM) to automatically evaluate and score the effectiveness of deployed guardrails. By providing a rubric and clear examples, the LLM acts as an impartial evaluator, determining whether user inputs are correctly classified according to defined criteria (e.g., banking-topic relevance). This approach enables scalable, consistent, and automated assessment of guardrail performance, ensuring that only appropriate and relevant inputs are processed by downstream applications.\n",
+    "The \"LLM as a judge\" monitoring application leverages a large language model (LLM) to automatically evaluate and score the effectiveness of deployed guardrails. By providing a rubric and clear examples, the LLM acts as an impartial evaluator, determining whether user inputs are correctly classified according to defined criteria (e.g., banking-topic relevance). This approach enables scalable, consistent, and automated assessment of guardrail performance, ensuring that only appropriate and relevant inputs are processed by downstream applications.\n",
     "\n",
     "This implementation is pulled from another [MLRun demo - LLM monitoring and feedback loop: Banking](https://github.com/mlrun/demo-monitoring-and-feedback-loop/tree/main)."
    ]
@@ -121,7 +121,7 @@
    "id": "a1566010",
    "metadata": {},
    "source": [
-    "More information about model monitoring applications in the context of LLM's can be found in the [documentation](https://docs.mlrun.org/en/stable/tutorials/genai-02-model-monitor-llm.html#genai-02-mm-llm)."
+    "More information about model monitoring applications in the context of LLM's can be found in the [documentation's Model monitoring using LLM tutorial](https://docs.mlrun.org/en/stable/tutorials/genai-02-model-monitor-llm.html)."
    ]
   },
   {
@@ -187,9 +187,9 @@
    "id": "73812b75-e859-43c4-8ed1-3bd88b2d1f28",
    "metadata": {},
    "source": [
-    "### Banking Topic Guardrail\n",
+    "### Banking topic guardrail\n",
     "\n",
-    "The Banking Topic Guardrail is an LLM-powered filter designed to ensure that only banking-related user inputs are processed by downstream applications. It acts as a first line of defense, automatically classifying each user message as either relevant (`True`) or irrelevant (`False`) to banking topics, based on the context of the entire conversation.\n",
+    "The Banking topic guardrail is an LLM-powered filter designed to ensure that only banking-related user inputs are processed by downstream applications. It acts as a first line of defense, automatically classifying each user message as either relevant (`True`) or irrelevant (`False`) to banking topics, based on the context of the entire conversation.\n",
     "\n",
     "It's important to distinguish between the guardrail itself (this component), which enforces topic adherence in real time within the application, and the monitoring application described above. The monitoring application uses an LLM as a \"judge\" to independently evaluate and score the effectiveness of this guardrail, providing oversight and ensuring that the guardrail is functioning as intended. This separation allows for both proactive filtering and ongoing quality assurance of user input handling."
    ]
@@ -202,7 +202,7 @@
    "outputs": [],
    "source": [
     "SYSTEM_PROMPT_GUARDRAILS_V2 = \"\"\"\n",
-    "You are input guardrails for an AI banking agent that responds exclusively to questions pertaining to banking topics. Respond only with a boolean true/false value on whether the input adheres to banking topics. Consider the most recent input in the context of the whole conversation. Do not include any pre or post amble.\n",
+    "You are input-guardrails for an AI banking agent that responds exclusively to questions pertaining to banking topics. Respond only with a boolean true/false value on whether the input adheres to banking topics. Consider the most recent input in the context of the whole conversation. Do not include any pre or post amble.\n",
     "\n",
     "Examples:\n",
     "Q: What is the process to apply for a mortgage?\n",
@@ -467,9 +467,9 @@
    "id": "ac63e4d8-e9c7-487f-b4a8-6f410c0df1cb",
    "metadata": {},
    "source": [
-    "### Toxicity Filter Guardrail\n",
+    "### Toxicity filter guardrail\n",
     "\n",
-    "The Toxicity Filter Guardrail is designed to automatically detect and filter out user inputs that contain toxic, offensive, or inappropriate language. By leveraging a toxicity classification model, this guardrail ensures that only safe and respectful messages are processed by downstream applications. This helps maintain a positive user experience and protects the system from harmful or disruptive content. The toxicity filter can be customized with a threshold to determine the sensitivity of the filter, allowing for flexible adaptation to different application requirements.\n",
+    "The Toxicity filter guardrail is designed to automatically detect and filter out user inputs that contain toxic, offensive, or inappropriate language. By leveraging a toxicity classification model, this guardrail ensures that only safe and respectful messages are processed by downstream applications. This helps maintain a positive user experience and protects the system from harmful or disruptive content. The toxicity filter can be customized with a threshold to determine the sensitivity of the filter, allowing for flexible adaptation to different application requirements.\n",
     "\n",
     "The output of the toxicity guardrail is a boolean value (`True` or `False`). A result of `True` means the input passes the guardrail (i.e., is non-toxic and allowed through), while `False` indicates the input is flagged as toxic and is blocked from further processing."
    ]