👋 Welcome! This repo contains demos showcasing TrustyAI's guardrailing and model evaluation features within Red Hat Openshift AI.
- Evaluation Quickstart: This demo will quickly get you started running an evaluation against a deployed model.
- Garak Quickstart: Run Garak security scans on Kubeflow Pipelines to test LLMs for vulnerabilities.
- Guardrails Quickstart: This demo will quickly get you started with three detectors, for detecting hate speech, gibberish, and jailbreaking respectively.
- Custom Detectors: This demo shows off how to create custom detectors via Python, and provides an example of LLM self-reflection guardrailing.
- Lemonade Stand: A demo showing manual configuration of guardrails, as shown in the Guardrails for AI models video on the RH YouTube channel
- NeMo-Guardrails Quickstart: This demo will quickly get you started using NVIDIA's NeMo-Guardrails project to perform guardrailing on OpenShift AI.
- Language Detector Demo: Attacking LLMs using non-English prompts is a common approach. This demo showing how to use a language classification model to guardrail against these kinds of attacks.
If you run into issues, see the troubleshooting guide for common issues and their solutions.