Skip to content

Commit 7e47d38

Browse files
nathan-weinbergnathan-weinberg
committed
[WIP]feat: add API level information to distro docs
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
1 parent 37762d4 commit 7e47d38

File tree

3 files changed

+58
-34
lines changed

3 files changed

+58
-34
lines changed

‎.pre-commit-config.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,3 +67,4 @@ repos:
6767
files: ^distribution/run\.yaml$
6868
additional_dependencies:
6969
- pyyaml==6.0.2
70+
- git+https://github.com/opendatahub-io/llama-stack.git@v0.3.0rc3+rhai0

‎distribution/README.md‎

Lines changed: 28 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -8,31 +8,31 @@ The image is currently shipping with the Open Data Hub version of Llama Stack ve
88

99
You can see an overview of the APIs and Providers the image ships with in the table below.
1010

11-
| API | Provider | External? | Enabled by default? | How to enable |
12-
|-----|----------|-----------|---------------------|---------------|
13-
| agents | inline::meta-reference | No | ✅ | N/A |
14-
| datasetio | inline::localfs | No | ✅ | N/A |
15-
| datasetio | remote::huggingface | No | ✅ | N/A |
16-
| eval | inline::trustyai_ragas | Yes (version 0.4.2) | ❌ | Set the `EMBEDDING_MODEL` environment variable |
17-
| eval | remote::trustyai_lmeval | Yes (version 0.3.1) | ✅ | N/A |
18-
| eval | remote::trustyai_ragas | Yes (version 0.4.2) | ❌ | Set the `KUBEFLOW_LLAMA_STACK_URL` environment variable |
19-
| files | inline::localfs | No | ✅ | N/A |
20-
| inference | inline::sentence-transformers | No | ✅ | N/A |
21-
| inference | remote::azure | No | ❌ | Set the `AZURE_API_KEY` environment variable |
22-
| inference | remote::bedrock | No | ❌ | Set the `AWS_ACCESS_KEY_ID` environment variable |
23-
| inference | remote::openai | No | ❌ | Set the `OPENAI_API_KEY` environment variable |
24-
| inference | remote::vertexai | No | ❌ | Set the `VERTEX_AI_PROJECT` environment variable |
25-
| inference | remote::vllm | No | ❌ | Set the `VLLM_URL` environment variable |
26-
| inference | remote::watsonx | No | ❌ | Set the `WATSONX_API_KEY` environment variable |
27-
| safety | remote::trustyai_fms | Yes (version 0.2.3) | ✅ | N/A |
28-
| scoring | inline::basic | No | ✅ | N/A |
29-
| scoring | inline::braintrust | No | ✅ | N/A |
30-
| scoring | inline::llm-as-judge | No | ✅ | N/A |
31-
| telemetry | inline::meta-reference | No | ✅ | N/A |
32-
| tool_runtime | inline::rag-runtime | No | ✅ | N/A |
33-
| tool_runtime | remote::brave-search | No | ✅ | N/A |
34-
| tool_runtime | remote::model-context-protocol | No | ✅ | N/A |
35-
| tool_runtime | remote::tavily-search | No | ✅ | N/A |
36-
| vector_io | inline::faiss | No | ❌ | Set the `ENABLE_FAISS` environment variable |
37-
| vector_io | inline::milvus | No | ✅ | N/A |
38-
| vector_io | remote::milvus | No | ❌ | Set the `MILVUS_ENDPOINT` environment variable |
11+
| Provider API | Provider | REST APIs | External? | Enabled by default? | How to enable |
12+
|--------------|----------|-----------|-----------|---------------------|---------------|
13+
| agents | inline::meta-reference | | No | ✅ | N/A |
14+
| datasetio | inline::localfs | /v1/files | No | ✅ | N/A |
15+
| datasetio | remote::huggingface | | No | ✅ | N/A |
16+
| eval | inline::trustyai_ragas | | Yes (version 0.4.2) | ❌ | Set the `EMBEDDING_MODEL` environment variable |
17+
| eval | remote::trustyai_lmeval | | Yes (version 0.3.1) | ✅ | N/A |
18+
| eval | remote::trustyai_ragas | | Yes (version 0.4.2) | ❌ | Set the `KUBEFLOW_LLAMA_STACK_URL` environment variable |
19+
| files | inline::localfs | /v1/files | No | ✅ | N/A |
20+
| inference | inline::sentence-transformers | /v1/completions<br>/v1/embeddings | No | ✅ | N/A |
21+
| inference | remote::azure | /v1/completions<br>/v1/embeddings | No | ❌ | Set the `AZURE_API_KEY` environment variable |
22+
| inference | remote::bedrock | /v1/completions<br>/v1/embeddings | No | ❌ | Set the `AWS_ACCESS_KEY_ID` environment variable |
23+
| inference | remote::openai | /v1/completions<br>/v1/embeddings | No | ❌ | Set the `OPENAI_API_KEY` environment variable |
24+
| inference | remote::vertexai | /v1/completions<br>/v1/embeddings | No | ❌ | Set the `VERTEX_AI_PROJECT` environment variable |
25+
| inference | remote::vllm | /v1/completions<br>/v1/embeddings | No | ❌ | Set the `VLLM_URL` environment variable |
26+
| inference | remote::watsonx | | No | ❌ | Set the `WATSONX_API_KEY` environment variable |
27+
| safety | remote::trustyai_fms | | Yes (version 0.2.3) | ✅ | N/A |
28+
| scoring | inline::basic | | No | ✅ | N/A |
29+
| scoring | inline::braintrust | | No | ✅ | N/A |
30+
| scoring | inline::llm-as-judge | | No | ✅ | N/A |
31+
| telemetry | inline::meta-reference | | No | ✅ | N/A |
32+
| tool_runtime | inline::rag-runtime | | No | ✅ | N/A |
33+
| tool_runtime | remote::brave-search | | No | ✅ | N/A |
34+
| tool_runtime | remote::model-context-protocol | | No | ✅ | N/A |
35+
| tool_runtime | remote::tavily-search | | No | ✅ | N/A |
36+
| vector_io | inline::faiss | /v1/vector_stores | No | ❌ | Set the `ENABLE_FAISS` environment variable |
37+
| vector_io | inline::milvus | /v1/vector_stores | No | ✅ | N/A |
38+
| vector_io | remote::milvus | | No | ❌ | Set the `MILVUS_ENDPOINT` environment variable |

‎scripts/gen_distro_docs.py‎

Lines changed: 29 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,19 @@
33
import yaml
44
import re
55
from pathlib import Path
6+
from llama_stack.core.library_client import LlamaStackAsLibraryClient
67

78

89
REPO_ROOT = Path(__file__).parent.parent
910

1011

12+
def extract_llama_stack_routes(distro_path: str):
13+
"""Extract Llama Stack route information"""
14+
client = LlamaStackAsLibraryClient(str(distro_path))
15+
routes = client.routes.list()
16+
return routes
17+
18+
1119
def extract_llama_stack_version():
1220
"""Extract Llama Stack version from the Containerfile."""
1321
containerfile_path = REPO_ROOT / "distribution" / "Containerfile"
@@ -87,11 +95,11 @@ def load_external_providers_info():
8795
exit(1)
8896

8997

90-
def gen_distro_table(providers_data):
98+
def gen_distro_table(providers_data, routes):
9199
# Start with table header
92100
table_lines = [
93-
"| API | Provider | External? | Enabled by default? | How to enable |",
94-
"|-----|----------|-----------|---------------------|---------------|",
101+
"| Provider API | Provider | REST APIs | External? | Enabled by default? | How to enable |",
102+
"|--------------|----------|-----------|-----------|---------------------|---------------|",
95103
]
96104

97105
# Load external provider information from build.yaml
@@ -108,6 +116,17 @@ def gen_distro_table(providers_data):
108116
provider_type = provider["provider_type"]
109117
provider_id = provider.get("provider_id", "")
110118

119+
rest_api_pairs = []
120+
for route in routes:
121+
if (provider_type != "inline::meta-reference") and \
122+
(provider_type in route.provider_types) and \
123+
(len(route.route.split("/")) < 4 ) and \
124+
(route.route not in rest_api_pairs):
125+
rest_api_pairs.append(
126+
route.route
127+
)
128+
rest_apis = "<br>".join(rest_api_pairs)
129+
111130
# Check if provider_id contains the conditional syntax ${<something>:+<something>}
112131
# This regex matches the pattern ${...} containing :+
113132
conditional_match = re.search(
@@ -133,6 +152,7 @@ def gen_distro_table(providers_data):
133152
(
134153
api_name,
135154
provider_type,
155+
rest_apis,
136156
external_status,
137157
enabled_by_default,
138158
how_to_enable,
@@ -146,12 +166,13 @@ def gen_distro_table(providers_data):
146166
for (
147167
api_name,
148168
provider_type,
169+
rest_apis,
149170
external_status,
150171
enabled_by_default,
151172
how_to_enable,
152173
) in api_provider_pairs:
153174
table_lines.append(
154-
f"| {api_name} | {provider_type} | {external_status} | {enabled_by_default} | {how_to_enable} |"
175+
f"| {api_name} | {provider_type} | {rest_apis} | {external_status} | {enabled_by_default} | {how_to_enable} |"
155176
)
156177

157178
return "\n".join(table_lines)
@@ -190,13 +211,15 @@ def gen_distro_docs():
190211

191212
# Extract providers section
192213
providers = run_yaml_data.get("providers", {})
193-
194214
if not providers:
195215
print("Error: No providers found in run.yaml")
196216
return 1
197217

218+
# Extract routes info
219+
routes = extract_llama_stack_routes(str(run_yaml_path))
220+
198221
# Generate the Markdown table
199-
table_content = gen_distro_table(providers)
222+
table_content = gen_distro_table(providers, routes)
200223

201224
# Write to README.md
202225
with open(readme_path, "w") as readme_file:

0 commit comments

Comments
 (0)