Skip to content

Commit 54dbf46

Browse files
Guaristysoekong
Guaris
authored and
tysoekong
committed
custom model dosc
1 parent 7601490 commit 54dbf46

File tree

14 files changed

+81
-0
lines changed

14 files changed

+81
-0
lines changed

‎app/_hub/kong-inc/ai-proxy-advanced/how-to/llm-provider-integration-guides/_anthropic.md

+1
Original file line numberDiff line numberDiff line change
@@ -66,3 +66,4 @@ curl -X POST http://localhost:8000/anthropic-chat \
6666
-H 'Content-Type: application/json' \
6767
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
6868
```
69+
{% include_cached /md/plugins-hub/ai-custom-model-advanced.md %}

‎app/_hub/kong-inc/ai-proxy-advanced/how-to/llm-provider-integration-guides/_azure.md

+2
Original file line numberDiff line numberDiff line change
@@ -77,3 +77,5 @@ curl -X POST http://localhost:8000/azure-chat \
7777
-H 'Content-Type: application/json' \
7878
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
7979
```
80+
81+
{% include_cached /md/plugins-hub/ai-custom-model-advanced.md %}

‎app/_hub/kong-inc/ai-proxy-advanced/how-to/llm-provider-integration-guides/_cohere.md

+2
Original file line numberDiff line numberDiff line change
@@ -64,3 +64,5 @@ curl -X POST http://localhost:8000/cohere-chat \
6464
-H 'Content-Type: application/json' \
6565
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
6666
```
67+
68+
{% include_cached /md/plugins-hub/ai-custom-model-advanced.md %}

‎app/_hub/kong-inc/ai-proxy-advanced/how-to/llm-provider-integration-guides/_llama2.md

+1
Original file line numberDiff line numberDiff line change
@@ -112,3 +112,4 @@ curl -X POST http://localhost:8000/llama2-chat \
112112
-H 'Content-Type: application/json' \
113113
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
114114
```
115+
{% include_cached /md/plugins-hub/ai-custom-model-advanced.md %}

‎app/_hub/kong-inc/ai-proxy-advanced/how-to/llm-provider-integration-guides/_mistral.md

+1
Original file line numberDiff line numberDiff line change
@@ -103,3 +103,4 @@ curl -X POST http://localhost:8000/mistral-chat \
103103
-H 'Content-Type: application/json' \
104104
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
105105
```
106+
{% include_cached /md/plugins-hub/ai-custom-model-advanced.md %}

‎app/_hub/kong-inc/ai-proxy-advanced/how-to/llm-provider-integration-guides/_openai.md

+2
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ formats:
5555
{% endplugin_example %}
5656
<!--vale on-->
5757

58+
5859
### Test the configuration
5960

6061
Make an `llm/v1/chat` type request to test your new endpoint:
@@ -64,3 +65,4 @@ curl -X POST http://localhost:8000/openai-chat \
6465
-H 'Content-Type: application/json' \
6566
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
6667
```
68+
{% include_cached /md/plugins-hub/ai-custom-model-advanced.md %}

‎app/_hub/kong-inc/ai-proxy/how-to/llm-provider-integration-guides/_anthropic.md

+1
Original file line numberDiff line numberDiff line change
@@ -82,3 +82,4 @@ curl -X POST http://localhost:8000/anthropic-chat \
8282
-H 'Content-Type: application/json' \
8383
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
8484
```
85+
{% include_cached /md/plugins-hub/ai-custom-model.md %}

‎app/_hub/kong-inc/ai-proxy/how-to/llm-provider-integration-guides/_azure.md

+1
Original file line numberDiff line numberDiff line change
@@ -89,3 +89,4 @@ curl -X POST http://localhost:8000/azure-chat \
8989
-H 'Content-Type: application/json' \
9090
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
9191
```
92+
{% include_cached /md/plugins-hub/ai-custom-model.md %}

‎app/_hub/kong-inc/ai-proxy/how-to/llm-provider-integration-guides/_cohere.md

+2
Original file line numberDiff line numberDiff line change
@@ -76,3 +76,5 @@ curl -X POST http://localhost:8000/cohere-chat \
7676
-H 'Content-Type: application/json' \
7777
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
7878
```
79+
80+
{% include_cached /md/plugins-hub/ai-custom-model.md %}

‎app/_hub/kong-inc/ai-proxy/how-to/llm-provider-integration-guides/_llama2.md

+2
Original file line numberDiff line numberDiff line change
@@ -123,3 +123,5 @@ curl -X POST http://localhost:8000/llama2-chat \
123123
-H 'Content-Type: application/json' \
124124
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
125125
```
126+
127+
{% include_cached /md/plugins-hub/ai-custom-model.md %}

‎app/_hub/kong-inc/ai-proxy/how-to/llm-provider-integration-guides/_mistral.md

+1
Original file line numberDiff line numberDiff line change
@@ -115,3 +115,4 @@ curl -X POST http://localhost:8000/mistral-chat \
115115
-H 'Content-Type: application/json' \
116116
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
117117
```
118+
{% include_cached /md/plugins-hub/ai-custom-model.md %}

‎app/_hub/kong-inc/ai-proxy/how-to/llm-provider-integration-guides/_openai.md

+1
Original file line numberDiff line numberDiff line change
@@ -76,3 +76,4 @@ curl -X POST http://localhost:8000/openai-chat \
7676
-H 'Content-Type: application/json' \
7777
--data-raw '{ "messages": [ { "role": "system", "content": "You are a mathematician" }, { "role": "user", "content": "What is 1+1?"} ] }'
7878
```
79+
{% include_cached /md/plugins-hub/ai-custom-model.md %}

‎app/_includes/md/plugins-hub/ai-custom-model-advanced.md

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
2+
For all providers, the Kong AI Proxy Advanced plugin attaches to **route** entities.
3+
4+
### Custom model
5+
6+
You can configure the AI Proxy Advanced Plugin using a custom model of your choice by setting the `name` and `upstream_url` when configuring the model.
7+
8+
<!--vale off-->
9+
{% plugin_example %}
10+
plugin: kong-inc/ai-proxy-advanced
11+
name: ai-proxy-advanced
12+
config:
13+
targets:
14+
- route_type: "llm/v1/chat"
15+
auth:
16+
header_name: "Authorization"
17+
header_value: "Bearer <openai_key>"
18+
- model:
19+
name: custom_model_name
20+
provider: openai|azure|anthropic|cohere|mistral|llama2|gemini|bedrock
21+
options:
22+
upstream_url: http://localhost:8000/vi/chat/completions
23+
targets:
24+
- route
25+
formats:
26+
- curl
27+
- konnect
28+
- yaml
29+
- kubernetes
30+
- terraform
31+
{% endplugin_example %}
32+
<!--vale on-->

‎app/_includes/md/plugins-hub/ai-custom-model.md

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
2+
For all providers, the Kong AI Proxy Advanced plugin attaches to **route** entities.
3+
4+
### Custom model
5+
6+
You can configure the AI Proxy Advanced Plugin using a custom model of your choice by setting the `name` and `upstream_url` when configuring the model.
7+
8+
<!--vale off-->
9+
{% plugin_example %}
10+
plugin: kong-inc/ai-proxy-advanced
11+
name: ai-proxy-advanced
12+
config:
13+
targets:
14+
- route_type: "llm/v1/chat"
15+
auth:
16+
header_name: "Authorization"
17+
header_value: "Bearer <openai_key>"
18+
- model:
19+
name: custom_model_name
20+
provider: openai|azure|anthropic|cohere|mistral|llama2|gemini|bedrock
21+
options:
22+
upstream_url: http://localhost:8000/vi/chat/completions
23+
targets:
24+
- route
25+
formats:
26+
- curl
27+
- konnect
28+
- yaml
29+
- kubernetes
30+
- terraform
31+
{% endplugin_example %}
32+
<!--vale on-->

0 commit comments

Comments
 (0)