Skip to content

Updating deployment guides for REST, GraphQL, AI API and gRPC #726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Apr 24, 2025
Merged

Updating deployment guides for REST, GraphQL, AI API and gRPC #726

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
dd36aa1
DP to CP QSG fixes
vinupa Mar 20, 2025
f71607e
Updating hostname in generating access token guide
vinupa Mar 21, 2025
9dc4fd5
Updating Create REST API guides
vinupa Mar 21, 2025
73a5edb
Updating Create GraphQL API guides
vinupa Mar 21, 2025
16bad24
Updating Create AI API guides
vinupa Mar 26, 2025
41b940a
Updating Create gRPC API guides
vinupa Mar 27, 2025
625e293
Updating token link in weighted routing
vinupa Mar 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 32 additions & 38 deletions en/docs/create-api/create-and-deploy-apis/ai/create-ai-api-using-crs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,12 @@ spec:
apiName: "Azure OpenAI Service API"
apiType: "REST"
apiVersion: "1.0.0"
basePath: "/QXp1cmUgT3BlbkFJIFNlcnZpY2UgQVBJMjAyNC0wNi0wMQ/1.0.0"
basePath: "/azure-open-ai/1.0.0"
organization: "default"
isDefaultVersion: false
definitionPath: "/docs"
production:
- routeRefs:
- "8bcdea77634e2d4c827a7ea88f8075c4dd538834-production-httproute-1"
- "production-httproute-1"
apiProperties: []
```

Expand All @@ -47,17 +46,16 @@ spec:

## Create HTTPRoute CR

This is the resource where you define resources of your API. This `HTTPRoute` is linked to the API by referring to this resource name from the `API` resource.
This `HTTPRoute` resource is linked to the API by referring to this resource name from the `API` resource.

```
---
apiVersion: "gateway.networking.k8s.io/v1beta1"
kind: "HTTPRoute"
metadata:
name: "8bcdea77634e2d4c827a7ea88f8075c4dd538834-production-httproute-1"
name: "production-httproute-1"
spec:
hostnames:
- "default.gw.wso2.com"
- "default.gw.example.com"
rules:
- matches:
- path:
Expand All @@ -73,7 +71,7 @@ spec:
backendRefs:
- group: "dp.wso2.com"
kind: "Backend"
name: "backend-33eb53282e93f5fd3f26935311af727d58bd42c3-api"
name: "backend-api"
- matches:
- path:
type: "RegularExpression"
Expand All @@ -88,7 +86,7 @@ spec:
backendRefs:
- group: "dp.wso2.com"
kind: "Backend"
name: "backend-33eb53282e93f5fd3f26935311af727d58bd42c3-api"
name: "backend-api"
- matches:
- path:
type: "RegularExpression"
Expand All @@ -103,7 +101,7 @@ spec:
backendRefs:
- group: "dp.wso2.com"
kind: "Backend"
name: "backend-33eb53282e93f5fd3f26935311af727d58bd42c3-api"
name: "backend-api"
- matches:
- path:
type: "RegularExpression"
Expand All @@ -118,7 +116,7 @@ spec:
backendRefs:
- group: "dp.wso2.com"
kind: "Backend"
name: "backend-33eb53282e93f5fd3f26935311af727d58bd42c3-api"
name: "backend-api"
- matches:
- path:
type: "RegularExpression"
Expand All @@ -133,7 +131,7 @@ spec:
backendRefs:
- group: "dp.wso2.com"
kind: "Backend"
name: "backend-33eb53282e93f5fd3f26935311af727d58bd42c3-api"
name: "backend-api"
- matches:
- path:
type: "RegularExpression"
Expand All @@ -148,7 +146,7 @@ spec:
backendRefs:
- group: "dp.wso2.com"
kind: "Backend"
name: "backend-33eb53282e93f5fd3f26935311af727d58bd42c3-api"
name: "backend-api"
parentRefs:
- group: "gateway.networking.k8s.io"
kind: "Gateway"
Expand All @@ -157,7 +155,7 @@ spec:

```

Here, we have used `default.gw.wso2.com` as the virtual hostname for this API. The `spec.parentRefs[0]` parameter defines the Gateway to which this `HTTPRoute` is deployed. We have defined a single rule with a `PathPrefix` match type, and the `URLRewrite` filter with `ReplacePrefixMatch` to rewrite the API context prefix so that only the remainder of the path is sent to the actual backend.
Here, we have used `default.gw.example.com` as the virtual hostname for this API. The `spec.parentRefs[0]` parameter defines the Gateway to which this `HTTPRoute` is deployed. We have defined a single rule with a `PathPrefix` match type, and the `URLRewrite` filter with `ReplacePrefixMatch` to rewrite the API context prefix so that only the remainder of the path is sent to the actual backend.

## Create Backend CR

Expand All @@ -167,19 +165,19 @@ In the above created HTTPRoute resource we have referred to a `Backend` resource
apiVersion: "dp.wso2.com/v1alpha2"
kind: "Backend"
metadata:
name: "backend-33eb53282e93f5fd3f26935311af727d58bd42c3-api"
name: "backend-api"
spec:
services:
- host: "xxxx.openai.azure.com"
- host: "{endpoint}.openai.azure.com"
port: 443
basePath: "/openai/deployments/yyyyyy"
basePath: "/openai/deployments/{deployment-id}"
protocol: "https"
security:
apiKey:
in: "Header"
name: "api-key"
valueFrom:
name: "mysecret"
name: "azure-ai-secret"
valueKey: "apiKey"
```

Expand All @@ -193,32 +191,29 @@ The following CR defines the API policy for the API. In this CR, we have overrid
apiVersion: dp.wso2.com/v1alpha3
kind: APIPolicy
metadata:
name: ai-provider-8bcdea77634e2d4c827a7ea88f8075c4dd538834-api-policy
namespace: apk
name: ai-provider-api-policy
spec:
override:
aiProvider:
name: "ai-provider-azure-ai"
targetRef:
group: gateway.networking.k8s.io
group: dp.wso2.com
kind: API
name: 8bcdea77634e2d4c827a7ea88f8075c4dd538834
name: azure-open-ai
```

## Create Secret CR

The following CR defines the secret that contains the API key for the Azure AI provider.
Create a secret containing the API Key of the LLM Service Provider using the following command. Replace the ```api key of LLM Service``` value with your API Key generated for LLM Service Provider.

```
apiVersion: v1
kind: Secret
metadata:
name: mysecret
namespace: apk
type: Opaque
data:
apiKey: base64_encoded_api_key
```
=== "Sample Command"
```bash
kubectl create secret generic azure-ai-secret --from-literal=apiKey='xxxxxxxxxxxxxxxxxxx'
```
=== "Command Format"
```bash
kubectl create secret generic azure-ai-secret --from-literal=apiKey='<<api key of LLM Service>>' --namespace=<<namespace>>
```

## Create AIRatelimitPolicy CR

Expand All @@ -229,7 +224,6 @@ apiVersion: dp.wso2.com/v1alpha3
kind: AIRateLimitPolicy
metadata:
name: llm-backend-rl
namespace: apk
spec:
override:
organization: default
Expand All @@ -242,16 +236,16 @@ spec:
requestsPerUnit: 6000
unit: Minute
targetRef:
group: dp.wso2.com
kind: Backend
name: backend-33eb53282e93f5fd3f26935311af727d58bd42c3-api
group: gateway.networking.k8s.io
name: backend-api
```

## Deploy the CRs
Once you have designed your APIs using these essential CRs, the next step is to apply them to the Kubernetes API server. APK will process and deploy your APIs seamlessly, taking full advantage of the Kubernetes infrastructure.

```
kubectl apply -f <path_to_CR_files> -n apk
kubectl apply -f <path_to_CR_files>
```

## Verify the API Invocation
Expand All @@ -260,7 +254,7 @@ kubectl apply -f <path_to_CR_files> -n apk

=== "Sample Request"
```
curl -X POST "https://default.gw.wso2.com:9095/QXp1cmUgT3BlbkFJIFNlcnZpY2UgQVBJMjAyNC0wNi0wMQ/1.0.0/chat/completions?api-version=2024-06-01" \
curl -X POST "https://default.gw.example.com:9095/azure-open-ai/1.0.0/chat/completions?api-version=2024-06-01" \
-H "Content-Type: application/json" \
-H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsICJ0eXAiOiJKV1QiLCAia2lkIjoiZ2F0ZXdheV9jZXJ0aWZpY2F0ZV9hbGlhcyJ9.eyJpc3MiOiJodHRwczovL2lkcC5hbS53c28yLmNvbS90b2tlbiIsICJzdWIiOiI0NWYxYzVjOC1hOTJlLTExZWQtYWZhMS0wMjQyYWMxMjAwMDIiLCAiYXVkIjoiYXVkMSIsICJleHAiOjE3MzAwMTU0NDIsICJuYmYiOjE3MzAwMTE4NDIsICJpYXQiOjE3MzAwMTE4NDIsICJqdGkiOiIwMWVmOTQyZi1jODgwLTE0ZDYtYTMzNC0yNTMyMDEzNzhkOWUiLCAiY2xpZW50SWQiOiI0NWYxYzVjOC1hOTJlLTExZWQtYWZhMS0wMjQyYWMxMjAwMDIiLCAic2NvcGUiOiJhcGs6YXBpX2NyZWF0ZSJ9.Lumx8tBDTNhwgfUHWwgiSQEwcjz6ZF-5f3UJfJlCNv7feAnIEsdGmb5sFw6wRQBZklSVsZYffj1uK7ManfSR6gfws-W1qo5itwYFixvkoOMU5HcxtVdTsYOl8CzN4u76hbbk_r7I3vov-2g4iMT2Lfu45g1u1sEj1JgjbpOnTZdZ2c2jWHal35idLSEBhULMElGPjce1uCwTS2zsEZQond1q3HvMouIJ58q2oGaD4qpcx-FTlbYKsBD5_h4v4U2PV3kGkxLzos4eXoeM88vbVhIew-z8NxZuxiuP3dS4AAIbHevkQgmueMdN6E0Y5xXoYbcTDVuB8dMYAuctof6b4A' \
-d '{
Expand Down
30 changes: 15 additions & 15 deletions en/docs/create-api/create-and-deploy-apis/ai/create-ai-api-using-rest-api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,13 @@ For this use case, we will use the Azure OpenAI service and will use the API Key

This service generates an APK configuration file by providing your OAS file. APK configuration file includes important API metadata, rate limiting details, security settings, and other necessary information about your API.

The OpenAPI specification file can be provided as a local file or as a URL containing a definition file. For this use case, we will use the Azure OpenAI OAS definition therefore download and save the azure OAS file as <b>azure_api.yaml</b>.
The OpenAPI specification file can be provided as a local file or as a URL containing a definition file. For this use case, we will use the Azure OpenAI OAS definition therefore download and save the azure OAS file as <b>azure-api.yaml</b>.

=== "Sample Request"
```
curl -k --location 'https://api.am.wso2.com:9095/api/configurator/1.3.0/apis/generate-configuration' \
--header 'Host: api.am.wso2.com' \
--form 'definition=@"/Users/user/azure_api.yaml"'
curl -k --location 'https://api.example.com:9095/api/configurator/1.3.0/apis/generate-configuration' \
--header 'Host: api.example.com' \
--form 'definition=@"/Users/user/azure-api.yaml"'
```
=== "Request Format"
```
Expand All @@ -43,7 +43,7 @@ The OpenAPI specification file can be provided as a local file or as a URL conta
=== "Sample Response"
```yaml
name: "Azure OpenAI Service API"
basePath: "/QXp1cmUgT3BlbkFJIFNlcnZpY2UgQVBJMjAyNC0wNi0wMQ"
basePath: "/azure-open-ai"
version: "2024-06-01"
type: "REST"
defaultVersion: false
Expand Down Expand Up @@ -78,7 +78,7 @@ The OpenAPI specification file can be provided as a local file or as a URL conta
scopes: []
```

You will get the apk-conf file content as the response, as seen in the above sample response. Save this content into a file with the `.apk-conf` file extension. For example, you can save under the name, azure.apk-conf.
You will get the apk-conf file content as the response, as seen in the above sample response. Save this content into a file with the `.apk-conf` file extension. For example, you can save under the name <b>azure.apk-conf</b>.

!!! note
You will need to fill in the AI Provider and Endpoint Configuration fields such as endpoint and security, before deploying the API.
Expand All @@ -87,18 +87,18 @@ You will get the apk-conf file content as the response, as seen in the above sam

Review the content inside the apk-conf file and update it with additional API configurations as needed, such as configuring AI Provider, LLM Service URl in the Endpoint Configuration, Secret which stores the API key provided from LLM Service and rate limits, CORS configurations, etc.

Refer to the sample configuration file below:
Refer to the sample configuration file below. For this guide, replace the contents of the apk-conf file that you generated with the contents of this sample configuration and replace the `{endpoint}` and `{deployment-id}` in the endpoint field with the actual values provided by the LLM Service Provider.

```yaml
name: "Azure OpenAI Service API"
basePath: "/QXp1cmUgT3BlbkFJIFNlcnZpY2UgQVBJMjAyNC0wNi0wMQ"
basePath: "/azure-open-ai"
version: "1.0.0"
type: "REST"
defaultVersion: false
subscriptionValidation: false
aiProvider:
name: "AzureAI"
apiVersion: "2021-06-01"
apiVersion: "2024-06-01"
endpointConfigurations:
production:
- endpoint: "https://{endpoint}/openai/deployments/{deployment-id}"
Expand Down Expand Up @@ -174,11 +174,11 @@ Create a secret containing the API Key of the LLM Service Provider using the fol

=== "Sample Command"
```bash
kubectl create secret generic azure-ai-secret --from-literal=apiKey='xxxxxxxxxxxxxxxxxxx'
kubectl create secret generic azure-ai-secret --from-literal=apiKey='xxxxxxxxxxxxxxxxxxx'
```
=== "Command Format"
```bash
kubectl create secret generic azure-ai-secret --from-literal=apiKey='<<api key of LLM Service>>' --namespace=<<namespace>>
kubectl create secret generic azure-ai-secret --from-literal=apiKey='<<api key of LLM Service>>' --namespace=<<namespace>>
```

## Step 6. Deploy the API to a Kubernetes cluster.
Expand All @@ -193,7 +193,7 @@ After generating the token, you can deploy the API directly into APK using API S

=== "Sample Request"
```
curl -k --location 'https://api.am.wso2.com:9095/api/deployer/1.3.0/apis/deploy' \
curl -k --location 'https://api.example.com:9095/api/deployer/1.3.0/apis/deploy' \
--header 'Content-Type: multipart/form-data' \
--header 'Accept: application/yaml' \
--header 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsICJ0eXAiOiJKV1QiLCAia2lkIjoiZ2F0ZXdheV9jZXJ0aWZpY2F0ZV9hbGlhcyJ9.eyJpc3MiOiJodHRwczovL2lkcC5hbS53c28yLmNvbS90b2tlbiIsICJzdWIiOiI0NWYxYzVjOC1hOTJlLTExZWQtYWZhMS0wMjQyYWMxMjAwMDIiLCAiYXVkIjoiYXVkMSIsICJleHAiOjE3MzAwMTQ0MjksICJuYmYiOjE3MzAwMTA4MjksICJpYXQiOjE3MzAwMTA4MjksICJqdGkiOiIwMWVmOTQyZC02Y2I0LTFjNDgtYmFmYS04Yzg5NGFlZWZkYzIiLCAiY2xpZW50SWQiOiI0NWYxYzVjOC1hOTJlLTExZWQtYWZhMS0wMjQyYWMxMjAwMDIiLCAic2NvcGUiOiJhcGs6YXBpX2NyZWF0ZSJ9.isgFKKbdmIApTpCcZEhn1aWpHPFs4ZRhEva9Hjj6WMQWtwlYYlcgBK_g08p1zA0HOxCOZuAsXBcEMLnJ9HzgBq0bWKegWwAt_oTHCvDu4yi4gNYR0TLPc_8goa6fXS-iLckrG22Csi7ODoj84agW6rwLEq4G62dVWt4dNMSSO91cy2HdYtVjcnbKxefVJ942uwJOqqL4HqDc4a-u1rHeLchvwn_b1ezZIyWHcZQFsY2PP3UBZ30t_5-08V3w2AbkVXvppYpHKNqg647MzdOChz66nF0va5FxmlTr71uY0Q4Gufv-tT6QENWX_GcYhDmdH4OQXTFt9jHgm8lcXxyK7g' \
Expand All @@ -212,7 +212,7 @@ After generating the token, you can deploy the API directly into APK using API S
```
id: "8bcdea77634e2d4c827a7ea88f8075c4dd538834"
name: "Azure OpenAI Service API"
basePath: "/QXp1cmUgT3BlbkFJIFNlcnZpY2UgQVBJMjAyNC0wNi0wMQ"
basePath: "/azure-open-ai"
version: "1.0.0"
type: "REST"
defaultVersion: false
Expand Down Expand Up @@ -272,7 +272,7 @@ After generating the token, you can deploy the API directly into APK using API S
You can generate K8s resources as a zip file from config-deployer service.

```
curl --location 'https://api.am.wso2.com:9095/api/configurator/1.3.0/apis/generate-k8s-resources' \
curl --location 'https://api.example.com:9095/api/configurator/1.3.0/apis/generate-k8s-resources' \
--header 'Content-Type: multipart/form-data' \
--header 'Accept: application/zip' \
--form 'apkConfiguration=@"/Users/user/azure.apk-conf"' \
Expand All @@ -292,7 +292,7 @@ kubectl apply -f <path_to_extracted_zip_file>

=== "Sample Request"
```
curl -X POST "https://default.gw.wso2.com:9095/QXp1cmUgT3BlbkFJIFNlcnZpY2UgQVBJMjAyNC0wNi0wMQ/1.0.0/chat/completions?api-version=2024-06-01" \
curl -X POST "https://default.gw.example.com:9095/azure-open-ai/1.0.0/chat/completions?api-version=2024-06-01" \
-H "Content-Type: application/json" \
-H 'Authorization: Bearer eyJhbGciOiJSUzI1NiIsICJ0eXAiOiJKV1QiLCAia2lkIjoiZ2F0ZXdheV9jZXJ0aWZpY2F0ZV9hbGlhcyJ9.eyJpc3MiOiJodHRwczovL2lkcC5hbS53c28yLmNvbS90b2tlbiIsICJzdWIiOiI0NWYxYzVjOC1hOTJlLTExZWQtYWZhMS0wMjQyYWMxMjAwMDIiLCAiYXVkIjoiYXVkMSIsICJleHAiOjE3MzAwMTU0NDIsICJuYmYiOjE3MzAwMTE4NDIsICJpYXQiOjE3MzAwMTE4NDIsICJqdGkiOiIwMWVmOTQyZi1jODgwLTE0ZDYtYTMzNC0yNTMyMDEzNzhkOWUiLCAiY2xpZW50SWQiOiI0NWYxYzVjOC1hOTJlLTExZWQtYWZhMS0wMjQyYWMxMjAwMDIiLCAic2NvcGUiOiJhcGs6YXBpX2NyZWF0ZSJ9.Lumx8tBDTNhwgfUHWwgiSQEwcjz6ZF-5f3UJfJlCNv7feAnIEsdGmb5sFw6wRQBZklSVsZYffj1uK7ManfSR6gfws-W1qo5itwYFixvkoOMU5HcxtVdTsYOl8CzN4u76hbbk_r7I3vov-2g4iMT2Lfu45g1u1sEj1JgjbpOnTZdZ2c2jWHal35idLSEBhULMElGPjce1uCwTS2zsEZQond1q3HvMouIJ58q2oGaD4qpcx-FTlbYKsBD5_h4v4U2PV3kGkxLzos4eXoeM88vbVhIew-z8NxZuxiuP3dS4AAIbHevkQgmueMdN6E0Y5xXoYbcTDVuB8dMYAuctof6b4A' \
-d '{
Expand Down
Loading