Skip to content

Feat: Custom plugin streaming #8556

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Apr 23, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions app/_data/docs_nav_konnect.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,8 @@
url: /gateway-manager/dedicated-cloud-gateways/azure-peering
- text: Custom Domains
url: /gateway-manager/dedicated-cloud-gateways/custom-dns/
- text: Custom Plugins
url: /gateway-manager/dedicated-cloud-gateways/custom-plugins/
- text: Data plane logs
url: /gateway-manager/dedicated-cloud-gateways/data-plane-logs
- text: Serverless Gateways
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
title: Custom plugins in Dedicated Cloud Gateways
content_type: reference
---

With Dedicated Cloud Gateways, {{site.konnect_short_name}} can stream custom plugins from the control plane to the data plane.

This means that the control plane becomes a single source of truth for plugin versions. You only need to upload a plugin once, to the control plane, and {{site.konnect_short_name}} handles distributing the plugin code to all data planes in that control plane.

Compared to the manual process required in a regular hybrid {{site.konnect_short_name}} deployment, custom plugin streaming in Dedicated Cloud Gateways provides the following benefits:
* Faster custom plugin distribution
* Minimal manual maintenance
* The control plane is the single source of truth

## Prerequisites

* Your [custom plugin](/gateway/latest/plugin-development/) meets the following requirements:
* Each custom plugin must have a unique name.
* Each custom plugin can have a maximum of 1 `handler.lua` file and 1 `schema.lua` file.
* The plugin can't execute in the `init_worker` phase and can't set any timers.
* The plugin must be written in Lua. No other languages are supported.
* You have a [personal access token or system access token](/konnect/org-management/access-tokens) for the {{site.konnect_short_name}} API

## Adding a custom plugin to a Dedicated Cloud Gateway deployment

Upload custom plugin schema and handler files to create a configurable entity in {{site.konnect_short_name}}.
If you prefer using the {{site.konnect_short_name}} UI, you can also upload these files through the Plugins menu in [Gateway Manager](https://cloud.konghq.com/gateway-manager/).

{:.important}
> The name you give the plugin must be identical to the name of the plugin in the schema file.

Using the following command, make a `POST` request to the [`/custom-plugins`](/konnect/api/control-plane-configuration/latest/#/operations/list-custom-plugin) endpoint of the {{site.konnect_short_name}} Control Plane Config API:

```sh
curl -X POST https://{region}.api.konghq.com/v2/control-planes/{control-plane-id}/core-entities/custom-plugins \
-H "Content-Type: application/json" \
-H "Authorization: Bearer {your-access-token}" \
-d "$(jq -n \
--arg handler "$(cat handler.lua)" \
--arg schema "$(cat schema.lua)" \
'{"handler":$handler,"name":"streaming-headers","schema":$schema}')" \
| jq
```

This request returns an `HTTP 200` response with the schema and handler for your plugin as a JSON object.

Once a custom plugin is uploaded to a Dedicated Cloud Gateway control plane, it can be managed like any other plugin, using any of the following tools:
* [decK](/konnect/gateway-manager/declarative-config/)
* [{{site.konnect_short_name}} Control Plane Config API](/konnect/api/control-plane-configuration/latest/#/operations/list-custom-plugin)
* [{{site.konnect_short_name}} UI](https://cloud.konghq.com/)

## More information

* [Custom plugin schema endpoints (Control Plane Config API)](/konnect/api/control-plane-configuration/latest/#/Custom%20Plugin%20Schemas)
* [Custom plugin template](https://github.com/Kong/kong-plugin)
* [Plugin development guide](/gateway/latest/plugin-development/)
* [PDK reference](/gateway/latest/plugin-development/pdk/)
Original file line number Diff line number Diff line change
Expand Up @@ -60,5 +60,6 @@ This includes [Data Plane Resilience](/gateway/latest/kong-enterprise/cp-outage-
* [Data Plane logs](/konnect/gateway-manager/dedicated-cloud-gateways/)
* [Transit Gateways](/konnect/gateway-manager/dedicated-cloud-gateways/transit-gateways/)
* [How to upgrade data planes](/konnect/gateway-manager/data-plane-nodes/upgrade/)
* [Custom Domains](/konnect/gateway-manager/dedicated-cloud-gateways/custom-dns/)
* [Custom domains](/konnect/gateway-manager/dedicated-cloud-gateways/custom-dns/)
* [Custom plugins](/konnect/gateway-manager/dedicated-cloud-gateways/custom-plugins/)
* [Cloud Gateways API](/konnect/api/cloud-gateways/latest/)
4 changes: 3 additions & 1 deletion app/konnect/gateway-manager/plugins/add-custom-plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ See [Editing or deleting a custom plugin's schema](/konnect/gateway-manager/plug
for more information.

{:.note}
> **Note**: Custom plugins are not supported in Dedicated Cloud Gateways.
> **Note**: For adding custom plugins to a Dedicated Cloud Gateway, see
[Custom plugins in Dedicated Cloud Gateways](/konnect/gateway-manager/dedicated-cloud-gateways/custom-plugins/).
Comment on lines +20 to +21
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to have: for Dev site, I think the DCG custom plugin section could be an include that we'd use here instead of the note. But I think the way you have it now it okay for this release.


## Prerequisites

Expand Down Expand Up @@ -186,6 +187,7 @@ or {{site.base_gateway}} deployments.
## More information

* [Edit or delete custom plugins in {{site.konnect_short_name}}](/konnect/gateway-manager/plugins/update-custom-plugin/)
* [Custom plugins in Dedicated Cloud Gateways](/konnect/gateway-manager/dedicated-cloud-gateways/custom-plugins/)
* [Custom plugin schema endpoints (Control Plane Config API)](/konnect/api/control-plane-configuration/latest/#/Custom%20Plugin%20Schemas)
* [Custom plugin template](https://github.com/Kong/kong-plugin)
* [Plugin development guide](/gateway/latest/plugin-development/)
Expand Down
4 changes: 3 additions & 1 deletion app/konnect/gateway-manager/plugins/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: Kong Gateway Plugins in Konnect

Check failure on line 2 in app/konnect/gateway-manager/plugins/index.md

View workflow job for this annotation

GitHub Actions / Vale

[vale] reported by reviewdog 🐶 [kong.kongterms] Use '{{site.base_gateway}}' instead of 'Kong Gateway'. Raw Output: {"message": "[kong.kongterms] Use '{{site.base_gateway}}' instead of 'Kong Gateway'.", "location": {"path": "app/konnect/gateway-manager/plugins/index.md", "range": {"start": {"line": 2, "column": 8}}}, "severity": "ERROR"}
content_type: reference
---

Expand Down Expand Up @@ -40,7 +40,8 @@
only sees a custom plugin's configuration options, and does not see any other data.

{:.note}
> **Note**: Custom plugins are not supported in Dedicated Cloud Gateways.
> **Note**: For adding custom plugins to a Dedicated Cloud Gateway, see
[Custom plugins in Dedicated Cloud Gateways](/konnect/gateway-manager/dedicated-cloud-gateways/custom-plugins/).

To run in {{site.konnect_short_name}}, every custom plugin must meet the following requirements:

Expand Down Expand Up @@ -88,6 +89,7 @@
### Custom plugins
* [Add a custom plugin in {{site.konnect_short_name}}](/konnect/gateway-manager/plugins/add-custom-plugin/)
* [Edit or delete custom plugins in {{site.konnect_short_name}}](/konnect/gateway-manager/plugins/update-custom-plugin/)
* [Custom plugins in Dedicated Cloud Gateways](/konnect/gateway-manager/dedicated-cloud-gateways/custom-plugins/)
* [Custom plugin schema endpoints (Control Plane Config API)](/konnect/api/control-plane-configuration/latest/#/Custom%20Plugin%20Schemas)
* [Custom plugin template](https://github.com/Kong/kong-plugin)
* [Plugin development guide](/gateway/latest/plugin-development/)
Expand Down
Loading