Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add support for externalized consumers #7657

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

jschmid1
Copy link
Contributor

@jschmid1 jschmid1 commented Jul 18, 2024

Copy link

netlify bot commented Jul 18, 2024

Deploy Preview for kongdocs ready!

Name Link
🔨 Latest commit bbc2109
🔍 Latest deploy log https://app.netlify.com/sites/kongdocs/deploys/67d472a2f0671d0008163178
😎 Deploy Preview https://deploy-preview-7657--kongdocs.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
Lighthouse
Lighthouse
9 paths audited
Performance: 93 (🟢 up 3 from production)
Accessibility: 93 (no change from production)
Best Practices: 98 (🟢 up 8 from production)
SEO: 99 (no change from production)
PWA: -
View the detailed breakdown and full score reports

To edit notification comments on pull requests, go to your Netlify site configuration.

@jschmid1 jschmid1 added the do not merge Issues/ PRs whose changes should not be merged at this time label Jul 18, 2024
@Guaris Guaris added the review:general Review for general accuracy and presentation. Does the doc work? Does it output correctly? label Jul 22, 2024
@Guaris Guaris requested a review from lena-larionova July 22, 2024 16:12
@lena-larionova lena-larionova added this to the Gateway 3.8 milestone Jul 23, 2024
@Guaris Guaris self-assigned this Aug 19, 2024
@Guaris Guaris marked this pull request as ready for review August 19, 2024 16:29
@Guaris Guaris requested a review from a team as a code owner August 19, 2024 16:29
Comment on lines 366 to 367
- geo: null
id: null
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

geo and id should be optional (default null) imo


-- Dummy/Link that points to Konnect Docs when ready.

With the `pool_id` you obtained from the previous step, you can configure the key-auth plugin to validate API keys against the Identity Service.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jschmid1 where does the pool_id come from? Konnect?

Copy link
Contributor

@smritikjaggi smritikjaggi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to change all references of pools to realms, and update the field terminology

@@ -0,0 +1,71 @@
---
nav_title:
title: How to configure pools
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to Realms

---


With `pool_id` you can configure the key-auth plugin to validate API keys against the Identity Service.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to "realm_id"


With `pool_id` you can configure the key-auth plugin to validate API keys against the Identity Service.

### Configuring Multiple Pools
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change to "Realms"


### Configuring Multiple Pools

In the key-auth plugin configuration, add the `pools` option as shown below:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change to "realms"

In the key-auth plugin configuration, add the `pools` option as shown below:

```yaml
pools:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change to "identity_realms"

type: remote
```

In this configuration, the dataplane will initially check the local pool (LMDB) before querying the remote Identity Service.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cp-scoped consumers first


In this configuration, the dataplane will initially check the local pool (LMDB) before querying the remote Identity Service.

If a matching key is found in any of these pools, the request will be authenticated. If the key is not found in any of the configured pools, the request will be blocked.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"realms"


If a matching key is found in any of these pools, the request will be authenticated. If the key is not found in any of the configured pools, the request will be blocked.

### Configuring Single Pools
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Single Realm


### Configuring Single Pools

It is also possible to configure only a single pool, either local or remote. However, only one of each type can be configured.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change pool to realm
change "either local or remote" to "consumers can be scoped to the geo or cp".


It is also possible to configure only a single pool, either local or remote. However, only one of each type can be configured.

To configure only a remote pool:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change "remote pool" to "realm"

@lena-larionova lena-larionova removed this from the Gateway 3.8 milestone Sep 10, 2024
smritikjaggi and others added 4 commits March 6, 2025 11:49
Testing edit function
Updated the terminology used for realms and the yaml file examples
@cloudjumpercat
Copy link
Contributor

cloudjumpercat commented Mar 11, 2025

@brentos Hi! I just wanted to let you know I'll be working on external consumers and I just added some changes to the PR, including a new Konnect page that explains what they are and how to configure them, as well as a revised section to key auth. I had a few questions while I was working on it:

  • We mention that a kong dev build is required, but this is for Konnect. I don't understand how that custom dev build is required unless we're using it for data plane nodes?
  • I tried using the /realms endpoint, but wasn't able to access it. Maybe something to do with not having the custom build?
  • Are we publishing the API specs for this release? Can I have a copy of them so I can look at the endpoints and parameters?
  • So key auth only associates the plugin with a realm, it doesn't actually create the realm, you use the /realm endpoint to do that?

Signed-off-by: Diana <[email protected]>
Signed-off-by: Diana <[email protected]>
@cloudjumpercat
Copy link
Contributor

Broken sales link was a false failure, I manually approved link validation for now

@cloudjumpercat
Copy link
Contributor


The following diagram shows how you can configure centralized Consumers in realms as well as scope Consumers to a Control Plane:

<!--vale off -->
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really understand what this diagram is showing me.
For example:

Consumer: Ira
Consumer Groups: Gold for appointments

Does this consumer only have access to the appointments control plane? Or does it have access to both control planes, but since the consumer group Gold only exists on appointments, it can only access appointments? Or something else?

Are control planes located inside realms, or alongside them?

* The Consumer identity can be setup centrally instead of being defined across multiple control planes.
* Share Consumers across multiple control planes. Users don't need to replicate changes to Consumer identity in multiple control planes and Consumer configuration doesn't conflict.
* Reduces configuration sync issues between the control plane and the dataplane. Centralized Consumers aren't part of the configuration that is pushed down from the control plane to the dataplane, so it reduces config size and latency.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add the required Gateway version here as a note, somewhere.

Comment on lines 338 to 341
* **Realm is listed first:** The dataplane will first reach out to the realm. If the API key is not found in the realm, the dataplane will look for the API key in the control plane config.
* **Control Plane scope listed first:** The dataplane will initially check the control plane configuration (LMDB) for the API key before looking up the API Key in the realm.
* **Realm isn't associated with a control plane:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the dataplane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked.
* **Realm is only associated with a Control Plane:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the dataplane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* **Realm is listed first:** The dataplane will first reach out to the realm. If the API key is not found in the realm, the dataplane will look for the API key in the control plane config.
* **Control Plane scope listed first:** The dataplane will initially check the control plane configuration (LMDB) for the API key before looking up the API Key in the realm.
* **Realm isn't associated with a control plane:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the dataplane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked.
* **Realm is only associated with a Control Plane:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the dataplane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked.
* **Realm is listed first:** The data plane will first reach out to the realm. If the API key is not found in the realm, the data plane will look for the API key in the control plane config.
* **Control Plane scope listed first:** The data plane will initially check the control plane configuration (LMDB) for the API key before looking up the API Key in the realm.
* **Realm isn't associated with a control plane:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the data plane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked.
* **Realm is only associated with a Control Plane:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the data plane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked.

Should this info actually be on the Konnect page instead, except genericized to "credentials" instead of "API key"?

<!--vale off -->
{% mermaid %}
flowchart TD
A(Consumer: Cruz<br>Consumer Groups: Gold for appointments, Silver for billing)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each consumer can be in multiple consumer groups, but you can't scope the consumer group to a service.

If billing respected the Gold group, then Cruz would be Gold for billing too.

* {region}: Region for your {{site.konnect_short_name}} instance.
* $KONNECT_TOKEN: Replace with your {{site.konnect_short_name}} personal access token.
* $CONTROL_PLANE_UUID: (Optional) Replace with your Control Plane UUID.
* `ttl` and `negative_ttl`: (Optional) Time-to-live in minutes of the consumer or negative consumer for this realm in the {{site.base_gateway}} cache.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's a negative consumer? I don't think that this is what negative_ttl means. @zekth can you help us out here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's no concept of negative_consumer. negative_ttl represents the TTL of a bad login cache entry. Meaning api key foobar is sent to the gateway; gateway reaches the service to authenticate it; if it fails the api key foobar will be labelled as non working in gateway cache to the time of the TTL. During that time it won't try to reache the authentication server.

* $KONNECT_TOKEN: Replace with your {{site.konnect_short_name}} personal access token.
* {realmId}: The ID of the realm you created previously.
* $CONSUMER_NAME: Replace with the name of the consumer.
* $CONSUMER_GROUP: (Optional) Replace with the name of the consumer groups you want to associate with the consumer.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I specify consumer_groups on a consumer does that replace consumer_groups from the realm, or are they added together?

If I specify consumer_groups: [] does that mean no consumer groups, or use the groups from the realm?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is additive here, I'll add that info to the doc.

Comment on lines 340 to 341
* **Realm isn't associated with a control plane:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the dataplane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked.
* **Realm is only associated with a Control Plane:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the dataplane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* **Realm isn't associated with a control plane:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the dataplane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked.
* **Realm is only associated with a Control Plane:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the dataplane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked.
* **Realm only:** You can also configure a single `identity_realms` by omitting the `scope: cp` from the example. In this case, the dataplane will only attempt to authenticate API keys against the realm. If the API key isn't found, the request will be blocked.
* **Control Plane only:** You can configure a look up only in the control plane config by only specifying `scope: cp` for `identity_realms`. In this scenario, the dataplane will only check the control plane configuration (LMDB) for API key authentication. If the API key isn't found, the request will be blocked.

@@ -287,19 +295,55 @@ Response:
}
```

{% if_version gte: 3.10.x %}

## Configure realms for centralized Consumers in {{site.konnect_short_name}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be an example rather than on the overview?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I'll convert it to an example.

cloudjumpercat and others added 2 commits March 14, 2025 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci:manual-approve:link-validation do not merge Issues/ PRs whose changes should not be merged at this time review:general Review for general accuracy and presentation. Does the doc work? Does it output correctly?
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants