Simplify management of Kong Plugins

[programmer04]

Problem Statement

Currently, there are two dedicated CRDs

• KongPlugin
• KongClusterPlugin

How Kong Plugins can be configured

• Plugins can be associated with the Ingress, Service, HTTPRoute, KongConsumer, or KongConsumerGroup object in Kubernetes using konghq.com/plugins annotation.
• For KongClusterPlugins global label is used, which is not well documented, you can find the description in this example and PR KIC #112.
• For KongPlugin cross-namespace reference with a ReferenceGrant is allowed too, which basically can replace KongClusterPlugins entirely. See KIC issue #5355 for more context.

Everything aforementioned happens in annotations and labels.

Proposed Solution

Leave only KongPlugin CRD that works with ReferenceGrant and enhance KongPluginBinding to handle all cases that used to be handled with annotations explicitly with CRDs.

Additional Information

Problems with the current approach that will be resolved

• Specifying a nonexistent plugin name via konghq.com/plugins annotation is silently ignored, configuration is applied with that plugin kubernetes-ingress-controller#6139
• Consider plugin bindings combinations created by annotating KongService, KongRoute, KongConsumer and KongConsumerGroup objects #660

Acceptance Criteria

• only KongPlugin that works with ReferenceGrant and enhanced KongPluginBinding

[czeslavo]

@programmer04 will update the issue description with links to the design doc with more details about the proposed solution.

[sandromodarelli]

punting to KO 2.1.0 due to other priorities, maybe can land in future milestones

[sandromodarelli]

we are still waiting for product feedback about how to deal with plugin CRDs. punting to KO 2.2

[sandromodarelli]

@justindavies WDYT about getting the rid of annotations to attach plugins?