Revise GEP-1713: Update gateway status and conformance test details for ListenerSets

[davidjumani]

What type of PR is this?
Add one of the following kinds:
/kind documentation
/kind gep

Optionally add one or more of the following kinds if applicable:
/area conformance-test

What this PR does / why we need it:
This PR does the following :

• Adds the list of conformance tests to be added to ListenerSets (GEP-1713)
• Updates the status of the Gateway to include AttachedListeners that is the count of successful ListenerSet attachments to the gateway. Ref: slack theraad

The aim is to get consensus for the set of conformance tests required to validate this feature against implementations, and move towards promoting this to Standard

Does this PR introduce a user-facing change?:

Adds the `AttachedListeners` conditions to the Gateway status for the GEP and details for ListenerSets conformance tests

[davidjumani]

cc @rikatz @dprotaso

[robscott]

cc @rostislavbobo @mkosieradzki

[rikatz]

@davidjumani thanks, this is a great write of conformance. Do you think it makes sense to also add the expected conditions on each conformance item, and also the ancestorStatus expected, to make it clear to implementations what we will look for?

[davidjumani]

Do you think it makes sense to also add the expected conditions on each conformance item

I intentionally left them out right now in case there's any discussion to be had when the conformance tests are in review. Happy to add them now if that would clarify things and make reviewing the tests easier.

Added the expected status as well as added the AttachedListeners status that is the count of successful ListenerSet attachments to the gateway. Ref: slack theraad

[rikatz]

should this accepted here be false?

[davidjumani]

Based on the GEP

	// This can be the reason when "Accepted" is "True" or "False", depending on whether
	// the listener being invalid causes the entire Gateway to not be accepted.

I believe this should be true since the ListenerSet is configured but does not adversely affect the parent Gateway.
Additionally, there can be a scenario where only a subset of the listeners is invalid and this would more accurately reflect it

[rikatz]

but then the reason should be empty or something else, right? It was accepted but not programmed, I guess

[rostislavbobo]

+1. If the status is Accepted, there should not be a reason for that.

I wouldn't also mix Accepted: True and Programmed: False in this case. The Programmed condition indicates whether the controller was physically able to apply the requested resources (e.g. allocate an IP or open a port). Listener conflicts are configuration errors, not the controller ability to program them.

[davidjumani]

I don't think so. A subset of listeners can be accepted, so the status should reflect it accordingly.
If not all listeners are accepted, then the listenerSet Accepted condition should be false.

Additionally, the Programmed condition should also reflect the same, since a subset of listeners was applied

[rikatz]

I think there's some confusion here, let me know if the below makes sense

• If I have one listener conflicted:
  • This listener should be accepted (means a controller SAW it), no Reason set
  • This listener should have a status Conflicted=True, Reason explaining if it is a Protocol conflict, etc (and a good message)
  • Programmed will be False, because the proxy won't receive this configuration

As the status conditions is per listener, I think this would make sense, wdyt?

[rikatz]

(specifically on this case, we can think about extending the valid Reasons to something like ConflictWithParent to let the user know there's already a conflicting listener on the Gateway, without leaking any kind of information)

[youngnick]

I think the language in this part is a little confusing:

	// This can be the reason when "Accepted" is "True" or "False", depending on whether
	// the listener being invalid causes the entire Gateway to not be accepted.

I think it should be something like:

	// This can be used as a reason when "Accepted" is "True" or "False", because Accepted
    // requires that at least one Listener present on the object is valid. If zero Listeners in the
    // ListenerSet are valid, then Accepted is False, and this reason is used to explain why.
    // If at least one Listener in the ListenerSet is valid, then Accepted is True, and this reason
    // is used to indicate that the ListenerSet is only _partially_ valid.

Acceptance is about being locally correct, where Programmed is about full correctness. Conflicted is then used to indicate that at least one Listener in the ListenerSet is in conflict with a Listener on another object (no matter which one it is).

I don't think that saying that the conflicted Listener is on the Gateway gives too much information away, since the ListenerSet already includes a reference to that parent anyway. That said, adding a ConflictWithParent reason sounds good to me. It's okay to have more options for Reasons, they don't have the overhead that adding extra whole Conditions does.

[rikatz]

do we really need this condition on the parent gateway? I am wondering if in Gateway case we can just not have any count of attached listener, as mostly the user will have its own condition

[davidjumani]

I think the condition should be set since :

• The user has explicitly disallowed ListenerSets
• ListenerSets have been attached (at least tried) to the Gateway

The user should know about this in the status

[rikatz]

I am not aware of the whole conditions reasoning here tbh, will defer to @youngnick for some better view on it

[rikatz]

I am still not fully convinced we should have the condition if we would have a counter as well, but I also do not have a very strong opinion on it

[youngnick]

In this case, I think it's useful for the reason that @davidjumani said - the Gateway owner has said "no ListenerSets allowed", and some ListenerSets have attempted to attach. Regardless of exactly how that situation has come up, the Gateway owner needs to know. attachedListeners doesn't tell us that, because it's a count of ListenerSets that have successfully attached.

[rikatz]

again, should we have an accepted here? And a reason? Not sure if this doesn't get more confusing

[davidjumani]

I think this is a better indication that the listenerSet is correct (the parentref is valid, etc.) but one or more of the listeners is invalid which is indicated by the ListnerStatus above

[rostislavbobo]

but one or more of the listeners is invalid which is indicated by the ListnerStatus above

If all listeners in the ListenerSet have conflicts, shouldn't we report the ListenerSet as not accepted? This is consistent with other resources (e.g. if an HTTPRoute has no hostname intersection with a Gateway listener, it's reported a not accepted).

[davidjumani]

Yes, I agree and will update it accordingly. I'll make this clearer with the total number of conflicting listeners

[rikatz]

@davidjumani overall lgtm! I would like us to keep separated a PR that changes the GEP from a PR that changes the API, this way if we need to track any (or revert) we have proper changes

Thanks again for the effort here!

[rikatz]

Just a headsup, I got a bit confused here to understand that AttachedListenerSets is a condition and AttachedListeners is a .status field, maybe worth just clarifying it

[youngnick]

Yeah, I think these might need to be separate tables or something, or maybe something like:

      | Type    | Status | Reason |
      | -------- | ------- | ------- |
      | conditions.AttachedListenerSets  | Unknown    | // any reason // |
      | AttachedListeners  | 0    |  |

[rikatz]

On the port? Or on the tuple? If you have a Listener defined on the same port, using a different host (eg.: some HTTP listener) this shouldn't fail.

I think you can make a call out for now that all the tests here are executed with at least one listener defined on gateway (because we enforce this...) and then have a separate set of tests for when we remove the need of Listeners on Gateway, wdyt?

[youngnick]

Should probably be "the request to that distinct Listener fails" (that's what the word "distinct" is useful for when talking about Listeners.)

[youngnick]

And yes, we will definitely need a bunch more tests if and when required Listeners is 0 on a Gateway, both when ListenerSets are attached and when they are not.

[youngnick]

Hmm, this means that the winning ListenerSet owner will have way to know that there is another ListenerSet that's in conflict with this one.

It feels a little better to me to have this ListenerSet have a Conflicted: True, with a reason of something like ConflictWinner (or some other better name), to indicate that there was a conflict, but it won. Then the losing ListnenerSet would get "ConflictLoser" or something (wow, that's a terrible Reason, but you get what I mean).

That way "Conflicted" means "there is a conflict with something", and there is a Reason to indicate what the result of the conflict was.

I suppose, though, that Accepted is probably also carrying the same information, so maybe those Reasons are redundant. Interested in other's thoughts here.

[youngnick]

This is really great @davidjumani, I think that we should try and do this exercise more often for conformance tests for other, this will make it easier to ensure we don't miss things in testing.

[rikatz]

do we want to mark this feature as well on supported features? like a Gateway SHOULD report if it supports dynamic port assignment using the feature name "ListenerSetDynamic" (be more creative than I am with names!)

[davidjumani]

Is there an existing feature name for dynamic port assignment on Gateway? If so we should use that - since ListenerSets are just an extension of the gateway

[youngnick]

There is currently no dynamic port assignment on Gateway, port is a required field there. We are trying out dynamic port assignment on ListenerSet.

[davidjumani]

Added the ListenerSetDynamicPortAssignment supported feature flag

[rikatz]

IIRC ReferenceGrant is also a specific supported feature (

gateway-api/pkg/features/referencegrant.go

Line 27 in a863bd3

SupportReferenceGrant FeatureName = "ReferenceGrant"

) so the test needs to be sure that it only happens when ReferenceGrant is supported

[davidjumani]

Agreed. Will add this as a requirement on the specific test

[rikatz]

not sure if this is me being too cautious, but I would like for every test if possible that we also make a request and guarantee that it is working as desired.

eg.: on a conflicting ListenerSet we need to be sure the request goes to the right route attached to the right LS on the right namespace.

Also, optionally, do we want a test on a ListenerSet that has 2 different listeners (eg.: http and https) just to be sure that everything is working fine on some more complex scenarios?

[davidjumani]

Yes, we can do that, and I intend to do so.
I removed it from the doc for now, as I see that not all Gateway conformance tests send out requests but only check the status and I wasn't sure if we wanted to just do it that way.

[rikatz]

/cc
/assign

[rikatz]

Just did a pair review and update with David, I think this is good to go.

/lgtm
/approve

Will wait @youngnick for a second pass, but we can work on conformance based on this already

Thanks!

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: davidjumani, rikatz
Once this PR has been reviewed and has the lgtm label, please assign aojea for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• geps/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[youngnick]

Is it useful in this case to record that the Listener is actually Conflicted? (It is, it's just the winner in the conflict stakes).

That would allow Gateway owners to tell that, for example, ListenerSet owners are attempting to configure a Listener that conflicts, and that removing this Listener will allow the ListenerSet Listener to win instead.

This would have to apply to all conflicts between Listeners, including between sibling ListenerSets to work though.

[kflynn]

I feel like it probably is worth marking the Listener as conflicted.

More broadly, I think that it's a good idea to start thinking of a Listener as a sort of degenerate LIstenerSet in basically all cases.

[youngnick]

Doesn't need to be resolved now, but we will need to come back to what constitutes a "unique" port. If the Protocol is HTTPS, does this port need to be unique if the hostname is distinct? (That would substantially complicate implementing this, and may be a bad idea. But if we are ruling it out, we need to be explicit about that).

[youngnick]

The list of conformance tests looks great to me, as long as we are can answer the two questions I added there.