Migrate Access Logging support

[npolshakova]

K8sGateway Version

main

Is your feature request related to a problem? Please describe.

Migrate Gloo access logging functionality and APIs to kgateway.

Describe the solution you'd like

kgateway would be able to configure access logs on ListenerPolicy similar to how Gloo is able to with the ListenerOptions: https://docs.solo.io/gloo-edge/latest/guides/security/access_logging/

Describe alternatives you've considered

N/A

Additional Context

No response

[lgadban]

I think we need to reevaluate the Gloo API before porting over as-is, i.e. should AccessLogging be its own policy etc?

[lgadban]

related to (possibly a child of): #10489

[npolshakova]

I think we need to reevaluate the Gloo API before porting over as-is, i.e. should AccessLogging be its own policy etc?

I think the consensus was to avoid having CRs per policy, so ideally all the Access Logging config would live on an existing kgateway policy (such as ListenerPolicy)

[lgadban]

I think the consensus was to avoid having CRs per policy, so ideally all the Access Logging config would live on an existing kgateway policy (such as ListenerPolicy)

Yes that is the general plan but we still have some analysis to do and there may be exceptions.

[lgadban]

summarizing current state of discussion:

• gloo ALS plugin is a ListenerOption level config
• the presence of an ALS config at the Listener "tier" in the hierarchy will be applied to Listener (i.e. downstream), HCM, and tcp_proxy depending on the context
  • https://github.com/solo-io/gloo/blob/main/projects/gloo/pkg/plugins/als/plugin.go#L73
  • https://github.com/solo-io/gloo/blob/main/projects/gloo/pkg/plugins/als/plugin.go#L57
  • https://github.com/solo-io/gloo/blob/main/projects/gloo/pkg/plugins/tcp/plugin.go#L201
  • notably this means the same log format is used in all these contexts, with command directives that are not applicable to the context warned on in logs
• open question from me: does this mean you get 2 access log entries now when you enable ALS on ListenerOptions in GG?

outlining my concern:

• this is confusing behavior (magic)
• having a policy that is "inherited" by more granular config should ONLY be done via attachment point
  • this is the same concern I have with VHostOptions
  • if we move forward with it on ListenerOptions, do other configs on the ListenerOptions propagate to other contexts?
• with something like AccessLogging, do we not expect defaults for e.g. a GatewayClass to be defined?
  • i.e. if i want to turn on access logging for all Gateways how do I do it?
  • we do not have a semantic for default policies (other than RoutePolicies with route delegation)
    • one option is policies targeting/attaching to a GatewayClass
  • currently, "defaults" for gateway config lives on GatewayParameters with the ability for individual Gateways to provide their own customization

[npolshakova]

Summary:

Envoy exposes a lot of data that can be used when customizing access logs. The following data properties are available for both TCP and HTTP access logging:
*The downstream (client) address, connection information, TLS configuration, and timing
*The upstream (service) address, connection information, TLS configuration, timing, and Envoy routing information
*Relevant Envoy configuration, such as rate of sampling (if used)
*Filter-specific context that is published to Envoy’s dynamic metadata during the filter chain

While there's also additional HTTP-only information is available for access logging:
*Request data, including the method, path, scheme, port, user agent, headers, body, and more
*Response data, including the response code, headers, body, and trailers, as well as a string representation of the response code
*Protocol version

Some users may want to explicitly have access logs configured on both the listener and hcm. This is partially due to the command operators that exist at the listener level versus hcm as well as to catch different data in failure cases.

Open Questions:

• Is it weird/confusing to have the same access log config (in this case ListenerOptions.als) used in different places (listener, hcm, tcp_proxy) knowing that the unknown commands in the context will just be ignored? This is similar to the issue where users try to use operators from versions of envoy that we aren't running and therefore don't get the output they would want.
• Is having separate CRs to configure the HCM vs. Listener better UX? (Introducing a new HttpListenerPolicy) vs. how the listener plugin already separates http and non-http policies
  • If we have a new HttpListenerPolicy, what other HCM specific config would go here? -

[npolshakova]

Discussed offline:

• Initial access logging support will only support HCM access logging
• Will introduce new HTTPListenerPolicy to handle this instead of ListenerOption
• Default configuration will be handled at a later point