From https://github.com/kubermatic/kubelb-ee/issues/22:

Since KubeLB is a mission-critical part of potential a lot of Clusters to work, we need MLA in place.

This also includes metrics and visibility of traffic flow.
A good start would be to check out what metrics cillium and hubble are providing to help us out.

Requirements

• Meaningful metrics for CCM, Manager, and Envoy Control Plane
• Grafana Dashboards to visualize metrics
• Recording and Alerting Rules

Limitations

• KKP doesn't support PrometheusRule CRDs

References

1. Controller-runtime metrics
2. Autogenerate metric docs
3. Envoy Gateway Metrics