Operator should configure route server peers

[daxelrod-rh]

Thank you very much for working on this!

The README indicates that Terraform should handle

Route Server peers (one per router node)

However, ROSA clusters do not maintain stable worker node IP addresses over time. Workers are swapped out for new workers with different IP addresses during upgrades, scaling events, or in cases of failure. There is no good way to trigger Terraform runs when these events happen.

Therefore, this operator should add and delete peers within the Route Server as router nodes join and leave the cluster.

For reference, see the create-bgp-peers container of this daemonset, and this cleanup cronjob. I am not suggesting you use those implementations exactly, but pointing you to them as examples of something accomplishing what needs to be done.

For more prior art, see also this implementation for GCP https://github.com/rh-mobb/osd-gcp-cudn-routing/tree/real-vms/operator

For things you might want to do differently than the above implementations:

• The cleanup assumes that the only peers for this Route Server belong to the cluster, even though the Route Server itself is not a cluster-managed resource. Ideally, it would only clean up the peers that were added by the cluster
• When considering how this implementation might be generalized to eventually work in other hyperscalers, it may make sense to hook node lifecycle events rather than using a daemonset and a cronjob. For example, on GCP, worker node instances cannot actually be terminated unless they have been removed from the list of peers, so the peer removal needs to be done in a finalizer.