Efficient Kubernetes API Updates using Informers (#75)

* Add new Datasource based on Kubernetes Informers

Within the Go API client, the Kubernetes project has created tools for
keeping local references to API resources up to date with the server
state. By creating an Informer, the kubernetes API client can perform an
intial sync of objects we are interested in and then watch for updates
to those objects instead of polling the server on a regular interval.

Changing the access pattern of the config reloader is important for
scaling kubernetes clusters. If the config reloader is set to poll for
updates on a short interval (such as 5 seconds) it can generate quite
a significant amount of traffic on the API server even in relatively
modest clusters composed of a couple dozen compute nodes. This increase
in request volume does not provide any value to the config reloader, and
is a potentially expensive side-effect on AWS if some nodes are in a
different availability zone from the kubernetes API server(s).

By creating this new Datasource, the config reloader can be configured
with an arbitrarily short update interval without having to worry about
drowning the kubernetes API server in a flood of pointless requests. The
Informer datasource maintains watches on important API objects in a
separate goroutine, and will make changes available to configreloader
updates very quickly, usually on the update immediately following the
change in the upstream API.

The new Datasource has been designed to be able to replace both existing
Kubernetes Datasources, but has only been tested against the behavior of
the Kubernetes Multimap Datasource.

Signed-off-by: Richard Jackson <[email protected]>

* Replace usage of existing Kubernetes Datasources

Update the controller to only use the new Informer-based datasource to
replace both existing kubernetes datasource implementations.

Note: This change has been tested in multiple production-quality
clusters for several weeks, but only in a configuration that formerly
used the Multimap datasource. Only cursory testing has been done to
verify that the new datasource is compatible with the default Kubernetes
datasource.

Signed-off-by: Richard Jackson <[email protected]>

* Remove unused datasources

The new Informer-based datasource for Kubernetes has clear benefits in
terms of scalability, and should be compatible with the behavior of both
existing datasources.

Signed-off-by: Richard Jackson <[email protected]>

Author: rjackson90

config-reloader/controller/controller.go

@@ -49,11 +49,7 @@ func New(cfg *config.Config) (*Controller, error) {
 	} else if cfg.Datasource == "fs" {
 		ds = datasource.NewFileSystemDatasource(cfg.FsDatasourceDir, cfg.OutputDir)
 	} else {
-		if cfg.Datasource == "multimap" {
-			ds, err = datasource.NewKubernetesMultimapDatasource(cfg)
-		} else {
-			ds, err = datasource.NewKubernetesDatasource(cfg)
-		}
+		ds, err = datasource.NewKubernetesInformerDatasource(cfg)
 		if err != nil {
 			return nil, err
 		}