|
1 | | -# Scaler |
2 | | -[](https://travis-ci.org/v3io/scaler) |
| 1 | +# Scale to Zero |
3 | 2 |
|
4 | | -Building blocks to create a scaler (to and from zero instances) based on custom metrics. |
| 3 | +Infrastructure to scale any resource to and from zero |
5 | 4 |
|
6 | | -Prerequisites: |
| 5 | +## Design: |
| 6 | + |
7 | 7 |
|
8 | | -https://github.com/prometheus/prometheus <br> |
9 | | -https://github.com/kubernetes-incubator/metrics-server <br> |
10 | | -https://github.com/DirectXMan12/k8s-prometheus-adapter <br> |
| 8 | +**Resource** - A generic name for the component which is being scaled by this mechanism. |
| 9 | +Will mostly include pod/s (that will be scaled when needed) wrapped by one or many deployment/daemon set/replica set, |
| 10 | +and k8s service/s that can be used to route incoming requests. |
11 | 11 |
|
12 | | -Schema: |
13 | | -``` |
| 12 | +**Resource-scaler** - An implementation of the `ResourceScaler` interface defined in |
| 13 | +[scaler-types](https://github.com/v3io/scaler-types). It is transplanted inside the autoscaler and dlx using |
| 14 | +[Go plugins](https://appliedgo.net/plugins/). They use them to perform actions on the specific resource.<br> |
| 15 | +For example, when the autoscaler decides it needs to scale some resource to zero, it executes the resource-scaler's |
| 16 | +`SetScale` function which has the knowledge how to scale to zero its specific resource. |
14 | 17 |
|
15 | | - Your Pod |
16 | | - To scale |
17 | | - ------------ ------------ ------------ ------------ ------------ ------------ |
18 | | -| | | /metrics | | Prometheus | | Prometheus | | Metrics | | Metrics | |
19 | | -| Service |-> | |<--- | |<--- | Adapter |<--- | Aggregator |---> | Server | |
20 | | -| | | | | | | | | | | | |
21 | | - ------------ ------------ ------------ ------------ ------------ ------------ |
22 | | - | /\ |
23 | | - | ------------ |
24 | | - | Resource | | |
25 | | - | Scaled to zero | Autoscaler | |
26 | | - | | | |
27 | | - | ------------ |
28 | | - | |
29 | | - | ------------ |
30 | | - | | | |
31 | | - ------------------> | DLX | |
32 | | - | | |
33 | | - ------------ |
34 | | - |
35 | | - |
36 | | - |
37 | | -``` |
| 18 | +**The autoscaler** - Responsible for periodically checking whether some resources should be scaled to zero. this is |
| 19 | +performed by by querying the custom metrics API. Upon deciding a resource should be scaled to zero, it uses the internal |
| 20 | +resource-scaler module to scale the resource to zero. |
| 21 | +The resource-scaler will first route all incoming traffic to the DLX, which in terms of K8s is done by changing a |
| 22 | +service selector, after that, it will scale the resource to zero. |
38 | 23 |
|
39 | | -### Autoscaler |
40 | | -Based on custom metric name, will call `ResourceScaler` interface function `Scale` with a `Resource` from a list of known `Resource`'s (`GetResources` function) |
41 | | -Config for service includes: |
42 | | -``` |
43 | | - Namespace - kubernetes namespace of the resources |
44 | | - ScaleWindow - an allowed period for the resource to be inactive before calling the `Scale` function |
45 | | - MetricName - name of the metric to monitor for the resource |
46 | | - Threshold - A threshold for a metric to concider the resource being inactive |
47 | | - ScaleInterval - an interval to call scale function which checks if a resource has passed the scale window, if not the scale window is being considered the longest duration of time where the metric for the resource is under the threshold |
| 24 | +**The DLX** - Responsible for receiving and buffering requests of scaled to zero resources, Upon receiving an incoming |
| 25 | +request it creates a buffer for the messages, and tells the resource-scaler to scale the service back from zero. |
| 26 | +The resource-scaler will scale the resource back up and then route the traffic back to the service (by modifying the k8s |
| 27 | +service selector). |
| 28 | + |
| 29 | +## Prerequisites |
| 30 | + |
| 31 | +**Custom metrics API implementation:** |
| 32 | + |
| 33 | +The Autoscaler makes decisions based on data queried from Kubernetes |
| 34 | +[custom metrics API.](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/custom-metrics-api.md) |
| 35 | +There are several possible tools that implement it, we internally use |
| 36 | +[Prometheus](https://prometheus.io/) with the [Prometheus-Adapter](https://github.com/DirectXMan12/k8s-prometheus-adapter) |
| 37 | +but you can use which ever you want! You can find some recommended implementations |
| 38 | +[here](https://github.com/kubernetes/metrics/blob/master/IMPLEMENTATIONS.md#custom-metrics-api) |
| 39 | + |
| 40 | +## Getting Started |
| 41 | +The infrastructure is designed to be generic, flexible and extendable, so as to serve any resource we'd wish to scale |
| 42 | +to/from zero. All you have to do is implement the specific resource-scaler for your resource. The interface between your |
| 43 | +resource-scaler and the scale-to-zero infrastructure's components is defined in |
| 44 | +[scaler-types](https://github.com/v3io/scaler-types) |
| 45 | + |
| 46 | +**Note:** Incompatibility between this scaler vendor dir and your resource-scale vendor dir may break things, |
| 47 | +therefore it's suggested to put your resource-scaler in its own repo |
| 48 | + |
| 49 | +Examples: |
| 50 | +* [Nuclio functions resource-scaler](https://github.com/nuclio/nuclio/blob/master/pkg/platform/kube/resourcescaler/resourcescaler.go) |
| 51 | +* [Iguazio's app service resource-scaler](https://github.com/v3io/app-resource-scaler/blob/development/resourcescaler.go) |
| 52 | + |
| 53 | +## Installing |
| 54 | +[Go plugins](https://appliedgo.net/plugins/) is the magic that glues the resource-scaler and this infrastructure |
| 55 | +components together.<br> |
| 56 | +First you'll need to build the resource-scaler as a Go plugin, for example: <br> |
| 57 | +```sh |
| 58 | +GOOS=linux GOARCH=amd64 go build -buildmode=plugin -a -installsuffix cgo -ldflags="-s -w" -o ./plugin.so ./resourcescaler.go |
48 | 59 | ``` |
49 | | -Scaler should change the resource service to point to DLX service and not to the scaled down pod (to be changed by `ResourceScaler` or some other external entity) |
| 60 | +The autoscaler/dlx looks for the plugin using this path (from the execution directory) `./plugins/*.so` so you should |
| 61 | +move the binary artifact of the build command (the `plugin.so` file) to the `plugins` directory |
| 62 | +It is much easier to do everything using Dockerfiles, here are some great examples: |
| 63 | +* [Nuclio function Autoscaler dockerfile](https://github.com/nuclio/nuclio/blob/master/cmd/autoscaler/Dockerfile) |
| 64 | +* [Nuclio function DLX dockerfile](https://github.com/nuclio/nuclio/blob/master/cmd/dlx/Dockerfile) |
| 65 | +* [Iguazio's app service Autoscaler dockerfile](https://github.com/v3io/app-resource-scaler/blob/development/autoscaler/Dockerfile) |
| 66 | +* [Iguazio's app service DLX dockerfile](https://github.com/v3io/app-resource-scaler/blob/development/dlx/Dockerfile) |
| 67 | + |
| 68 | +You can install the components using the [scaler helm chart](https://github.com/v3io/helm-charts/tree/development/stable/scaler)<br> |
| 69 | +`$ helm install --name my-release v3io-stable/scaler` |
| 70 | + |
50 | 71 |
|
51 | | -### DLX |
52 | | -A service that listens on http requests for a given namespace, when a request is received, it's being saved in-memory until `Scale` function returns, this is an indication that the resource is ready and any changes that were made by autoscaler were reverted (i.e. changing where service points to), that request is then being proxied to the new instance and the response from it returns to the user. |
| 72 | +## Versioning |
53 | 73 |
|
| 74 | +We use [SemVer](http://semver.org/) for versioning. For the versions available, see the |
| 75 | +[releases on this repository](https://github.com/v3io/scaler/releases). |
0 commit comments