Resource requests - defaults and documentation on typical resource usages

[consideRatio]

Should we really define a default value for the singleuser resource requests?

I remember reading about providing defaults are bad practice, and also that mybinder.org is providing a lot less guaranteed memory atm.

[manics]

I don't think defaults are needed, but it would be useful to have some example settings, either commented out in values.yaml or in the docs, including for the hub and proxy (e.g. suggested settings for "50 simultaneous users").

[consideRatio]

Vertical pod autoscalers are very much related to this discussion, they can empirically find out whats suitable: https://www.youtube.com/watch?v=Y4vnYaqhS74&t=0s&list=PLj6h78yzYM2PZf9eA7bhWnIh_mK1vyOfU&index=68

[minrk]

I think z2jh should have opinionated, best-practices defaults, not necessarily the most generic, forcing every deployment to have lots of the same configuration just to get going. I feel pretty strongly that we should at least have default limits, but I think default requests are still a good idea. Without default requests, it's awfully easy for too many users to exhaust memory on a node.

The question for defaults, I think, is: what's the failure mode when the default is wrong and how easy is it to change? Since it's very easy to change, I'm not worried about that so much, but thinking about the failure modes:

1. default no request means that ~100 users will go on each node by default. With small nodes, that makes it quite easy for users to trigger OOM errors on the node, even with the default limits
2. default request too high means autoscaling will happen sooner than needed, increasing cost and autoscale events. Or, if autoscale is disabled, causing launch to fail when too many users try to login.
3. default request too low can lead to 1., but only if request is not the same as limit and it will be strictly better than 1 in terms of OOM possibility.

[consideRatio]

Btw, if we define limits without requests explicitly, the requests will become the limits, i think.

I think also it could be nice to still have some default requests and limits. I suggested the following configuration could make sense for autoscaling in the optimization docs, see #1074.

# The resources requested is very important to consider in
# relation to your machine type. If you have a n1-highmem-4 node
# on Google Cloud for example you get 4 cores and 26 GB of
# memory. With the configuration below you would  be able to have
# at most about 50 users per node. This can be reasonable, but it
# may not be, it will depend on your users. Are they mostly
# writing and reading or are they mostly executing code?
singleuser:
  cpu:
    limit: 4
    guarantee: 0.05
  memory:
    limit: 4G
    guarantee: 512M

[consideRatio]

No matter if we provide defaults or not, I think it would be good to provide documentation describing the CPU/memory profile of various pods we deploy, and then let the user decide what to set, and perhaps provide defaults as well.

Prometheus + Grafana has helped a lot over the time to provide insights.

The following pods would be nice to have some smart recommendations for: hub/proxy/autohttps/user-scheduler