On-premise cluster considerations

[SEJeff]

What would be required to support on-premise clusters? Obviously it would be some code changes, but given remote management such as IPMI and some decent tooling, this seems possible.

[mikkeloscar]

Right now this tool is very much designed around AWS (or more generally cloud infrastructure). So we make some assumptions:

• Nodes are immutable and can be replaced during rolling cluster upgrades
• Base infrastructure like network are not directly the responsibility of CLM e.g. we use Cloudformation to provision this in AWS.

These things are usually different on-premise so the question is if this is the right tool for the job?

What features are you most interested in, maybe we can think of ways that could be adapted to on-premise use cases?

[SEJeff]

We netboot the kubelets, so they can be considered immutable as well. The network isn't something we need to worry about as well. What I'm most interested is the ability to manage a kubernetes cluster (we run everything sans the kubelet and etcd in pods), and upgrade / provision / re-provision it. This tool looks a lot like the type of thing I'd build, so I figured why not try to use an existing mechanism.

[szuecs]

@mikkeloscar maybe you can create an interface for the aws specific handling and use provider packages and let others implement these interfaces. I guess it’s a bit of work. Maybe you only need to guide and review code.
Maybe @SEJeff would like to do the work to make his work less and get our experience.
Thoughts?