Separate supported Public API from Private

[sthaha]

Problem

Currently, to provide a supported version of Kepler API and to prevent API breakage (bad UX), we disallow features such as

• setting an image
• Deploying more than one version of Kepler
• Specifying a namespace where kepler is deployed
• Long discussions to get the API right.

Proposal: new Internal API

The proposal is to introduce a new internal api - KeplerInternal that allows all the above + more. Users of Kepler Operator can use any internal api at their own risk but the internal API/Custom Resource should be treated as implementation details (that can change without notice).

As features stablise in internal API, they can be migrated to the Public API - Kepler

Advantages

• Allows kepler deployment in production to remain stable
• Allows features to be added to internal API quickly
• Allows experimental features to be shipped to users quickly (like any regular release)
• Allows clear separation of what is supported vs unsupprted
• No code duplication
• Allows multiple kepler to be deployments (eg. bcc, libbpf)

Usecase

• New Model Server deployment can be added to Internal API. This would allows users to who want to experiment with ModelServer by creating a kepler-internal CR, and provide feedback.

[sthaha]

I have the implementation of the proposal here - #306