there's no one "right" answer to this question.
this issue is mainly for thoughts sharing.
I was trying to compare ExternalModel to LLMInferenceService.
in llmisvc - the CRD + reconciler doesn't live in this repo.
the integration point: MaaSModelRef references llmisvc in its spec.
if we export ExternalModel CRD and reconciler to payload processing repo, we can follow the same pattern and MaaSModelRef can reference it.
from the payload processing repo point of view:
we have a set of plugins, all of them can work only if ExternalModel CRD exists. there is a tight coupling here.
e.g., in the README.md, the first instruction is to deploy ExternalModel CRD (otherwise plugins will not work).
Adding few more details:
from checking through maas repo, we can see the existing reconcilers are:
maasmodelref, maasauthpolicy, maassubscription (up until here, all is core maas, which is good) and then we have externalmodel (which violates the pattern).
given the fact that ExternalModel plugins are tightly coupled with ExternalModel CRD + following llmisvc pattern shows it's possible,
I'd recommend we move that CRD to the other repo (Post 3.4).
ExternalModel serving becomes self contained and can be tested and deployed without maas.
happy to hear your thoughts:
cc: @jland-redhat @noyitz @bartoszmajsak @nerdalert @ishitasequeira
there's no one "right" answer to this question.
this issue is mainly for thoughts sharing.
I was trying to compare ExternalModel to LLMInferenceService.
in llmisvc - the CRD + reconciler doesn't live in this repo.
the integration point: MaaSModelRef references llmisvc in its spec.
if we export ExternalModel CRD and reconciler to payload processing repo, we can follow the same pattern and MaaSModelRef can reference it.
from the payload processing repo point of view:
we have a set of plugins, all of them can work only if ExternalModel CRD exists. there is a tight coupling here.
e.g., in the README.md, the first instruction is to deploy ExternalModel CRD (otherwise plugins will not work).
Adding few more details:
from checking through maas repo, we can see the existing reconcilers are:
maasmodelref, maasauthpolicy, maassubscription (up until here, all is core maas, which is good) and then we have externalmodel (which violates the pattern).
given the fact that ExternalModel plugins are tightly coupled with ExternalModel CRD + following llmisvc pattern shows it's possible,
I'd recommend we move that CRD to the other repo (Post 3.4).
ExternalModel serving becomes self contained and can be tested and deployed without maas.
happy to hear your thoughts:
cc: @jland-redhat @noyitz @bartoszmajsak @nerdalert @ishitasequeira