[EPIC] Serving Foundation Models

[Shreyanand]

Serving foundation models on an OpenShift cluster is a crucial step for using them in production. As a part of this issue we will deploy demo models using open source tools and/or the stack recommended by RHODS. Following questions would be further explored as a part of this epic:

• Serving LLMs on Open Data Hub / RHODS with ModelMesh #46

• Optimizing LLMs for max performance when serving on ODH #48

• text-generation-webui as a platform for serving LLMs #47