wip: experiment running the UI from fastapi

[leseb]

@ashwinb this is a followup on our previous conversation. By using the script attached in f9095ce, I could run a static version of the file. This eliminates the need to call npm to run the UI. We can include the static content in a new route on the distro server.

I've run both the static version and the one with npm dev run, and I get two identical pages—except the dev tool debug thingy.

Am I missing something? If not, I feel we still have a case where we build the static version of the site (from .next, we would need to track that directory) and package it along with our current package. Then users can decide to activate the UI or not through a server spec config.

Thanks for considering this!

[leseb]

cc @ehhuang

[ashwinb]

This precludes running anything on the server as part of Next.JS. This is the true benefit of Next where pieces of the UI can move transparently between the backend and the frontend. It is extremely beneficial as the UI scales.

I need to understand the root cause of the concern: is the concern the need for a Node runtime for the Stack?

[leseb]

This precludes running anything on the server as part of Next.JS. This is the true benefit of Next where pieces of the UI can move transparently between the backend and the frontend. It is extremely beneficial as the UI scales.

I need to understand the root cause of the concern: is the concern the need for a Node runtime for the Stack?

The short answer is yes. My main concern is how we can elegantly include it as part of the server unless we say, " The UI runs standalone as another service next to the distro server."

[ashwinb]

I think it is important to understand the overarching primary requirement: The Stack shall come with UI. No ifs and buts. You can of course configure (via YAML) to say "nope, don't want that s**t." But the default is that the Stack comes with UI.

Given that requirement, the next goal is: we shall not handicap that UI from the get go. Forcing premature constraints on how a proper modern UI should be architected will get us something half baked which serves nobody's purpose.

So yes:

• we will require a Node runtime if you have a UI.
• this shall be the default.
• and yes, you will have a way to say "NO thank you" but via the YAML configuration.

[ashwinb]

unless we say, " The UI runs standalone as another service next to the distro server."

Correct that's how it should run in the cluster.

[ashwinb]

By the way, while we are iterating on the UI and haven't quite gotten the Node / NPM stuff completely worked out, it is reasonable to say "UI is not yet the default and you must do a --enable-ui to llama stack run"

cc @ehhuang

[leseb]

unless we say, " The UI runs standalone as another service next to the distro server."

Correct that's how it should run in the cluster.

This clears out all of my concerns then. However, that's not what this PR does. It runs it as a subprocess of the distro server :/
Anyways, I'm with you on the fact that we need a UI and a well-designed one, if we need a Node runtime so be it, my "issue" is more the tigh-coupling with the distro server.