refactor: remove container from list of run image types

[Bobbins228]

What does this PR do?

[Provide a short summary of what this PR does and why. Link to relevant issues if applicable.]
Removes the ability to run llama stack container images through the llama stack CLI
Closes #2110

Test Plan

[Describe the tests you ran to verify your changes with result summaries. Provide clear instructions so the plan can be easily re-executed.]
Run:

llama stack run /path/to/run.yaml --image-type container

Expected outcome:

llama stack run: error: argument --image-type: invalid choice: 'container' (choose from 'conda', 'venv')

[mattf]

it's a good idea to reduce the number of ways users can run a containerized distribution.

constructing the correct docker command-line is non-trivial, platform & config specific, and easy to get wrong.

this change leaves dead code in start_stack.sh, see https://github.com/meta-llama/llama-stack/blob/main/llama_stack/distribution/start_stack.sh#L114-L163

what about turning https://github.com/meta-llama/llama-stack/blob/main/llama_stack/distribution/start_stack.sh#L152 into a message about what docker command to run?

users may also want a --please-run option to run that docker command for them.

[Bobbins228]

@mattf I am not sure I understand what you mean.
docker run ... should still run with no issue.

It is only cases where users run llama stack run with a container image type will there be an error.

[mattf]

docker run ... should still run with no issue.

... is difficult to figure out. llama stack run --image-type container does the heavy lifting to figure out what ... should be.

[Bobbins228]

I see so you are saying its worth keeping the ability to run containers through llama stack run as it can reduce complexity for users who may not be comfortable with configuring their run commands.

I can see a reason to go either way but if it is the case of keeping the functionality to the best of my knowledge there is no mention of this capability in the LS docs. Mostly when container images are concerned the examples given are docker commands.

[mattf]

https://llama-stack.readthedocs.io/en/latest/distributions/building_distro.html talks about containers in the context of llama stack run, but all current templates have image_type: conda and it only gives examples of --image-type conda/venv

the "via docker" examples (e.g. https://llama-stack.readthedocs.io/en/latest/distributions/self_hosted_distro/ollama.html#via-docker, https://llama-stack.readthedocs.io/en/latest/distributions/self_hosted_distro/meta-reference-gpu.html#via-docker, ...) are inconsistent (e.g. mount handling) and lack info about selinux or when --gpus is needed.

all i'm advocating for is an easy, consistent way for users to construct a docker (or podman) command line.

[Bobbins228]

@mattf As you've said it is difficult for us to provide a one command to suit all. Given the user's platform and specs they may want to run a container with different settings/configs or have a vastly different way they want their container to be configured.

I understand your concerns with regards to the "via docker" examples, we can work to improving documentation there with comments on each flag where needed so people are given clearer instructions and are less likely to get confused.

I'll update https://github.com/meta-llama/llama-stack/blob/main/llama_stack/distribution/start_stack.sh#L152 to include a message to use an appropriate docker command.

[mattf]

@mattf As you've said it is difficult for us to provide a one command to suit all. Given the user's platform and specs they may want to run a container with different settings/configs or have a vastly different way they want their container to be configured.

it's definitely difficult for users. if we can make it easier for users, how difficult it is for us should be irrelevant.

how to run these containers on a platform like k8s is even more difficult and currently left as an exercise for the reader.

[leseb]

@mattf thanks for the review, my initial thought about running a containerized distro via the llama stack run command is that it feels strange to require the llama-stack package to run a distro.
I think we should do a better job at documenting how to execute a Docker/Podman command if needed.

Going a bit further, even supporting things like conda/venv makes no sense to me. That is a llama-stack developer convenience more than something "real" users will ever do. Only llama stack run makes sense, it then relies on the presence of the package on the system.

So coming back to this one, I'd:

• remove code from llama_stack/distribution/start_stack.sh
• revisit the doc to make sure the container run commands are correct

Thoughts? @Bobbins228 @mattf
Thanks!

[Bobbins228]

When I went to write up a warning/example docker command to replace https://github.com/meta-llama/llama-stack/blob/main/llama_stack/distribution/start_stack.sh#L114-L163. I tested it first and it ran just fine.

This script is creating a docker command to run the distribution rather than use llama stack run so this script should always run without getting the llama stack run: error: argument --image-type: invalid choice: 'container' (choose from 'conda', 'venv').

I am on board with your idea to be more clear with users with regards to building docker commands but it may be out of scope for this issue? I can create a new issue and update the documentation in a follow on PR to clearly explain why the example docker commands are configured the way that they are so we can limit confusion. WDYT?

[leseb]

@Bobbins228 yes, please, we can do that as a followup. But why this reluctance to delete the code to start the container from llama_stack/distribution/start_stack.sh?

[Bobbins228]

@leseb I have no problem with removing the code I'll update the PR. 👍

[leseb]

nits, @ashwinb I'd like your signoff on this, thanks!

[ashwinb]

Yes this makes sense to me. Run dockers directly. The CLI is really for the "single node" user. If the CLI auto downloaded the dockers we officially distribute, then it would make sense but that's too complicated and doesn't serve anybody's purposes.

[leseb]

Final nits

[leseb]

@mattf I'll soon merge this, PTAL to make sure all your requested changes have been addressed, thanks!

[leseb]

Thanks!

[booxter]

the wording here is a bit confusing. "how come it doesn't support running containers? I just built one!" (here "Llama Stack probably stands for llama stack command, but it may be not clear to a new user.

[ChristianZaccaria]

One nit, thanks!

[ChristianZaccaria]

/lgtm

stale pending review - please open an issue if any concern arises.