RFE: allow running ramalama from toolbox

[juhp]

Thought maybe I could use my podman flatpak-spawn hack to run ramalama from within toolbox
but it looks harder since it bind-mounts /usr/bin/ramalama.

Anyway this is an rfe to support toolbox: though I suppose ideally we would like the tools to be packaged in Fedora afap.

[ericcurtin]

ramalama essentially has it's own toolbox implementation. ramalama by default executes podman internally just like toolbox does.

Now one could create a ramalama toolbox if they were hell-bent on doing things via toolbox, that would be fine.

The other thing one must be wary of when using toolbox is ensuring toolbox pokes the correct holes in the toolbox container for GPU access.

Another feature I think we need sometime is some local config file, lets say someone doesn't want to type --nocontainer every time, they could do something like this:

$ cat ~/.ramalamaconfig
[ramalama]
	use_containers = false
        runtime = vllm

[rhatdan]

So this would be more about ramalama not being available on the host as silverblue. So I install a toolbox and I want to dnf install ramalama within the toolbox. As I understand toolbox, this should be possible and toolbox allows running of podman containers within them. so it should all work.

[ericcurtin]

Where toolbox is inconvenient right now is GPU access/AI API enablement whatever we want to call it... We should make contributions to toolbox to make that better though...

https://github.com/search?q=repo%3Acontainers%2Ftoolbox+%2Fdev%2Fkfd&type=code

"/dev/kfd" should be exposed for AMD GPUs for example... and there's no reference to it

I think we should do both... toolbox and "/usr/local" installs... Even if "/usr/local" installs are undocumented...

[debarshiray]

Where toolbox is inconvenient right now is GPU access/AI API enablement whatever we want to call it... We should make contributions to toolbox to make that better though...

GPU access was never a problem with the free software stacks like those of AMD and Intel. The proprietary NVIDIA stack should work from Toolbx 0.0.99.6, which is what's there on all Fedoras and CentOS Stream 9 and 10.

https://github.com/search?q=repo%3Acontainers%2Ftoolbox+%2Fdev%2Fkfd&type=code

That's because Toolbx containers have the entire /dev from the host.

[rhatdan]

Moving this to a discussion, since this is not something I believe can be fixed in RamaLama.