Skip to content

add containerfile that use native fedora-43 rocm (6.4.4)#50

Open
Djip007 wants to merge 3 commits intokyuz0:mainfrom
Djip007:fedora-rocm-6.4.4
Open

add containerfile that use native fedora-43 rocm (6.4.4)#50
Djip007 wants to merge 3 commits intokyuz0:mainfrom
Djip007:fedora-rocm-6.4.4

Conversation

@Djip007
Copy link
Copy Markdown

@Djip007 Djip007 commented Jan 22, 2026

This is 2 containerfile:
I build and pre configure llama.cpp.
llama.cpp is installed in /opt/llama.cpp

  • Containerfile.rocm-6.4.4
    this is a simple image, to use with podman. It run the llama-server by default, can be use:
# build:
podman build -t rocm-serve:6.4.4 -f Containerfile.rocm-6.4.4

# run llama-server with podman:
#  - mount simple model file (the toolbox start by default "/models/default.gguf":
podman run --rm --device /dev/dri --device /dev/kfd --privileged --network=host -v ~/LLM/openai_gpt-oss-120b/MXFP4.gguf:/models/default.gguf rocm-serve:6.4.4 --ctx-size 0 --jinja

#  - mount models path you need to define the "-m <model path>":
podman run --rm --device /dev/dri --device /dev/kfd --privileged --network=host -v ~/LLM:/models rocm-serve:6.4.4 --ctx-size 0 -m /models/openai_gpt-oss-120b/MXFP4.gguf --jinja

# run other llama exec:
podman run --rm --device /dev/dri --device /dev/kfd --privileged --network=host -v ~/LLM:/models --entrypoint="/opt/llama.cpp/bin/llama-bench" rocm-serve:6.4.4 -m /models/openai_gpt-oss-120b/MXFP4.gguf -b 8192 -ub 2048 -fa 1 -ngl 999 --mmap 0 -p "1,1,2,4,8,16,32,64,128,256,512,1024,2048,4096,8192"

# for interactive run you can change the entrypoint to bash:
podman run --rm -it --device /dev/dri --device /dev/kfd --privileged --network=host -v ~/LLM:/models --entrypoint="/usr/bin/bash" rocm-serve:6.4.4
# - next in the container:
/opt/llama.cpp/bin/llama-cli -m /models/openai_gpt-oss-120b/MXFP4.gguf --jinja --ctx-size 0
  • Toolboxfile.rocm-6.4.4
    it is a true toolbox to use with ... toolbox with no config need on fedora:
# build:
podman build -t rocm-toolbox:6.4.4 -f Toolboxfile.rocm-6.4.4

# run (no need for "--device" .. etc.:
toolbox create -i rocm-toolbox:6.4.4 rocm-toolbox-6.4.4
toolbox enter rocm-toolbox-6.4.4

# example use in the toolbox:
/opt/llama.cpp/bin/llama-cli -m openai_gpt-oss-120b/MXFP4.gguf --jinja --ctx-size 0

Comment thread toolboxes/Containerfile.rocm-6.4.4 Outdated
#

# rocm testing or std update: https://bugzilla.redhat.com/show_bug.cgi?id=2430582 / https://src.fedoraproject.org/rpms/rocm-runtime
ARG DNF_UPDATE="upgrade --enablerepo=updates-testing --refresh --advisory=FEDORA-2026-e8113d69d1"
Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to remove when published on updates repos.

@Djip007
Copy link
Copy Markdown
Author

Djip007 commented Jan 22, 2026

for now pick the testing package, follow : https://src.fedoraproject.org/rpms/rocm-runtime
patch is in rocm-runtime-6.4.2-3.fc43

it is no more needed
@Djip007
Copy link
Copy Markdown
Author

Djip007 commented Jan 23, 2026

look fedora-43 patch is now up-stream => no need for testing.
fedora-43+rocm-6.4.4 today update (23/01/2026) is back stable/working. With all latest:

  • kernel: 6.18.5-200.fc43.x86_64
  • rocm: 6.4.4 (with rocm-runtime-6.4.2-3.fc43)
  • amd-gpu-firmware: 20260110-1.fc43

@kyuz0
Copy link
Copy Markdown
Owner

kyuz0 commented Jan 24, 2026

Thanks for the work on the PR. I might not merge this at the moment, as I want to update the Dockerfiles I already have, which are part of my automated build and test pipeline.

@Djip007
Copy link
Copy Markdown
Author

Djip007 commented Jan 24, 2026

Take your time. Create this PR only to share it.

Note: in this PR I install llama.cpp in /opt/llama.cpp did not take time to figure what to do to install on /usr/...

And thanks for all you work/time on this nice stric-halo !

@Djip007
Copy link
Copy Markdown
Author

Djip007 commented Jan 25, 2026

  • add fedora-44/rocm-7.1.1
  • for podman/Containerfile install llama.cpp in /usr/local (live it in /op/llama.cpp for Toolboxfile ...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants