docs: add Model Backends page and document AMD ROCm support - #3900
Closed
zetyquickly wants to merge 1 commit into
Closed
docs: add Model Backends page and document AMD ROCm support#3900zetyquickly wants to merge 1 commit into
zetyquickly wants to merge 1 commit into
Conversation
Introduces docs/usage/model_backends.md, organizing Docling's models by runtime backend (Pure PyTorch, ONNX, MLX, vLLM, other), wired into the nav and cross-linked from the model catalog. Adds an AMD ROCm section to gpu.md: device="cuda" is correct on a ROCm-built PyTorch too (it aliases HIP under torch.cuda), with install instructions for both the standard ROCm wheel index and AMD's multi-arch per-gfx index for newer APUs, how to verify a wheel actually ships kernels for your GPU's gfx target before trusting it, and known caveats (ONNX has no ROCm execution provider yet, cuda_use_flash_attention2 assumes the CUDA-only flash-attn wheel, Nemotron OCR and WhisperS2T are CUDA/CTranslate2-only). Verified end-to-end on an AMD Ryzen AI 7 PRO 350 (Radeon 860M, gfx1152): default layout model and TableFormer ran on-GPU through the standard PDF pipeline with correct output. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
|
❌ DCO Check Failed Hi @zetyquickly, your pull request has failed the Developer Certificate of Origin (DCO) check. This repository supports remediation commits, so you can fix this without rewriting history — but you must follow the required message format. 🛠 Quick Fix: Add a remediation commitRun this command: git commit --allow-empty -s -m "DCO Remediation Commit for zetyquickly <zetyquickly@googlemail.com>
I, zetyquickly <zetyquickly@googlemail.com>, hereby add my Signed-off-by to this commit: bcddf8cb2009d4cf1b1e1f633fecdb679063f532"
git push🔧 Advanced: Sign off each commit directlyFor the latest commit: git commit --amend --signoff
git push --force-with-leaseFor multiple commits: git rebase --signoff origin/main
git push --force-with-leaseMore info: DCO check report |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/usage/model_backends.md, organizing Docling's models along the runtime-backend axis (Pure PyTorch, ONNX, MLX, vLLM, other), wired into the nav and cross-linked from the model catalog.gpu.md:device="cuda"is correct on a ROCm-built PyTorch too (it aliases HIP undertorch.cuda), with install instructions for the standard ROCm wheel index and AMD's multi-arch per-gfxindex for newer APUs, how to verify a wheel actually ships kernels for your GPU'sgfxtarget before trusting it (mismatches segfault/hang rather than raising cleanly), and known caveats (ONNX has no ROCm execution provider yet,cuda_use_flash_attention2assumes the CUDA-onlyflash-attnwheel, Nemotron OCR and WhisperS2T are CUDA/CTranslate2-only).gfx1152, ROCm 7.14): the default layout model and TableFormer ran their real GPU kernels through the standard PDF pipeline and produced correct output (text extraction + a structured table), with zero Docling code changes.Test plan
torch.cuda.get_arch_list()includesgfx1152after installing fromrepo.amd.com/rocm/whl-multi-archDocumentConverterwithAcceleratorOptions(device=AcceleratorDevice.CUDA)against a real PDF on the AMD iGPU;ConversionStatus.SUCCESS, text + 1 table extracted correctly🤖 Generated with Claude Code