Skip to content

docs: add Model Backends page and document AMD ROCm support - #3900

Closed
zetyquickly wants to merge 1 commit into
docling-project:mainfrom
zetyquickly:docs/model-backends-rocm
Closed

docs: add Model Backends page and document AMD ROCm support#3900
zetyquickly wants to merge 1 commit into
docling-project:mainfrom
zetyquickly:docs/model-backends-rocm

Conversation

@zetyquickly

Copy link
Copy Markdown

Summary

  • Adds docs/usage/model_backends.md, organizing Docling's models along the runtime-backend axis (Pure PyTorch, ONNX, MLX, vLLM, other), wired into the nav and cross-linked from the model catalog.
  • Adds an AMD ROCm section to gpu.md: device="cuda" is correct on a ROCm-built PyTorch too (it aliases HIP under torch.cuda), with install instructions for the standard ROCm wheel index and AMD's multi-arch per-gfx index for newer APUs, how to verify a wheel actually ships kernels for your GPU's gfx target before trusting it (mismatches segfault/hang rather than raising cleanly), and known caveats (ONNX has no ROCm execution provider yet, cuda_use_flash_attention2 assumes the CUDA-only flash-attn wheel, Nemotron OCR and WhisperS2T are CUDA/CTranslate2-only).
  • Verified end-to-end on an AMD Ryzen AI 7 PRO 350 (Radeon 860M, gfx1152, ROCm 7.14): the default layout model and TableFormer ran their real GPU kernels through the standard PDF pipeline and produced correct output (text extraction + a structured table), with zero Docling code changes.

Test plan

  • Manually verified torch.cuda.get_arch_list() includes gfx1152 after installing from repo.amd.com/rocm/whl-multi-arch
  • Ran DocumentConverter with AcceleratorOptions(device=AcceleratorDevice.CUDA) against a real PDF on the AMD iGPU; ConversionStatus.SUCCESS, text + 1 table extracted correctly
  • Docs-only change; no code paths modified

🤖 Generated with Claude Code

Introduces docs/usage/model_backends.md, organizing Docling's models by
runtime backend (Pure PyTorch, ONNX, MLX, vLLM, other), wired into the
nav and cross-linked from the model catalog.

Adds an AMD ROCm section to gpu.md: device="cuda" is correct on a
ROCm-built PyTorch too (it aliases HIP under torch.cuda), with install
instructions for both the standard ROCm wheel index and AMD's
multi-arch per-gfx index for newer APUs, how to verify a wheel actually
ships kernels for your GPU's gfx target before trusting it, and known
caveats (ONNX has no ROCm execution provider yet, cuda_use_flash_attention2
assumes the CUDA-only flash-attn wheel, Nemotron OCR and WhisperS2T are
CUDA/CTranslate2-only). Verified end-to-end on an AMD Ryzen AI 7 PRO 350
(Radeon 860M, gfx1152): default layout model and TableFormer ran on-GPU
through the standard PDF pipeline with correct output.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

DCO Check Failed

Hi @zetyquickly, your pull request has failed the Developer Certificate of Origin (DCO) check.

This repository supports remediation commits, so you can fix this without rewriting history — but you must follow the required message format.


🛠 Quick Fix: Add a remediation commit

Run this command:

git commit --allow-empty -s -m "DCO Remediation Commit for zetyquickly <zetyquickly@googlemail.com>

I, zetyquickly <zetyquickly@googlemail.com>, hereby add my Signed-off-by to this commit: bcddf8cb2009d4cf1b1e1f633fecdb679063f532"
git push

🔧 Advanced: Sign off each commit directly

For the latest commit:

git commit --amend --signoff
git push --force-with-lease

For multiple commits:

git rebase --signoff origin/main
git push --force-with-lease

More info: DCO check report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant