[Bug]: AMD a395 gfx1151 (Radeon 8060S / Strix Halo): Cannot look up function runtime_get_rand_states_buffer_size on gs.amdgpu backend

[a0983115883-boop]

Bug Description

On AMD gfx1151 (Radeon 8060S, Ryzen AI Max+ 395 / Strix Halo), initializing Genesis with the AMD ROCm backend fails with a missing runtime function:

AMDGPU Error hipErrorNotFound: named symbol not found while calling module_get_function (hipModuleGetFunction)
[jit_amdgpu.h:lookup_function@73] Cannot look up function runtime_get_rand_states_buffer_size
RuntimeError: [jit_amdgpu.h:lookup_function@73] Cannot look up function runtime_get_rand_states_buffer_size

The failure occurs during impl.get_runtime().prog.materialize_runtime() inside qd.init().

Steps to Reproduce

import genesis as gs
gs.init(backend=gs.amdgpu) # official AMD backend per README

What I have verified
1.Correct backend is used. Per the README, AMD users should call gs.init(backend=gs.amdgpu). hasattr(gs, 'amdgpu') is True, and the error path goes through jit_amdgpu.h — so the ROCm/HIP backend is correctly selected (not the CUDA fallback).
2.PyTorch ROCm works on this GPU. torch.cuda.is_available() returns True, device is correctly identified as "Radeon 8060S Graphics", and GPU matrix multiplication runs successfully. So the ROCm driver stack and hardware are functional.
3.Not a Genesis version issue. Tested three combinations, all fail identically:

Genesis 1.2.2 + Quadrants 1.0.2 → same error
Genesis 1.2.2 + Quadrants 1.1.0 → same error
Genesis 1.2.0 + Quadrants 1.0.2 → same error

4.Standalone qd.init(arch=qd.gpu) succeeds, including with individual parameters that Genesis passes (enable_fallback, force_scalarize_matrix, advanced_optimization, fast_math, random_seed, default_ip/fp, ad_stack_experimental_enabled, etc.). The failure only surfaces at materialize_runtime() when the full runtime is instantiated.
5.Meets all stated system requirements (Python 3.10–3.13, Ubuntu 22.04–24.04, ROCm ≥ 5.2).

Expected Behavior

Is runtime_get_rand_states_buffer_size expected to be available in the ROCm backend for gfx1151? Is there a way to build/enable it, or is gfx1151 not yet fully supported by the Quadrants ROCm backend?

Happy to provide further logs, run diagnostics, or test patches — I have the hardware available.

Screenshots/Videos

No response

Relevant log output

Environment

GPU | AMD Radeon 8060S Graphics (gfx1151, Strix Halo / Ryzen AI Max+ 395)
OS | Ubuntu 24.04
Python | 3.12
ROCm/HIP | 7.13
PyTorch | 2.12.0a0+rocm7.13 (GPU verified working)
Genesis | 1.2.2 (also tested 1.2.0)
Quadrants | 1.1.0 (also tested 1.0.2)

Release version or Commit ID

Genesis: 1.2.2
Quadrants: 1.1.0

Additional Context

No response

[hughperkins]

Hi @a0983115883-boop ,

Here is what agent says:

Could you try the suggestion from agent on what to try next, and report the results? ie:

[vybhav-ibr]

I encountered the same issue on this machine(hosted by digital ocean), even with older genesis versions like 0.4.6

+------------------------------------------------------------------------------+
| AMD-SMI            26.5.0+2b22ab01                                           |
| amdgpu Version:    6.16.13                                                   |
| ROCm Version:      7.14.0                                                    |
| VBIOS Version:     00123529                                                  |
| Platform:          Linux Guest                                               |
|-------------------------------------+----------------------------------------|
| BDF                        GPU-Name | Mem-Uti   Temp   UEC       Power-Usage |
| GPU  HIP-ID  OAM-ID  Partition-Mode | GFX-Uti    Fan               Mem-Usage |
|=====================================+========================================|
| 0000:83:00.0 AMD Instinct MI300X VF | 0 %      50 °C   0           169/750 W |
|   0       0       1        SPX/NPS1 | 0 %        N/A           285/196288 MB |
+-------------------------------------+----------------------------------------+
+------------------------------------------------------------------------------+
| Processes:                                                                   |
|  GPU      PID  Process Name       GTT_MEM  VRAM_MEM  MEM_USAGE  CU %  SDMA   |
|==============================================================================|
|    0     1814  gpuagent             0.0 B     0.0 B      0.0 B  0.0 %   0 us |
+------------------------------------------------------------------------------+

The error:

Cannot look up function runtime_get_memory_requirements

solution:

import genesis as gs
import quadrants as qd

qd.init(arch=qd.amdgpu)
gs.init()

[hughperkins]

@vybhav-ibr Nice! Do you also have a script to reproduce the issue please?

[hughperkins]

oh, to reproduce, I just do:

import genesis as gs
gs.init(backend=gs.amdgpu)

?

interseting.

[vybhav-ibr]

@hughperkins
Yep, just doing a gs.init() with the backend param as empty/gs.gpu/gs.amdgpu causes this.
I think this is caused by a newer ROCm version(7.14), using ROCm 6.4(based on the provided AMD docker instructions) does not seem to cause this issue even with the same instinct GPU.

[hughperkins]

Interesting. Thank you!

[hughperkins]

AI unable to reproduce. See report below ("I" == "AI"). Thoughts:

Attempted reproduction of gs.init(backend=gs.amdgpu) -> "Cannot look up function runtime_get_*"

TL;DR

I tried hard to reproduce this on the exact same hardware as @vybhav-ibr (a DigitalOcean
MI300X VF, ROCm 7.14.0) using the reporters' exact software versions, and I could not
reproduce it. gs.init(backend=gs.amdgpu) initializes cleanly and GPU kernels (including the RNG
runtime that owns the "missing" runtime_get_rand_states_buffer_size symbol) run correctly.

The only variable I could not match is the amdgpu kernel-mode driver (KMD) version:

	ROCm (userspace)	amdgpu KMD	Result
@vybhav-ibr (fails)	7.14.0	6.16.13	Cannot look up function runtime_get_memory_requirements
my droplet (works)	7.14.0	6.19.14	initializes + runs fine

So my current best hypothesis is that the failure tracks the KMD / GPU driver build, not the
ROCm userspace version alone, and that a newer amdgpu driver (>= 6.19) already resolves it. Details
and next steps below.

---

Environment I tested (matched to the reporters)

Component	Reporter values	What I ran
GPU / host	MI300X VF, DigitalOcean (gfx942)	MI300X VF, DigitalOcean (gfx942) - identical
ROCm userspace	7.14 (@vybhav-ibr), 7.13 (@a0983115883-boop)	real 7.14 (libamdhip64.so.7.14.60850) and real 7.13 (7.13.60800)
PyTorch	2.12.0a0+rocm7.13 (gfx1151); MI300X torch unstated	both 2.9.1+rocm7.14 and the exact 2.12.0a0+rocm7.13 (AMD TheRock nightly, gfx94X-dcgpu)
Quadrants	1.0.2 and 1.1.0	both 1.0.2 and 1.1.0 (PyPI wheels)
Genesis	1.2.2 (also 1.2.0 / 0.4.6)	1.2.2
Python	3.12	3.12
Call	gs.init(backend=gs.amdgpu)	gs.amdgpu, gs.gpu, and empty - all tested

amd-smi on my droplet:

AMDSMI Tool: 26.5.0 | ROCm version: 7.14.0 | amdgpu version: 6.19.14.31400000
MARKET_NAME: AMD Instinct MI300X VF

---

Result

All backends initialize successfully and run kernels:

[Genesis] Running on [AMD Instinct MI300X VF] with backend gs.amdgpu. Device memory: 191.69 GB.
[Genesis] Genesis initialized. version: 1.2.2

Every version combination I tested passes (no Cannot look up function ...):

Quadrants	PyTorch	ROCm runtime actually loaded	Result
1.0.2	2.9.1+rocm7.14	7.14.60850	PASS
1.1.0	2.9.1+rocm7.14	7.14.60850	PASS
1.0.2	2.12.0a0+rocm7.13 (reporter's exact torch)	7.13.60800	PASS
1.1.0	2.12.0a0+rocm7.13 (reporter's exact torch)	7.13.60800	PASS

In the 7.13 runs, torch and Quadrants share a single libamdhip64.so (7.13.60800) - i.e. torch
initializes the exact HIP runtime first and Quadrants then loads its runtime module into it, which is
precisely the ordering the reporters' qd.init-first workaround sidesteps. It still succeeds here.

---

How I verified I was actually testing the right thing

Because a "works on my machine" is only useful if it is really the same thing, I checked:

1. Real PyPI wheels, not a dev/editable checkout

Name: quadrants        Version: 1.1.0 (and 1.0.2)   Location: .../repro312/.../site-packages
Name: genesis-world    Version: 1.2.2               Location: .../repro312/.../site-packages
Name: torch            Version: 2.9.1+rocm7.14...   Location: .../repro312/.../site-packages

• PYTHONPATH empty; no .pth / .egg-link / __editable__ pointing at any source tree.
• quadrants-1.1.0.dist-info/INSTALLER present; the compiled backend is the wheel binary
  quadrants/_lib/core/quadrants_python.cpython-312-x86_64-linux-gnu.so.
• At runtime quadrants.__file__ and genesis.__file__ both resolve under site-packages.

2. The real ROCm 7.14 HIP runtime is actually loaded (from /proc/<pid>/maps):

.../_rocm_sdk_core/lib/libamdhip64.so.7            (torch's bundled 7.14)
/opt/rocm/core-7.14/lib/libamdhip64.so.7.14.60850  (system 7.14)

Both are 7.14.60850. The HSACO is linked with /opt/rocm/llvm/bin/ld.lld = AMD LLD 23.0.0
(the same 7.14-era LLVM build as core-7.14), and is a gfx942, code-object ABI v4 (COV4) object.

3. The exact failing symbols resolve - trace log at the same jit_amdgpu.h:lookup_function
site as the reporters' error:

[jit_amdgpu.h:lookup_function@76] AMDGPU module_get_function runtime_get_rand_states_buffer_size costs 0.005 ms
[jit_amdgpu.h:lookup_function@76] AMDGPU module_get_function runtime_get_memory_requirements costs 0.001 ms
[llvm_runtime_executor.cpp:materialize_runtime@796] Initializing 19922944 random states (parallel GPU)

4. End-to-end GPU execution via the real gs.init(backend=gs.amdgpu) path (which initializes
the torch/HIP runtime first, then qd.init -> materialize_runtime):

• qd.random(qd.f32) reduction over 2^20 elements -> mean 0.5001 (proves the RNG runtime
  buffer, i.e. the runtime_get_rand_states_buffer_size machinery, compiled + loaded + ran on GPU).
• deterministic fill+sum -> exact.

In other words, the specific code path that fails for the reporters runs correctly here.

---

Hypothesis

• @vybhav-ibr already noted ROCm 6.4 is fine, 7.14 fails. On my box, 7.14 userspace is fine too,
  which points away from "the 7.14 userspace runtime is broken".
• The one thing different between my (working) box and @vybhav-ibr's (failing) box on otherwise
  identical hardware is the amdgpu KMD: mine is 6.19.14, theirs was 6.16.13.
• "named symbol not found" from hipModuleGetFunction on a freshly loaded, valid code object is
  consistent with a driver-side code-object load issue (the module loads but its kernels are not
  registered), which can be sensitive to the KMD build.
• The fact that the qd.init(arch=qd.amdgpu) first workaround fixes it (initializing Quadrants'
  HIP context before torch does) suggests an initialization-order / context interaction that a
  newer KMD tolerates and an older one does not.

I cannot change the KMD from inside a VF guest (it is set by the host image), so I cannot
directly confirm this on my droplet.

---

Secondary finding (possibly a separate issue)

With the current PyPI genesis-world==1.2.2 wheel and an alpha torch nightly
(torch.__version__ == "2.12.0a0+rocm7.13..."), import genesis itself fails before any GPU
work:

File ".../genesis/__init__.py", line 37, in <module>
  _TORCH_MPS_SUPPORT_DLPACK_FIELD = tuple(map(int, torch.__version__.replace("+", ".").split(".")[:3])) > (2, 9, 1)
ValueError: invalid literal for int() with base 10: '0a0'

The parser does int("0a0") on the alpha tag. This is worth flagging because
@a0983115883-boop reports torch 2.12.0a0 and reaching materialize_runtime, which should be
impossible if this line raised at import - so either their exact torch version string differs, or the
Genesis version differs from the current wheel. (I neutralized only the version string to run the
torch-first tests above; the torch binary/HIP runtime were untouched.)

---

Next steps

For the reporters (would really help narrow this):

1. Please post amd-smi version (or cat /sys/module/amdgpu/version) so we can see the amdgpu KMD
   version. Hypothesis: failing boxes are on an older amdgpu (~6.16.x).
2. @vybhav-ibr: if your DigitalOcean droplet can be re-imaged / the GPU driver updated to
   amdgpu >= 6.19 (still ROCm 7.14), please retest gs.init(backend=gs.amdgpu) - I expect it to
   start working.
3. Please try the AMD TheRock nightly torch that matches your GPU and confirm whether it changes
   anything, e.g. for MI300X:

   pip install torch --index-url https://rocm.nightlies.amd.com/v2/gfx94X-dcgpu/ \
       --extra-index-url https://pypi.org/simple --prerelease=allow --index-strategy unsafe-best-match
   

   (gfx1151 users: use https://rocm.nightlies.amd.com/v2/gfx1151/.)
4. @a0983115883-boop (gfx1151 / Strix Halo): I do not have that hardware, so I cannot test it. Could
   you post the full traceback, amd-smi version, rocminfo | grep -iE 'Name|gfx|Runtime', and
   whether the qd.init(arch=qd.amdgpu) -> gs.init() workaround also fixes it on gfx1151?

On the Quadrants side (to make it robust regardless of KMD):
5. Since qd.init first fixes it, the runtime module load in materialize_runtime is likely running
before the HIP device/primary-context is in the state Quadrants expects when torch initialized HIP
first. Worth trying: explicitly hipSetDevice / retain+set the primary context (or reuse torch's
current context) immediately before hipModuleLoadData + hipModuleGetFunction.
6. Improve the diagnostic at jit_amdgpu.h:lookup_function: on failure, also log hipGetLastError(),
the code-object version of the loaded module, torch.version.hip, and the amdgpu KMD version.
That single log line would confirm/deny the driver hypothesis immediately.
7. Check the code-object version: my HSACO is COV4 and loads fine under KMD 6.19. If an older KMD
rejects COV4 (or a specific COV emitted by the reporter's ld.lld), emitting a
broadly-compatible COV would be a clean fix.

On my side (if we want a definitive repro):
8. Get a box with ROCm 7.14 + amdgpu ~6.16.x (an older DigitalOcean image or a bare-metal MI300X
with that driver) and re-run the exact stack above; then bisect the KMD version. This is the most
likely path to an actual reproduction.

---

Reproduction recipe I used (for reference)

uv venv --python 3.12 repro312
uv pip install --python repro312/bin/python "genesis-world==1.2.2" "quadrants==1.0.2"   # or ==1.1.0

# Option A: torch built for ROCm 7.14 (highest on the MI300X index)
uv pip install --python repro312/bin/python --prerelease=allow --index-strategy unsafe-best-match \
    "torch==2.9.1+rocm7.14.0a20260611" \
    --index-url https://rocm.nightlies.amd.com/v2/gfx94X-dcgpu/ \
    --extra-index-url https://pypi.org/simple
LD_LIBRARY_PATH="$PWD/repro312/lib/python3.12/site-packages/_rocm_sdk_core/lib:/opt/rocm/core-7.14/lib"

# Option B: the reporter's EXACT torch, built for ROCm 7.13
uv pip install --python repro312/bin/python --prerelease=allow --index-strategy unsafe-best-match \
    "torch==2.12.0a0+rocm7.13.0a20260325" \
    --index-url https://rocm.nightlies.amd.com/v2/gfx94X-dcgpu/ \
    --extra-index-url https://pypi.org/simple
# make Quadrants share torch's single bundled 7.13 runtime (its lib dir has only libamdhip64.so.7)
SDK="$PWD/repro312/lib/python3.12/site-packages/_rocm_sdk_core/lib"; ln -sf libamdhip64.so.7 "$SDK/libamdhip64.so"
LD_LIBRARY_PATH="$SDK"

uv pip install --python repro312/bin/python "numpy<2.5"   # numba compat

QD_OFFLINE_CACHE=0 LD_LIBRARY_PATH="$LD_LIBRARY_PATH" \
  repro312/bin/python -c "import genesis as gs; gs.init(backend=gs.amdgpu)"