[Bug] Raise an actionable error when libmlc_llm shared dependencies are missing

[pjdurden]

Issue #3506 — OSError: libtvm.so: cannot open shared object file

1. Root cause

libtvm.so is not part of the mlc-llm wheel. It is shipped by the separate
mlc-ai / TVM package. This is deliberate: ci/task/build_lib.sh runs
auditwheel repair with --exclude libtvm --exclude libtvm_runtime --exclude libtvm_ffi ..., so the mlc-llm wheel only bundles libmlc_llm.so /
libmlc_llm_module.so, both of which link against libtvm.so at load time.

At import, python/mlc_llm/base.py loads libmlc_llm.so via ctypes.CDLL. When the
matching mlc-ai package is missing, incomplete, or its CUDA variant does not match
(the exact situation reported for the CUDA 13.0 nightly wheels), the dynamic linker
cannot resolve the libtvm.so dependency and ctypes.CDLL raises a bare
OSError: libtvm.so: cannot open shared object file: No such file or directory.

The underlying missing-file problem lives in the mlc-ai/TVM wheel packaging, which is
built and published outside this repository — there is no libtvm.so-producing
build in mlc-llm to fix here. What is in this repo's control is the loader's behavior:
it turns a resolvable-diagnosis situation ("your mlc-ai install is missing/mismatched")
into an opaque OSError that gives the user no path forward. That unhelpful failure is
the fixable defect.

2. The fix and why

Added load_lib(path) to python/mlc_llm/libinfo.py and routed base.py's load
through it. It wraps ctypes.CDLL and, on OSError, re-raises a RuntimeError that:

• preserves the original loader error (kept as __cause__ and in the message) for
  debugging, and
• explains the actual cause and fix: a matching mlc-ai package must be installed, and
  for pip wheels the CUDA variant must match (mlc-ai-nightly-cuXYZ alongside
  mlc-llm-nightly-cuXYZ), with a link to the install docs.

This mirrors the existing convention in the same file: find_lib_path(..., optional=False)
already raises a descriptive RuntimeError (with candidate paths) when the mlc-llm
library itself is absent. The change extends the same "clear, actionable error" treatment
to the missing-dependency case.

The helper was placed in libinfo.py (not base.py) on purpose: libinfo.py is
standalone — it imports only os/sys/ctypes, has no tvm dependency, and is even
exec-ed directly by setup.py. That keeps the new logic unit-testable without a fully
built mlc_llm/tvm install. base.py no longer references ctypes directly, so that
now-unused import was removed.

3. Files changed

• python/mlc_llm/libinfo.py — add import ctypes; add load_lib(path) helper.
• python/mlc_llm/base.py — call libinfo.load_lib(...) instead of ctypes.CDLL(...);
  drop the now-unused import ctypes.
• tests/python/test_libinfo.py — new focused unit test.

4. Risk / uncertainty

• Low behavioral risk. On the success path load_lib returns exactly what
  ctypes.CDLL returned; only the error path changes (a clearer RuntimeError in place
  of the raw OSError). Callers of _load_mlc_llm_lib did not catch OSError
  specifically, so no error-handling contract is broken.
• Scope caveat (honest): this does not make CUDA 13.0 wheels ship libtvm.so — that
  requires a change in the mlc-ai/TVM wheel build, which is not in this repository. This
  fix converts the confusing symptom into a self-service diagnostic that points users to
  the real remedy; it is a robustness/UX fix, not a repackaging of the upstream wheel.
• I could not exercise the real dlopen failure locally because tvm/mlc_llm are not
  installed in this environment; the test simulates the OSError from ctypes.CDLL
  instead (see below).

5. How I verified

• python3 -m pytest tests/python/test_libinfo.py -v → 2 passed. Covers both the
  missing-dependency error path (asserts the message retains libtvm.so, mentions
  mlc-ai, and chains the original OSError as __cause__) and the success path.
• Confirmed libinfo.py still loads two ways: via importlib and via the exact
  exec(compile(...)) pattern setup.py uses (so packaging is unaffected).
• python3 -m py_compile on all three files, and ruff check on them → all checks passed.

[gemini-code-assist]

Code Review

This pull request improves error handling when loading the MLC LLM library by wrapping ctypes.CDLL in a new load_lib helper, which catches OSError and raises a descriptive RuntimeError if a dependency like libtvm.so is missing. The feedback recommends dynamically determining the platform-specific TVM library name (e.g., .dll or .dylib) in the error message to support Windows and macOS, and removing the redundant NOTES.md file to keep the repository clean.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[Copilot]

Pull request overview

Improves the Python-side shared library loader experience by turning an opaque OSError (missing libtvm.so / mismatched mlc-ai install) into a clearer, actionable RuntimeError, matching the existing “actionable error” behavior already used when the MLC library itself cannot be found.

Changes:

• Add mlc_llm.libinfo.load_lib(path) that wraps ctypes.CDLL and re-raises a descriptive RuntimeError (with the original OSError chained).
• Route python/mlc_llm/base.py library loading through libinfo.load_lib(...) and remove the now-unused ctypes import.
• Add a focused unit test validating both the error-path messaging/cause chaining and the success path.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
python/mlc_llm/libinfo.py	Adds load_lib() wrapper to provide actionable diagnostics when ctypes.CDLL fails due to missing dependencies like libtvm.so.
python/mlc_llm/base.py	Uses the new libinfo.load_lib() helper for consistent error reporting during import-time library loading.
tests/python/test_libinfo.py	Adds unit coverage for load_lib() success and missing-dependency error behavior.
NOTES.md	Documents the issue context, root cause, and rationale for the loader behavior change.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.