[Bug] Raise an actionable error when libmlc_llm shared dependencies are missing

NOTES.md

@@ -0,0 +1,74 @@
+# Issue #3506 — `OSError: libtvm.so: cannot open shared object file`
+
+## 1. Root cause
+
+`libtvm.so` is **not** part of the `mlc-llm` wheel. It is shipped by the separate
+`mlc-ai` / TVM package. This is deliberate: `ci/task/build_lib.sh` runs
+`auditwheel repair` with `--exclude libtvm --exclude libtvm_runtime --exclude
+libtvm_ffi ...`, so the mlc-llm wheel only bundles `libmlc_llm.so` /
+`libmlc_llm_module.so`, both of which link against `libtvm.so` at load time.
+
+At import, `python/mlc_llm/base.py` loads `libmlc_llm.so` via `ctypes.CDLL`. When the
+matching `mlc-ai` package is missing, incomplete, or its CUDA variant does not match
+(the exact situation reported for the CUDA 13.0 nightly wheels), the dynamic linker
+cannot resolve the `libtvm.so` dependency and `ctypes.CDLL` raises a bare
+`OSError: libtvm.so: cannot open shared object file: No such file or directory`.
+
+The underlying missing-file problem lives in the `mlc-ai`/TVM wheel packaging, which is
+built and published **outside this repository** — there is no `libtvm.so`-producing
+build in mlc-llm to fix here. What *is* in this repo's control is the loader's behavior:
+it turns a resolvable-diagnosis situation ("your `mlc-ai` install is missing/mismatched")
+into an opaque `OSError` that gives the user no path forward. That unhelpful failure is
+the fixable defect.
+
+## 2. The fix and why
+
+Added `load_lib(path)` to `python/mlc_llm/libinfo.py` and routed `base.py`'s load
+through it. It wraps `ctypes.CDLL` and, on `OSError`, re-raises a `RuntimeError` that:
+
+- preserves the original loader error (kept as `__cause__` and in the message) for
+  debugging, and
+- explains the actual cause and fix: a matching `mlc-ai` package must be installed, and
+  for pip wheels the CUDA variant must match (`mlc-ai-nightly-cuXYZ` alongside
+  `mlc-llm-nightly-cuXYZ`), with a link to the install docs.
+
+This mirrors the existing convention in the same file: `find_lib_path(..., optional=False)`
+already raises a descriptive `RuntimeError` (with candidate paths) when the mlc-llm
+library itself is absent. The change extends the same "clear, actionable error" treatment
+to the missing-**dependency** case.
+
+The helper was placed in `libinfo.py` (not `base.py`) on purpose: `libinfo.py` is
+standalone — it imports only `os`/`sys`/`ctypes`, has no `tvm` dependency, and is even
+`exec`-ed directly by `setup.py`. That keeps the new logic unit-testable without a fully
+built `mlc_llm`/`tvm` install. `base.py` no longer references `ctypes` directly, so that
+now-unused import was removed.
+
+## 3. Files changed
+
+- `python/mlc_llm/libinfo.py` — add `import ctypes`; add `load_lib(path)` helper.
+- `python/mlc_llm/base.py` — call `libinfo.load_lib(...)` instead of `ctypes.CDLL(...)`;
+  drop the now-unused `import ctypes`.
+- `tests/python/test_libinfo.py` — new focused unit test.
+
+## 4. Risk / uncertainty
+
+- **Low behavioral risk.** On the success path `load_lib` returns exactly what
+  `ctypes.CDLL` returned; only the *error* path changes (a clearer `RuntimeError` in place
+  of the raw `OSError`). Callers of `_load_mlc_llm_lib` did not catch `OSError`
+  specifically, so no error-handling contract is broken.
+- **Scope caveat (honest):** this does not make CUDA 13.0 wheels ship `libtvm.so` — that
+  requires a change in the `mlc-ai`/TVM wheel build, which is not in this repository. This
+  fix converts the confusing symptom into a self-service diagnostic that points users to
+  the real remedy; it is a robustness/UX fix, not a repackaging of the upstream wheel.
+- I could not exercise the real dlopen failure locally because `tvm`/`mlc_llm` are not
+  installed in this environment; the test simulates the `OSError` from `ctypes.CDLL`
+  instead (see below).
+
+## 5. How I verified
+
+- `python3 -m pytest tests/python/test_libinfo.py -v` → 2 passed. Covers both the
+  missing-dependency error path (asserts the message retains `libtvm.so`, mentions
+  `mlc-ai`, and chains the original `OSError` as `__cause__`) and the success path.
+- Confirmed `libinfo.py` still loads two ways: via `importlib` and via the exact
+  `exec(compile(...))` pattern `setup.py` uses (so packaging is unaffected).
+- `python3 -m py_compile` on all three files, and `ruff check` on them → all checks passed.

python/mlc_llm/base.py

@@ -1,6 +1,5 @@
 """Load MLC LLM library and _ffi_api functions."""
 
-import ctypes
 import os
 import sys
 
@@ -19,7 +18,7 @@ def _load_mlc_llm_lib():
             os.add_dll_directory(path)
     lib_name = "mlc_llm" if tvm.base._RUNTIME_ONLY else "mlc_llm_module"
     lib_path = libinfo.find_lib_path(lib_name, optional=False)
-    return ctypes.CDLL(lib_path[0]), lib_path[0]
+    return libinfo.load_lib(lib_path[0]), lib_path[0]
 
 
 @tvm.register_global_func("mlc.debug_cuda_profiler_start")

python/mlc_llm/libinfo.py

@@ -1,6 +1,7 @@
 """Library information. This is a standalone file that can be used to get various info"""
 
 #! pylint: disable=protected-access
+import ctypes
 import os
 import sys
 
@@ -69,3 +70,39 @@ def find_lib_path(name, optional=False):
             )
             raise RuntimeError(message)
     return lib_found
+
+
+def load_lib(path):
+    """Load a shared library, raising a clear error when a dependency is missing.
+
+    ``ctypes.CDLL`` raises a bare ``OSError`` (e.g. ``libtvm.so: cannot open shared
+    object file``) when the library itself is present but one of its shared
+    dependencies cannot be resolved. ``libtvm.so`` is provided by the ``mlc-ai`` /
+    TVM package rather than by ``mlc-llm`` itself, so this typically points at a
+    missing or mismatched ``mlc-ai`` installation. Wrap the load to turn that opaque
+    failure into an actionable message.
+
+    Parameters
+    ----------
+    path : str
+        The full path to the shared library to load.
+
+    Returns
+    -------
+    lib : ctypes.CDLL
+        The loaded library handle.
+    """
+    try:
+        return ctypes.CDLL(path)
+    except OSError as error:
+        raise RuntimeError(
+            f"Failed to load the MLC LLM library at '{path}'.\n"
+            f"Underlying error: {error}\n"
+            "This usually means one of its shared dependencies (for example "
+            "libtvm.so, which is provided by the `mlc-ai` / TVM package) could not "
+            "be found or is missing from the installation. Please make sure a "
+            "matching `mlc-ai` package is installed, and when installing pip wheels "
+            "ensure its CUDA variant matches (e.g. install `mlc-ai-nightly-cuXYZ` "
+            "alongside `mlc-llm-nightly-cuXYZ`). See "
+            "https://llm.mlc.ai/docs/install/mlc_llm.html for details."
+        ) from error

tests/python/test_libinfo.py

@@ -0,0 +1,53 @@
+"""Unit tests for :mod:`mlc_llm.libinfo`.
+
+``libinfo.py`` is intentionally standalone (it is also ``exec``-ed by ``setup.py``
+before the package is importable), so it is loaded here directly from its file path.
+This keeps the test independent of a fully built ``mlc_llm`` / ``tvm`` install.
+"""
+
+import importlib.util
+import os
+
+import pytest
+
+pytestmark = [pytest.mark.unittest]
+
+_LIBINFO_PATH = os.path.join(
+    os.path.dirname(__file__), "..", "..", "python", "mlc_llm", "libinfo.py"
+)
+
+
+def _load_libinfo():
+    spec = importlib.util.spec_from_file_location("mlc_llm_libinfo_standalone", _LIBINFO_PATH)
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+def test_load_lib_reports_missing_dependency(monkeypatch):
+    """A failed dependency resolution should surface an actionable RuntimeError."""
+    libinfo = _load_libinfo()
+
+    def _raise_missing_dep(_path):
+        raise OSError("libtvm.so: cannot open shared object file: No such file or directory")
+
+    monkeypatch.setattr(libinfo.ctypes, "CDLL", _raise_missing_dep)
+
+    with pytest.raises(RuntimeError) as exc_info:
+        libinfo.load_lib("/some/path/libmlc_llm.so")
+
+    message = str(exc_info.value)
+    # The original loader error is preserved for debugging...
+    assert "libtvm.so" in message
+    # ...and the message points at the real fix: a matching mlc-ai install.
+    assert "mlc-ai" in message
+    assert isinstance(exc_info.value.__cause__, OSError)
+
+
+def test_load_lib_success(monkeypatch):
+    """On success ``load_lib`` returns the loaded handle unchanged."""
+    libinfo = _load_libinfo()
+    sentinel = object()
+    monkeypatch.setattr(libinfo.ctypes, "CDLL", lambda path: sentinel)
+
+    assert libinfo.load_lib("/some/path/libmlc_llm.so") is sentinel