llama_model_load fails with "failed to read magic" on Windows MSVC for models > 4GB (mmap/file I/O bug)

[White2Cat]

Describe the bug
When using llama-cpp-2 (v0.1.138) on Windows with the MSVC toolchain (x86_64-pc-windows-msvc), loading any GGUF model larger than 4GB (e.g., a 4.5GB 7B model) fails immediately with:
gguf_init_from_file_impl: failed to read magic
llama_model_load_from_file_impl: failed to load model

Steps to Reproduce

1. Environment: Windows 11, Rust x86_64-pc-windows-msvc (64-bit).
2. Model: Any GGUF file > 4GB (e.g., qwen2.5-coder-7b-instruct-q4_k_m.gguf which is ~4.5GB).
3. Try to load it using LlamaModel::load_from_file(&backend, &model_path, &model_params).
4. It fails.
5. Models < 2GB (e.g., 1.5B models) load perfectly fine with the exact same code and environment.
6. Crucial baseline: The exact same 4.5GB physical file loads perfectly in < 1 second using llama-cpp-python on the exact same machine.

Attempts to workaround
I suspected a Windows mmap 4GB boundary issue, so I tried bypassing the Rust API to forcefully disable mmap using unsafe:

unsafe {
    let raw_ptr = &mut model_params as *mut _ as *mut llama_cpp_sys_2::llama_model_params;
    (*raw_ptr).use_mmap = false; 
}

However, the exact same failed to read magic error persists even with standard I/O forced.

Root Cause Suspicion
Since Python (llama-cpp-python) handles this flawlessly and 1.5B models work in Rust, it strongly implies a 32-bit integer truncation issue within the llama-cpp-sys-2 build process on Windows MSVC. In MSVC, the C long type is 32-bit. It is highly likely that the build.rs or CMake configuration is missing large-file support macros (like _FILE_OFFSET_BITS=64 equivalent for MSVC), causing file pointers or mmap offsets to overflow/truncate when addressing files larger than 4GB.

Expected behavior
Models > 4GB should load on Windows MSVC Rust exactly as they do in Python or Linux.

[Aidan24]

I think the issue may have been fixed in llama.cpp build b8149
Details
gguf : fix ftell/fseek for Windows (ggml-org/llama.cpp#19870)

Regression introduced in ggml-org/llama.cpp#19856.

This changes the ftell/fseek calls to use _ftelli64/_fseeki64 on Windows, and ftello/fseeko for POSIX systems.

long on Windows is always 32-bit. Since that would cause an overflow on large files, ftell/fseek fails and nbytes_remain() returns 0.

fixes ggml-org/llama.cpp#19862
fixes ggml-org/llama.cpp#19868

[Aidan24]

I should have added, I think it is 0.1.137 that introduced b8148, unfortunately one behind the fix, so downgrading to 0.1.136 (making sure you also downgrade llama-cpp-sys-2 to 0.1.136) should fix it until a later build of llama.cpp is pinned.

0.1.137
What's Changed
Bumped version to 0.1.137 by @github-actions[bot] in #939
Use CARGO_CFG_TARGET_FEATURE to detect CRT linkage on MSVC by @jbg in #943
Resolve toktrie version conflicts by @zhufucdev in #944
chore(deps): bump clap from 4.5.58 to 4.5.60 by https://github.com/dependabot[bot] in #934
chore: update llama.cpp to b8148 by @vigsterkr in #942

[Aidan24]

0.1.139 seems to fix the ftell/fseek issue on windows with build b8233 of llama.cpp. However I am having an issue with tool calls for the Minimax 2.5 model with this build that I did not have with 0.1.136. I receive the error "chat template error: ffi error -3" on any tool call.
I have been checking my code to try and make sure I conform to the correct Jinja chat template, in case the previous version was a bit more tolerant of a problem in my code, but to no avail so far. I've also downloaded a slightly different version of the gguf model file in case there was something specific in the version I used. So far I am unable narrow it down. Reverting to 0.1.136 works and I can't find a specific reference to a this issue in llama.cpp.

From Grok:
b8233 is relatively recent (early March 2026), but upstream llama.cpp had several Jinja/tool-related fixes and stability improvements landing in the days/weeks after that date:

Better handling of complex custom templates (including stricter exception propagation that can surface as ffi -3).
Improvements to fallback tool description injection when the template "supports tool calls but does not natively describe tools" (a very common log message for MiniMax-M2.5).
Fixes for edge cases in tool schema parsing, multi-turn tool messages, and thinking/reasoning tags.

[fhars]

@Aidan24 If you model your tool calls on examples/tools_reasoning.rs, the FfiError(-3) you see may be the issue I noticed in #992. That is unrelated to any windows file size issues.