Chat templates not working on published 0.4 version

[D4RKAR117]

Problem

The same as with #451, while the fix already exists on #452, projects leveraging the 0.4 architecture cannot use it due that the huge amount of changes, like the transition to Fusor as the backend.

Steps To Reproduce

Steps to reproduce the behavior. Include a code sample or a link to a repository with the code sample if applicable:

• Create a project with kalosm 0.4 from crates.io
• Configure the project to leverage any LLM model family different from Llama
• Try to use llm.chat().with_system_prompt(<YOUR TEST SYSTEM PROMPT>)
• Same symptoms as described in detail on System messages render with role "developer", breaking chat templates #451

Expected behavior

We would be able to use system prompts without drifting with any LLM model family.

Theoretical/Desired Solution
To have a patch version of 0.4 (0.4.1) fixing this issue without pulling all of the main HEAD changes for the project available on crates.io, and to mark all of the fusor non-prod-ready changes as a different version to avoid confusion. Why? Some projects (like mine) are using the 0.4 CUDA-based architecture with good performance but just got problems using the system prompts, and while Fusor is still WIP, it would be great to have the fix on the existing battle tested architecture.

Environment:

• Crate version: [Kalosm 0.4.0]
• Rust version: [1.99.0, nightly]
• OS info: [Arch Linux via WSL2]

Questionnaire

• I'm interested in fixing this myself but don't know where to start.
• I would like to fix it, and I have a solution.
• I don't have time to fix this right now, but maybe later

[D4RKAR117]

I created a fork and cherry-picked the fix of #452 here: https://github.com/D4RKAR117/kalosm/tree/hf-template-backport

[D4RKAR117]

While bumping tokenizers in this fork so it'd compile with the version my downstream project resolves (0.21.4, vs the 0.21.0 this workspace had pinned), I ran into two build breaks and fixed both:

1. tokenizers 0.21.4 needs AHashMap, not HashMap. Once I bumped the version, gguf_tokenizer.rs failed to compile because BpeBuilder::vocab_and_merges now requires Into<AHashMap<String, u32>>. Upstream already fixed this in 60c9e6ac ("convert vocab to AHashMap Convert vocab to AHashMap #389"), so I reapplied the same fix here (added ahash and converted the vocab before building).

2. Bumping tokenizers pulled in a newer docx-rs, which broke docx.rs. The newer docx-rs added a few new enum variants to DocumentChild/ParagraphChild/RunChild, and our exhaustive matches didn't cover them (E0004). Upstream had already handled this in 790ac0d8 by collapsing the no-op arms into a single _ => {}. I copied that fix over.

So both changes are just this fork catching up to upstream fixes so it keeps building against current dependency versions.