Any reason why chunking with llama vision model result in gibberish in the chunk ?

[junhong-AI]

This is version v0.19.1, that was only working version for for Ollama integration, when I was using Ollama llama3.2-vision:11b as img2txt, It was working for the previous few images then it start producing gibberish. Any idea why it happen ?

[dosubot[bot]]

There are a few likely reasons why chunking with the llama vision model in RAGFlow v0.19.1 starts producing gibberish after initially working:

• Ollama vision models expect raw base64 image strings without headers like data/png;base64,. If the chunking or parsing step passes images with headers, it can cause errors or gibberish output. Manually removing the header before sending to Ollama can help as a workaround [source].
• The chunking logic in v0.19.1 uses a "general" method that splits content (including images) based on token size and delimiter. If chunking parameters are misconfigured or if the chunking method is incompatible with the vision model, it can lead to malformed input and gibberish output [source].
• The parser always saves images as JPEG before sending to the vision model. If the model expects a different format or if the conversion corrupts the image, this could also cause issues [source].
• Resource constraints (like high VRAM/CPU usage) or Ollama configuration issues can cause the model to output gibberish or repeat tokens, especially with large models. Setting OLLAMA_NUM_GPU=1 and disabling keepalive may help [source].
• Integration bugs or version mismatches between RAGFlow and Ollama can cause erratic output. Upgrading to a newer RAGFlow version or using a different deployment method (like vLLM) may resolve the issue [source].

If the gibberish output started suddenly, double-check if the image format or chunking parameters changed, or if system resources are being maxed out. If possible, try removing base64 headers, adjusting chunking parameters, or testing with a different chunking method (like "Paper" instead of "General") to see if the output improves.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[ZhenhangTung]

How about changing to another model?

[junhong-AI]

gpt models will work