[Bug]: response field larger than field limit (131072)

[theauAg]

Do you need to file an issue?

• I have searched the existing issues and this bug is not already filed.
• I believe this is a legitimate bug, not just a question or feature request.

Describe the bug

Hello,

Current situation

When I use the rag.query in LightRAG Vanilla with the parameter response_type="Multiple Paragraphs" there is a bug if the response is too long.

Steps to reproduce

Take a document with a long description, initialize LightRAG vanilla and ask to repeat the description.
Then launch:

rag = LightRAG(
            working_dir=WORKING_DIR,
            llm_model_func=llm_model,
            embedding_func=EmbeddingFunc(
                embedding_dim=768,
                max_token_size=8192,
                func=embedding,
            ),
        )

 with open(name_of_element_, "r") as file:
        text = file.read()
        # Inserting the text into LightRAG
        rag.insert(text)


rag.query(
        query=query,
        param=QueryParam(
            mode="mix",
            top_k=5,
            response_type="Multiple Paragraphs"
        ),
    )

Expected Behavior

raise ValueError(f"Failed to parse CSV string: {str(e)}")
ValueError: Failed to parse CSV string: field larger than field limit (131072)

LightRAG Config Used

Paste your config here

rag = LightRAG(
working_dir=WORKING_DIR,
llm_model_func=llm_model,
embedding_func=EmbeddingFunc(
embedding_dim=768,
max_token_size=8192,
func=embedding,
),
)

Logs and screenshots

No response

Additional Information

• LightRAG Version: 1.3.2
• Operating System:
• Python Version: 3.11.12
• Related Issues:

[danielaskdd]

The information you provided is insufficient to determine where the error occurs in the program. Additionally, the new version of LightRAG has replaced CSV format with JSON format when constructing context-aware queries. Please check the latest version to see if the same issue persists.

[sunjiamin1]

我的是csv有一单元格内容长度超出限制，扫描列出所有超限文件

import csv
import sys
from pathlib import Path

csv.field_size_limit(sys.maxsize)

ROOT_DIR = Path("your_csv_folder")
LIMIT = 131072

found = False

for csv_file in ROOT_DIR.rglob("*.csv"):
try:
with open(csv_file, "r", encoding="utf-8", errors="replace") as f:
reader = csv.reader(f)
for row_idx, row in enumerate(reader, start=1):
for col_idx, field in enumerate(row, start=1):
if len(field) > LIMIT:
found = True
print(
f"[OVERSIZE] file={csv_file} "
f"row={row_idx} col={col_idx} len={len(field)}"
)
except Exception as e:
print(f"[ERROR] file={csv_file} err={e}")

if not found:
print("✅ No oversized fields found")