Adding Citation Support in LightRAG

[LarFii]

We are considering adding a citation feature to LightRAG. Does anyone have any good ideas or suggestions on how to implement this?

[BireleyX]

I had to modified the code (operate.py and prompt.py) to add support for citation on modes other than Mix.
But I just updated to the latest version 1.2.6 of the source code, and it seems they added citations/references on all modes already...
although there's still a bug on the file_id.

[mballav]

Really? Citations code is already there in v1.2.6? Could you please confirm, @LarFii ?

[BireleyX]

yes, but seems to have a bug. I raised a ticket for it.
#1116

you can my screenshots there and see if those are the citations you were asking for.

[ntsarb]

Citations (currently down to the chunk id, I suppose?) are needed inline with the text from the LightRAG response, i.e. at the end of each sentense/paragraph, so that the information can be cross-checked/validated.

Citations should also refer to the relevant chapter/section/numbered_paragraph of a document, so that they are easy to find. To implement these, we need to add the relevant fields/columns in the chunk records, and build a parser that is capable of extracting this information from the document. I need to look into the code, to better understand what's needed; I wonder if Apache Tika can help.

[johnshearing]

The more I tinker and bang on the LightRAG Server, the more I appreciate how little meaning we can get from citations. That's because there isn't all that much direct correspondence between the response and the material it comes from. Better to ask for an audit trail of how LightRAG forms responses. By putting a question into the retrieval tab and checking the Only Need Context option and then running the query once for each of the following options: (mix, hybrid, global, local, naive), I get the source material back in several ways that make it possible to understand where the answers are coming from. A lot of looking through the data and thinking about it is required to make sense of what LightRAG is doing but the information is there. More interesting to me is the idea of automating the audit so that less time and effort is required to see how the responses are formed.

[ntsarb]

The purpose of any RAG is to ground the responses to a reliable source of information. Without in-line citations to the paragraphs used from the source material, a response may be pretty, may be fun to read, but neither reliable, nor really useful.

When I experimented with custom prompts from Microsoft's GraphRAG, I was able to extract original text, to describe entities and relationships. I need to get my head around LightRAG and how it can be tinkered accordingly.

[johnshearing]

Greetings @ntsarb

ntsarb writes:

When I experimented with custom prompts from Microsoft's GraphRAG, I was able to extract original text, to describe entities and relationships.<

I just discovered you can do that with LightRAG too.
Have you tried the LightRAG Server which is in the LightRAG repository?
You won't believe what it does.

• Ingest documents,
• Explore the Knowledge Graph,
• Query in several different ways including several views of the source material,
• Work directly with the API.

This is what made me realize that richness of answers returned can not be validated by simply citing sources.
It doesn't really return information. It interprets information and returns meaning.
An audit of how the system arrives at it's answers is required to prove what it says is correct.

---

[ntsarb]

This is very useful. Unfortunately, the server crashes on my system, I can't see any of this (or the graph), and I have to figure out why. I've reported the issue here.