Custom Steering of nodes extraction

[ParisNeo]

In general, the graph contains alot of noise (many completely out of scope or useless nodes are generated along useful nodes).
How about we add in the interface a user input that details the kind of nodes we want to extract or to suppress.
this can be done by making a part of the extraction prompt modifiable. We can use the webui to steer the extraction with two texts that can either be set together or just one of them:

• What kind of information we want to extract
• What kind of information to ignore

This can be cool and allow enhancing the quality of te extraction and qurying.

@LarFii @danielaskdd @ArnoChenFx .
What do you think?

[LarFii]

I think it might be possible to add an optional customizable prompt section, allowing users to control which content to focus on extracting.

[danielaskdd]

I think after indexing the documents, we should perform an optimization scan of nodes and relationships to mitigate issues. During this phase, we can do the following:

• Remove unnecessary nodes
• Community detection and community summarization
• Identify and merge nodes representing the same entities
• Node and edge summary compression (to prevent LLM context overflow during queries)