Skip to content

Commit 3acdeeb

Browse files
authored
Merge pull request #1693 from HKUDS/delete_doc
Feat: Add comprehensive document deletion feature with integrated knowledge graph reconstruction
2 parents d1aeb29 + 2946bbd commit 3acdeeb

13 files changed

Lines changed: 1313 additions & 287 deletions

README-zh.md

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -932,6 +932,94 @@ rag.insert_custom_kg(custom_kg)
932932

933933
</details>
934934

935+
## 删除功能
936+
937+
LightRAG提供了全面的删除功能,允许您删除文档、实体和关系。
938+
939+
<details>
940+
<summary> <b>删除实体</b> </summary>
941+
942+
您可以通过实体名称删除实体及其所有关联关系:
943+
944+
```python
945+
# 删除实体及其所有关系(同步版本)
946+
rag.delete_by_entity("Google")
947+
948+
# 异步版本
949+
await rag.adelete_by_entity("Google")
950+
```
951+
952+
删除实体时会:
953+
- 从知识图谱中移除该实体节点
954+
- 删除该实体的所有关联关系
955+
- 从向量数据库中移除相关的嵌入向量
956+
- 保持知识图谱的完整性
957+
958+
</details>
959+
960+
<details>
961+
<summary> <b>删除关系</b> </summary>
962+
963+
您可以删除两个特定实体之间的关系:
964+
965+
```python
966+
# 删除两个实体之间的关系(同步版本)
967+
rag.delete_by_relation("Google", "Gmail")
968+
969+
# 异步版本
970+
await rag.adelete_by_relation("Google", "Gmail")
971+
```
972+
973+
删除关系时会:
974+
- 移除指定的关系边
975+
- 从向量数据库中删除关系的嵌入向量
976+
- 保留两个实体节点及其他关系
977+
978+
</details>
979+
980+
<details>
981+
<summary> <b>通过文档ID删除</b> </summary>
982+
983+
您可以通过文档ID删除整个文档及其相关的所有知识:
984+
985+
```python
986+
# 通过文档ID删除(异步版本)
987+
await rag.adelete_by_doc_id("doc-12345")
988+
```
989+
990+
通过文档ID删除时的优化处理:
991+
- **智能清理**:自动识别并删除仅属于该文档的实体和关系
992+
- **保留共享知识**:如果实体或关系在其他文档中也存在,则会保留并重新构建描述
993+
- **缓存优化**:清理相关的LLM缓存以减少存储开销
994+
- **增量重建**:从剩余文档重新构建受影响的实体和关系描述
995+
996+
删除过程包括:
997+
1. 删除文档相关的所有文本块
998+
2. 识别仅属于该文档的实体和关系并删除
999+
3. 重新构建在其他文档中仍存在的实体和关系
1000+
4. 更新所有相关的向量索引
1001+
5. 清理文档状态记录
1002+
1003+
注意:通过文档ID删除是一个异步操作,因为它涉及复杂的知识图谱重构过程。
1004+
1005+
</details>
1006+
1007+
<details>
1008+
<summary> <b>删除注意事项</b> </summary>
1009+
1010+
**重要提醒:**
1011+
1012+
1. **不可逆操作**:所有删除操作都是不可逆的,请谨慎使用
1013+
2. **性能考虑**:删除大量数据时可能需要一些时间,特别是通过文档ID删除
1014+
3. **数据一致性**:删除操作会自动维护知识图谱和向量数据库之间的一致性
1015+
4. **备份建议**:在执行重要删除操作前建议备份数据
1016+
1017+
**批量删除建议:**
1018+
- 对于批量删除操作,建议使用异步方法以获得更好的性能
1019+
- 大规模删除时,考虑分批进行以避免系统负载过高
1020+
1021+
</details>
1022+
9351023
## 实体合并
9361024

9371025
<details>

README.md

Lines changed: 83 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -988,6 +988,89 @@ These operations maintain data consistency across both the graph database and ve
988988

989989
</details>
990990

991+
## Delete Functions
992+
993+
LightRAG provides comprehensive deletion capabilities, allowing you to delete documents, entities, and relationships.
994+
995+
<details>
996+
<summary> <b>Delete Entities</b> </summary>
997+
998+
You can delete entities by their name along with all associated relationships:
999+
1000+
```python
1001+
# Delete entity and all its relationships (synchronous version)
1002+
rag.delete_by_entity("Google")
1003+
1004+
# Asynchronous version
1005+
await rag.adelete_by_entity("Google")
1006+
```
1007+
1008+
When deleting an entity:
1009+
- Removes the entity node from the knowledge graph
1010+
- Deletes all associated relationships
1011+
- Removes related embedding vectors from the vector database
1012+
- Maintains knowledge graph integrity
1013+
1014+
</details>
1015+
1016+
<details>
1017+
<summary> <b>Delete Relations</b> </summary>
1018+
1019+
You can delete relationships between two specific entities:
1020+
1021+
```python
1022+
# Delete relationship between two entities (synchronous version)
1023+
rag.delete_by_relation("Google", "Gmail")
1024+
1025+
# Asynchronous version
1026+
await rag.adelete_by_relation("Google", "Gmail")
1027+
```
1028+
1029+
When deleting a relationship:
1030+
- Removes the specified relationship edge
1031+
- Deletes the relationship's embedding vector from the vector database
1032+
- Preserves both entity nodes and their other relationships
1033+
1034+
</details>
1035+
1036+
<details>
1037+
<summary> <b>Delete by Document ID</b> </summary>
1038+
1039+
You can delete an entire document and all its related knowledge through document ID:
1040+
1041+
```python
1042+
# Delete by document ID (asynchronous version)
1043+
await rag.adelete_by_doc_id("doc-12345")
1044+
```
1045+
1046+
Optimized processing when deleting by document ID:
1047+
- **Smart Cleanup**: Automatically identifies and removes entities and relationships that belong only to this document
1048+
- **Preserve Shared Knowledge**: If entities or relationships exist in other documents, they are preserved and their descriptions are rebuilt
1049+
- **Cache Optimization**: Clears related LLM cache to reduce storage overhead
1050+
- **Incremental Rebuilding**: Reconstructs affected entity and relationship descriptions from remaining documents
1051+
1052+
The deletion process includes:
1053+
1. Delete all text chunks related to the document
1054+
2. Identify and delete entities and relationships that belong only to this document
1055+
3. Rebuild entities and relationships that still exist in other documents
1056+
4. Update all related vector indexes
1057+
5. Clean up document status records
1058+
1059+
Note: Deletion by document ID is an asynchronous operation as it involves complex knowledge graph reconstruction processes.
1060+
1061+
</details>
1062+
1063+
**Important Reminders:**
1064+
1065+
1. **Irreversible Operations**: All deletion operations are irreversible, please use with caution
1066+
2. **Performance Considerations**: Deleting large amounts of data may take some time, especially deletion by document ID
1067+
3. **Data Consistency**: Deletion operations automatically maintain consistency between the knowledge graph and vector database
1068+
4. **Backup Recommendations**: Consider backing up data before performing important deletion operations
1069+
1070+
**Batch Deletion Recommendations:**
1071+
- For batch deletion operations, consider using asynchronous methods for better performance
1072+
- For large-scale deletions, consider processing in batches to avoid excessive system load
1073+
9911074
## Entity Merging
9921075

9931076
<details>

lightrag/api/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
__api_version__ = "0173"
1+
__api_version__ = "0174"

lightrag/api/lightrag_server.py

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -355,7 +355,13 @@ async def azure_openai_model_complete(
355355
)
356356

357357
# Add routes
358-
app.include_router(create_document_routes(rag, doc_manager, api_key))
358+
app.include_router(
359+
create_document_routes(
360+
rag,
361+
doc_manager,
362+
api_key,
363+
)
364+
)
359365
app.include_router(create_query_routes(rag, api_key, args.top_k))
360366
app.include_router(create_graph_routes(rag, api_key))
361367

0 commit comments

Comments
 (0)