Open
Description
Currently if you upload the same document twice, it will create duplicated chunks of the same documents.
Before add a document to the database, we should remove all chunks related to the document using its filename as ID to avoid duplication and allow updates.
Tasks
- Remove all chunks with the same filename of uploaded document in POST /documents
- for Azure ComosDB for MongoDB vCore path
- for FAISS path when running locally