How-to clear or set-up separate Knowledge sources?

[GunnarStrandbergAB]

Hi all!
I am doing a project where it is important that data between runs is not stored in the same ChromaDB as data gets mixed up.

It worked well to solve it using:

crewai reset-memories --knowledge

But I want to be able to clear (or create new, separate collections) within the code, as I have built an Streamlit app that re-runs the flow with new data given what is selected.

Tried this:

doc_id = os.path.basename(os.path.dirname(cust_path)) # e.g. "DOC-003"
collection_name = f"collection_{doc_id}"

self.customer_profile = JSONKnowledgeSource(
file_paths=[cust_path],
# instruct or override the default collection name here, if Crew.ai allows:
collection_name=collection_name
)

But it did not work, guess that crew.ai does not expose the collection_name?

Any tips?

Best regards,

Gunnar

[GunnarStrandbergAB]

Solved it myself.

Just did:

subprocess.run(["crewai", "reset-memories", "--knowledge"], check=True)

[Komal-99]

Hi @GunnarStrandber

Have you found any other solution? Do we not have to restart the crew every time? Also I have implemented Docling knowledge source but can't see where crewai is saving the vector db.

[lucasgomide]

The collection name is defined based on Agent.rule. If you added many knowledge sources to the agent, all those source will be stored in the same collection.
Given that you could add a random value as a timestamp, UUID in your Agent rule. Just be careful. the collection_name is sanitized to attend chromadb's collection rules

[Komal-99]

Okay, can you share some references? I am not clear. Basically, what I want is for different agents to use different knowledge sources. My main concern is where Crewai saves the embeddings generated. I can't find any Directory made for it.

[Vidit-Ostwal]

Okay, can you share some references? I am not clear. Basically, what I want is for different agents to use different knowledge sources. My main concern is where Crewai saves the embeddings generated. I can't find any Directory made for it.

You should be able to see this,
Can you set this os env variable to your directory and run the crew again, I think you will be able to see that
CREWAI_STORAGE_DIR

[komal-work]

Hi ,
tried the Environment variable as CREWAI_STORAGE_DIR="./storage_DB" but still can't see the vector db created. Also Found some idea that I can pass Storage parameter but It requires Knowledge storage instance not sure how