Canopy And Pinecone Serverless

[gfleetwood]

Hello. With the release of Pinecone Serverless I upgraded from the free to the standard plan. While canopy will now create a new serverless database, it appears under the free tier instead of my current plan. I submitted a support ticket last week which hasn't been answered, so I thought I'd ask here.

For reference, this is the code I'm using:

from canopy.tokenizer import Tokenizer
from canopy.knowledge_base import KnowledgeBase
from canopy.knowledge_base import list_canopy_indexes

Tokenizer.initialize()
INDEX_NAME = 'from-python'
kb = KnowledgeBase(index_name = INDEX_NAME)

if not any(name.endswith(INDEX_NAME) for name in list_canopy_indexes()):
    kb.create_canopy_index()
    #kb.create_canopy_index(index_params = {'spec': 'serverless'})

#help(kb.create_canopy_index)

I also have two more questions:

• How would I direct canopy to create a db in my old pod based project.
• I read in some documentation that it's recommended to use canopy with a db created by canopy. Is there any way to use it with a db created by the pinecone API?

[coreation]

@gfleetwood to use canopy in your own made index, I think it's more of a hardcoded set of metadata properties that Canopy expects to be there...as far as I know. I'm using Canopy with an index I made manually, using canopy with that index failed because it couldn't find some of the metadata properties it needed.

I've changed the metadata in the documents to match the ones in the Canopy code and I'm getting RAG based responses from the Canopy REST API ...so that seems to work just fine.

[izellevy]

As @coreation stated canopy expects a specific metadata structure so it is not possible to use it with existing indexes created beforehand.

You can direct canopy to create indexes in the pod based project by passing a Pod Spec to the create_canopy_index function. Just make sure you are in the latest canopy version.

You can pass something similar to this as a spec:

{
      "replicas": 1,
      "pod_type": "p1.x1",
      "shards": 1,
      "environment": "us-east1-gcp"
 }

More info on how to create this dictionary is documented here: https://docs.pinecone.io/reference/create_index

@gfleetwood please let me know if this answers your questions.

[coreation]

@izellevy is it possible to mess up the meta-data structure once it is created?