Skip to content

Embedded images with custom width #62

@bcmohr

Description

@bcmohr

It looks like nodes containing embedded images like:

![[image.png]]

are successfully ignored as tokens when you run the cannoli and have vision disabled. This is normal. No image content goes to the tokenizer.

But then, the image's content gets sent to the tokenizer (even when vision is disabled) when you give the image embedding a custom width (such as 123 pixels wide) like so:

![[image.png|123]] 

I was wondering why I kept getting told I had hundreds of thousands of tokens getting sent out. This was the culprit. It was resolved when I replaced all custom-width image embeddings (![[image.png|123]]) in my document with regular image embeddings (![[image.png]] ). Apparently this is a regex issue.

No real emergency here, just an odd behaviour that tripped me up. Easy to fix on my end.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions