Skip to content

fastest way to create and store embeddings for data in a SQLite table #189

Open
@punkish

Description

I have a SQLite table like so (simplified schema below) with ~950K rows

CREATE TABLE t (id INTEGER PRIMARY KEY, fulltext TEXT);

What is the quickest and easiest way to generate embeddings and store them in a vectors table? I am using Ollama with llama 3.2 running locally using the nomic-embed-text embeddings model. I would like to do this once and then create a web interface to query the data. Additionally, as new rows get added to the table t, I would like to TRIGGER embeddings for them as well. Is that possible? If yes, any hint would be very welcome.

Metadata

Assignees

Labels

questionFurther information is requestedstale

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions