Matryoshka Embeddings to skip UMAP

[e-hossam96]

Rather than performing your downstream task (e.g., dimensionality reduction) on the full embeddings, you can shrink the embeddings to a smaller size and very efficiently "shortlist" your embeddings. Afterwards, you can process the remaining embeddings using their full dimensionality.

Article on HF

[MaartenGr]

That is correct. It's becoming a more popular technique these days for creating embeddings. However, I haven't seen many embeddings that go that small (e.g., 5 or 10 dimensions). Do you perhaps have an example of performant embeddings that go this small?

[e-hossam96]

I haven't seen anything small yet either, but I thought about bringing that up.