Feature: Manual Annotation and Tagging

[arthur-ver]

Hi,

Embedding Atlas is a very powerful tool for visualizing embeddings. However, more often than not, we want to do more than just observe the embeddings produced by our models, we want to actively make adjustments to the generated cluster labels. As of now, there is a significant lack of tools in the ecosystem that seamlessly bridge the gap between visualization and manual annotation.

It would be highly beneficial to have the ability to create, delete, or merge categories using the existing selection mechanisms. Adding this feature (and ideally a way to export the updated labels, or store them in a database) would allow ML developers and researchers to immediately act on the insights they discover in their data.

Best regards,
Arthur

[domoritz]

Thanks for noting this. While it's not built in, you could probably easily build such a feature today without changing embedding atlas.

The idea is to wrap embedding atlas in an application (could be Jupyter/marimo/streamlit) and add a listener for the selection. Then when you select points in embedding atlas and could have a button in your wrapper UI that labels the points in the dataset. I'm not 100% sure we have a clean way to update the dataset without totally the rerendering the UI but we could look into that.

I personally would way the wrapper approach is better than building in custom labeling etc since the former is much more customizable.

What do you think?

[arthur-ver]

@domoritz Thanks for your reply! I haven’t actually thought of such approach that you proposed, it sounds pretty reasonable to me.

On the other hand, I also like the way how you can fully customise the EmbeddingView component and have a standalone web app. Not 100% sure the python widget offers the same capabilities 🤔

[domoritz]

The python widget is running the same code as the web app so it should be very similar in capabilities.

We designed embedding atlas as an interoperable component/building block so composition should allow you to prototype complex workflows.

We may support labeling in the future but it requires so careful thinking and wouldn't be as flexible as wrapping the widget (since you can write your own logic to process a selection).