ClusterLab

ClusterLab is a browser-based workspace for image embeddings and clustering.
You can run DINOv2 models from Hugging Face or local ONNX files, extract embeddings, visualize patch features in 3D, and explore clusters interactively.

Project Positioning

This repository is an experimental playground for image clustering and visual data exploration.
It is intended for fast iteration, concept validation, and UI/algorithm experimentation.

Roadmap Direction

This playground informs a future production-grade asset platform with image clustering and visual search, designed to operate across multi-source storage environments, including local file systems, S3-compatible object storage, and major online file services.

Live Demo

https://promto-c.github.io/cluster-lab/

Features

• 4-step pipeline: Initialize -> Embed -> Cluster -> Visualize
• Model sources: Hugging Face (Xenova/dinov2-small, base, large, or custom repo ID), local ONNX model folder, or classical CV descriptors (Color Histogram, LBP, GLCM, HOG)
• Hugging Face ONNX variant selector (auto-discovers files in onnx/, e.g. model.onnx, model_q4.onnx, model_quantized.onnx)
• Preprocessing controls: Letterbox Pad (blur, mirror/reflect, or solid), Stretch, Center Crop
• Dataset tooling: add files/folders, example datasets, enable/disable/focus batch controls
• Embedding studio: batch embedding generation, 3D patch visualization (PCA or channel mode), and local vs global PCA color basis
• Clustering lab: Agglomerative, KMeans, HDBSCAN-style density clustering, and BIRCH with cosine/euclidean metrics, normalization, auto-tuning, and dendrogram thresholding
• Hierarchical drill-down clustering across sub-clusters
• Import/export embeddings as JSON (lite or full with patches)

Tech Stack

• React 18 + TypeScript + Vite
• @huggingface/transformers for in-browser model inference
• ONNX Runtime Web (WASM assets in public/onnxruntime)
• Three.js + React Three Fiber for 3D visualization

Architecture Notes

• Runs fully in-browser (no backend required)
• Uses ONNX Runtime Web (WASM) for model inference
• Supports both Hugging Face–hosted models and local ONNX files
• Embeddings and clustering computed client-side

Run Locally

Prerequisites: Node.js 18 or higher

1. Install dependencies:

npm install

2. Start dev server:

npm run dev

Build

1. Sync ONNX Runtime WebAssembly assets (optional; automatically runs before build):

npm run sync:onnx-wasm

2. Create production build:

npm run build

3. Preview production build:

npm run preview

ONNX Runtime WASM Assets

• The app self-hosts ONNX Runtime WASM binaries in public/onnxruntime.
• npm run build automatically syncs required variants from onnxruntime-web via prebuild.
• Runtime path uses import.meta.env.BASE_URL, so this works on GitHub Pages project URLs.

Local Model Folder Requirements

When using Local Folder mode, include:

• model.onnx (or another .onnx file selected in the UI)
• config.json
• preprocessor_config.json

Workflow Quick Start

1. Initialize: choose model source and preprocessing, then initialize model/algorithms.
2. Embed: import images (or load examples) and run embeddings.
3. Cluster: pick algorithm/settings, run clustering, and drill down into sub-clusters.
4. Visualize: explore patch-level features in 3D and inspect clusters.

Acknowledgments

This project builds on top of several excellent open-source libraries and models:

• React, Vite, and the modern frontend ecosystem
• Three.js and the React Three Fiber ecosystem for 3D visualization
• Hugging Face ecosystem and contributors for pretrained models (e.g., DINOv2)
• DINOv2 models developed by Meta AI
• ONNX Runtime Web for efficient model execution
• Lucide for iconography

We thank the open-source community for making these tools available.

License

Licensed under the MIT License. See LICENSE for details.

Note

Some features rely on third-party models (e.g., DINOv2 via Hugging Face) that are subject to their own licenses. See THIRD_PARTY_LICENSES.md.