research: benchmark quantized Florence-2 and lightweight captioning alternatives

[Abhash-Chakraborty]

Summary

Benchmark lower-footprint captioning paths rather than replacing Florence-2 based on parameter count alone.

Evidence

Microsoft lists Florence-2-base at 0.23B parameters: https://huggingface.co/microsoft/Florence-2-base. Hugging Face's SmolVLM-256M card reports under 1 GB GPU RAM for one-image inference: https://huggingface.co/HuggingFaceTB/SmolVLM-256M-Instruct. SmolVLM is not automatically smaller in parameters, so quantized Florence and runtime overhead must also be measured.

Where to work

• backend/src/find_api/ml/captioner.py
• model manager/cache work in feat: design installer model downloads and cache management #45
• caption/search evaluation fixtures

Acceptance criteria

• Benchmark current Florence-2-base FP32 CPU/FP16 GPU baseline.
• Evaluate supported quantized/ONNX/OpenVINO Florence paths and one small VLM candidate.
• Compare download size, runtime dependencies, cold load, RAM/VRAM, latency, caption quality, OCR leakage, and failure rate.
• Verify caption output still supports hybrid search and user-facing metadata.
• Record license and offline-download behavior for every candidate.
• Recommend a model/runtime matrix instead of one unqualified default.

Dependency

Use measurable fixtures from #100 where captions affect retrieval.

[palak170306-design]

hii i would like to work on this issue under gssoc
please assign this to me
@Abhash-Chakraborty

[Abhash-Chakraborty]

@palak170306-design assigned — sorry for the slow reply. Before you start, post your benchmark plan here (which quantised Florence-2 builds you'll compare, what hardware you'll measure on, and how you'll score caption quality) so we agree on the method before you spend time on runs. #341's PR (#364) is a good template for what the writeup should look like: measured numbers, raw results committed, caveats stated.