Problem with get_nexusscore.py

[yiwen-song]

Edit:

I just realized that nexusscore is only computed on the other 2 domains. NVM!
Feel free to close this issue.

---

Hey OpenS2V team,

Recently I've tried to run OpenS2V evals with my own model following the instructions here.

I have encountered an issue with get_nexusscore.py

For example, when evaluating the single-domain, there are some samples such as singleface_30.
In this part of the code, since "face" in prefix & image_label == "Woman" always holds true, which results in all_prompt_images=[], then causes gme.get_image_embeddings to throw out this error:

torch.cat(): expected a non-empty list of Tensors

Because of this issue, out of the 60 samples in Single-Domain_Eval.json, about half errored out, so the result json file "nexusscore.json" ends up being incomplete (Only 28/60 samples are recorded.)

I'm wondering if this is intended, or am i missing some important details here?
Would appreciate if you can take a look, thanks!

[SHYuanBest]

Thanks for your interest! This model (YOLO-World-Image) has low accuracy in face detection, so we skipped it.