Skip to content

COCO metrics only evaluated on image_ids with predictions #2504

Open
@digital-nomad-cheng

Description

coco_dt = coco_gt.loadRes(predictions=coco_predictions)
image_ids = [ann["image_id"] for ann in coco_predictions]
coco_eval = COCOeval(coco_gt, coco_dt, iouType="bbox")
coco_eval.params.imgIds = image_ids

The pycoco_wrapper only evaluates on images where there is a prediction, and leave out image_ids where there is not prediction.
This is a serious bug if I understand it correctly. The coco metrics should be evaluated on all the images.
Correct way to do this should be below, without filtering image_ids.

    coco_dt = coco_gt.loadRes(predictions=coco_predictions)
    coco_eval = COCOeval(coco_gt, coco_dt, iouType="bbox")

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions