CP-Cluster is improving Yolov5 accuracy without retraining official models

[shenyi0220]

CP-Cluster outperforms all NMS methods when applied to Yolov5 official models without getting them retrained, gaining ~0.3 improvement on average.
https://github.com/shenyi0220/cp-cluster

[glenn-jocher]

@shenyi0220 yes I'd seen that! Those are impressive mAP improvements. What's the best way to offer this an an NMS option? Our NMS function is here, it's mainly a wrapper around the torch vision nms function, which itself is compiled in C I think:

yolov5/utils/general.py

Lines 647 to 669 in f80c463

	def non_max_suppression(prediction, conf_thres=0.25, iou_thres=0.45, classes=None, agnostic=False, multi_label=False,
	labels=(), max_det=300):
	"""Runs Non-Maximum Suppression (NMS) on inference results
	Returns:
	list of detections, on (n,6) tensor per image [xyxy, conf, cls]
	"""
	nc = prediction.shape[2] - 5 # number of classes
	xc = prediction[..., 4] > conf_thres # candidates
	# Checks
	assert 0 <= conf_thres <= 1, f'Invalid Confidence threshold {conf_thres}, valid values are between 0.0 and 1.0'
	assert 0 <= iou_thres <= 1, f'Invalid IoU {iou_thres}, valid values are between 0.0 and 1.0'
	# Settings
	min_wh, max_wh = 2, 7680 # (pixels) minimum and maximum box width and height
	max_nms = 30000 # maximum number of boxes into torchvision.ops.nms()
	time_limit = 10.0 # seconds to quit after
	redundant = True # require redundant detections
	multi_label &= nc > 1 # multiple labels per box (adds 0.5ms/img)
	merge = False # use merge-NMS

[shenyi0220]

In my exp impl, I directly replaced the torchvision.ops.nms with the mmcv-based API of CP-Cluster.

So later for yolov5, when I got green light from the NV to fully open source GPU versions of this method, maybe I can contribute like this:
-Opt1: upstreamed the CP-Cluster cpu/gpu versions to mmcv, and support optional call of mmcv API in addition to torchvision nms.
-Opt2: directly implemented gpu versions in yolov5 repo, while this may need optional compile step
-Opt3: try to upstream the gpu version of cp-cluster to torchvision

[zhiqwang]

Hi @shenyi0220 , Do you think it is possible to make this as a separate package?

[glenn-jocher]

@shenyi0220 opt3 would be awesome! But I know the torch team doesn't like to include new development very fast. It took them a very long time to add the Swish activation for example.

We can't support any compilation steps in YOLOv5, so then Opt1 might be best while you work on Opt3. For Opt3 please raise an issue on the torch repo and @ me and I'll throw my support behind it.