-
Notifications
You must be signed in to change notification settings - Fork 766
Open
Description
Hi, I found a small problem with the field main_metric_name in dataclass Evaluator in the file dinov3/eval/linear.py. When I tried to test multiple metrics for the same testing dataset, the main_metric_name for different metrics are same then only one metric's result can be saved to disk. I don't know if there is any specific purpose of designing in this way or is there any other method to save result for all metrics. If not, I think it would be better to add metric name into main_metric_name. Thanks for your time and for this great project and the link to the file and code snippet are attached below.
link: https://github.com/facebookresearch/dinov3/blob/main/dinov3/eval/linear.py#L292
@dataclass
class Evaluator:
batch_size: int
num_workers: int
transform_config: TransformConfig
dataset_str: str
metric_type: ClassificationMetricType
metrics_file_path: str
training_num_classes: int
save_results_func: Optional[Callable]
def __post_init__(self):
self.data_loader, self.class_mapping = make_eval_data_loader(
test_dataset_str=self.dataset_str,
batch_size=self.batch_size,
num_workers=self.num_workers,
transform_config=self.transform_config,
metric_type=self.metric_type,
)
self.main_metric_name = f"{self.dataset_str}_accuracy"
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels