- * *key* corresponds to the `metric_name | String`. When processed, the value is split by the underscore ("_") character. The first part of the split, or the entire string if no underscore exists, is used as the `metric_key`. This must match one the name of a class in the `metrics` folder that inherits from `MetricABC`. By default, the available options are: `["Precision", "PrecisionWeighted", "TPI", "TPS", "NMI", "Recall", "RecallPerEntityType", "TRIR"]`. Custom metrics added following the [Extensions guidelines](#extension-guidelines) will also be considered. If the `metric_key` is invalid (for instance, because `metric_name` starts with an underscore) this triggers a warning log and prevents the metric from being computed. Any text following the first underscore is independent of the `metric_key`, and can be used to indicate variations of the same metric (*e.g.*, "TPS", "TPS_TA=4", "TPS_TA=4_bert").
0 commit comments