Skip to content

Commit 1fdee05

Browse files
fix: Dampen class weights to avoid Medium recall collapse
Full balanced weights gave Low ~6x the weight of Medium, dropping Medium recall by 20pp. Use sqrt-dampened weights for a gentler minority boost. Related to #19 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent b1679fb commit 1fdee05

1 file changed

Lines changed: 6 additions & 3 deletions

File tree

vulntrain/trainers/classify_severity_cnvd.py

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -175,7 +175,10 @@ def train(base_model, dataset_id, repo_id, model_save_dir="./vulnerability-class
175175
label_counter = Counter([ex["severity_label"] for ex in dataset["train"]])
176176
logger.info(f"Label distribution after filtering: {label_counter}")
177177

178-
# Compute balanced class weights for the training set
178+
# Compute dampened class weights for the training set.
179+
# Full "balanced" weights are too aggressive (Low gets ~6x Medium), causing
180+
# massive Medium recall loss. Square-root dampening provides a gentler boost
181+
# to the minority Low class without overwhelming Medium/High.
179182
all_labels = np.array(
180183
[SEVERITY_MAPPING[ex["severity_label"]] for ex in dataset["train"]]
181184
)
@@ -185,9 +188,9 @@ def train(base_model, dataset_id, repo_id, model_save_dir="./vulnerability-class
185188
)
186189
class_weights_array = np.ones(len(SEVERITY_MAPPING), dtype=np.float32)
187190
for cls, w in zip(present_classes, weights):
188-
class_weights_array[cls] = w
191+
class_weights_array[cls] = np.sqrt(w)
189192
class_weights = torch.tensor(class_weights_array, dtype=torch.float)
190-
logger.info(f"Class weights: {dict(zip(SEVERITY_MAPPING.keys(), class_weights_array))}")
193+
logger.info(f"Class weights (sqrt-dampened): {dict(zip(SEVERITY_MAPPING.keys(), class_weights_array))}")
191194

192195
tokenizer = AutoTokenizer.from_pretrained(base_model)
193196
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)

0 commit comments

Comments
 (0)