chg: [doc] Document the optional CWE knowledge base refresh before training

Clarify that vulntrain-train-cwe-classification works out of the box
with the versioned mapping, and that refreshing the CWE knowledge base
from the Vulnerability-Lookup API is an optional two-step pipeline
(update_cwe_knowledge_base.py then build_child_to_ancestor.py) to run
before training.

Author: cedricbonhomme

docs/index.md

@@ -132,6 +132,23 @@ Predict CWE classifications from vulnerability descriptions and associated patch
 vulntrain-train-cwe-classification --base-model roberta-base --dataset-id CIRCL/vulnerability-cwe-patch --repo-id CIRCL/cwe-parent-vulnerability-classification-roberta-base
 ```
 
+The trainer maps each CWE of the dataset to an ancestor CWE via
+`vulntrain/data/deep_child_to_ancestor.json`, built so that every training
+label has an *Allowed* or *Allowed-with-Review* MITRE mapping usage: the model
+can never suggest a Discouraged or Prohibited CWE.
+
+This mapping is versioned and shipped with VulnTrain, so no extra step is
+required before training. To optionally refresh it against the latest CWE
+data from [Vulnerability-Lookup](https://vulnerability.circl.lu), run the
+following commands before training and commit the regenerated files:
+
+```bash
+python tools/cwe/update_cwe_knowledge_base.py   # refresh the CWE knowledge base from the API
+python tools/cwe/build_child_to_ancestor.py     # regenerate vulntrain/data/deep_child_to_ancestor.json
+```
+
+See `tools/cwe/README.md` for details.
+
 
 ### Text generation
 

tools/cwe/README.md

@@ -1,8 +1,16 @@
 # CWE mapping generation tools
 
-One-off scripts used to build the CWE mapping files. They are not part of the
-installed `vulntrain` package and are only needed when the CWE data must be
-regenerated.
+Scripts used to build the CWE mapping consumed by
+`vulntrain-train-cwe-classification`. They are not part of the installed
+`vulntrain` package.
+
+Running them is **optional**: the resulting
+`vulntrain/data/deep_child_to_ancestor.json` is versioned and shipped with
+VulnTrain, so training works out of the box. Run the pipeline below before
+training only when you want to pick up the latest CWE data (new CWEs, changed
+mapping usages) from Vulnerability-Lookup, and commit the regenerated files.
+Both steps are needed, in order: the first only refreshes the knowledge base,
+the second regenerates the mapping the trainer reads.
 
 ## Pipeline