Random folding in Auto3DSeg `AutoRunner` doesn't acknowledge `num_fold` input

[bhashemian]

Describe the bug
When setting num_fold in AutoRunner, if the input datalist doesn't have "fold" field, it uses a hard coded number of folds, which is 5, and create randomly create folds for the datasets and save it in the datalist.

MONAI/monai/apps/auto3dseg/auto_runner.py

Lines 418 to 433 in 782c1e6

	num_fold = 5
	warnings.warn(
	f"Datalist has no folds specified {datalist_filename}..."
	f"Generating {num_fold} folds randomly."
	f"Please consider presaving fold numbers beforehand for repeated experiments."
	)
	from sklearn.model_selection import KFold
	kf = KFold(n_splits=num_fold, shuffle=True, random_state=0)
	for i, (_, valid_idx) in enumerate(kf.split(datalist["training"])):
	for vi in valid_idx:
	datalist["training"][vi]["fold"] = i
	ConfigParser.export_config_file(datalist, datalist_filename, fmt="json", indent=4)

Although the input parameter is acknowledged and overwrite the five folds, the datalist is not created again.

MONAI/monai/apps/auto3dseg/auto_runner.py

Lines 292 to 294 in 782c1e6

	num_fold = self.inspect_datalist_folds(datalist_filename=datalist_filename)
	if "num_fold" in self.data_src_cfg:
	num_fold = int(self.data_src_cfg["num_fold"]) # override from config

[bhashemian]

CC: @dongyang0122 @myron @heyufan1995

[mattlin1124]

Hi, I’m interested in working on this issue if it’s still relevant.

I checked the current dev branch and confirmed that when the datalist has no existing fold assignments, a user-specified num_fold is still not used when the fold assignments are initially generated. For example, with num_fold=2, the datalist is still assigned 5 folds while runner.num_fold is later set to 2.

I also noticed the previous attempt in AlexanderSanin#1, which was closed shortly after creation without review or being merged upstream.

I’d like to add a regression test and submit a minimal backward-compatible fix. Would this still be useful? Thanks!