Problem running -m test with held out dataset

[jjdevega]

We have generated a prototype subset to use the -m test mode after training the models.
The training works ok, then we run test with the same config file, but changing the data and metadata to the held out subset

We are getting the error pasted below, we have tried multiple times and options resulting in the same error.
We have tried different inputs, but we finished with the same error. I am wondering if we are failing to provide the correct options for the test mode in the config file. Thanks

$ singularity exec ~/autoxai4omics-1.0.0.img autoxai4omics.sh -m test -c multiclass_eval.json
Getting flags
Registering config
/usr/local/lib/python3.9/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
from pandas import MultiIndex, Int64Index
/usr/local/lib/python3.9/site-packages/sklearn/utils/validation.py:70: FutureWarning: Pass n_repeats=1, random_state=None as keyword args. From version 1.0 (renaming of 0.25) passing these as positional arguments will result in an error
warnings.warn(f"Pass {args_msg} as keyword args. From version "
/usr/local/lib/python3.9/site-packages/sklearn/utils/validation.py:70: FutureWarning: Pass n_splits=3, n_repeats=1 as keyword args. From version 1.0 (renaming of 0.25) passing these as positional arguments will result in an error
warnings.warn(f"Pass {args_msg} as keyword args. From version "
experiments/results/seed_classifier_prototype1 exists - results may be overwritten!
Output file: ./data/1_gt_mc_processed.csv
2024-10-21 15:21:19 ERROR Invalid file path or buffer object type: <class 'NoneType'>
Traceback (most recent call last):
File "/opt/software/AutoXAI4Omics/src/mode_testing_holdout.py", line 49, in
x_heldout, y_heldout, features_names = utils.load.load_data(
File "/opt/software/AutoXAI4Omics/src/utils/load.py", line 243, in load_data
x_heldout, y_heldout, features_names = load_data_holdout(config_dict)
File "/opt/software/AutoXAI4Omics/src/utils/load.py", line 165, in load_data_holdout
x_heldout, y_heldout, features_names = tabular.get_data_tabular(
File "/opt/software/AutoXAI4Omics/src/omics/tabular.py", line 58, in get_data_tabular
filtered_data, genestokeep = rrep.preprocessing_LO(
File "/opt/software/AutoXAI4Omics/src/omics/R_replacement.py", line 42, in preprocessing_LO
data_file = pd.read_csv(data_dict[file], index_col=0) # sampleID as index
File "/usr/local/lib/python3.9/site-packages/pandas/util/_decorators.py", line 211, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/pandas/util/_decorators.py", line 317, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 950, in read_csv
return _read(filepath_or_buffer, kwds)
File "/usr/local/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 605, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/usr/local/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1442, in init
self._engine = self._make_engine(f, self.engine)
File "/usr/local/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1729, in _make_engine
self.handles = get_handle(
File "/usr/local/lib/python3.9/site-packages/pandas/io/common.py", line 714, in get_handle
ioargs = _get_filepath_or_buffer(
File "/usr/local/lib/python3.9/site-packages/pandas/io/common.py", line 452, in _get_filepath_or_buffer
raise ValueError(msg)
ValueError: Invalid file path or buffer object type: <class 'NoneType'>
Traceback (most recent call last):
File "/opt/software/AutoXAI4Omics/src/mode_testing_holdout.py", line 167, in
raise e
File "/opt/software/AutoXAI4Omics/src/mode_testing_holdout.py", line 49, in
x_heldout, y_heldout, features_names = utils.load.load_data(
File "/opt/software/AutoXAI4Omics/src/utils/load.py", line 243, in load_data
x_heldout, y_heldout, features_names = load_data_holdout(config_dict)
File "/opt/software/AutoXAI4Omics/src/utils/load.py", line 165, in load_data_holdout
x_heldout, y_heldout, features_names = tabular.get_data_tabular(
File "/opt/software/AutoXAI4Omics/src/omics/tabular.py", line 58, in get_data_tabular
filtered_data, genestokeep = rrep.preprocessing_LO(
File "/opt/software/AutoXAI4Omics/src/omics/R_replacement.py", line 42, in preprocessing_LO
data_file = pd.read_csv(data_dict[file], index_col=0) # sampleID as index
File "/usr/local/lib/python3.9/site-packages/pandas/util/_decorators.py", line 211, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/pandas/util/_decorators.py", line 317, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 950, in read_csv
return _read(filepath_or_buffer, kwds)
File "/usr/local/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 605, in _read
parser = TextFileReader(filepath_or_buffer, **kwds)
File "/usr/local/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1442, in init
self._engine = self._make_engine(f, self.engine)
File "/usr/local/lib/python3.9/site-packages/pandas/io/parsers/readers.py", line 1729, in _make_engine
self.handles = get_handle(
File "/usr/local/lib/python3.9/site-packages/pandas/io/common.py", line 714, in get_handle
ioargs = _get_filepath_or_buffer(
File "/usr/local/lib/python3.9/site-packages/pandas/io/common.py", line 452, in _get_filepath_or_buffer
raise ValueError(msg)
ValueError: Invalid file path or buffer object type: <class 'NoneType'>

proto_eval.metadata.csv
proto_eval.transposed.csv
1_md_mc_processed.csv
1_gt_mc_processed.csv
proto_train.metadata2.csv
proto_train.csv
multiclass_eval.json
multiclass.json