Bug in routines.split_sets

[ak-nss]

train_pats_list = random.sample(patient_list, int(np.round(len(patient_list)*0.75)))
val_pats_list = random.sample(patient_list, int(np.round(len(patient_list)*0.15)))
test_pats_list = random.sample(patient_list, int(np.round(len(patient_list)-len(train_pats_list)-len(val_pats_list))))

train_pats_list, val_pats_list, test_pats_list with a high probability will overlap, which will cause target leakage and incorrect metric values.

[ak-nss]

I would write this code like this

random.seed(random_seed)
random.shuffle(patient_list)
n_pats = len(patient_list)
train_idx = int(np.round(n_pats * 0.75))
val_idx = train_idx + int(np.round(n_pats * 0.15))
train_pats_list = patient_list[:train_idx]
val_pats_list = patient_list[train_idx:val_idx]
test_pats_list = patient_list[val_idx:]

[whubaichuan]

@ak-nss Thanks for your solution. I can provide an alternative solution with random selection.

    train_pats_list = random.sample(patient_list, int(np.round(len(patient_list)*0.75)))
    val_pats_list = random.sample(set(patient_list)-set(train_pats_list), int(np.round(len(patient_list)*0.15)))
    test_pats_list = set(patient_list)-set(train_pats_list)-set(val_pats_list)

[ak-nss]

In your new code test_pats_list will be of type Set

[whubaichuan]

@ak-nss Yes, but it does not influence something. Because the test_pas_list is only used here elif pat in test_pats_list: and it works actually.