'patience' count resets when tuning with Hyperband

[mountainML]

I'm using keras-tuner to perform a hyperparameter optimization of a neural network by using Hyperband as:

import keras_tuner as kt
tuner = kt.Hyperband(ann_model,
                     objective=Objective('val_loss', direction="min"),
                     max_epochs=100, factor=2, 
                     directory=/path/to/folder,
                     project_name="project_name", seed=0)
tuner.search(training_gen(), epochs=50, validation_data=valid_gen(),
             callbacks=[stop_early], steps_per_epoch=1000,
             validation_freq=1, validation_steps=100)

where I defined the EarlyStopping callback as:

stop_early = tf.keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0.1,
                                          mode='min', patience=15)

By setting max_epochs=100, I noticed that (during the Hyperband algorithm) the training of a specific model is not performed in one shot, but it's perfomed by steps ("Runnning trials").

1. firstly, the model is trained from epoch 1 to epoch 3;
2. secondly, from epoch 4 to epoch 7;
3. then, from 8 to 13;
4. then, from 14 to 25;
5. then, from 26 to 50;
6. and finally, from 51 to 100.

So, at the end of each "Running trial", Keras saves the state of the training, in order to continue, at the next "Running trial", the training from that state.

By setting patience=15:

I noticed that the EarlyStopping callback stops the training during "Running trial" 5), in particular at epoch 41 (the first epoch at which EarlyStopping is able to operate, because: start_epoch + patience = 26 + 15 = 41).

Then, the training continues in the "Running trial" 6), and EarlyStopping arrests the training at epoch 66 (thus, at the epoch: start_epoch + patience = 51 + 15 = 66).

Thus:

• even though EarlyStopping stops training during running trial 5), the training continues in running trial 6);
• the patience count, resets itself at the beginning of each "Running trial" (I thought that it should be started at epoch 1 and should never reset itself).

Is it the normal/expected behavior that the patience count automatically resets itself at the beginning of each "Running trial" when tuning with Keras Hyperband?