Reusing weights when training on dataset with different number of classes

[ajuric]

I have the following scenario.

I train the model on some dataset and save the weights of the trained model. Later, I would like to reuse those weights as a good starting point for training on another dataset which has a different number of classes.

Is there an option to load weights but ignore only weights in the last layer (because the models for those two datasets differ only in the last layer)? If no, is there a plan to add it?

[qubvel]

Hi @ajuric
You can do it another way around:

# Loading trained model
base_model = load_model('path/to/trained/model.h5')

# Modify base model (change number of output classes)
x = base_model.layers[-2].output  # remove activation and last conv layer
x = keras.layers.Conv2D(n_classes, (1, 1))(x)
output = keras.layers.Activation(<activation_name>)(x)

new_model = keras.models.Model(base_model.input, output)

P.S. May be it is a good feature to add 😄

[ajuric]

The procedure you mention is for the case when you save the whole model.

But, if you save only weights, you cannot use load_model.

[ajuric]

If you want to read only weights, maybe you could create a new model. And then just call load_weights method on that model.

But the problem with this is that your new model and weights from old model have different number of output classes, so I assume that this method would fail at last layer.

Maybe workaround could be this: https://stackoverflow.com/questions/51173580/how-to-read-keras-model-weights-without-a-model
You read the weights file and then manually set the weights of the model layers (model.layers[i].set_weights()) except for the last conv layer?

[qubvel]

just create the previous model and load weights, then modify it.

[Dendrophylax]

What about using "by_name" and "skip_mismatch" when loading?
model.load_weights('seresnet34_imagenet_1000_no_top.h5', by_name=True, skip_mismatch=True)