Keras implementation?

[michetonu]

Hey,

First of all thanks a lot for this. I was wondering whether there is an easy way to make the gradient flipping work in Keras. Someone has done it for the Theano backend, but not for the Tensorflow. Would it be feasible to combine the two?

Thanks!

[pumpikano]

I see two options, but I'm not knowledgeable enough with Keras to judge how easy they would be.

First, you could define a GradientReversal layer in Keras - this PR attempted that in the Keras 1.0 API, but was never finished. I believe this is feasible and would be best in terms of usability by other Keras users.

Second, I believe that Keras 2.0 and TF can interoperate relatively seamlessly, so you could use Keras to define most of your model and use TF for the gradient reversal portion. Here is a simple example of mixing the two. Again, I haven't used this myself, but it seems like a feasible option.

[kilickaya]

Adding to pumpikano's answer, you can implement gradient reversal layer as 'maximizing' the objective of interest rather than minimization. This can be implemented as minimizing the negative of the cost function in Tensorflow, and could be very easy with Keras also. So your objective may look like this:

min (object_cost - domain_cost)

where you favor good object recognition (object_cost) and hurt your model's ability to differentiate images from two different domains (domain_cost).

This leaves no need for explicitly implementing Gradient Reversal Layer, as it also achieves the same objective after all.

[michetonu]

@pumpikano Thanks, that looks feasible - I'll take a look and keep you updated.

@kilickaya That's true! Should I then basically just define that as a custom loss function for the domain adaptation part? Would that produce the same output as the paper?

[kilickaya]

In the paper, the Gradient Reversal Layer is used to go in reverse direction of Gradients. This can be achievable in two ways. You can either reverse the sign of gradients and minimize the same objective, or you can reverse the loss function, as I mentioned above. Implementation wise it is slightly different, but they serve for the same purpose (optimizes the same objective).

Either the authors did not recognize that while writing the paper, that this idea is that simple to implement, or they wanted to make it look more complex than necessary.

In their later manuscript, they also state that this can be implemented by maximization.

I don't know about Keras, but you can implement in Tensorflow like this (assume you use Adam optimizer):

optimizer = tf.train.AdamOptimizer(learning_rate=learningrate).minimize(object_cost - domain_cost, var_list= vars)

[pumpikano]

Just to clarify, you will need two alternating optimizations with this approach

min_D(domain_cost)
min_P(object_cost - domain_cost)

where D is the domain prediction subnetwork, P is the class prediction subnetwork, and min_X means "minimize w.r.t. function X". This is a classic GAN setup.

It is worth noting that it also gives you the freedom to allow the domain cost to differ between the two optimizations. For instance, many GAN implementations actually minimize the discriminator objective with flipped labels rather than maximizing it with correct labels. This amounts to a different objective and usually works better in practice in GANS.

[michetonu]

Could I just define a custom loss such as:

def custom_categorical_crossentropy(y_true, y_pred):
    return -K.categorical_crossentropy(y_pred, y_true)

?

@pumpikano what exactly do you mean by 'alternating'? I would just use one cost function in the label branch and the other one in the domain subbranch, correct?

[pumpikano]

I just mean that usually you will take a step with one loss function and then a step with the other for each batch, usually updating the discriminator first.

[michetonu]

@pumpikano So I managed to implement the layer for the TF backend expanding on the link you provided, you can find it here: https://github.com/michetonu/gradient_inversion_keras_tf/blob/master/flipGradientTF.py

Regarding the loss function steps, Keras works with multi-output models, and I think the loss functions are just additive. Correct me if I'm wrong, but I think I can just create one model (it seems to work). The way I'm doing it is I'm feeding both distributions as input, but setting the target samples' loss weights to zero in the classifier, so that they don't get considered in the backprop update, while removing the need to alternate inputs. I'll happily share my code once I'm sure it works properly :)

[pumpikano]

Cool! Yeah that seems like a reasonable approach.

@michetonu Did you got your code working properly? Would love a Keras DANN implementation example. (Especially how you set the target samples' loss weights to zero in the classifier)

[michetonu]

@Wojova yes! Here is the gradient inversion layer: https://github.com/michetonu/gradient_reversal_keras_tf

For the sample weights you just need to create your input batches by alternating samples from the source and target domain, and pass a sample_weights array of 1s and 0s accordingly.

Thanks!

[erlendd]

Responding to some of the previous comments on here, I don't think that simply reversing the loss function (on the domain part of the network) is a replacement for the gradient reversing layer. The reason is that simply maximising the domain loss won't necessarily lead to domain-invariant features in the shared feature layer, as the weights of the domain-classifier layer will be able to give a bad loss even if the shared feature layer isn't domain invariant.

[michetonu]

@erlendd if you read the original al paper by Ganin et al. (and the few other papers implementing the same domain adversarial training approach), that's exactly what the gradient inversion layer does. It does nothing on the forward pass, and it multiplies the loss by a negative constant during backprop.

[erlendd]

@michetonu yeah I know, I was replying to @kilickaya 's comment above.

Am I correct in saying that the implementation of DANN here using the target labels during training? I.e. it isn't unsupervised?