Code to Visualize Attention Weights

[ni9elf]

Need some help in writing the code to obtain and visualize the attention weights like that in the HAN paper (heat map). To obtain the attention weights, I'm currently thinking of obtaining the hidden representations of the GRUs (h_it) and then manually using h_it to compute the attention weights using the equations from the call function of the attention layer.

layer_name = 'GRU'
intermediate_layer_model = Model(input=model.input, output=model.get_layer(layer_name).output)
intermediate_output = intermediate_layer_model.predict(input_variable)
h_it = intermediate_output
#use h_it from above to compute attention weights

If there is a more direct way (direct function call in Keras or some existing code available), it will be helpful.

[PedroCardoso]

This is open for a while, but if you are still interested:

In line 98 you have the attentionSent layer. If you want to see it you can just add it to you output model.
I would advice to keep the "normal" model for training, and create a new one with extra outputs for evaluation :

The way it would look is : model = Model(input=[documentInputs], output=[documentOut, attentionSent])

For attention inside the sentences, you can try the same, but might be more complicated with all the TimeDistributed. Did not try.

FYI: this is code for keras 1.x, not 2