Question about the randomness of Gumbel softmax in Differentiable Reward Optimization (DiffRO)

[kayden-kim0702]

I am currently reproducing the Cosy3 model and have a question regarding the DiffRO process described in your paper.

When I applied Gumbel-Softmax, the results changed depending on the randomness of the sampling in Gumbel-Softmax, even though the input features were the same. Could you provide any tips on how to handle this randomness?
for example, adjusting temperature of gumbel softmax ?

[gaochangfeng]

This randomness is necessary, it can sample output token according to the probability similar to the softmax distribution . You can see details in https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.gumbel_softmax.html
For our training, we set the temperature to 1.0.