Reproducibility and Perplexity calculation on test data

[Eirsteir]

I've noticed that some CRSs like TG-ReDial and INSPIRED do not calculate Perplexity on the test dataloaders in their current implementation in CRSLab. Therefore, I wondered how you calculated the PPL results in the README.

For now, I've changed these system classes to call the same procedure for training and validation to calculate PLL for the test data. However, that results in a very high BLEU@1 of 0.5841 when using the default config and training for 5 epochs, which is inconsistent with your reports.

How did you calculate PPL in your testing and what might be the issue here?