How to build gloss2ids.pkl ?

[enrico310786]

Hi,

thanks for sharing the code of this interesting work. I am trying to reproduce the code on my own dataset, whose struicture is similar to the Phoenix_2014T. Thus, given the files phoenix-2014t.train, phoenix-2014t.dev and phoenix-2014t.test with gloss annotation, how did you built the gloss2ids.pkl file? I see that inside there are additional tokens beyond the standard glosses so I don't want to be wrong. Could you share the script with which you generate your file?

Thanks in advance for your answer,
Enrico