Pytorch 1.4+ and no NLTK

• Works with Pytorch versions after Pytorch 1.3
• Reduces memory footprint by converting LongTensors to IntTensors
• Removes NLTK dependency (self-contained tokenization)

2018-2020

This version is the last version fully compatible with models trained before 2021. Updates after this use a different tokenizer and tensor type, which breaks compatibility with earlier models.

EMNLP 2018

This version should enable replication of van Schijndel and Linzen (CogSci 2018,EMNLP 2018,SCiL 2019)