-
Notifications
You must be signed in to change notification settings - Fork 56
Projects
mjpost edited this page Aug 14, 2011
·
20 revisions
-
✓ Fix multithreaded Joshua (sentences should be placed in a queue that threads pop and deposit somewhere; deposits would then be assembled sequentially) --- currently on the fix_threads branch
-
✓ Clean up the input handling routines (HackishSegmentParser, SAXSegmentParser, PlainSegmentParser)
-
Configuration parameters should be overridable from the command line. This is especially true of runtime related parameters such as the number of threads.
- ✓ Rudimentary support has been added for -threads...
- ...but it should be rewritten in a more general fashion: (1) load the configuration file, then (2) process command line arguments and let anything be overridden.
-
Fix KenLM integration
- KenLM typically scores between 0.5 and 1.0 BLEU points less than SRILM using the same model
- fix the vocabulary mapping
- use a proper UNK
- get rid of SRILM
-
Pruning: we should be able to prune simply by specifying a pop-limit on cube growing