tokenizer not correctly splitting some contractions

[francolq]

The tokenizer is not following standard contraction tokenization [0], expected by the Stanford POS tagger. Contractions are not splitted and should be.

Also, the apostrophe character ´ is not handled.

[0] http://www.cis.upenn.edu/~treebank/tokenization.html

[francolq]

Reproducing:

from iepy.tokenizer import en_tokenize_and_segment
s = u"The bleeding wasn't from a miscarriage and she still tests positive for pregnancy."
en_tokenize_and_segment(s)['tokens']
[u'The', u'bleeding', u"wasn't", u'from', u'a', u'miscarriage', u'and', u'she', u'still', u'tests', u'positive', u'for', u'pregnancy', u'.']
s = u"The bleeding wasn’t from a miscarriage and she still tests positive for pregnancy."
en_tokenize_and_segment(s)['tokens']
[u'The', u'bleeding', u'wasn', u'\u2019t', u'from', u'a', u'miscarriage', u'and', u'she', u'still', u'tests', u'positive', u'for', u'pregnancy', u'.']