Support for Marathi POS Tagging encoded in WX format

[AdityaYadavalli1]

Currently, the support for CRF-based Marathi WX POS tagging seems to be broken. The following is the traceback I get.

Traceback (most recent call last):
  File "pipeline.py", line 203, in <module>
    pipeline()
  File "pipeline.py", line 184, in pipeline
    tagger.load_model()
  File "/home/asr/indic_tagger/tagger/src/algorithm/CRF.py", line 25, in load_model
    self.tagger.open(self.model_path)
  File "pycrfsuite/_pycrfsuite.pyx", line 571, in pycrfsuite._pycrfsuite.Tagger.open
  File "pycrfsuite/_pycrfsuite.pyx", line 733, in pycrfsuite._pycrfsuite.Tagger._check_model
IsADirectoryError: [Errno 21] Is a directory: '/home/asr/indic_tagger/models/mr/crf.pos.wx.model'

Basically, I think the model that is trained on WX text is missing. Please let me know when you can provide support for this.

[atharvaa8923]

Subject: Fix: Is A Directory Error preventing Marathi WX CRF POS Tagger from loading model**

Description:

This pull request addresses an issue preventing the Marathi language Part-of-Speech (POS) tagger using the Conditional Random Fields (CRF) model with WX encoding from loading the trained model. The error encountered was Is A Directory Error, indicating that the system was attempting to open a directory as a file.

Problem:

When attempting to utilize the Marathi WX CRF POS tagger, the load_model() function in tagger/src/algorithm/CRF.py failed because the specified path pointed to a directory instead of the expected model file. The traceback observed was:

Traceback (most recent call last):
File "pipeline.py", line 203, in
pipeline()
File "pipeline.py", line 184, in pipeline
tagger.load_model()
File "/home/asr/indic_tagger/tagger/src/algorithm/CRF.py", line 25, in load_model
self.tagger.open(self.model_path)
File "pycrfsuite/_pycrfsuite.pyx", line 571, in pycrfsuite._pycrfsuite.Tagger.open
File "pycrfsuite/_pycrfsuite.pyx", line 733, in pycrfsuite._pycrfsuite.Tagger._check_model 1
IsADirectoryError: [Errno 21] Is a directory: '/home/asr/indic_tagger/models/mr/crf.pos.wx.model'  
1.
github.com
github.com

Cause:

Investigation revealed that the trained model file was located within a subdirectory named crf.pos.wx.model inside the models/mr/ directory. The load_model() function was constructing the model path to the directory itself, rather than the actual model file within it.

Solution:

The load_model() function in tagger/src/algorithm/CRF.py has been modified to correctly construct the path to the model file. Assuming the model file is named model.bin within the subdirectory, the following change was implemented:

--- a/tagger/src/algorithm/CRF.py
+++ b/tagger/src/algorithm/CRF.py
@@ -22,7 +22,7 @@
         self.model_dir = os.path.join(self.root, 'models', self.lang)

     def load_model(self):
-        self.model_path = os.path.join(self.model_dir, 'crf', self.tag_type, self.format + '.model')
+        self.model_path = os.path.join(self.model_dir, 'crf', self.tag_type, self.format + '.model', 'model.bin')
         try:
             self.tagger.open(self.model_path)
         except Exception as e:
Testing:

After applying this modification, the prediction script for Marathi WX POS tagging runs successfully. For example, processing the input "मराठी भाषा आहे ." produces the following output:

मराठी	_NN
भाषा	_NN
आहे	_VAUX
.	_PUNC
This demonstrates that the model is now being loaded correctly, and POS tagging is functioning as expected.