Issue #8, convert hunspell output to spellchecker JSON format by teners · Pull Request #9 · bardsoftware/papeeria

teners · 2016-08-10T17:32:54Z

No description provided.

dbarashev · 2016-08-10T23:59:01Z

backend/spellchecker/huntojson.sh

+done
+shift $((OPTIND-1))
+
+if [ "$#" != 1 ]


any sense to surround 1 with double-quotes, to compare string and string?

fixed hunspell options to interpret input as latex file

dbarashev · 2016-08-11T12:30:51Z

backend/spellchecker/huntojson.sh

+INFILE=$1
+
+JSON=$(
+    cat $INFILE    | 


Enquote $INFILE, otherwise you're going have troubles with spaces in the path

dbarashev · 2016-08-11T12:53:18Z

One interesting question is duplicate keys:
echo "documeent documeent documeent" | hunspell -a

This script will produce JSON with three equals keys. I suggest filtering duplicates with | jq '.'

dbarashev · 2016-08-11T12:54:20Z

backend/spellchecker/huntojson.sh

+        options_number=split(split_string[2], options, ", ");
+
+        for (i = 1; i <= options_number; i++) 
+            {print "\t\t\""options[i]"\","} 


Comma after the last element of the array produces invalid JSON, at least from jq perspective.The same applies to the commma after the last entry

removed "pretty-printing efforts"

fixed it with an additional variable in awk

dbarashev · 2016-08-11T21:00:09Z

backend/spellchecker/huntojson.sh


-echo $JSON
+echo "$JSON" | 
+sed ':a;N;$!ba;s/,\n}/\n}/g' 


This can probably be better solved by introducing a global variable which is empty in the beginning and "," after the first use. Use it like a prefix of the serialized array:

print $2 + PREFIX + [

Heck, that's a really good point. Shame on me for such a dummy solution.

Merge branch 'tkt_8_convert_hunspell_to_json' of https://github.com/bardsoftware/papeeria into tkt_8_convert_hunspell_to_json

dbarashev · 2016-08-31T14:20:07Z

backend/spellchecker/proto/spellchecker.proto

+}
+
+message Suggestions {
+    string json = 1;


one of the purposes of protocol buffers is to provide typed interface for data exchange between servers. Please replace this json with typed interface, like map<string, Replacements> suggestions, use it in the spellchecker and remove your own json-serialization

Makes a lot of sense, right.

dbarashev · 2016-08-31T14:36:01Z

backend/spellchecker/server/parser.py

+        # note: these fields' names doesn't start with "_" since deleting of this 
+        # object isn't so straightforward -- for some reason interpreter destroys 
+        # these "private" fields before __del__ is invoked.
+        self.parser_lib_ = cdll.LoadLibrary("../libparser/build/libparser.so")


what's the purpose of the trailing underscores?

also, please parameterize this class with the path to the shared library

Oh, I thought I removed them.

Yep, that's what I forgot to do.

dbarashev · 2016-08-31T14:41:55Z

backend/spellchecker/server/server.py

+
+class SpellcheckerServicer(spellchecker_pb2.SpellcheckServicer):
+    """
+    gRPC service to check text that comes from stubs.


it is service client who sends you the text, not a stub. Stub is a purely technical thing which provides typed interface on the client side.

dbarashev · 2016-08-31T21:14:33Z

backend/spellchecker/proto/spellchecker.proto

 message Suggestions {
-    string json = 1;
+    message Suggestion {
+        string key = 1;


you don't need the key anymore

dbarashev · 2016-08-31T21:30:33Z

[bard@bardtop3 build (tkt_8_convert_hunspell_to_json)]$ make
CMake Error: The source directory "/home/tener/git/papeeria/backend/spellchecker/libparser" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.

teners · 2016-08-31T21:44:48Z

@dbarashev meh, didn't realize that CMake generates such Makefiles, I'll remove the Makefile. Of course you want to use cmake.

dbarashev · 2016-09-01T01:01:44Z

backend/spellchecker/server/server.py

+                          type=str,
+                          metavar="PATH",
+                          help="path to .dic and .aff files")
+    required.add_argument("-L", "--language",


language is defined in the request, no?

Request from client? Nope, in the request we ask spellchecker to use those languages and that demand is satisfyed only if spellchecker already has hunspell instance initialized with a dictionaries of that languages.

which means that we're restricted to one language, since this argument is single-valued.

Why don't you initialize hunspell instance with the languagw requested bythe client on demand?

Alrighty then, I'll work that out.

Initial commit

7833457

teners self-assigned this Aug 10, 2016

dbarashev reviewed Aug 10, 2016
View reviewed changes

Removed output file option, added corrections with spaces support,

a197b6e

fixed hunspell options to interpret input as latex file

dbarashev reviewed Aug 11, 2016
View reviewed changes

root and others added 2 commits August 11, 2016 16:34

Fixed duplicates issue, comma-after-last-element issue,

ccada06

removed "pretty-printing efforts"

Removed dummy sed solution for the last comma issue,

d225ac5

fixed it with an additional variable in awk

dbarashev reviewed Aug 11, 2016
View reviewed changes

teners added 6 commits August 12, 2016 22:39

Deleted shell script and readme, added gitignore for .idea stuff

d1a31dd

Iniital commit for Java implementation

768d0e2

Merged something for some reason

2429ed8

Merge branch 'tkt_8_convert_hunspell_to_json' of https://github.com/bardsoftware/papeeria into tkt_8_convert_hunspell_to_json

Removed Java spellchecker implementation

f3f1597

Initial commit for python grpc version

1a2ee88

Deleted .pyc file and added that extension to gitignore

039864a

dbarashev reviewed Aug 31, 2016
View reviewed changes

Added author's name, removed unneeded test.py

54a60d2

dbarashev reviewed Aug 31, 2016
View reviewed changes

teners added 3 commits August 31, 2016 18:10

Replaced json-string with typed interface in .proto file

81adc92

Added path to shared library as constructor argument in Parser

77b1f9f

Replaced json with protobuf interface, fixed small flaws

893fca6

dbarashev reviewed Aug 31, 2016
View reviewed changes

teners added 5 commits September 1, 2016 02:48

Fixed Suggestions interface in proto file

8c0e6fd

Replaced all string literals with command line arguments, minor fixes

9e4af1a

Fixed docstring

01d8e7a

Reformatted parser.cpp

9e79d4d

Removed unused ujson import

eb1d37e

dbarashev reviewed Sep 1, 2016
View reviewed changes

teners added 3 commits September 1, 2016 04:08

Added requirements.txt

d8aaff2

Moved new-hunspell-instance-initialization on request from check_text

9ae2d84

Removed "language" command line argument

a7e2db5

Comments

Conversation

teners commented Aug 10, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dbarashev commented Aug 11, 2016

Uh oh!

dbarashev Aug 11, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dbarashev commented Aug 31, 2016

Uh oh!

teners commented Aug 31, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dbarashev Aug 11, 2016 •

edited

Loading