Getting ", or } expected while parsing object/hash" when converting a PrusaSlicer bundle

[jjramsey]

To be specific, the error is , or } expected while parsing object/hash, at character offset 2 (before "(end of string)") at /home/jjramsey/bin/superslicer_to_orca.pl line 1295.

I get this when running the command superslicer_to_orca.pl --input PrusaSlicer_config_bundle.ini.

I've uploaded the config bundle that's caused me problems.

PrusaSlicer_config_bundle_ini.txt

[theophile]

I couldn't reproduce the error, although testing on your config bundle exposed an issue with parsing profile names in config bundles so I've pushed a fix to address that. On my machine, your bundle converts successfully:

Double check and make sure you've got all the necessary perl modules installed:
cpan Getopt::Long File::Basename File::Glob File::HomeDir Path::Class Path::Tiny String::Escape Term::Choose Term::Form::ReadLine Text::SimpleTable JSON

[jjramsey]

Even with the fix you made to the script, I still get the error , or } expected while parsing object/hash, at character offset 2 (before "(end of string)") at /home/jjramsey/bin/superslicer_to_orca.pl line 1322. Of course, with your recent changes, the line number where the error happens is different.

On my end, the script seems to choke when it gets to the printer profiles. The print profiles and filament profiles seem to have been parsed without error.

Originally, I had installed all needed Perl packages via CPAN, but this time I installed only Term::Choose and Term::Form::Readline via CPAN, with versions 1.762 and 0.554, respectively. The Perl packages I had installed as regular Arch Linux packages were File::Homedir, version 1.006, Path::Class, version 0.37, and JSON, version 4.10. The packages installed from the AUR were String::Escape, version 2010.002, and Text::SimpleTable, version 2.07.

All other packages seem to have just come with my Perl package. As far as I can tell, Getopt::Long has version 2.54, File::Basename has version 2.86, File::Glob has version 1.40, and Path::Tiny has version 0.144. My Perl executable itself has version 5.38.1.

[theophile]

As best I can figure, the script is choking on one of the .json files located in /home/jjramsey/.config/OrcaSlicer/system/. Please take a look at the files in that directory with a .json extension and see if any of them is missing the final } at the end. If there's only one .json file there, please upload it here.

[jjramsey]

So far, I haven't had any luck finding bad JSON files. I've used this script find possible bad ones:

#!/usr/bin/env python3

import json
import argparse

parser = argparse.ArgumentParser()

parser.add_argument("json_files", nargs = "+",
                    help = "JSON files to parse")

args = parser.parse_args()

for fname in args.json_files:
    with open(fname, "r") as f:
        try:
            json.load(f)
        except Exception as e:
            print(f"Error loading file {fname}")
            print(e)

[theophile]

Try renaming them all to *.json.bak and try the script again. If it works, restore the original .json filenames one at a time to test whether one or more of them is causing the error.

[jjramsey]

FYI, I ran the superslicer_to_orca.pl script through the Perl debugger and got more detailed output. Here it is:

, or } expected while parsing object/hash, at character offset 2 (before "(end of string)") at /home/jjramsey/bin/superslicer_to_orca.pl line 1322.
 at /usr/share/perl5/core_perl/JSON/PP.pm line 1363.
	JSON::PP::decode_error(", or } expected while parsing object/hash") called at /usr/share/perl5/core_perl/JSON/PP.pm line 1143
	JSON::PP::object() called at /usr/share/perl5/core_perl/JSON/PP.pm line 833
	JSON::PP::value() called at /usr/share/perl5/core_perl/JSON/PP.pm line 802
	JSON::PP::PP_decode_json(JSON::PP=HASH(0x564cfe044258), "{\x{a}", 0) called at /usr/share/perl5/core_perl/JSON/PP.pm line 153
	JSON::PP::decode(JSON::PP=HASH(0x564cfe044258), "{\x{a}", "    \"name\": \"Custom Printer\",\x{a}", "    \"version\": \"01.08.00.00\",\x{a}", "    \"force_update\": \"0\",\x{a}", "    \"description\": \"My configurations\",\x{a}", "    \"machine_model_list\": [\x{a}", "        {\x{a}", ...) called at /usr/share/perl5/core_perl/JSON/PP.pm line 116
	JSON::PP::decode_json("{\x{a}", "    \"name\": \"Custom Printer\",\x{a}", "    \"version\": \"01.08.00.00\",\x{a}", "    \"force_update\": \"0\",\x{a}", "    \"description\": \"My configurations\",\x{a}", "    \"machine_model_list\": [\x{a}", "        {\x{a}", "            \"name\": \"Generic Klipper Printer\",\x{a}", ...) called at /home/jjramsey/bin/superslicer_to_orca.pl line 1322
	main::link_system_printer("RatRig V-Core-3-400 - 0.6mm nozzle") called at /home/jjramsey/bin/superslicer_to_orca.pl line 1662

Not sure if that helps.

[jjramsey]

Looks like this is the file that superslicer_to_orca.pl is choking on: Custom.json

[theophile]

Thanks. I'll test with it and see if I can figure out what's going on.

[jjramsey]

Judging from the debugger output, it looks like decode_json is being fed a list of strings "{\x{a}", " \"name\": \"Custom Printer\",\x{a}", etc., instead of a single string, and then passes on only the first element of that list to the internal function PP_decode_json. Maybe Perl sees $file->slurp as operating in a list context instead of a scalar one? Maybe the API of the JSON module got updated so that $file->slurp used to always return a string no matter what the context but does so no longer?

[jjramsey]

Looks like my guess was mostly correct, at least in that $file->slurp as an argument of decode_json was operating in a list context instead of the desired scalar one. My fix was to use the scalar operator to force the intended context where it wasn't already present. I ended up adding scalar in two places, as indicated in this patch: fix-scalar-context_patch.txt

[theophile]

You're right that, according to the debugger output, $file->slurp wasn't operating in scalar context as expected. That's very odd though because it's clearly being called in scalar context. Also, $file is a Path::Class::File object, and it has its own slurp() method that should be operating in scalar context.

I'm not sure why the script would be calling slurp in a list context on your system but not on others. Only thing I can think of is that maybe somehow your perl environment has modified the record separator ($/)?

Anyway, your solution of forcing scalar context should work. I'll test when I get a chance later then push the update. Thanks for your help.

[jjramsey]

I think it was putting $file->slurp in parentheses, or maybe even just having it as a function argument, that led it to be interpreted in list context.

[theophile]

It shouldn't. In that situation, since it's an argument of decode_json(), the decode_json() would determine the context it invokes. This script uses the standard built-in JSON module, which invokes the scalar context. And the result of decode_json() is being assigned to a scalar so that would also invoke the scalar context. Besides, if there were something in the script itself causing slurp() to be invoked in list context, then everyone would be getting this error.

Curious whether you have other .json files in /home/jjramsey/.config/OrcaSlicer/system/ that do not throw this error with the unpatched script. The content of the .json file shouldn't have any effect on whether it's slurped in scalar context or list, but if other files are read correctly, something else may be going on.

[jjramsey]

I think this open issue in the JSON module may shed light on why we got different results: makamaka/JSON#54. If you're using a different parser backend (XS instead of PP), then you shouldn't see the problem that I did.

I suspect the reason the script choked on "Custom.json" is simply because it was the first JSON file read in the link_system_printer subroutine, rather than anything special about its contents.

(ETA: And it turns out that I did not have JSON::XS installed, only the JSON::PP included in Arch Linux's perl-json package.)

[theophile]

Incredible. That's it exactly. The funny thing about this is that I originally developed the script using JSON::XS but decided the built-in module would suffice and eliminate one dependency. I'm thinking that I may go back to requiring JSON::XS. Forcing the scalar context may work in this scenario, but now I'm wondering what other potential pitfalls could be lurking if different JSON backends behave differently. Requiring everyone use the same backend should at least make it easier to test/reproduce any problems that may arise.

Thanks again for your help in investigating this.