UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 510: ordinal not in range(128)

[BioComSoftware]

Upon starting apache2 on carina-dev, and attempting to access the website, access failed with an error (in /var/log/apache2/error.log) as follows:

Fri Jan 19 13:48:04.425583 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512] mod_wsgi (pid=25443): Target WSGI script '/home/carina/webdorina/webdorina/webdorina.wsgi' cannot be loaded as Python module.
[Fri Jan 19 13:48:04.425711 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512] mod_wsgi (pid=25443): Exception occurred processing WSGI script '/home/carina/webdorina/webdorina/webdorina.wsgi'.
[Fri Jan 19 13:48:04.425785 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512] Traceback (most recent call last):
[Fri Jan 19 13:48:04.426417 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]   File "/home/carina/webdorina/webdorina/webdorina.wsgi", line 16, in <module>
[Fri Jan 19 13:48:04.426475 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]     from webdorina.app import app as application
[Fri Jan 19 13:48:04.426937 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]   File "/home/carina/webdorina/webdorina/app.py", line 41, in <module>
[Fri Jan 19 13:48:04.426997 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]     Regulator.init(app.config['DATA_PATH'])
[Fri Jan 19 13:48:04.427659 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]   File "/usr/local/lib/python3.4/dist-packages/dorina/regulator.py", line 65, in init
[Fri Jan 19 13:48:04.427722 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]     parse_func)
[Fri Jan 19 13:48:04.427957 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]   File "/usr/local/lib/python3.4/dist-packages/dorina/utils.py", line 33, in walk_assembly_tree
[Fri Jan 19 13:48:04.428087 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]     species_dict[assembly] = parse_func(assembly_path)
[Fri Jan 19 13:48:04.428151 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]   File "/usr/local/lib/python3.4/dist-packages/dorina/regulator.py", line 55, in parse_func
[Fri Jan 19 13:48:04.428304 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]     experiments = parse_experiment(experiment_path)
[Fri Jan 19 13:48:04.428372 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]   File "/usr/local/lib/python3.4/dist-packages/dorina/regulator.py", line 32, in parse_experiment
[Fri Jan 19 13:48:04.428718 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]     return json.load(fh)
[Fri Jan 19 13:48:04.429431 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]   File "/usr/lib/python3.4/json/__init__.py", line 265, in load
[Fri Jan 19 13:48:04.429514 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]     return loads(fp.read(),
[Fri Jan 19 13:48:04.429729 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]   File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
[Fri Jan 19 13:48:04.429855 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512]     return codecs.ascii_decode(input, self.errors)[0]
[Fri Jan 19 13:48:04.429918 2018] [wsgi:error] [pid 25443:tid 140426388211456] [remote 129.206.148.116:512] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 510: ordinal not in range(128)

[BioComSoftware]

This is an issue of conversion between Python2.7 and Python3.

Within /usr/local/lib/python3.4/dist-packages/dorina/regulator.py the parse_func(root) whereroot is the path to a json config file (in this case the file was /home/carina/webdorina/data/regulators/h_sapiens/hg19/HITSCLIP_DGCR8Macias2012c_hg19.json ... although the file itself is not the problem)

This method then calls the parse_experiment(filename) method using the above filename.

In Python2.7 filename is a ASCII string. Python3 expects this to be bytestring. Thus, in line 31 of regulator.py the code...

            with open(filename) as fh:
                return json.load(fh)

...attempts to open filename as a textstring. The corrected code...

        with open(filename, "rb") as fh:
            return json.load(fh)

... tells Python to open this as a "readbytes"...and the error disappears.

UNFORTUNATELY...this leaves Dorina in need of refactoring.

Because although that error is gone, the return return json.load(fh) chokes because "fh" is now a bytes and not a ASCII text file handle.

Error...

TypeError: the JSON object must be str, not 'bytes'

[Fri Jan 19 15:48:43.504166 2018] [wsgi:error] [pid 30321:tid 140037985048320] [remote 129.206.205.123:512]     experiments = parse_experiment(experiment_path)
[Fri Jan 19 15:48:43.504214 2018] [wsgi:error] [pid 30321:tid 140037985048320] [remote 129.206.205.123:512]   File "/usr/local/lib/python3.4/dist-packages/dorina/regulator.py", line 32, in parse_experiment
[Fri Jan 19 15:48:43.504242 2018] [wsgi:error] [pid 30321:tid 140037985048320] [remote 129.206.205.123:512]     return json.load(fh)
[Fri Jan 19 15:48:43.505004 2018] [wsgi:error] [pid 30321:tid 140037985048320] [remote 129.206.205.123:512]   File "/usr/lib/python3.4/json/__init__.py", line 268, in load
[Fri Jan 19 15:48:43.505072 2018] [wsgi:error] [pid 30321:tid 140037985048320] [remote 129.206.205.123:512]     parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
[Fri Jan 19 15:48:43.505126 2018] [wsgi:error] [pid 30321:tid 140037985048320] [remote 129.206.205.123:512]   File "/usr/lib/python3.4/json/__init__.py", line 312, in loads
[Fri Jan 19 15:48:43.505179 2018] [wsgi:error] [pid 30321:tid 140037985048320] [remote 129.206.205.123:512]     s.__class__.__name__))
[Fri Jan 19 15:48:43.505480 2018] [wsgi:error] [pid 30321:tid 140037985048320] [remote 129.206.205.123:512] TypeError: the JSON object must be str, not 'bytes'