UnicodeDecodeError / Change strict_decode value of trim_label function

[afrigeri]

Hi,

from a script, when pdr.read, we got:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa1 in position 11853: invalid start byte

from a pdr-unsupported dataset.

I would like to change the strict_decode argument of the trim_label function from True to False, so that text.decode's errors arg goes from errors="strict" to="replace".

Is there a suggested way to change the value of the trim_label argument when importing the modules in a script?

[m-stclair]

We do not currently expose this in the high-level Data API. This is really only because no one has asked for it, though. As this will be very easy to implement, we will put it on the feature request shortlist.

[afrigeri]

Ah great! looking forward for the new feature.

Maybe we can also include the option to specify the encoding of the input file (e.g. iso-8859-1 instead of utf-8).

[m-stclair]

This one is partly intentional. Because every byte value is valid latin-1, a decode op is not even a soft check for "is this 'text' or 'binary'". And because of this fact, some early versions of pdr could spend painful amounts of time trying to 'parse' massive binary arrays with attached latin-1 headers. So we have historically treated it very, very gingerly.

That really bad behavior is no longer possible, but there still might be some cases in which a latin-1 attached PVL label has a slightly nonstandard end statement, and pdr parses it quickly but incorrectly.

Anyway: with that caveat, we don't mind exposing it as an option. Similarly easy and can be done at the same time. Adding it to the list.