Skip to content

UnicodeDecodeError with vcrpy 6.0.2 when using httpx #893

Open
@ramnes

Description

@ramnes

Hey there, thanks for the great project!

I'm the maintainer of notion-sdk-py and when I update vcrpy to 6.0.2 for our tests, I can't use the cassettes it creates anymore. I can still successfully create the cassettes through pytest-vcr, but using them gives UnicodeDecodeErrors.

I first thought it would be an issue with pytest-vcr (especially as it's unmaintained), but replacing it with pytest-recording doesn't help, hence this issue. If it's an issue with both pytest-vcr and pytest-recording, just tell me and I'll open the issues accordingly. :)

Here is a PR that showcases the diff and the resulting CI errors:

Example error in spoiler
______________________________ test_is_full_user _______________________________

client = <notion_client.client.Client object at 0x7f41d6e502d0>

    @pytest.mark.vcr()
    def test_is_full_user(client):
>       response = client.users.me()

tests/test_helpers.py:213: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
notion_client/api_endpoints.py:267: in me
    return self.parent.request(
notion_client/client.py:194: in request
    return self._parse_response(response)
notion_client/client.py:129: in _parse_response
    body = response.json()
/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/httpx/_models.py:832: in json
    return jsonlib.loads(self.content, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

s = b"\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03l\x8fM\x8a\xc30\x14\x83\xef\xa2\xb5\x1f\xb8\xb5\xeb\xa4\xbeL\xf0\xcf3t\x9a\x...98M\xf4\xd2P\x18\x83'\x1d} o\x9c\xa2\x834\xde\xc4t\x8c:I\xec\xbf\x00\x00\x00\xff\xff\x03\x00\x12>\x87B\xfe\x00\x00\x00"
cls = None, object_hook = None, parse_float = None, parse_int = None
parse_constant = None, object_pairs_hook = None, kw = {}

    def loads(s, *, cls=None, object_hook=None, parse_float=None,
            parse_int=None, parse_constant=None, object_pairs_hook=None, **kw):
        """Deserialize ``s`` (a ``str``, ``bytes`` or ``bytearray`` instance
        containing a JSON document) to a Python object.
    
        ``object_hook`` is an optional function that will be called with the
        result of any object literal decode (a ``dict``). The return value of
        ``object_hook`` will be used instead of the ``dict``. This feature
        can be used to implement custom decoders (e.g. JSON-RPC class hinting).
    
        ``object_pairs_hook`` is an optional function that will be called with the
        result of any object literal decoded with an ordered list of pairs.  The
        return value of ``object_pairs_hook`` will be used instead of the ``dict``.
        This feature can be used to implement custom decoders.  If ``object_hook``
        is also defined, the ``object_pairs_hook`` takes priority.
    
        ``parse_float``, if specified, will be called with the string
        of every JSON float to be decoded. By default this is equivalent to
        float(num_str). This can be used to use another datatype or parser
        for JSON floats (e.g. decimal.Decimal).
    
        ``parse_int``, if specified, will be called with the string
        of every JSON int to be decoded. By default this is equivalent to
        int(num_str). This can be used to use another datatype or parser
        for JSON integers (e.g. float).
    
        ``parse_constant``, if specified, will be called with one of the
        following strings: -Infinity, Infinity, NaN.
        This can be used to raise an exception if invalid JSON numbers
        are encountered.
    
        To use a custom ``JSONDecoder`` subclass, specify it with the ``cls``
        kwarg; otherwise ``JSONDecoder`` is used.
        """
        if isinstance(s, str):
            if s.startswith('\ufeff'):
                raise JSONDecodeError("Unexpected UTF-8 BOM (decode using utf-8-sig)",
                                      s, 0)
        else:
            if not isinstance(s, (bytes, bytearray)):
                raise TypeError(f'the JSON object must be str, bytes or bytearray, '
                                f'not {s.__class__.__name__}')
>           s = s.decode(detect_encoding(s), 'surrogatepass')
E           UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/json/__init__.py:341: UnicodeDecodeError

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions