Skip to content

UnicodeDecodeError when using MsgPackSerializer #834

Open
@nottmtt

Description

@nottmtt

Hi!

When using MsgPack as the (de)serializer for aiocache (no matter the backend), the following error occurs when reading an item from the cache:

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 in position 0: invalid start byte

(I'm using Python 3.12.2, aiocache 0.12.2, aiomcache 0.8.1, redis 4.6.0 and msgpack 1.0.8.)

The error seems to stem from the fact that most backends (prematurely?) decode values from the cache before passing them to the (de)serializer (see backends/memcached.py:24). Replacing this line with return value makes it work (for MsgPack, at least).

But I guess simply removing the value decoding logic isn't a solution. If anyone can give me a clue of what's going wrong here (besides the value.decode()), I'll gladly create a merge request with (my attempt at) a fix.

Minimal test case (using MemcachedCache, use RedisCache if you wish to):

import asyncio

from aiocache               import MemcachedCache
from aiocache.serializers   import MsgPackSerializer

async def test():
    cache = MemcachedCache(
        serializer = MsgPackSerializer(),
    )

    await cache.set("test_case", { "key1": "test", "key2": 3 })

    test_case = await cache.get("test_case") # UnicodeDecodeError here
    print(test_case)

if __name__ == "__main__":
    asyncio.run(test())

Full error:

Traceback (most recent call last):
  File "/home/spike/aiocache_test.py", line 17, in <module>
    asyncio.run(test())
  File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/base_events.py", line 685, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/spike/aiocache_test.py", line 13, in test
    test_case = await cache.get("test_case")
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/spike/.cache/pypoetry/virtualenvs/api--KCnhHZq-py3.12/lib/python3.12/site-packages/aiocache/base.py", line 64, in _enabled
    return await func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/spike/.cache/pypoetry/virtualenvs/api--KCnhHZq-py3.12/lib/python3.12/site-packages/aiocache/base.py", line 48, in _timeout
    return await asyncio.wait_for(func(self, *args, **kwargs), timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/tasks.py", line 520, in wait_for
    return await fut
           ^^^^^^^^^
  File "/home/spike/.cache/pypoetry/virtualenvs/api--KCnhHZq-py3.12/lib/python3.12/site-packages/aiocache/base.py", line 78, in _plugins
    ret = await func(self, *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/spike/.cache/pypoetry/virtualenvs/api--KCnhHZq-py3.12/lib/python3.12/site-packages/aiocache/base.py", line 199, in get
    value = loads(await self._get(ns_key, encoding=self.serializer.encoding, _conn=_conn))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/spike/.cache/pypoetry/virtualenvs/api--KCnhHZq-py3.12/lib/python3.12/site-packages/aiocache/backends/memcached.py", line 77, in _get
    return value.decode(encoding)
           ^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 in position 0: invalid start byte

Thanks a lot in advance :)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions