Consider asyncio and aioredis

[WisdomPill]

Problem Statement
As django 4 is on his way, with support for async cache, a question came to my mind, to support a... methods, should be just wrap sync methods in the async call, without making it really async or start to use some async client like aioredis?

Describe the solution you'd like
Start using aioredis to make django-redis async compatible.

Describe alternatives you've considered
Create async methods to wrap sync ones.

@terencehonles what do you think would be the best way to go after this inevitable change?

[terencehonles]

ooo, this is cool (Django support coming down the pipeline). I haven't looked into aioredis, but whatever we choose it should probably be as compatible with our previous redis driver options. I'm going to be tied up for at least a week or so, but this is definitely something I'd be interested in discussing. We probably need to look at how much the two different redis drivers differ.

I'm going to mark this as unread and I'll try to respond soon 🙂

[WisdomPill]

From version 2, it should have the same api.

I am working on move to version 2 already on channels_redis when I get a little bit of time.

I will let you know how comfortable I find it in the next one or two weeks

[WisdomPill]

Seems like async is already supported because BaseCache wraps async calls in sync_to_async, look here

[Andrew-Chen-Wang]

It should be noted that redis-py now supports asyncio and asyncio clusters

Would be a great addition here. Ref notes here: https://code.djangoproject.com/ticket/33573

I'm currently debating whether a second class should be used or not. In my belief, yes to 2 classes, to separate resources in given environments, but what do you think @WisdomPill on a class based on environment (async vs. sync)?

[Andrew-Chen-Wang]

Also FYI published a package for async support that models django-redis: https://github.com/Andrew-Chen-Wang/django-async-redis

[some1ataplace]

Did not test this but maybe this can give someone some ideas.

1. Install the required packages:

pip install aioredis django-redis channels_redis

2. Update your Django settings.py file to include the following:

INSTALLED_APPS = [
    ...
    'django_redis',
    'channels',
]

# Django-Redis settings
CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
        }
    }
}

# Channels settings
CHANNEL_LAYERS = {
    "default": {
        "BACKEND": "channels_redis.core.RedisChannelLayer",
        "CONFIG": {
            "hosts": [("127.0.0.1", 6379)],
        },
    },
}

3. Create an async_cache.py file in your Django app folder with the following content:

import asyncio
from django.core.cache import cache
from channels.db import database_sync_to_async

@database_sync_to_async
def get_cache(key):
    return cache.get(key)

@database_sync_to_async
def set_cache(key, value, timeout=None):
    return cache.set(key, value, timeout)

async def async_cache_example():
    await set_cache("my_key", "my_value")
    value = await get_cache("my_key")
    print(value)

# Run the async_cache_example function
loop = asyncio.get_event_loop()
loop.run_until_complete(async_cache_example())

This file demonstrates how to use the Django cache with asyncio by creating two wrapper functions, get_cache and set_cache, which are marked as database_sync_to_async. These functions allow you to interact with the cache using async/await syntax.

4. Add a new Django view that uses the async cache functions:

In your views.py file, create a new view that uses the async_cache_example function:

from django.http import JsonResponse
from .async_cache import async_cache_example

async def async_cache_view(request):
    await async_cache_example()
    return JsonResponse({"message": "Cache accessed successfully"})

5. Update your Django app's urls.py file to include the new view:

from django.urls import path
from . import views

urlpatterns = [
    ...
    path("async_cache/", views.async_cache_view),
]

6. Configure Django to use the ASGI server:

Create an asgi.py file in your Django project folder with the following content:

import os
from django.core.asgi import get_asgi_application
from channels.routing import ProtocolTypeRouter, URLRouter
from channels.auth import AuthMiddlewareStack

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "your_project_name.settings")

application = ProtocolTypeRouter(
    {
        "http": AuthMiddlewareStack(
            URLRouter(
                your_app_name.urls.urlpatterns
            )
        ),
    }
)

7. Run your Django project using the ASGI server:

To run your Django project using the ASGI server, use the following command:

daphne your_project_name.asgi:application
Daphne is an ASGI server that will allow your Django project to handle asynchronous requests.

8. Test the async cache view:

Now that your Django project is running with the ASGI server, you can test the async cache view by visiting the following URL in your browser or using a tool like curl or Postman:

http://localhost:8000/async_cache/

If everything is set up correctly, you should receive a JSON response with the message "Cache accessed successfully."

By following these steps, you've successfully integrated asyncio and Django cache in your Django project. This will allow you to take advantage of asynchronous programming when accessing the cache, which can lead to improved performance and responsiveness in your application.

[WisdomPill]

hello @some1ataplace,

thanks for the effort but database_sync_to_async should not be used for cache, and @Andrew-Chen-Wang solution is the preferred way, it is just a matter of understanding how to integrate it without duplicating the code.

Would be nice to come with some sansIO approach

[Andrew-Chen-Wang]

would be difficult to do it sansio (I'm not that experienced in designing sansio in mind either), but the connection classes are totally different between sync and async.

the client classes are also mixing logic and IO everywhere, so might not be possible to do it sansio style.

I do believe the client classes should mix both async and sync.