Help understanding cache

[ShayBox]

So I have an endpoint I need to cache for a short time because it can get a lot of the same request at the same time.
I use InMemory cache, and the request takes a second or more to process.
If I get say 100 requests for the route "at the same time" would they all run the process because the cache wasn't ready, or would they all wait for the first one to finish and use the cache from that one?

[jameswinegar]

You are async processing the data. If they don't hit the cache on the ingestion into the middleware then they will repeat all the processing.

[ShayBox]

Okay, I have to add my own race condition then but how would I fetch the cached value from within an endpoint?

[mjpieters]

There is no easy, clean way to get the currently cached value for a route; you'd need references to the function itself, the current request and response, the positional and keyword arguments passed into the endpoint function. If you are using the default key builder you can, currently, use None for the request and response objects. You'd then have to call the key builder to get a key, check the backed for the cached value, then access the coder to decode the cached value into an actual object.

We can make that process easier by adding utility functions onto the wrapper returned by the @cache decorator, but you'd still have to deal with locking and race conditions.

I'd instead create an uncached endpoint that then uses an asyncio.lock object to serialise access the actual cached function:

lock = asyncio.Lock()

@app.get(...)
async def uncached_outer_endpoint(...):
    async with lock:
         return await cached_inner_function(...)


@cache(...)
async def cached_inner_function(...):
    ...

If you want FastAPICache to still set HTTP cache-related headers and handle If-None-Match conditional requests you'll need to pass along a request and response object (add request: Request, response: Response annotated arguments to your outer endpoint function).

I'll see if this is something the documentation should have an example for.