Description
Lines 167 to 170 in 1cb4623
If you call all() on a table with many rows, it will retrieve all the data at once, which will likely result in out-of-memory (OOM) errors.
As a workaround, make sure to set the
SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENTvery low (0.01 for example so it runs every 10_000 requests) and theSILKY_MAX_RECORDED_REQUESTShigh (100_000 or more).This way, the chances of running into this issue are much smaller (but still possible, depending on your load).
An even better workaround is to set
SILKY_MAX_RECORDED_REQUESTS_CHECK_PERCENTto 0 and run thesilky.Request.garbage_collect(force=True)method in a scheduled task through celery.Originally posted by @okke-formsma in #265 (comment)
I previously used Celery to call this function to avoid deadlock, as suggested in the referenced comment. However, due to OOM errors, our scheduler failed to clear the database, leading to excessive database usage.
I recommend setting a limit and deleting the rows recursively instead. I will submit a pull request to address this issue.