perf: restore real TTLs on cached list fetchers

[lokesh]

Closes #

Performance fix: makes two "cached" list fetchers actually cache, instead of spawning a background recompute thread on every single request.

Estimated performance gain

Redundant backend recomputes drop from one per request to at most a few per hour per process — a ~98–99% reduction in this workload. Each recompute is a site.things query + a get_many over ~120 list documents plus owner/seed preloads, executed in a spawned thread. The gain is primarily backend load and thread churn (the recompute is off-request), which frees infobase/Postgres capacity on the shared request path.

xychart-beta
    title "Recomputes per hour per process (assuming 10 /lists views/min)"
    x-axis ["Before (timeout=0)", "After (5-min TTL)", "After (30-min TTL)"]
    y-axis "background recomputes / hour" 0 --> 600
    bar [600, 12, 2]

Loading

Estimates scale linearly with traffic: at N views/min the "before" bar is 60×N recomputes/hour, while "after" stays capped by the TTL regardless of traffic.

Technical

Problem. In openlibrary/plugins/openlibrary/lists.py, both get_cached_recently_modified_lists and get_active_lists_in_random passed timeout=0 to cache.memcache_memoize:

f = cache.memcache_memoize(
    _get_recently_modified_lists,
    key_prefix="lists.get_recently_modified_lists",
    timeout=0,
)  # dateutil.HALF_HOUR_SECS)

memcache_memoize's staleness check in openlibrary/core/cache.py is if t + self.timeout < time.time(): self.update_async(...). With timeout=0 that is true on every call, so every request that touches these functions spawns a background thread that re-runs the underlying work — _get_recently_modified_lists does a site.things query plus a get_many over the results, and _get_active_lists_in_random loops calling it with limit*5 = 120 lists per iteration and then preloads owner + seed documents.

The stranded # dateutil.HALF_HOUR_SECS) comment is the tell: a 30-minute TTL was replaced with timeout=0 (presumably a temporary cache-disable during debugging) and never reverted.

Where it hits. get_active_lists_in_random is @public and rendered on the /lists landing page (templates/lists/home.html, limit=24). Every view of that page currently kicks off redundant infobase/DB work in background threads (rate-limited only by memcache add-flag dedup), and the "cache" never actually shields the backend.

Fix.

• get_cached_recently_modified_lists: timeout=dateutil.HALF_HOUR_SECS — restoring exactly what the orphaned comment says was intended.
• get_active_lists_in_random: timeout=5 * dateutil.MINUTE_SECS — a shorter TTL so the random selection on /lists keeps rotating; matches the 5-minute convention used elsewhere (lending.py groundtruth availability, home.py homepage cache).

Behavior notes. memcache_memoize is stale-while-revalidate: it serves the cached value and refreshes in a single background thread after the TTL, so no user request ever blocks on recompute. Worst-case staleness on /lists becomes ~30 min for the recently-modified feed — acceptable for that surface, and the previous behavior (timeout=0) provided no fresher guarantees to users anyway, since reads still came from memcache.

Testing

New test test_cached_list_fetchers_use_real_ttls in openlibrary/plugins/openlibrary/tests/test_lists.py asserts both memoize wrappers are constructed with timeout > 0. The stale-while-revalidate timeout mechanism itself is covered by the existing openlibrary/tests/core/test_cache.py::Test_memcache_memoize::test_timeout.

Reproduce: make test-py-uv PYTEST_ARGS="openlibrary/plugins/openlibrary/tests/test_lists.py openlibrary/tests/core/test_cache.py" — 34 passed. pre-commit — all hooks pass.

Screenshot

N/A — no UI change (only cache freshness on /lists).

Stakeholders

Part of a performance-review series; see the review report branch claude/performance-review-top-changes-5wcs7g.

🤖 Generated with Claude Code

https://claude.ai/code/session_015ky5Sc19MGPkDgbgTWWWET