Context
PR #9 fixed the worst performance issues (homepage N+1, /problems/429/ 502 crash). But there are remaining N+1 patterns worth addressing if page load times matter.
Remaining N+1 patterns
ProblemsView (views.py:48-50)
The for provider in providers loop runs a separate query per provider to fetch links. For /problems/429/ with 10 providers, that's 10 queries. Currently masked by the [:100] limit per provider — page loads in 0.2s — but the pattern is inefficient.
ProviderView (views.py:74-78)
Same pattern: loops through codes, runs a query per code to fetch links.
PublisherView (views.py:130-133)
Same pattern again.
Options considered
A. Regroup in Python — single query, itertools.groupby in Python. Simple but loads all matching objects into memory (19K+ for 429 page).
B. Two queries — one for counts, one for links with select_related, group in Python. Better memory profile.
C. Django Prefetch — doesn't work cleanly with values() querysets used here.
D. Pagination — add Django pagination instead of rendering all links. Changes UX but fundamentally solves the problem.
No slam dunk — each has tradeoffs. Current fix ([:100] limit + select_related) is good enough for now.
Related
Context
PR #9 fixed the worst performance issues (homepage N+1, /problems/429/ 502 crash). But there are remaining N+1 patterns worth addressing if page load times matter.
Remaining N+1 patterns
ProblemsView (views.py:48-50)
The
for provider in providersloop runs a separate query per provider to fetch links. For /problems/429/ with 10 providers, that's 10 queries. Currently masked by the[:100]limit per provider — page loads in 0.2s — but the pattern is inefficient.ProviderView (views.py:74-78)
Same pattern: loops through codes, runs a query per code to fetch links.
PublisherView (views.py:130-133)
Same pattern again.
Options considered
A. Regroup in Python — single query,
itertools.groupbyin Python. Simple but loads all matching objects into memory (19K+ for 429 page).B. Two queries — one for counts, one for links with
select_related, group in Python. Better memory profile.C. Django Prefetch — doesn't work cleanly with
values()querysets used here.D. Pagination — add Django pagination instead of rendering all links. Changes UX but fundamentally solves the problem.
No slam dunk — each has tradeoffs. Current fix ([:100] limit + select_related) is good enough for now.
Related