Improve perceived loading performance on heavy pages (Squadrons, Lists, Ships, Cards)

[Francespo]

Objective

Improve perceived performance on the heaviest pages by reducing time-to-usable-content.

Context & Symptoms

• Squadrons, Lists, Ships, and Cards pages feel slow during initial load.
• Initial waiting time is high and negatively impacts navigation.
• A progressive loading behavior is needed: render useful first-page content quickly, then continue loading remaining data in background.

Expected Outcome

• Heavy pages render a useful first content slice quickly.
• Remaining data loads progressively without blocking interaction.
• Navigation feels noticeably more responsive on the listed routes.

[Francespo]

Approved Technical Implementation Plan

Scope

• Improve perceived loading performance on heavy index routes: Squadrons, Lists, Ships, Cards.
• Extend progressive behavior to related detail routes where users currently wait for multiple data blocks before the page feels usable.
• Refactor frontend-backend query contract to use corrected parameter names only (no legacy compatibility layer), so filtering is deterministic and request load is reduced.
• Apply focused backend optimizations that reduce avoidable per-request work on hot paths.

Non-Goals

• No full analytics architecture rewrite.
• No infinite-scroll or virtualization redesign in this issue.

Proposed Approach

Phase 1: Query Contract Correction and Baseline

• Define one canonical query parameter contract between frontend and backend.
• Refactor route query builders and API handlers to accept/use only corrected parameter names.
• Remove legacy-name fallback behavior to reduce ambiguity and hidden logic branches.
• Add lightweight request timing instrumentation on heavy endpoints for baseline comparison.

Phase 2: Backend First-Slice Acceleration

• Remove repeated expensive operations from hot loops on heavy endpoints.
• Ensure first-page payload behavior is tuned for faster time-to-first-useful-data.
• Keep aggregation semantics stable while reducing avoidable processing overhead.
• Verify sorting/filtering behavior remains correct with the corrected query contract.

Phase 3: Progressive Rendering on Index Routes

• Reduce redundant refetch cycles caused by immediate URL-sync navigation patterns.
• Add debounced filter-to-query synchronization for typing-based and numeric inputs.
• Render useful first-page content immediately, then fetch secondary/non-critical data in background.
• Defer non-critical sidebar/filter data loading so it does not block first interaction.

Phase 4: Progressive Rendering on Detail Routes

• Split data loading into critical-first and secondary-later sections.
• Show primary stats/info as soon as available.
• Load related secondary blocks (lists/squadrons/auxiliary panels) progressively with section-level loading states.

Phase 5: Stability and Regression Guardrails

• Add/extend backend tests to validate corrected-parameter behavior and response consistency.
• Add frontend checks to ensure route/query/pagination state integrity after refactor.
• Compare before/after latency on representative heavy endpoints and user flows.

Test and Validation Strategy

• Backend:
  • Validate filtering/sorting correctness under corrected parameter names.
  • Ensure deprecated names no longer alter behavior.
  • Capture before/after timings for heavy endpoints.
• Frontend:
  • Validate progressive first content slice appears quickly.
  • Confirm background loading does not block interaction.
  • Confirm URL state remains stable/shareable and pagination remains consistent.
• End-to-end UX checks:
  • Heavy routes should feel responsive under normal navigation.
  • Detail routes should be usable before all secondary sections finish loading.

Risks and Rollback

• Risk: Breaking old bookmarked URLs or clients that still send deprecated query names.
  • Mitigation: coordinated frontend/backend rollout in one pass, explicit regression checks.
• Risk: Progressive sections can temporarily display partial state.
  • Mitigation: explicit loading placeholders and deterministic section update order.
• Risk: Backend micro-optimizations could impact ranking/output semantics.
  • Mitigation: preserve aggregation rules and compare representative outputs before release.
• Rollback:
  • Revert progressive rendering per route if needed.
  • Keep corrected naming contract as the single source of truth.

Delivery Sequence

• Milestone A: Phase 1
• Milestone B: Phase 2
• Milestone C: Phase 3
• Milestone D: Phase 4
• Milestone E: Phase 5

[Francespo]

Revised Technical Implementation Plan (Post-Split)

Scope

• Continue performance work for heavy routes: Squadrons, Lists, Ships, Cards.
• Improve perceived speed by showing useful content earlier and reducing avoidable refetch churn.
• Keep frontend/backend query contract deterministic (sources, search_text, canonical params only).
• Apply backend hot-path optimizations and reliability fixes tied to current heavy flows.

Explicit Split

• Work related to a dedicated ship-list page has been split out into a separate issue: Dedicated ship list page for /ship/.../lists route #103.
• This issue (Improve perceived loading performance on heavy pages (Squadrons, Lists, Ships, Cards) #96) no longer includes ship-list-page delivery scope.

Non-Goals

• No infrastructure/VPS plan changes.
• No full analytics architecture rewrite.
• No redesign of infinite scroll/virtualization.

Proposed Approach

1. Progressive Loading Where It Reduces Initial Blocking

   • Keep critical-first data rendering and defer non-critical sections.
   • For ship chassis filter data, move from monolithic fetch behavior toward incremental retrieval so first usable options render earlier.

2. Request-Churn Reduction

   • Preserve/extend debounced URL synchronization and avoid duplicate refetches caused by equivalent query state.
   • Ensure stale async responses do not overwrite latest UI state.

3. Backend Efficiency and Stability

   • Remove repeated expensive operations from hot loops.
   • Harden heavy endpoints against malformed/partial data to prevent 500s on secondary blocks.

4. Contract and Compatibility Discipline

   • Keep canonical query keys only, no legacy fallback paths.
   • Validate filtering/sorting semantics remain consistent after optimization changes.

5. Regression Guardrails

   • Extend focused backend tests for contract and heavy endpoint behavior.
   • Add checks for progressive/deferred frontend flows to avoid UX regressions.

Validation Strategy

• Compare before/after latency on representative heavy endpoints and flows.
• Verify first useful content appears earlier on heavy pages.
• Verify no stale-section overwrite under rapid navigation/filter changes.
• Verify ship-related secondary endpoints do not produce 500s under normal inputs.

Risks and Rollback

• Risk: Progressive behavior introduces partial-state timing edge cases.
  • Mitigation: explicit section loading states and stale-response guards.
• Risk: Optimization changes alter ordering/filter semantics.
  • Mitigation: contract tests and output comparisons on representative datasets.
• Rollback: route-level progressive behavior can be reverted independently while preserving canonical query contract.