Cache sitemap discovery results to avoid re-fetching on every run

[brylie]

Discovery for a sitemap-backed source (discovery.source: sitemap) re-fetches every sitemap document from scratch on each discover <site> invocation — there is no caching keyed by discovery.cache_ttl_hours, even though docs/specs/crawler-improvements.md's example configuration and "Scale and timeline" section call this out explicitly: migri.fi and dvv.fi's sitemap index alone costs ~2.4–2.5 hours to enumerate at their mandated 5s Crawl-delay, and that cost "should be paid roughly once a day at most per source, not on every run."

Discovered while auditing #74's Phase 1 implementation before building Phase 2 rendering (#75) on top of it.

Scope

• Add a discovery.cache_ttl_hours field to DiscoveryConfig (crawler/tapio_crawler/config/config_models.py), matching the spec's example config.
• Cache the sitemap fetch/parse result (crawler/tapio_crawler/discovery/sitemap.py) for that TTL, so a discover run within the window reuses the manifest's existing discovery state instead of re-enumerating every child sitemap.
• A run past the TTL performs a full re-fetch as today.

Acceptance criteria

• Given a source was discovered less than cache_ttl_hours ago, when discover runs again, then no sitemap or child-sitemap HTTP requests are made.
• Given the TTL has elapsed, when discover runs, then it re-fetches as it does today.

[brylie]

Refinement from the 2026-08-05 discovery-only backfill review (kela, vero, tyomarkkinatori, migri, dvv):

Both migri.fi and dvv.fi came back from that run with complete=False — migri had 17 failed child-sitemap fetches (clean 404s) out of 1,741, dvv had 153 out of 1,616 (~9.5%, see #82). Under the current all-or-nothing discovery flow, recovering from that means re-running discover from scratch and repaying the entire ~2.4–2.5h cost, to re-fetch child sitemaps that mostly already succeeded.

Two changes to fold into this issue's scope, not just "cache the successful result for cache_ttl_hours":

1. Don't cache an incomplete run as if it were a valid, fresh result. A complete=False run should not satisfy cache_ttl_hours the same way a complete one does — otherwise an operator can get stuck serving a permanently-partial manifest state until they notice and force a re-discover.
2. Persist per-child-sitemap fetch status, not just a whole-run pass/fail, so a retry only re-fetches what previously failed. This is the more valuable half for cost: with dvv's ~90% success rate on child-sitemap fetches, a re-run that skips already-succeeded child sitemaps and only retries the failed subset turns a ~2.5h re-fetch into a few minutes. This is the same "make it replayable in shorter subsequent rounds" principle Phase 2's manifest-driven resumability already applies to rendering (crawler/tapio_crawler/crawler.py) — worth extending the same pattern to discovery rather than treating discovery as strictly all-or-nothing.

Related: #77 (persisting run metadata) and #82 (logging the actual exception on sitemap fetch failure, so a retry-eligible failure can be distinguished from e.g. a permanent 404).