Improvement: Persist downloaded schemas/schematrons to disk to speed up back-to-back executions

[jordanpadams]

Summary

validate already caches downloaded XSD schemas and schematrons in-memory via CachedLSResourceResolver and CachedEntityResolver, so within a single run each remote schema is fetched only once. However, that cache is discarded when the process exits. Every new invocation redownloads the same schemas from pds.nasa.gov.

For back-to-back executions — a data producer iterating on a label during development, or a CI pipeline invoking validate per-file rather than per-bundle — this adds measurable latency (typically 150–500ms per unique schema URL, per invocation) and fails entirely if the network is unavailable.

Proposed behavior

• On first fetch, write the downloaded schema/schematron bytes to a local disk cache directory (e.g. ~/.pds4/schema-cache/ or ${resources.home}/schema-cache/), keyed by a sanitized form of the source URL.
• On subsequent invocations, check the disk cache before making a network request. If found, load from disk.
• Add a --force-schema-download flag (or similar) to bypass the disk cache and re-fetch from the network. This is a correctness requirement: LDD extension schema URLs are sometimes unversioned and can be updated in place, so users need a way to pull fresh copies.

Notes

• PDS4 core schema URLs are version-pinned (PDS4_PDS_1D00.xsd, PDS4_PDS_1100.xsd, etc.) and never go stale — those cache entries can be kept indefinitely.
• LDD extension/discipline/mission dictionary schemas are the main TTL concern; --force-schema-download handles that.
• The hook points already exist: CachedLSResourceResolver.resolveResource() and CachedEntityResolver.resolveEntity() are where remote fetches occur. The disk cache layer slots in between the in-memory check and the network fetch.
• Users who need offline or air-gapped operation today can work around this with -C (catalog), -S (schema), and -s (schematron) flags pointing at local files. The disk cache would automate that workflow.

Expected benefit

Primarily useful for:

• Data producers doing iterative label development (fix → validate → repeat)
• CI pipelines that invoke validate per-file
• Users on slow, metered, or intermittent connections
• Air-gapped or offline environments

🤖 Generated with Claude Code