[feature] Resource-naming contract prototype: decisions 1, 2, 3, 5 (draft illustration for #6463)

[joewiz]

[This PR was prompted by Joe, drafted by Claude Code, and reviewed by Joe.]

Why this is a draft

This PR is an illustration, not a merge request. It implements the recommended answers in #6463 as running, tested code so that the contract discussion (which @dizzzz rightly flagged as complex, and worth a community call or workshop) has a concrete artifact to react to: you can see exactly what each decision costs, what it changes, and where the boundaries are. It is intentionally a draft; if the discussion wants it as a real merge candidate, we can un-draft it and add the remaining pre-merge gate (a full XQTS run).

It merges cleanly onto current develop today (no conflicts).

Which decisions of #6463 it reflects

#6463 decision	This PR	Where
1 — apps speak raw decoded UTF-8; REST/WebDAV keep wire encoding but must round-trip	XML-RPC now decodes resource/collection names on read; REST/WebDAV verified to already converge (Jackrabbit), aligned the codec to RFC 3986 pchar to keep them in lockstep	RpcConnection.java, conformance test
2 — literal % (the non-injective / silent-collision defect)	fully handled, not postponed (per @duncdrum's request to see the % fallout in one branch). A bijective lenient codec (encodeForURILenient/decodeForURI) escapes %→%25 at the store boundary, so café.xml and the literal caf%C3%A9.xml no longer collide; and the read path resolves a literal-% name by its decoded form (doc("/db/x/50%.xml") → 50%25.xml) across doc(), collection(), and the xmldb: reads	URIUtils.java, DocUtils.java, ExtCollection.java, XMLDBAbstractCollectionManipulator.java, codec test
3 — allow full Unicode; reject only / and control/NUL	xmldb:store/create-collection accept a raw space (and other Unicode) instead of throwing FORG0001 — and the read path now resolves a raw-space db-path too (doc() / doc-available()), closing the write-accepts/read-rejects gap	XMLDBStore.java, URIUtils.java, DocUtils.java, FunDocAvailable.java
5 — doc()/collection() normalize a bare db-path the way store does	doc() and collection() canonicalize a bare db-path to its stored key (decode-then-encode, idempotent on already-encoded paths); extended to the xmldb: collection read functions, so a decoded or descriptor-derived literal path resolves the stored key	DocUtils.java, ExtCollection.java, XMLDBAbstractCollectionManipulator.java

Decision 4 (guard non-/db paths at the API surface, leave core alone) is deliberately not in this PR — it lives at the application-API layer (e.g., existdb-openapi), per the issue.

The codec is proven to converge across surfaces: a stored-form oracle asserts every surface stores exactly encodeForURILenient(name) and decodes back to the original, over an awkward-name corpus (space, non-ASCII, sub-delimiters, parentheses, apostrophe, literal %, and filesystem-hostile chars).

Benefits

• Ends the store-normalizes / doc-doesn't asymmetry. doc($col || "/" || $name) after xmldb:store($col, $name, …) now resolves for any name — no more doc(xmldb:encode-uri($n)) dance.
• Fixes silent data loss. The non-injective encoding (café.xml vs literal caf%C3%A9.xml → same key, one destroyed) becomes bijective; distinct names get distinct keys.
• One canonical codec across xmldb:store/create-collection, doc(), collection(), the xmldb: read functions, and XML-RPC — verified converging, not asserted.
• Raw spaces and full Unicode just work end to end — accepted at the write boundary (no FORG0001 surprise) and resolvable on the read path (doc("/db/x/with space.xml") after xmldb:store("/db/x", "with space.xml", …) now hits).
• Literal % is fully handled — stored without data loss and resolvable by its decoded name on read (doc(), collection(), xmldb: reads), not postponed.
• A package's EXPath repo collection is addressable by its own descriptor-derived name (the manifestation written up in Define eXist's resource-naming contract: name vs URI (5 decisions) #6463's latest comment).

Is it a breaking change? What would a developer need to adjust?

No data migration. The stored form of every name that is creatable on develop today is unchanged — existing databases are unaffected. The only names whose stored form differs are ones that previously could not be stored at all (a raw space → FORG0001) or that were silently colliding (literal %).

The behavior changes a developer might notice, and how to adjust:

1. XML-RPC listings now return decoded UTF-8 names (decision 1). A client that consumed RpcConnection listing results and decoded them itself, or compared them against the percent-encoded form, will see the already-decoded name. Adjust: treat XML-RPC resource/collection names as raw decoded UTF-8 (the same contract REST clients follow after decoding the wire form).
2. xmldb:store / xmldb:create-collection no longer throw FORG0001 on a raw space (decision 3) and now store a literal-% name bijectively (decision 2). Code that relied on the throw to reject such names will no longer get the exception. Adjust: if rejection is intended, validate the name explicitly instead of depending on the codec to throw.
3. Literal-% names store and resolve distinctly. a%20b.xml now stores as a%2520b.xml instead of colliding with a b.xml, and a literal-% name resolves by its decoded form on read (see the full-fallout section). This affects only names that were previously unsafe or data-losing — no correctly-stored data moves. The one caveat is the residual ambiguity above: a name that literally is a valid escape is addressed by its encoded form.
4. doc() / collection() / xmldb: reads may now resolve a decoded path that previously missed. This is strictly additive for callers that currently get an empty result and want a hit; the honest risk is code that branched on the miss (treated empty as "absent"). Adjust: none expected, but the full XQTS run is the gate to confirm nothing leaned on the old miss.

On read-path validity: the doc() / doc-available() read path previously threw FODC0005 on a raw-space db-path (three separate java.net.URI parse points). All three now defer to the DB normalization for db-paths (/db or xmldb:), so a raw-space name resolves — while any genuinely malformed non-db URI (%gg, :/, a Windows backslash path) still raises FODC0005, exactly as the qt3tests fn/doc cases require.

Literal % — the full fallout (per @duncdrum's request)

The branch handles % rather than postponing it, so the trade-off is visible in one place rather than deferred to a Phase 2.

• Write — solved, no data loss. encodeForURILenient escapes a literal %→%25, so a name containing % and a name that is a percent-escape get distinct stored keys. The old non-injective encoding silently overwrote one with the other; that is gone.
• Read — resolved by decoded name, via decode-then-encode. The read path canonicalizes a db-path by decodeForURI then encodeForURILenient — treating it as decoded UTF-8 (decision 1) and re-encoding to the stored key. Because decodeForURI is the exact inverse of encodeForURILenient, this maps both a decoded display name and an already-encoded path to the same key, so pre-encoding callers (doc(xmldb:encode-uri($n)), TEI Publisher) keep working unchanged.
• The one irreducible residual. A name that literally is a valid escape (a%20b.xml, %25.xml) is read as its decoded meaning (a b.xml, %.xml), so it is addressed by its encoded form, not that literal string. This is inherent to a single decoded-wire contract (you cannot have a%20b.xml mean both "the literal name" and "a b.xml" on the same wire). It is pinned exactly by the round-trip oracle: a name resolves by its decoded form iff decodeForURI(name) == name — i.e. every normal name, plus a literal % that is not itself a valid escape (50%.xml, 100%done%.xml resolve; a%20b.xml is reached by its encoded form).

No stored data migrates: the only names whose stored form is new are ones that previously could not be stored without collision (a literal %) — there was no correct prior form to migrate from.

Tests

• ResourceNameCodecTest — the codec in isolation: bijective round-trip of the full corpus incl. 50%.xml / a%20b.xml / %25.xml, pins canonical stored forms, demonstrates the legacy collision/rejection.
• ResourceNamingXmldbRoundTripTest — the xmldb: write boundary against a real database: every stored key equals encodeForURILenient(name) and decodes back.
• XmldbCollectionAddressByDecodedNameTest — the xmldb: read boundary: an ${…}-style awkward-named collection is addressable by its decoded literal (fails on the pre-fix baseline, passes here), plus literalPercentCollectionResolvesByDecodedName (a literal-% collection resolves via xmldb:collection-available and fn:collection).
• DocTest — read-path raw space and literal %: doc_resolvesRawSpaceDbPathOnRead, doc_resolvesLiteralPercentNameByDecodedForm (store 50%.xml → 50%25.xml with no data loss, then resolve by both the decoded and the encoded form), and doc_malformedNonDbUriStillRaisesFODC0005 (the qt3tests garbage cases stay FODC0005).
• XPathQueryTest (150) — regression guard for the shared collection() / getLocalCollection changes: stays green.
• ResourceNamingConformanceTest (webdav module, cross-surface) — the stored-form oracle: REST/WebDAV/XML-RPC converge on one key; documents the remaining KNOWN_FAILURES (: colon, etc.) with rationale.
• XmlRpcTest — XML-RPC read decode.

Pre-merge gate not yet run: a full XQTS pass (the issue's stated condition for the decision-5 core change). This is why it's a draft.

What this deliberately leaves for follow-up

Nothing in decisions 2, 3, or 5 is deferred. What this branch intentionally does not include, so the scope stays reviewable:

• Decoded xmldb:get-child-resources / -collections return. These now resolve a decoded path, but still return the encoded stored names (caf%C3%A9.xml, not café.xml) — the long-standing //TODO: decode names?. Changing the default return would break every caller that expects the encoded form, so the right shape is an opt-in $decode argument (already prototyped on a separate branch). Until then, callers decode the result themselves, exactly as eXide does (xmldb:decode-uri).
• XML-RPC write-side decode. XML-RPC read/listing methods decode names (decision 1, surface 3); the write methods (storeResource, createCollection, …) accepting decoded names is the symmetric follow-up.
• Colon (:) and the other KNOWN_FAILURES. a:b.xml still does not round-trip cross-surface — XmldbURI wraps java.net.URI, where a: parses as a scheme. This is recorded in the conformance ratchet, not silently broken; a real fix needs the storage-layer change below.
• The %-that-is-a-valid-escape residual. a%20b.xml is read as its decoded meaning (a b.xml); see the full-fallout section. Inherent to a single decoded-wire contract, not a TODO.
• Decision 4 (validate/400 a non-/db path) — by design this lives at the application-API layer (existdb-openapi), not core.
• The application-API half of decision 1 (existdb-openapi returning/accepting decoded UTF-8) — a separate PR in that repo.
• Phase 3 — storing names as raw UTF-8 in collections.dbx (no encoding at the persistence layer at all), which would also retire the colon residual. Large; captured for direction only, per the issue.
• The normative contract document — to be written once the decisions are ratified.