RAG: Plate editor pages chunk title-only — plate block missing from blocks_layout

[reekitconcept]

Reported downstream as kitconcept.intranet GitLab ticket 580: content authored in a WikiPage with the Plate editor is not found by the RAG search ("no documents show this information"), while the same content in a Workspace (classic Volto blocks) works.

Cause

The Plate editor registers its __somersault__ block in the blocks field without adding it to blocks_layout.items. extract_segments in rag/extraction.py walked only the layout, so the whole page body of an editor-authored WikiPage was invisible to the RAG chunking — chunks contained the title/description only, and body questions were answered with "not found".

Classic search was unaffected: plone.restapi's SearchableText indexer walks all blocks.values().

Fix

Walk the layout first (it carries the document order — the chunker merges consecutive segments), then any blocks not listed in it: same coverage as plone.restapi's visit_blocks. Verified end-to-end against the reporter's exact block structure.

[reekitconcept]

A note on the ordering of the fix, from review discussion:

Within the Plate block, segment order is exact: the block's value is an ordered array of top-level elements — the document order of the page as authored — and the extraction walks it front to back.

Across blocks, an off-layout block has no recoverable position: blocks_layout.items is the only place block ordering is stored, and the Plate block simply has no entry there. Appending unlisted blocks after the layout-listed ones is therefore by necessity, not an approximation of some available truth. In practice this is harmless — on editor-authored WikiPages the layout lists only empty scaffold blocks (a bare title and an empty slate), so the effective segment order is title, description, then the full body in document order.

The proper upstream fix belongs in kitconcept.plate: the editor stores its __somersault__ block in blocks without listing it in blocks_layout.items, which breaks the Volto blocks data contract — any consumer that follows the convention and walks the layout misses the whole page body (plone.restapi's SearchableText only survives because it iterates blocks.values()). If plate appended its block to the layout on save, this class of problem would disappear at the root, and this fix's append-fallback would remain only as a guard for legacy content. To be raised with the plate team (deferred — no urgency, this fix solves the problem for consumers of kitconcept.solr).

[davisagli]

@reekitconcept Not including the __somersault__ block in the layout is intentional. We want to phase out the layout eventually because it's not needed for plate-based pages.

[reekitconcept]

@davisagli so basically you are saying that the current way of ordering is sufficient? layout content - then plate content? And when layout is phased out, only plate content remains, so the current code will continue working.

Is this the case, or is this something else to consider? I'm asking because I want to make sure that the rag indexing is doing the right thing here.

[davisagli]

@reekitconcept Yes, I think that is right, as far as we know now.