[BUG] fn:transform input document conversion to XdmNode

[nverwer]

Description

When a document containing a org.exist.dom.memtree.ReferenceNode is transformed by fn:transform, the ReferenceNode is not passed on to Saxon. In other words, the input document becomes incomplete.

The problem occurs in /exist-core/src/main/java/org/exist/xquery/functions/fn/transform/Transform.java, from line 176.

                    final Document document;
                    Source source = sourceNode.get();
                    final Node node = ((DOMSource)sourceNode.get()).getNode();
                    if (!(node instanceof org.exist.dom.memtree.DocumentImpl) && !(node instanceof org.exist.dom.persistent.DocumentImpl)) {
                        //The source may not be a document
                        //If it isn't, it should be part of a document, so we build a DOMSource to use
                        document = node.getOwnerDocument();
                        source = new DOMSource(document);
                    }
                    final DocumentBuilder sourceBuilder = Transform.SAXON_PROCESSOR.newDocumentBuilder();
                    final XdmNode xdmNode = sourceBuilder.build(source);
                    xslt30Transformer.setGlobalContextItem(xdmNode);

The source variable contains a javax.xml.transform.dom.DOMSource, which contains a node field that is a DocumentImpl. The String representation (shown by the Java debugger in Eclipse) of this node is (edited for readability):

in-memory#document {
      in-memory#element {exercise}
         { in-memory#attribute {ch} {1}
           in-memory#element {chapter-title}
               { in-memory#text {The Nature of Language and Linguistics} }
           reference[ <question-wrap>
                    <question id="c1q1">
                    ...

When the xdmNode has been built, its String representation is:

<exercise ch="1" ex="1" next="c1q2" nextch="c2q1">
   <chapter-title>The Nature of Language and Linguistics</chapter-title>
</exercise>

The reference to <question-wrap> has disappeared, and is not seen by the XSLT stylesheet.

It seems that sourceBuilder.build(source) does not handle ReferenceNodes correctly.

This issue is not related to this earlier pull request.

Expected behavior
The complete document should be passed on to Saxon.

To Reproduce

Reproducing this depends on the exact construction of the input document. The input document comes from an eXist collection, but I have not found a way to easily make a document that has a ReferenceNode.

Context (please always complete the following information)

I am using the 'develop-6.x.x' branch of 2025-03-18 (3ae1884).

eXist Version: 6.4.0-SNAPSHOT
eXist Build: 2025-03-18T13:56:52Z
Operating System: Windows 10 10.0 amd64
Java Version: 11.0.14.1

Further remarks

I am investigating this, and may add more details to this issue.
If I find a solution, I will provide a pull request.

[nverwer]

Additional information.

When converting the document to a SAX document (XdmNode), the method net.sf.saxon.dom.DOMSender.walkNode(Node) (Saxon-HE 9.9.1-8)) is used.
When this method is called with the parent of the ReferenceNode, looping over its children, the type of the reference node is not recognized, and the reference node is processed by the default of a switch:

break;                  // should not happen

This is because child.getNodeType() is 100, which is not defined in org.w3c.dom.Node.
The method org.exist.dom.memtree.NodeImpl.getNodeType() determines the node type as document.nodeKind[nodeNumber]. Apparently, the node type is 100 for a reference node, and this makes it impossible to convert the node into something useful for Saxon.

In org.exist.dom.memtree.NodeImpl, there is a line

public static final short REFERENCE_NODE = 100;

which is specific to eXist's in-memory tree, and not compatible with org.w3c.dom.

At the moment, I have no idea how to solve this.