Broaden the simulated LSP session and register the Path watch types

The session that drives both the tracing agent and the JVM-vs-native
comparison left several code paths untouched, and four of its probes
compared an empty result against an empty result:

- completion on main.nf returned nothing, because member completion on a
  complete expression has no answer to give
- previewDag was passed a process name where the second argument is a
  workflow name, so it returned null and neither DataflowVisitor nor
  MermaidRenderer ever ran
- documentLink returned nothing, because links come from include
  statements and the workspace had a single file
- documentSymbol on the config file can only ever return nothing, as the
  config service has no symbol provider

So: point completion at positions that resolve (an identifier, the top
level, and a member of the lowercase channel namespace), pass null to
previewDag for the entry workflow, add a module that usage.nf includes,
and drop the config documentSymbol probe. Also cover the handlers that
were never exercised: didChange, incoming/outgoingCalls, and the three
remaining commands. The comparison goes from 27 messages to 42.

That immediately failed, which is the point of it:

  NoClassDefFoundError: Unable to configure java.nio.file.Path due to
  missing dependency java.nio.file.WatchEvent$Modifier

Groovy configures the Path class node for the Path shim type and
enumerates its methods, one of which is register(WatchService, Kind[],
Modifier...). Nothing calls it, so native-image left the watch types out
of the image entirely and the first completion request that resolved a
Path failed with an internal error. Registering that closure fixes it.

Also correct the ADR: tracing contributes the reflective entries behind
Groovy's indy call sites and five META-INF/services files, not the JSSE
provider graph or JDK icu resources -- no config directory registers
anything for those. Since nothing exercises the plugin registry, the
HTTPS path it needs is covered by neither source, which is now recorded
as a residual risk.

Author: bentsherman

adr/20260804-graalvm-native-image-build.md

@@ -70,7 +70,7 @@ Add a GraalVM 21 `native-image` build producing `nextflow-lsp` for linux-amd64 a
 
 ### Extracting reflection metadata
 
-`generateNativeImageMetadata` scans the resolved runtime classpath and emits `reflect-config.json`, `resource-config.json` and `proxy-config.json`, registering ~630 classes. Registration flags are set per group rather than uniformly, since image size is driven by reachability; the per-group reasoning lives in comments in `build.gradle`. The tracing agent remains a secondary source, for the tail that resists enumeration: Groovy indy call sites reaching `MethodHandleNatives`, JDK-internal resources with version-specific paths, and the JSSE provider graph loaded reflectively for the plugin-registry HTTPS call. `native-image` merges the two, static config first.
+`generateNativeImageMetadata` scans the resolved runtime classpath and emits `reflect-config.json`, `resource-config.json` and `proxy-config.json`, registering ~630 classes. Registration flags are set per group rather than uniformly, since image size is driven by reachability; the per-group reasoning lives in comments in `build.gradle`. The tracing agent remains a secondary source, for the tail that resists enumeration: the reflective entries behind Groovy's indy call sites, and the `META-INF/services` files that the JDK and Groovy load through `ServiceLoader`. `native-image` merges the two, static config first.
 
 A scan is exhaustive where a traced session is accidental. The reflected-over sets are *closed* — Gson reflects over every type in `org.eclipse.lsp4j`, and Groovy's `configureClassNode` over every class handed to `ClassHelper.makeCached` — and those are properties of the jars, so scanning enumerates them completely and picks up dependency upgrades automatically. Traced coverage is instead a function of the script: the session used here reaches 2 of the 35 `nextflow.script.types.**` classes, and whatever it misses is not reported at build time, because the `--report-unsupported-elements-at-runtime` that Groovy's indy call sites force turns what would be build errors into runtime no-ops.
 
@@ -88,6 +88,8 @@ The JAR is the only available oracle. Unit tests run on the JVM and say nothing
 
 - **A macOS-specific regression would go unnoticed**, since CI builds Linux only.
 
+- **The plugin registry is not covered.** Resolving `include { ... } from 'plugin/...'` fetches over HTTPS, which brings in `HttpClient` and the JSSE provider graph — reflection-heavy territory, and no config directory registers anything for it today. The session deliberately stays offline, so this would first fail for a user rather than in CI.
+
 ## Links
 
 - [GraalVM Native Image](https://www.graalvm.org/latest/reference-manual/native-image/)

native/native.gradle

@@ -75,6 +75,18 @@ def reflectionGroups = [
       'java.util.function.Consumer',
       'java.util.function.Function',
       'java.util.function.Predicate',
+      // Groovy configures the java.nio.file.Path class node for the Path shim
+      // type, which enumerates Path.register(WatchService, Kind[], Modifier...).
+      // Nothing calls that method, so without these the watch types are absent
+      // from the image and configuring Path fails with a NoClassDefFoundError
+      // -- surfacing as an internal error on the first completion request that
+      // resolves a Path.
+      'java.nio.file.Path',
+      'java.nio.file.WatchEvent',
+      'java.nio.file.WatchEvent$Kind',
+      'java.nio.file.WatchEvent$Modifier',
+      'java.nio.file.WatchKey',
+      'java.nio.file.WatchService',
     ],
     flags: ['allDeclaredFields', 'queryAllDeclaredMethods', 'queryAllDeclaredConstructors'],
   ],

native/simulate.sh

@@ -38,6 +38,7 @@
 #
 #   Document Synchronization:
 #     - textDocument/didOpen   : Open a document for editing
+#     - textDocument/didChange : Apply an incremental edit
 #     - textDocument/didClose  : Close a document
 #
 #   Language Features:
@@ -52,18 +53,27 @@
 #     - textDocument/documentLink       : Get clickable links in document
 #     - textDocument/rename             : Rename a symbol
 #     - textDocument/prepareCallHierarchy: Prepare call hierarchy
+#     - callHierarchy/incomingCalls     : Resolve callers
+#     - callHierarchy/outgoingCalls     : Resolve callees
 #
 #   Workspace Features:
 #     - workspace/symbol           : Search for symbols across workspace
 #     - workspace/didChangeConfiguration : Push client settings (this is what
 #                                    initializes the language services)
-#     - workspace/executeCommand   : Invoke a server-side command
+#     - workspace/executeCommand   : All four server commands
 #
-# TEST DOCUMENT:
-#   The script opens a sample Nextflow script containing:
-#     - A process definition (FOO) with input/output declarations
-#     - A workflow block that invokes the process
-#   This exercises parsing, symbol resolution, and most LSP features.
+#   Handlers deliberately left out because they only log: didSave,
+#   didChangeWatchedFiles, didCreateFiles, didDeleteFiles, didRenameFiles,
+#   $/setTrace. Also left out: didChangeWorkspaceFolders, and plugin includes
+#   (`from 'plugin/...'`), which would make the session depend on reaching the
+#   plugin registry over the network.
+#
+# TEST WORKSPACE:
+#   - main.nf           : a process (FOO) and an entry workflow that calls it
+#   - nextflow.config   : process and docker scopes
+#   - usage.nf          : includes module.nf and calls BAR
+#   - module.nf         : the included process (BAR)
+#   - lib/Helper.groovy : compiled by the Groovy compiler, not the Nextflow one
 #
 # NOTE:
 #   The sleep delays between messages ensure the server has time to process
@@ -142,9 +152,52 @@ docker {
 }
 EOF
 
+# A second script plus the module it includes. This is the only thing that
+# exercises include resolution, cross-file definition, and the document link
+# provider -- links come from include statements, so a single-file workspace
+# leaves that provider returning nothing.
+cat > "$WORKSPACE/module.nf" <<'EOF'
+process BAR {
+    input:
+    val y
+
+    output:
+    stdout
+
+    script:
+    """
+    echo ${y}
+    """
+}
+EOF
+
+cat > "$WORKSPACE/usage.nf" <<'EOF'
+include { BAR } from './module.nf'
+
+workflow NAMED {
+    ch = channel.of(1)
+    BAR(ch)
+}
+EOF
+
+# lib/*.groovy is compiled by GroovyLibCache through the Groovy compiler rather
+# than the Nextflow parser, which is a distinct code path -- and the one that
+# would reach Groovy's AST transformation machinery if anything does.
+mkdir -p "$WORKSPACE/lib"
+cat > "$WORKSPACE/lib/Helper.groovy" <<'EOF'
+class Helper {
+
+    static String greet(String name) {
+        return "hello ${name}"
+    }
+
+}
+EOF
+
 WORKSPACE_URI="file://$WORKSPACE"
 DOC_URI="$WORKSPACE_URI/main.nf"
 CONFIG_URI="$WORKSPACE_URI/nextflow.config"
+USAGE_URI="$WORKSPACE_URI/usage.nf"
 
 # JSON-escape a document so didOpen carries the same text that is on disk
 json_text() {
@@ -153,6 +206,16 @@ json_text() {
 
 DOC_TEXT=$(json_text "$WORKSPACE/main.nf")
 CONFIG_TEXT=$(json_text "$WORKSPACE/nextflow.config")
+USAGE_TEXT=$(json_text "$WORKSPACE/usage.nf")
+
+# The call hierarchy item that prepareCallHierarchy returns for `process FOO`.
+# incomingCalls/outgoingCalls take an item back as input, so it has to match
+# what the server produced -- these are the ranges of the process definition.
+FOO_ITEM='{"name":"FOO","kind":6,"uri":"'"$DOC_URI"'","range":{"start":{"line":0,"character":0},"end":{"line":11,"character":1}},"selectionRange":{"start":{"line":0,"character":0},"end":{"line":11,"character":1}}}'
+
+# The entry workflow, used for outgoingCalls: it calls FOO, whereas FOO itself
+# calls nothing, so asking for FOO's callees only ever returns an empty list.
+ENTRY_ITEM='{"name":"<entry>","kind":6,"uri":"'"$DOC_URI"'","range":{"start":{"line":13,"character":0},"end":{"line":17,"character":1}},"selectionRange":{"start":{"line":13,"character":0},"end":{"line":17,"character":1}}}'
 
 # =============================================================================
 # STEP 1: Initialize Connection
@@ -209,11 +272,24 @@ send_message '{"jsonrpc":"2.0","id":2,"method":"textDocument/hover","params":{"t
 sleep 0.5
 
 # =============================================================================
-# STEP 5: Completion Request
+# STEP 5: Completion Requests
 # =============================================================================
-# Request code completion suggestions after the `.` in `FOO.out.view()`.
+# On `Channel` in `ch = Channel.of(1, 2, 3)`, which resolves the channel type
+# and returns ~70 items -- the DSL and value-type class registrations are what
+# produce them, so this is the probe most likely to catch a metadata gap.
 #
-send_message '{"jsonrpc":"2.0","id":3,"method":"textDocument/completion","params":{"textDocument":{"uri":"'"$DOC_URI"'"},"position":{"line":16,"character":12},"context":{"triggerKind":2,"triggerCharacter":"."}}}'
+# Member completion on a complete expression (`FOO.out.view()`) returns nothing,
+# so do not probe there: it compares empty to empty and proves the two builds
+# agree about having no answer.
+#
+send_message '{"jsonrpc":"2.0","id":3,"method":"textDocument/completion","params":{"textDocument":{"uri":"'"$DOC_URI"'"},"position":{"line":14,"character":9},"context":{"triggerKind":1}}}'
+
+sleep 0.5
+
+# At the top level (blank line 12), which offers declaration keywords and
+# snippets instead -- a different branch of the completion provider.
+#
+send_message '{"jsonrpc":"2.0","id":17,"method":"textDocument/completion","params":{"textDocument":{"uri":"'"$DOC_URI"'"},"position":{"line":12,"character":0},"context":{"triggerKind":1}}}'
 
 sleep 0.5
 
@@ -317,6 +393,20 @@ send_message '{"jsonrpc":"2.0","id":13,"method":"textDocument/prepareCallHierarc
 
 sleep 0.5
 
+# =============================================================================
+# STEP 15b: Resolve the Call Hierarchy
+# =============================================================================
+# prepareCallHierarchy on its own only locates the item; these two requests are
+# what run ScriptCallHierarchyProvider and OutgoingCallsVisitor.
+#
+send_message '{"jsonrpc":"2.0","id":18,"method":"callHierarchy/incomingCalls","params":{"item":'"$FOO_ITEM"'}}'
+
+sleep 0.5
+
+send_message '{"jsonrpc":"2.0","id":19,"method":"callHierarchy/outgoingCalls","params":{"item":'"$ENTRY_ITEM"'}}'
+
+sleep 0.5
+
 # =============================================================================
 # STEP 16: Config File
 # =============================================================================
@@ -345,17 +435,84 @@ send_message '{"jsonrpc":"2.0","id":16,"method":"textDocument/completion","param
 
 sleep 0.5
 
-send_message '{"jsonrpc":"2.0","id":17,"method":"textDocument/documentSymbol","params":{"textDocument":{"uri":"'"$CONFIG_URI"'"}}}'
+# There is deliberately no documentSymbol probe for the config file: the config
+# service has no symbol provider, so it can only ever answer with an empty list.
+
+# =============================================================================
+# STEP 17: Execute Commands
+# =============================================================================
+# All four server commands. The second previewDag argument is the workflow name,
+# and `null` means the entry workflow -- which is exactly what the code lens
+# returned at id 9. Passing a process name instead matches no workflow, so the
+# command returns null and neither DataflowVisitor nor MermaidRenderer runs.
+#
+send_message '{"jsonrpc":"2.0","id":14,"method":"workspace/executeCommand","params":{"command":"nextflow.server.previewDag","arguments":["'"$DOC_URI"'",null]}}'
+
+sleep 0.5
+
+# previewWorkspace and convertPipelineToTyped take the workspace folder name.
+#
+send_message '{"jsonrpc":"2.0","id":20,"method":"workspace/executeCommand","params":{"command":"nextflow.server.previewWorkspace","arguments":["main"]}}'
 
 sleep 0.5
 
+send_message '{"jsonrpc":"2.0","id":21,"method":"workspace/executeCommand","params":{"command":"nextflow.server.convertPipelineToTyped","arguments":["main"]}}'
+
+sleep 1
+
+# The convert commands build a WorkspaceEdit and hand it to client.applyEdit,
+# so this also drives an outbound server-to-client request.
+#
+send_message '{"jsonrpc":"2.0","id":22,"method":"workspace/executeCommand","params":{"command":"nextflow.server.convertScriptToTyped","arguments":["'"$DOC_URI"'"]}}'
+
+sleep 1
+
 # =============================================================================
-# STEP 17: Execute Command
+# STEP 17b: Cross-file Include
 # =============================================================================
-# Invoke a server-side command. Exercises ExecuteCommandParams, which is
-# deserialized on a code path no other message reaches.
+# usage.nf includes module.nf, so these two requests cover include resolution:
+# the document link points at the included file, and the definition of `BAR`
+# resolves into it.
 #
-send_message '{"jsonrpc":"2.0","id":14,"method":"workspace/executeCommand","params":{"command":"nextflow.server.previewDag","arguments":["'"$DOC_URI"'","FOO"]}}'
+send_message '{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"'"$USAGE_URI"'","languageId":"nextflow","version":1,"text":"'"$USAGE_TEXT"'"}}}'
+
+sleep 3
+
+send_message '{"jsonrpc":"2.0","id":23,"method":"textDocument/documentLink","params":{"textDocument":{"uri":"'"$USAGE_URI"'"}}}'
+
+sleep 0.5
+
+send_message '{"jsonrpc":"2.0","id":24,"method":"textDocument/definition","params":{"textDocument":{"uri":"'"$USAGE_URI"'"},"position":{"line":4,"character":5}}}'
+
+sleep 0.5
+
+# Member completion, after the dot in `channel.of(1)`. This is a different
+# branch from the two completion probes above, which complete an identifier.
+# It must be the lowercase `channel` namespace: the deprecated uppercase
+# `Channel`, and channel-typed values such as `ch.` or `FOO.out.`, both return
+# nothing. Results are filtered by the prefix under the cursor, so this answers
+# with `of` rather than the whole member list.
+#
+send_message '{"jsonrpc":"2.0","id":26,"method":"textDocument/completion","params":{"textDocument":{"uri":"'"$USAGE_URI"'"},"position":{"line":3,"character":17},"context":{"triggerKind":2,"triggerCharacter":"."}}}'
+
+sleep 0.5
+
+# =============================================================================
+# STEP 17c: Edit a Document
+# =============================================================================
+# Sync kind is Incremental, so this is the message a real editor sends on every
+# keystroke, and nothing else here ever mutates a document: it exercises the
+# ranged patch, the re-parse, and the re-published diagnostics. The edit stays
+# valid, and the documentSymbol below observes the result.
+#
+# It comes after every other main.nf probe on purpose -- their positions are
+# fixed, and this shifts the lines below the insertion point.
+#
+send_message '{"jsonrpc":"2.0","method":"textDocument/didChange","params":{"textDocument":{"uri":"'"$DOC_URI"'","version":2},"contentChanges":[{"range":{"start":{"line":16,"character":18},"end":{"line":16,"character":18}},"text":"\n    ch.view()"}]}}'
+
+sleep 3
+
+send_message '{"jsonrpc":"2.0","id":25,"method":"textDocument/documentSymbol","params":{"textDocument":{"uri":"'"$DOC_URI"'"}}}'
 
 sleep 0.5
 

native/verify.py

@@ -29,7 +29,7 @@
 SIMULATOR = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'simulate.sh')
 
 # ids sent by simulate.sh; a missing one means the server never answered
-EXPECTED_IDS = set(range(1, 18)) | {99}
+EXPECTED_IDS = set(range(1, 27)) | {99}
 
 
 def run(cmd, workspace):