Rebase vendored Substrait proto from v0.23.0 to 0.98.0

[nielspardon]

Background

Gluten vendors its Substrait .proto files (gluten-substrait/src/main/resources/substrait/proto/substrait/) pinned near Substrait v0.23.0 with local patches, while upstream Substrait is now at 0.98.0. The JVM plan producer and both native consumers (Velox, ClickHouse) regenerate from this single proto source and ship together, and Gluten's Substrait plans are transient (built per query, serialized across JNI, discarded) — so there is no wire-compatibility constraint and the rebase is a pure code migration.

Goal

Bring the vendored proto to official 0.98.0 core (verbatim) plus Gluten's genuinely engine-specific additions grafted at non-colliding numbers, migrating the JVM producer and both native backends in lockstep.

Approach

Land as small, independently-green PRs, one per message family (each keeps protoc + JVM + both native builds compiling). Minimal rebase first — relocate only colliding fork fields and drop dead/decided-drop items; extracting engine-specific forks to formal Substrait extensions is deferred to a follow-up.

Increments

• 0 — Remove unused vendored proto files + dead proto-based type derivation + deprecated Expression.Enum
• 1 — plan.proto / extensions.proto → 0.98 (URI→URN, AdvancedExtension.optimization repeated)
• 2 — Type system: temporal types → PrecisionTimestamp*; relocate Type.Nothing (kind 33→40, kept — not dropped)
• 3 — CrossRel → NestedLoopJoinRel (+ join-type enum remap)
• 4 — Windowing: WindowRel → ConsistentPartitionWindowRel; WindowType → BoundsType
• 5 — FetchRel / AggregateRel.Grouping / ExpandRel
• 6 — TopNRel / WriteRel
• 7 — ReadRel (relocate stream_kafka; text-options redesign; VirtualTable)
• 8 — Rel oneof reconciliation to exact 0.98 layout

Note on Type.Nothing

Following the Substrait community sync (2026-07-29), the earlier plan to drop the Type.Nothing fork is superseded: upstream is adding a dedicated core null type (substrait-io/substrait#1147 — Type.Null at kind field 40). Increment 2 relocates Nothing to field 40 to match, so once #1147 releases the fork converges by renaming the message Nothing→Null — no field renumbering, no consumer-mapping changes. Gluten keeps Type.Nothing until then.

Related: upstream consolidation work is tracked under the apache-gluten label in substrait-io/substrait.

[nielspardon]

Update — Type.Nothing direction changed (Increment 2)

The Increment 2 checklist item previously read "drop Type.Nothing". That's been corrected to relocate (kind 33→40, kept).

Rationale: in the Substrait community sync on 2026-07-29 the PMCs decided upstream should add a dedicated core null type rather than have producers avoid a bottom type. That proposal is up as substrait-io/substrait#1147 (Type.Null at kind field 40). Increment 2 deliberately relocates Gluten's Nothing fork to field 40 to match, so once #1147 is merged and released upstream the fork converges by a pure message rename Nothing→Null — no field renumbering and no changes to the Velox/ClickHouse consumer mappings. Gluten keeps Type.Nothing as-is until then.

Net effect: Increment 2 is semantically neutral for Nothing (relocate only), and the former "drop" step becomes a future rename gated on the upstream Type.Null release.