docs: guidance on typing untyped nulls and empty collections - #1140
Closed
nielspardon wants to merge 2 commits into
Closed
docs: guidance on typing untyped nulls and empty collections#1140nielspardon wants to merge 2 commits into
nielspardon wants to merge 2 commits into
Conversation
Substrait has no null/bottom type: nulls are typed and empty collections carry a concrete element type. Add a non-normative note telling producers to resolve an untyped source value (a bare NULL, or an empty array/map from a source that models these with a null/unknown type) to a concrete type -- from context where possible, otherwise a widely-supported type such as i32 for portability. Distinguishes this from the unbound type.
nielspardon
requested review from
EpsilonPrime,
benbellick,
cpcloud,
jacques-n,
vbarua,
westonpace and
yongchul
as code owners
July 22, 2026 09:34
bvolpato
approved these changes
Jul 25, 2026
bvolpato
left a comment
Member
There was a problem hiding this comment.
LGTM. Non-blocking wording suggestion left inline.
…ections Address non-blocking review feedback: use the nullable form i32? for a bare NULL, and spell out collection type parameters (LIST<i32> for an empty list, MAP<i32, i32> for an empty map; a map carries key and value types, not a single element type).
Member
Author
|
Superseded by #1147. Per the community sync discussion, the agreed direction is to introduce a dedicated |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Substrait deliberately has no dedicated null or "bottom" type: a null is a value of a nullable concrete type, and an empty list/map carries a concrete element type. The type system docs already state this ("null is considered to be a special value of a nullable type, rather than the only value of a special null type") and that the type system is strict with no implicit coercion.
What the docs don't yet say is how a producer should satisfy that rule when its source models untyped nulls or empty collections with a dedicated null/unknown type — e.g. Spark
NullType, ClickHouseNothing, Apache ArrowNull. This is a recurring question, and real consumers have hit it: DataFusion's Substrait producer, for example, added typed-null handling (folding an explicit cast into a typed null, and mapping a bare ArrowNullto a concrete type) rather than introducing a bottom type — see apache/datafusion#15854 and apache/datafusion#18414.This PR adds a short, non-normative note documenting the intended handling, without changing the type system:
i32) for portability;It also explicitly distinguishes this case from the
unboundtype added in #1081 (which marks a type that is not yet known in a partially bound plan), since the two concepts now sit side by side in the docs and are easy to conflate.Context
This came out of the Apache Gluten ↔ Substrait fork-consolidation effort. Gluten carries a forked
Type.Nothingmessage to represent Spark'sNullType/ ClickHouse'sNothing(Gluten PR apache/gluten#791). Rather than upstream a bottom type — which would conflict with Substrait's explicitly-typed-null design and strict no-coercion rule — the resolution is for producers to emit typed nulls / typed empty collections, which this guidance documents. See Gluten's SubstraitModifications.md.Changes
site/docs/types/type_system.md: add a non-normative note, "Untyped nulls and empty collections."Docs-only; no proto or spec-behavior change.
🤖 Generated with AI
This change is