Skip to content

feat(protos): deprecate OuterReference.steps_out in favor of rel_reference - #1132

Open
nielspardon wants to merge 1 commit into
substrait-io:mainfrom
nielspardon:feat/deprecate-outer-reference-steps-out
Open

feat(protos): deprecate OuterReference.steps_out in favor of rel_reference#1132
nielspardon wants to merge 1 commit into
substrait-io:mainfrom
nielspardon:feat/deprecate-outer-reference-steps-out

Conversation

@nielspardon

@nielspardon nielspardon commented Jul 16, 2026

Copy link
Copy Markdown
Member

What / Why

PR #1031 introduced id-based outer reference resolution (OuterReference.rel_reference) with the stated intent of deprecating the offset-based steps_out mechanism at a later point.

In the community sync on 2026-07-15 we agreed that when we already have the intent to deprecate a field, we should mark it deprecated right away when introducing the alternative rather than delaying — giving producers the migration signal as early as possible. Worst case, we can walk back the deprecation in a future release.

Keeping both forms is awkward for implementations

Concretely, this ambiguity already surfaced in the substrait-java implementation (substrait-io/substrait-java#982). With both steps_out and rel_reference valid, it is unclear whether a roundtripped plan must preserve whichever form it arrived in, or whether an implementation is free to migrate between them (e.g. normalize steps_outrel_reference at the conversion boundary, as that PR does). Since id-based resolution is strictly more expressive than offset-based steps_out — it is unambiguous even in DAG-shaped plans with shared relations (ReferenceRel), where steps_out is not — there is little reason to keep both around long term. Deprecating steps_out now makes the intended direction explicit: id-based is the form to converge on.

Precedent

Substrait has repeatedly superseded a less expressive construct with a more expressive one and deprecated the original, following through to removal:

  • #1130FetchRel.offset/count (int64 literals) → offset_expr/count_expr (Expression)
  • #1129 — join left_keys/right_keys → the more general keys / expression-based conditions
  • #1002Aggregate.Grouping.grouping_expressions → the reference-based grouping mechanism
  • #994time/timestamp/timestamp_tz → the precision-carrying precision_* types

steps_outrel_reference is the same pattern; this PR takes the deprecation step.

Changes

  • Mark Expression.FieldReference.OuterReference.steps_out with [deprecated = true] and document the recommended rel_reference replacement.
  • Update the field reference and subquery docs to note steps_out is deprecated in favor of rel_reference.

Compatibility

steps_out remains fully functional for backward compatibility. This only adds the deprecated = true option — there is no wire-format change, and buf breaking passes against main, so this is not a breaking change.

🤖 Generated with AI


This change is Reviewable

…rence

PR substrait-io#1031 introduced id-based outer reference resolution
(`OuterReference.rel_reference`) with the stated intent of deprecating the
offset-based `steps_out` mechanism at a later point. Following the community
sync on 2026-07-15, we mark `steps_out` deprecated now, when the alternative
already exists, rather than delaying the signal to producers.

`steps_out` remains fully functional for backward compatibility; this only
adds the `deprecated = true` option and updates the docs. It is not a
wire-breaking change (`buf breaking` passes).
@benbellick

Copy link
Copy Markdown
Member

This may be because I missed the last sync and so I lack context, sorry about that. But I didn't know the intention was to deprecate this field. Is the reasoning for that captured anywhere?

@nielspardon

Copy link
Copy Markdown
Member Author

This may be because I missed the last sync and so I lack context, sorry about that. But I didn't know the intention was to deprecate this field. Is the reasoning for that captured anywhere?

I added some more context for you in the PR description above. You may also want to listen to the discussion in the sync recording when it becomes available.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants