feat(protos): deprecate OuterReference.steps_out in favor of rel_reference - #1132
Open
nielspardon wants to merge 1 commit into
Open
feat(protos): deprecate OuterReference.steps_out in favor of rel_reference#1132nielspardon wants to merge 1 commit into
nielspardon wants to merge 1 commit into
Conversation
…rence PR substrait-io#1031 introduced id-based outer reference resolution (`OuterReference.rel_reference`) with the stated intent of deprecating the offset-based `steps_out` mechanism at a later point. Following the community sync on 2026-07-15, we mark `steps_out` deprecated now, when the alternative already exists, rather than delaying the signal to producers. `steps_out` remains fully functional for backward compatibility; this only adds the `deprecated = true` option and updates the docs. It is not a wire-breaking change (`buf breaking` passes).
nielspardon
requested review from
EpsilonPrime,
benbellick,
cpcloud,
jacques-n,
vbarua,
westonpace and
yongchul
as code owners
July 16, 2026 11:47
Member
|
This may be because I missed the last sync and so I lack context, sorry about that. But I didn't know the intention was to deprecate this field. Is the reasoning for that captured anywhere? |
Member
Author
I added some more context for you in the PR description above. You may also want to listen to the discussion in the sync recording when it becomes available. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What / Why
PR #1031 introduced id-based outer reference resolution (
OuterReference.rel_reference) with the stated intent of deprecating the offset-basedsteps_outmechanism at a later point.In the community sync on 2026-07-15 we agreed that when we already have the intent to deprecate a field, we should mark it deprecated right away when introducing the alternative rather than delaying — giving producers the migration signal as early as possible. Worst case, we can walk back the deprecation in a future release.
Keeping both forms is awkward for implementations
Concretely, this ambiguity already surfaced in the substrait-java implementation (substrait-io/substrait-java#982). With both
steps_outandrel_referencevalid, it is unclear whether a roundtripped plan must preserve whichever form it arrived in, or whether an implementation is free to migrate between them (e.g. normalizesteps_out→rel_referenceat the conversion boundary, as that PR does). Since id-based resolution is strictly more expressive than offset-basedsteps_out— it is unambiguous even in DAG-shaped plans with shared relations (ReferenceRel), wheresteps_outis not — there is little reason to keep both around long term. Deprecatingsteps_outnow makes the intended direction explicit: id-based is the form to converge on.Precedent
Substrait has repeatedly superseded a less expressive construct with a more expressive one and deprecated the original, following through to removal:
FetchRel.offset/count(int64 literals) →offset_expr/count_expr(Expression)left_keys/right_keys→ the more generalkeys/ expression-based conditionsAggregate.Grouping.grouping_expressions→ the reference-based grouping mechanismtime/timestamp/timestamp_tz→ the precision-carryingprecision_*typessteps_out→rel_referenceis the same pattern; this PR takes the deprecation step.Changes
Expression.FieldReference.OuterReference.steps_outwith[deprecated = true]and document the recommendedrel_referencereplacement.steps_outis deprecated in favor ofrel_reference.Compatibility
steps_outremains fully functional for backward compatibility. This only adds thedeprecated = trueoption — there is no wire-format change, andbuf breakingpasses againstmain, so this is not a breaking change.🤖 Generated with AI
This change is