Skip to content

Streaming engine occasionally ignores maintain_order on right/full joins #29362

Description

@kdn36

Checks

  • I have checked that this issue has not already been reported.
  • I have confirmed this bug exists on the latest version of Polars.

Reproducible example

import polars as pl

a = pl.LazyFrame({"ak": [1]})
b = pl.LazyFrame({"bk": [1, 2, 3, 4, 5], "bv": [10, 20, 30, 40, 50]})

q = a.join(b, left_on="ak", right_on="bk", how="right", maintain_order="left_right")

print(q.collect(engine="in-memory"))  # ok
print(q.collect(engine="streaming"))  # sometimes not ok

run as

POLARS_MAX_THREADS=2 python tt.py

Log output

shape: (5, 2)
┌─────┬─────┐
│ bk  ┆ bv  │
│ --- ┆ --- │
│ i64 ┆ i64 │
╞═════╪═════╡
│ 1   ┆ 10  │
│ 2   ┆ 20  │
│ 3   ┆ 30  │
│ 4   ┆ 40  │
│ 5   ┆ 50  │
└─────┴─────┘
shape: (5, 2)
┌─────┬─────┐
│ bk  ┆ bv  │
│ --- ┆ --- │
│ i64 ┆ i64 │
╞═════╪═════╡
│ 1   ┆ 10  │
│ 3   ┆ 30  │
│ 2   ┆ 20  │
│ 5   ┆ 50  │
│ 4   ┆ 40  │
└─────┴─────┘

Issue description

maintain_order not respected in streaming engine

Expected behavior

results match

Installed versions

Reproduces on 1.44.2 and latest main

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

A-streamingRelated to the streaming enginebugSomething isn't workingneeds triageAwaiting prioritization by a maintainerpythonRelated to Python Polars

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions