Skip to content

Commit ad1a260

Browse files
authored
Docs: Add PartialSortExec documentation (apache#23092)
@mhilton mentioned the other day that he was looking for PartialSortExec but it wasn't well documented So let's fix that! ## Summary - expand PartialSortExec rustdoc with a concise explanation - add an ASCII example showing partial sorting from (a, b) to (a, b, c) cc @berkaysynnada and @akurmustafa perhaps you might have some time to review this PR
1 parent a1f56b7 commit ad1a260

1 file changed

Lines changed: 31 additions & 1 deletion

File tree

datafusion/physical-plan/src/sorts/partial_sort.rs

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,37 @@ use datafusion_physical_expr::LexOrdering;
7777
use futures::{Stream, StreamExt, ready};
7878
use log::trace;
7979

80-
/// Partial Sort execution plan.
80+
/// Sort execution plan for inputs that are already partially sorted.
81+
///
82+
/// This operator takes input ordered by a prefix of the required ordering, and
83+
/// produces output ordered by the required ordering. This is useful for
84+
/// unbounded or large inputs where a [`SortExec`] must buffer all rows before
85+
/// producing any output.
86+
///
87+
/// [`PartialSortExec`] relies on the property that rows with the same sort
88+
/// prefix are contiguous, so it can sort one prefix group at a time, emitting
89+
/// completed groups without reading (and buffering) the entire input.
90+
///
91+
/// For example, if the required output is `(a, b, c)`, but the input is only
92+
/// ordered by `(a, b)`, `PartialSortExec` sorts only within each `(a, b)`
93+
/// group to produce output ordered by `(a, b, c)`.
94+
///
95+
/// ```text
96+
/// input ordered by a, b output ordered by a, b, c
97+
///
98+
/// +---+---+---+ +---+---+---+
99+
/// | a | b | c | | a | b | c |
100+
/// +---+---+---+ +---+---+---+
101+
/// | 0 | 0 | 3 | -- same group --> | 0 | 0 | 2 |
102+
/// | 0 | 0 | 2 | | 0 | 0 | 3 |
103+
/// | 0 | 1 | 1 | -- single row --> | 0 | 1 | 1 |
104+
/// | 0 | 2 | 4 | -- same group --> | 0 | 2 | 0 |
105+
/// | 0 | 2 | 0 | | 0 | 2 | 4 |
106+
/// | 1 | 0 | 5 | -- single row --> | 1 | 0 | 5 |
107+
/// +---+---+---+ +---+---+---+
108+
/// ```
109+
///
110+
/// [`SortExec`]: crate::sorts::sort::SortExec
81111
#[derive(Debug, Clone)]
82112
pub struct PartialSortExec {
83113
/// Input schema

0 commit comments

Comments
 (0)