Commit f069a98
docs: log O5 — reductions persist results from incomplete member sets (NPIPE-204)
After an upstream pod's output changes, the first job.run() emits only the rows
recomputed in that pass. For a per-row pod that emission is complete; for a
many->one operator it is partial — the group is reduced over only the members
present, the reducing pod executes, and its result is persisted with nothing
marking it incomplete. It self-corrects on the next run.
Measured with two upstream pod stages: the per-row control converges in one run
while the grouped path needs two, executing on a one-member group first. So the
behaviour is NOT identical with and without GroupBy, which is how it was
previously characterised.
Batch behaves the same way, so this is pre-existing pipeline semantics surfaced
by reduction rather than something GroupBy introduced — but it matters more for
GroupBy, which exists precisely so a pod can reason over a complete group.
Not fixable inside the operator: an operator sees whatever its upstream emits
and cannot know whether a group is complete. Logged for reviewer judgment
rather than fixed here.
NPIPE-204
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent 2220a31 commit f069a98
1 file changed
Lines changed: 51 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
789 | 789 | | |
790 | 790 | | |
791 | 791 | | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
| 802 | + | |
| 803 | + | |
| 804 | + | |
| 805 | + | |
| 806 | + | |
| 807 | + | |
| 808 | + | |
| 809 | + | |
| 810 | + | |
| 811 | + | |
| 812 | + | |
| 813 | + | |
| 814 | + | |
| 815 | + | |
| 816 | + | |
| 817 | + | |
| 818 | + | |
| 819 | + | |
| 820 | + | |
| 821 | + | |
| 822 | + | |
| 823 | + | |
| 824 | + | |
| 825 | + | |
| 826 | + | |
| 827 | + | |
| 828 | + | |
| 829 | + | |
| 830 | + | |
| 831 | + | |
| 832 | + | |
| 833 | + | |
| 834 | + | |
| 835 | + | |
| 836 | + | |
| 837 | + | |
| 838 | + | |
| 839 | + | |
| 840 | + | |
| 841 | + | |
| 842 | + | |
792 | 843 | | |
793 | 844 | | |
794 | 845 | | |
| |||
0 commit comments