Remove the SetCommon restriction from Recall's ValidateOutput in all
three engines — Go, Java, C++. A Recall may now emit request-level
common state (e.g. a recall-generated request id) that downstream
operators consume. The DAG dependency layer already builds correct
RAW/WAW/WAR edges from CommonOutput regardless of operator type,
confirmed by a three-case probe before this change.
Recall's common write is a normal mutating hazard participant:
- Two recalls writing the same common field → WAW serialized.
- Two recalls writing different common fields → item additive
parallelism preserved.
- Downstream common reader → RAW edge from the recall (automatic).
Still forbidden: SetItem, RemoveItem, SetItemOrder (only AddItem for
items). The restriction half that guards data integrity is unchanged.
recall_static gains an optional `set_common` param across all three
engines + codegen, enabling fixture-driven testing and diff-fuzz
coverage of the new path.
differential-fuzz.py: ~30% of generated recall_static ops now write
random common fields, injecting them into prev_common_outputs for
downstream consumption. 250-round go+java smoke verified 0 divergences.
Tests:
- Go: ValidateOutput direct tests (new file), scheduler E2E
(recall writes common → downstream transform reads it)
- Java: ValidateOutputTest (6 cases mirroring Go)
- C++: test_engine.cpp reversed to "Recall may SetCommon" positive
test + new "Recall must not SetItem" negative test
- cross-validate section 1 (codegen schema parity) green
Summary
Removes the
SetCommonrestriction from Recall'sValidateOutputacross all three engines, allowing Recall operators to emit request-level common state (e.g. a recall-generated request id) that downstream operators consume.Motivation
The previous design forbade Recall from writing common fields. This meant recall-produced metadata (request ids, trace ids, source identifiers) had no legitimate exit — it either got stuffed into every item row (semantically wrong, N-fold redundant) or was discarded. The restriction existed for DAG dependency simplification, but investigation confirmed the DAG layer already builds correct edges from
CommonOutputregardless of operator type — the restriction was purely a runtime assertion with no structural justification.Design
Recall's common write is a normal mutating hazard participant in the DAG common pass:
This was verified by a three-case probe (now deleted — pure investigation, not production test) before any code change. The DAG
addEdgesfunction processesCommonOutputwithout inspecting operator type.Still forbidden:
SetItem,RemoveItem,SetItemOrder. Recall's only item action remainsAddItem(additive).Changes
Engine core (3 engines × ValidateOutput)
operator.go: removehasCWcheck from Recall branchOperatorType.java: removehasCommonWritescheck from RECALL caseengine.cpp: removehas_cwcheck from "recall" branchrecall_static enhancement (enables fixture + fuzz testing)
set_commonparam (typeany, default null): JSON object of common fields the recall writesDifferential fuzz enhancement
recall_staticops now write random common fieldsprev_common_outputsfor downstream consumptionTests
validate_output_test.go(6 cases: recall may write common, still forbids item mutations, transform/observe contracts, empty-output clean); scheduler E2ETestRunRecallWritesCommonConsumedDownstreamValidateOutputTest.java(6 mirrored cases)test_engine.cppreversed from "Recall must not SetCommon" error assertion to positive "Recall may SetCommon (downstream consumable)" + new "Recall must not SetItem" negative caseDocumentation
dag-engine.md: type constraint matrix updatedoperator-contract.md: type table + SetItemColumnFloat64 note correctedTest plan