Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apple/_version.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.10.11"
__version__ = "0.10.12"
4 changes: 4 additions & 0 deletions apple_generated/operators.py
Original file line number Diff line number Diff line change
Expand Up @@ -231,12 +231,14 @@ class RecallStaticOp(BaseOp):
_name = "recall_static"
_params_schema = {
"items": {"type": "any", "required": True},
"set_common": {"type": "any", "required": False},
}

def __call__(
self,
*,
items: Any = ...,
set_common: Any = None,
common_input: list[str] | None = None,
common_output: list[str] | None = None,
item_input: list[str] | None = None,
Expand All @@ -250,6 +252,8 @@ def __call__(
_params = {
"items": items,
}
if set_common is not None:
_params["set_common"] = set_common
return self._apply(
params=_params,
common_input=common_input,
Expand Down
2 changes: 2 additions & 0 deletions doc/operators/recall_static.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Emits a configurable static set of items for testing and validation.
| Name | Type | Required | Default | Description |
|------|------|----------|---------|-------------|
| items | any | Yes | - | JSON array of item maps to emit as candidates. |
| set_common | any | No | - | JSON object of common fields the recall writes. |

## Metadata Contract

Expand All @@ -24,6 +25,7 @@ Emits a configurable static set of items for testing and validation.
```python
flow.recall_static(
items=...,
set_common=...,
common_input=[...],
item_input=[...],
item_output=[...],
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/large_0100_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_a": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/large_0500_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_a": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/large_1000_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_a": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/large_5000_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_a": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/medium_0100_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_a": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/medium_0500_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_a": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/medium_1000_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_a": {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"_PINEAPPLE_CREATE_TIME": "2026-05-28T11:31:47Z",
"pipeline_config": {
"operators": {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"_PINEAPPLE_CREATE_TIME": "2026-05-28T11:31:47Z",
"pipeline_config": {
"operators": {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"_PINEAPPLE_CREATE_TIME": "2026-05-28T11:31:47Z",
"pipeline_config": {
"operators": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/realistic_for_you_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"_PINEAPPLE_CREATE_TIME": "2026-05-28T11:31:47Z",
"pipeline_config": {
"operators": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/realistic_for_you_latency_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"_PINEAPPLE_CREATE_TIME": "2026-05-28T11:31:47Z",
"pipeline_config": {
"operators": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/small_010_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/small_050_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/benchmarks/small_100_config.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/barrier_transform_reorder.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "barrier semantics (transform before reorder)",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"normalize_score": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/bench_cpu_work_parity.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "transform_bench_cpu byte-identical work output across runtimes (M12)",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"bench": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/control_op_nil_field_no_crash.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "control operator nil field does not crash — branch skipped",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"ctrl_if": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/data_parallel.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "data_parallel transform (sharded item execution)",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"copy_tag": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/data_parallel_lua.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "data_parallel Lua transform (verify shard-independent computation)",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_items": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/deeply_nested_subflow.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "deeply nested SubFlow (depth-3 expansion)",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"copy_tag": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/field_mode_nullable_vs_defaulted.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "three field modes — strict vs nullable vs defaulted",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"write_nil": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/field_mode_strict_nil_error.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "strict field mode — nil triggers error",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"write_nil": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/filter_removes_all.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "filter_condition removes all items — empty result",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"remove_banned": {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "implicit row-set: filter → common-only → item-field transform",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_items": {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "implicit row-set: filter then common-only transform (no dependency needed)",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_items": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/implicit_row_set_filter_then_item.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "implicit row-set: filter then item-field transform (no field overlap)",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_items": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/lua_e2e_conditional_sort.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "Lua E2E: recall → discount → stats → conditional sort",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_items": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/multi_recall_row_set_ordering.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "multi-recall row-set ordering (ConsumesRowSet semantics)",
"description": "Regression test for differential fuzz divergence #945: operators between recalls must wait for preceding recall via ConsumesRowSet before iterating items.",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_a": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/multi_skip.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "multiple independent skip flags",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"copy_tag": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/nested_subflow.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "nested SubFlow pipeline",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"copy_tag": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/nullable_field_pass_through.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "nullable field mode — explicit nil pass-through vs missing error",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"set_nil": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/recall_merge_filter_sort.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "recall → merge → filter → sort (full pipeline)",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_hot": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/recall_rank_truncate.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "recall → lua scoring → sort → truncate (recommendation pipeline)",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_candidates": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/reorder_topn_boost_parity.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "reorder_topn_boost byte-identical reorder output across runtimes (H8)",
"requires": ["bench"],
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"boost": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/resource_operators.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
]
},
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"recall_hot": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/size_dispatch_truncate.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "transform_size → dispatch → truncate (common↔item interaction)",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"count_items": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/skip_branch.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "skip/branch via Lua control flag",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"set_flag": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/subflow_if_else_control_flow.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "SubFlow if/else control flow with namespaced fields",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"my_sf::if_1": {
Expand Down
2 changes: 1 addition & 1 deletion fixtures/pipelines/transform_then_filter.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "transform_copy then filter_truncate",
"config": {
"_PINEAPPLE_VERSION": "0.10.11",
"_PINEAPPLE_VERSION": "0.10.12",
"pipeline_config": {
"operators": {
"copy_score": {
Expand Down
2 changes: 1 addition & 1 deletion llmdoc/architecture/dag-engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -538,7 +538,7 @@ RowFrame 与 ColumnFrame 必须保持稀疏字段语义一致:某一 item 未

| 算子类型 | 运行时角色 | 允许的输出方法 | 行集标记 |
|---|---|---|---|
| Recall | 产生新 item | 仅 `AddItem` | AdditiveWritesRowSet |
| Recall | 产生新 item,可选写 common | `AddItem`、`SetCommon` | AdditiveWritesRowSet |
| Transform | 变异字段值 | `SetCommon`、`SetItem` | 有 item 字段时自动注入 `_row_set_` 读;仅当 metadata 中 item 字段为空但仍结构性访问行集时需显式 ConsumesRowSet |
| Filter | 移除行 | 仅 `RemoveItem` | ConsumesRowSet + MutatesRowSet |
| Merge | 合并或去重行集 | `SetItem`、`RemoveItem` | ConsumesRowSet + MutatesRowSet |
Expand Down
4 changes: 2 additions & 2 deletions llmdoc/reference/operator-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -389,7 +389,7 @@ C++ 侧 `OperatorInput`(`include/pine/operator_input.hpp`)是 Frame + InputF
- **整列或全无**:`len(vals)` 必须等于 frame 当时的 item 数,不匹配报 `SetItemColumnFloat64 "f" length N does not match item count M`(跨引擎字节一致)
- **NaN/Inf 批量校验**先于任何写入(单次列写全有或全无),首错消息与逐元素路径相同:`item[i] write: field "f": NaN/Inf is not a valid JSON value`
- **所有权转移**:apply 时列存 frame 直接 adopt 底层数组为列存储(零拷贝,全槽位 present,typed 读快路径可命中);行存 frame 在一个锁窗口内逐行 scatter(装箱不可避免)。算子交出后**不得再读写该数组**
- 对 `ValidateOutput` 计为 `SetItem`(Transform/Filter 可用,Recall 违规);debug 快照折叠进 `item_writes` 视图;data_parallel 分片的列写由 merge 折叠为带 offset 的逐元素写(分片是窗口,无法 adopt)
- 对 `ValidateOutput` 计为 `SetItem`(Transform 可用,Recall/Filter/Observe 等非 item-write 类型违规);debug 快照折叠进 `item_writes` 视图;data_parallel 分片的列写由 merge 折叠为带 offset 的逐元素写(分片是窗口,无法 adopt)

算子作者指引:整列计算型算子(normalize 等)优先用批量写;稀疏/条件写仍用 `SetItem`。Go 侧校验注意:不要用 `validateValue(field, any(v))` 逐元素校验——`any` 转换每元素装箱一次,吃掉批量写的收益;直接内联 `math.IsNaN/IsInf`。

Expand All @@ -399,7 +399,7 @@ C++ 侧 `OperatorInput`(`include/pine/operator_input.hpp`)是 Frame + InputF

| 类型 | 预期角色 | 允许的输出方法 |
|---|---|---|
| Recall | 产生新行/item | `AddItem` |
| Recall | 产生新行/item,可选写 common | `AddItem`、`SetCommon` |
| Transform | 变异 common 或 item 字段值 | `SetCommon`、`SetItem` |
| Filter | 移除行/item | `RemoveItem` |
| Merge | 合并/去重行集 | `SetItem`、`RemoveItem` |
Expand Down
Loading