Handoff immutability policies, Phase 1: the policy mechanism#27
Conversation
Produced by han plan-implementation: specialist round (concurrency, architecture, testing, junior-dev stress test), decision log (22 decisions), iteration history, and the phased implementation plan for 0.6.0.
Every value crossing a worker boundary is now governed by a handoff
policy, applied at the consuming worker's intake:
- :frozen (new default): deep-freeze via Ractor.make_shareable, zero-copy;
mutation anywhere raises Shifty::PolicyViolation at the offending worker,
with a receiver heuristic (the value itself / reachable from it /
possibly unrelated) and the original FrozenError as cause.
- :isolated: private mutable deep copy via Marshal. (Empirical spike:
make_shareable(copy: true) returns a *frozen* copy, so it cannot satisfy
the mutable-scratch contract the spec assumed.)
- :shared: the raw reference, unchanged — the escape hatch for IO handles
and intentional shared mutation.
Declaration surface: policy: kwarg on Worker (the worker's contract),
with_policy(...) on a composed chain or Gang plus policy: kwarg on Gang
(pipeline default for undeclared workers), and
Shifty.configure { |c| c.default_policy = ... } (global). Precedence:
worker > pipeline > global.
IO values are rejected proactively under :frozen with
Shifty::UnshareableValue — make_shareable does NOT reject IO; it silently
freezes the live handle process-wide (verified by spike).
Also: policy-governed supply proxy covers mid-task supply.shift calls
(filter/batch/trailing); trailing_worker hands off a snapshot instead of
its live closure array; side_worker mode: :hardened is deprecated, mapped
to policy: :isolated (removal at 1.0.0); errors consolidated under
Shifty::Error; worker name: kwarg for diagnostics; required_ruby_version
>= 3.2; ostruct declared (no longer a default gem in Ruby 4); CI matrix
now 3.2/3.3/3.4; codeclimate-test-reporter dropped.
Eleven findings from the six-agent han review were confirmed by an adversarial validation pass (with live reproductions); all confirmed code findings are fixed here: - Validate policy names eagerly at every declaration site (Worker.new, Gang.new, Shifty.configure) via Policy.validate!, matching with_policy; a typo now fails where it was written, not at first shift inside a Fiber. - Remove @resolved_policy memoization so what effective_policy reports is always what intake enforces, even after late policy changes. - Rescue PolicyError in Worker#shift and discard the terminated Fiber, so a caller that rescues a PolicyViolation can keep shifting; the violating value is consumed and the pipeline continues with the next. - Govern Gang#shift's criteria-bypass path through the entry worker's intake, matching Worker#shift's bypass behavior. - Persist a Gang's declared pipeline policy and apply it to workers appended after the declaration. - Nil-guard Gang#supply so with_policy on an empty gang works. - Cycle-guard the with_policy supply-chain walk (self-referential pipes previously hung). - Deprecation-warn on ALL side_worker mode: values, with a message that names what the user actually typed; mode: :hardened maps to policy: :isolated without double-warning. - Delegate side_worker's scratch copy to Policy::Isolated (removes the duplicated Marshal round-trip). - Bound the PolicyViolation reachability walk (iterative, 50k-node cap) so a deeply nested value can't SystemStackError during error construction and mask the violation. - New specs: eager validation, violation recovery, gang bypass/append/ precedence/empty-gang, self-pipe termination, mode: :normal warning, Struct branch of the receiver heuristic.
Han code review — Phase 1 (medium change, 6 agents + adversarial validation)Automated checks: 141 examples / 0 failures · standardrb clean · Agents: junior-developer, adversarial-security-analyst, test-engineer, edge-case-explorer, structural-analyst, concurrency-analyst; consolidated findings re-attacked by an adversarial-validator with live reproductions. SecurityNo proven vulnerabilities. Marshal round-trips confirmed in-process only (never external bytes); dependencies current. Findings and outcomes (all fixed in 2d72412 unless noted)
YAGNI (advisory, intentionally not corrected)Reachability graph-walk machinery (serves the loud/local-violations goal; test-backed); What's good (per agents)The intake seam correctly governs all four Recommendation: merge. No critical findings; all confirmed warnings fixed and regression-tested. |
Part 1 of 4 for the 0.6.0 handoff-immutability release. Implements Phase 1 of
the implementation plan
(committed on this branch), which was produced from
the design doc via han planning.
What this does
Every value crossing a worker boundary is now governed by a handoff policy, applied at the consuming worker's intake:
:frozen(new default)Shifty::PolicyViolationat the offending worker:isolated:sharedDeclared per worker (
policy:kwarg — the worker's contract), per pipeline (.with_policy(...)on a chain or Gang,policy:kwarg on Gang — default for undeclared workers), or globally (Shifty.configure { |c| c.default_policy = ... }). Precedence: worker > pipeline > global.Design notes (differ from the design doc — empirically driven)
:isolateduses Marshal, notmake_shareable(copy: true): the latter returns a frozen copy, which can't satisfy the mutable-scratch-copy contract.:frozen(Shifty::UnshareableValue):Ractor.make_shareabledoes not reject IO — it silently freezes the live handle process-wide (spike verified; it froze$stdoutmid-experiment).supply.shift), so filter/batch/trailing workers and multi-yield splitter/source workers are covered with no task-body changes.Also in this PR
trailing_workernow hands off a snapshot instead of its live closure array (the exact aliasing bug class the policy exists to catch).side_worker mode: :hardeneddeprecated →policy: :isolated(warning; removal at 1.0.0). Under:isolated, a side worker's mutations evaporate — the behavior the docs always wished for.Shifty::Error;PolicyViolationcarries worker/policy/receiver/value/cause with a receiver-reachability heuristic.required_ruby_version >= 3.2;ostructdeclared (not a default gem since Ruby 4); CI matrix 2.6/2.7/3.0 → 3.2/3.3/3.4; codeclimate-test-reporter dropped (pinned simplecov to 2016).Verification
gem buildsucceeds.Phases 2–4 (testing harness & mutation detector; benchmarks &
#freeze!; docs/wiki/release) follow as separate PRs.