You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Reconcile concurrency docs with handoff immutability plan
Distinguish preemptive concurrency hazards (removed by single-threading)
from data-mutation hazards across handoffs (not removed); reframe the
Threads/Ractors note as a scoping decision that is Ractor-compatible; and
scope the thread-safety caveat to worker/pipeline objects and closure
state. Forward-reference docs/planning/handoff-immutability-policies.md.
Copy file name to clipboardExpand all lines: README.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -11,6 +11,8 @@ _"How many Ruby fibers does it take to screw in a lightbulb?"_
11
11
12
12
Shifty utilizes Ruby Fibers for cooperative multitasking. This means that all tasks (or "workers") run within a single operating system thread and explicitly yield control to one another. This model is intentionally chosen for its simplicity, which makes it easier to reason about and build sequential data processing pipelines.
13
13
14
+
Single-threading frees you from *preemptive* concurrency hazards (races, mutexes) within a pipeline. It does not, by itself, make the data passing between workers safe — a worker that mutates a value it was handed can still corrupt what later workers see. Governing those handoffs is a separate concern; see the planned [handoff immutability policies](docs/planning/handoff-immutability-policies.md). The Fiber model is also deliberately compatible with a future Ractor-backed worker type, rather than a rejection of parallelism.
15
+
14
16
For a more detailed explanation of Shifty's design and typical use cases, please see the [Use Cases Document](docs/use_cases.md).
Copy file name to clipboardExpand all lines: docs/use_cases.md
+13Lines changed: 13 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,6 +39,19 @@ Here are a few conceptual examples of how Shifty could be applied:
39
39
2. A `BatchWorker` (Shifty provides a `batch_worker` for this purpose) accumulates these items. It passes the accumulated batch to the next worker once a certain number of items are collected or a timeout occurs.
40
40
3. A `BatchProcessorWorker` then processes the entire batch of items at once (e.g., bulk database insert, writing to a file).
41
41
42
+
## A Note on Values Passed Between Workers
43
+
44
+
The examples above pass data from one worker to the next. Because Shifty runs
45
+
each value through every worker before starting the next value, workers should
46
+
treat a handed-off value as read-only and express changes as new values
47
+
(`arr + [x]`, `hash.merge(...)`, `value.with(...)`) rather than mutating in
48
+
place (`arr <<`, `hash[k] =`, `map!`). This keeps failures local and is the
49
+
direction Shifty is moving: a planned release makes deeply frozen handoffs the
50
+
default, with opt-in policies for workers that genuinely need a private scratch
51
+
copy or a shared mutable reference. See
52
+
[handoff immutability policies](planning/handoff-immutability-policies.md) for
53
+
the full design and migration guidance.
54
+
42
55
## Conclusion
43
56
44
57
Shifty aims to provide an intuitive and straightforward way to construct data processing systems within Ruby applications. By breaking down complex tasks into manageable, cooperatively multitasking workers, it helps in building modular, maintainable, and easy-to-understand data pipelines.
0 commit comments