Problem
In AgentRearrange, each node blocks until the previous one fully completes before starting. On long chains (A -> B -> C -> D) where each agent streams tokens, downstream agents could begin processing partial output — today they can't.
Proposed feature
Opt-in streaming between nodes: downstream agents receive the upstream agent's token stream incrementally and start generating output before the upstream agent finishes.
Design sketch
- New flow option:
stream_between_nodes=True on the AgentRearrange constructor.
- Each node produces a generator; the next node consumes from it via an async queue.
- Buffering strategy is configurable:
"all" (wait for full chunk), "line" (per newline), or "tokens" (every N tokens).
- Fallback to today's synchronous behavior when any node in the flow doesn't support streaming.
Files
swarms/structs/agent_rearrange.py
Why
Meaningful latency win on long chains. A 4-agent pipeline where each agent takes 10s drops end-to-end wall-clock from ~40s to closer to the slowest single stage.
Problem
In
AgentRearrange, each node blocks until the previous one fully completes before starting. On long chains (A -> B -> C -> D) where each agent streams tokens, downstream agents could begin processing partial output — today they can't.Proposed feature
Opt-in streaming between nodes: downstream agents receive the upstream agent's token stream incrementally and start generating output before the upstream agent finishes.
Design sketch
stream_between_nodes=Trueon theAgentRearrangeconstructor."all"(wait for full chunk),"line"(per newline), or"tokens"(every N tokens).Files
swarms/structs/agent_rearrange.pyWhy
Meaningful latency win on long chains. A 4-agent pipeline where each agent takes 10s drops end-to-end wall-clock from ~40s to closer to the slowest single stage.