fix: resolve O(n^3) performance degradation in workflow tester (#377)

[denisaditya0]

Fixes #377

Problem

The workflow tester exhibits O(n^3) performance degradation when using workflow.Go with many concurrent goroutines. A workflow spawning 256 goroutines takes ~71 seconds in the tester vs milliseconds with a real backend.

Root Cause

Two compounding factors:

1. Executor recreation on every task — The tester created a new executor for each workflow task, forcing full history replay. With n activities completing one at a time, this means n replays of growing history (O(n^2) total replay work).

2. Linear scan in CommandByScheduleEventID — During replay, each event lookup scans all commands linearly. With O(n) events each scanning O(n) commands, this adds another factor of n, yielding O(n^3) overall.

Changes

1. Cache executor per workflow instance in tester (tester/tester.go)

The real worker already caches executors via an LRU cache. The tester now does the same — stores the executor on the testWorkflow struct and reuses it across tasks. The executor is closed when the workflow finishes to prevent goroutine leaks.

2. O(1) command lookup via index map (internal/workflowstate/workflowstate.go)

Added a commandIndex map[int64]command.Command alongside the existing slice. AddCommand populates both, and CommandByScheduleEventID now returns directly from the map instead of scanning the slice.

Benchmark Results

Using the exact reproduction from #377 (n goroutines, each executing one activity):

n	Before	After	Speedup	Alloc reduction
1	1.67ms	1.66ms	1x	1x
32	301ms	15.6ms	19x	20x
64	1,268ms	38.6ms	33x	42x
128	8,587ms	209ms	41x	84x
256	70,859ms	507ms	140x	170x

At n=256: 8.5 GB heap allocations reduced to 50 MB per operation (measured via -benchmem).

Raw benchmark output

Before (main):

BenchmarkWorkflowGo/n=1-14            742     1672040 ns/op      34585 B/op      399 allocs/op
BenchmarkWorkflowGo/n=32-14             6   300645967 ns/op   20481288 B/op   253442 allocs/op
BenchmarkWorkflowGo/n=64-14             1  1268183200 ns/op  146344936 B/op  1789189 allocs/op
BenchmarkWorkflowGo/n=128-14            1  8586964900 ns/op 1103860864 B/op 13428877 allocs/op
BenchmarkWorkflowGo/n=256-14            1 70859178000 ns/op 8574337680 B/op 104063008 allocs/op

After (this PR):

BenchmarkWorkflowGo/n=1-14            720     1659500 ns/op      27911 B/op      294 allocs/op
BenchmarkWorkflowGo/n=32-14           100    15586562 ns/op    1006682 B/op    12635 allocs/op
BenchmarkWorkflowGo/n=64-14            32    38609597 ns/op    3522029 B/op    43515 allocs/op
BenchmarkWorkflowGo/n=128-14            9   209144400 ns/op   13125666 B/op   160591 allocs/op
BenchmarkWorkflowGo/n=256-14            2   507286200 ns/op   50688852 B/op   616224 allocs/op

Remaining O(n^2) behavior

The coroutine scheduler (internal/sync/scheduler.go) iterates all coroutines on every Continue() call. This is inherent to the cooperative scheduling model and would require a more invasive redesign to address. However, with the cubic factor eliminated, the tester is now practical for real-world use cases (256 goroutines in 500ms vs 71 seconds).

Testing

• All existing tester tests pass (42 tests)
• All internal package tests pass
• All workflow executor tests pass
• Benchmark included in tester/tester_bench_test.go