Summary
The config-generation prompt asks the LLM for total_simulation_hours and advises a 24–168 h range, but nothing in code enforces it. Any horizon the user names in their prompt is silently converted by the model to N×24 hours and accepted verbatim by the runner, one round per simulated hour. We observed a 90-day prompt produce total_simulation_hours = 2160 → a 2160-round commitment, which the pipeline cannot realistically complete (progressive degradation and eventual collapse; see also our separate memory-growth issue).
Reproduction
- Start a simulation whose user prompt names a long horizon, e.g. "over the next 90 days".
- Inspect the generated
simulation_config: total_simulation_hours = 2160 (90×24), minutes_per_round = 60.
- The runner accepts it:
total_rounds = int(total_hours * 60 / minutes_per_round) → 2160 rounds.
Relevant code (paths/lines from our read-only inspection of the backend):
app/services/simulation_config_generator.py:577 — prompt instructs the LLM to output total_simulation_hours (int) with advisory text "24-168 hours, short for breaking news, long for ongoing topics".
app/services/simulation_config_generator.py:578 — minutes_per_round (int): Time per round, 30-120 minutes, recommend 60 minutes.
app/services/simulation_runner.py:351-353 — total_hours = time_config.get("total_simulation_hours", 72); total_rounds = int(total_hours * 60 / minutes_per_round); no clamp anywhere between config generation and the runner.
- Code default is 72 h (
simulation_config_generator.py:86,565,598; simulation_runner.py:351).
Observed
- With the recommended 60 min/round, rounds = hours, so a prompt horizon of N days becomes N×24 rounds. This held on all four model families we tested (168 rounds for a 7-day horizon on every one of them).
- The advisory 24–168 range was exceeded whenever the prompt named a longer horizon: we observed 336 h (2× the documented max) and 2160 h (12.9× the documented max). The 2160-round run degraded and collapsed long before completion.
Expected
The documented range should be enforced, or at minimum an out-of-range value should require explicit confirmation. A config value 12.9× the documented maximum should not be silently accepted.
Suggested fix
- Add a code-level clamp (or validation error) on
total_simulation_hours after config generation — e.g. clamp to 24–168 with a logged warning, or make the max configurable.
- Surface the resulting
total_rounds in the UI before launch, with a warning above some threshold ("this will run N rounds; estimated duration/cost …"), so the user can consciously accept a long commitment.
Context / disclosure: We are an independent research team auditing multi-agent simulation pipelines. We found this while running a systematic 18-run study on MiroFish-Offline across four LLM families. We plan to publish a reproducible study referencing this issue in roughly 4 weeks, and we're happy to share drafts with maintainers beforehand. Thank you for open-sourcing this project — it made this kind of research possible in the first place.
Summary
The config-generation prompt asks the LLM for
total_simulation_hoursand advises a 24–168 h range, but nothing in code enforces it. Any horizon the user names in their prompt is silently converted by the model to N×24 hours and accepted verbatim by the runner, one round per simulated hour. We observed a 90-day prompt producetotal_simulation_hours = 2160→ a 2160-round commitment, which the pipeline cannot realistically complete (progressive degradation and eventual collapse; see also our separate memory-growth issue).Reproduction
simulation_config:total_simulation_hours= 2160 (90×24),minutes_per_round= 60.total_rounds = int(total_hours * 60 / minutes_per_round)→ 2160 rounds.Relevant code (paths/lines from our read-only inspection of the backend):
app/services/simulation_config_generator.py:577— prompt instructs the LLM to outputtotal_simulation_hours (int)with advisory text "24-168 hours, short for breaking news, long for ongoing topics".app/services/simulation_config_generator.py:578—minutes_per_round (int): Time per round, 30-120 minutes, recommend 60 minutes.app/services/simulation_runner.py:351-353—total_hours = time_config.get("total_simulation_hours", 72);total_rounds = int(total_hours * 60 / minutes_per_round); no clamp anywhere between config generation and the runner.simulation_config_generator.py:86,565,598;simulation_runner.py:351).Observed
Expected
The documented range should be enforced, or at minimum an out-of-range value should require explicit confirmation. A config value 12.9× the documented maximum should not be silently accepted.
Suggested fix
total_simulation_hoursafter config generation — e.g. clamp to 24–168 with a logged warning, or make the max configurable.total_roundsin the UI before launch, with a warning above some threshold ("this will run N rounds; estimated duration/cost …"), so the user can consciously accept a long commitment.Context / disclosure: We are an independent research team auditing multi-agent simulation pipelines. We found this while running a systematic 18-run study on MiroFish-Offline across four LLM families. We plan to publish a reproducible study referencing this issue in roughly 4 weeks, and we're happy to share drafts with maintainers beforehand. Thank you for open-sourcing this project — it made this kind of research possible in the first place.