fix: harden runtime API and simulation handling

[CyrusAuyeung]

Summary

Fixes #717.

This PR hardens runtime paths that can break end-to-end runs when using OpenAI-compatible third-party providers and long-running dual-platform simulations:

• pass configured API key/base URL and provider-safe headers to direct OpenAI SDK clients and CAMEL simulation clients
• make persisted simulation state reads tolerate UTF-8 BOMs
• validate real simulation process liveness before accepting interview/deep-interaction IPC commands
• preserve completed simulation state during shutdown cleanup, with regression coverage
• advance action cursors after initial seed posts so they are not duplicated into the next round
• preserve multiple initial posts assigned to the same agent in Twitter paths
• clean terminal-run logs before a restart while refusing to discard pending graph-memory updates
• suppress idle and empty action cards while retaining meaningful like/dislike/comment activity
• restore an existing simulation view instead of force-starting the simulation again on mount
• return validation-style 400s for malformed JSON request bodies instead of surfacing 500s
• declare psutil as a direct backend dependency because simulation liveness imports it directly

Rebase notes

Rebased onto main@60757b3c after #742 merged.

• fix: modernize Zep Cloud integration and dependency security #740 owns the shared Zep query normalization boundary used by graph search and profile enrichment; the duplicated local query slicing remains removed.
• fix: harden ontology JSON generation across LLM providers #742 owns ontology structured-output validation, truncation handling, bounded retry behavior, and safe 502 responses. This PR does not alter or reintroduce the superseded ontology response path.
• The only remaining llm_client.py delta is the provider-safe User-Agent header applied to the new fix: harden ontology JSON generation across LLM providers #742 client constructor. The graph API delta only makes malformed request bodies follow the existing validation path.

The provider configuration, IPC liveness, state handling, initial-post handling, malformed JSON, BOM, and simulation action fixes remain independent changes in this PR.

Related context

• interview/deep-interaction reports: 与世界中任意个体对话功能和发送调查问卷到世界中 两个功能报错 IPC 响应 failed #45, 无法和指定角色对话 #217, Step 5 "Send Survey" fails with 400 "environment not running" after simulation completes #551, fix(interview): repair interview-history endpoint + show past interviews in Step 5 #692
• low/odd posting behavior: 发帖数量过少 #322
• IPC/resource robustness: Improve database resource management and IPC input handling #578
• polling/process state: fix: stop polling when simulation fails #518
• shared Zep Cloud modernization and query normalization: fix: modernize Zep Cloud integration and dependency security #740
• ontology structured-output and safe provider-error handling: fix: harden ontology JSON generation across LLM providers #742

Validation

• uv run python -m pytest -q: 133 passed
• npm run build: passed
• git diff --check origin/main...HEAD: passed
• origin/main verified as an ancestor of the rebased head

The frontend build still emits the existing warning about a large JS chunk and a module that is both statically and dynamically imported. The backend tests pass without failures.

[666ghj]

Note

🤖 Automated maintainer response — Generated by the MiroFish triage agent and checked against main@d641759 on 2026-07-23. Posted with maintainer authorization; a maintainer may revise it.

Reviewed against current main. The Zep 400-character query cap in this PR has now landed through #740 at the shared service boundary with regression tests for both graph search and profile enrichment.

The provider configuration, IPC liveness, completed-state cleanup, initial-post handling, malformed JSON, BOM state, and psutil changes remain separate runtime work and have not been closed by #740. Please rebase and remove the now-duplicated Zep query change; this PR stays open for review of the remaining scope.

[CyrusAuyeung]

Addressed in the force-updated head 690a6a2:

• rebased fix: harden runtime API and simulation handling #718 onto main@d641759 (the fix: modernize Zep Cloud integration and dependency security #740 merge)
• removed the duplicated local Zep 400-character slicing from both graph search and profile enrichment; the PR now relies on fix: modernize Zep Cloud integration and dependency security #740's shared normalization boundary
• reconciled the simulation restart path with fix: modernize Zep Cloud integration and dependency security #740's graph lifecycle and pending-ingestion safeguards
• added regression coverage for terminal restart cleanup, pending graph updates, and preserving completed state during shutdown

Validation after the rebase: backend 113 passed, frontend production build passed, and git diff --check passed.

[666ghj]

Note

🤖 Automated maintainer response — Generated by the MiroFish maintenance agent after reviewing this PR against main@60757b3c on 2026-07-23. Posted with maintainer authorization; a maintainer may revise it.

Current main has advanced again through #742, which resolves the ontology structured-output, truncation, and opaque 500 path referenced in this PR and #717.

The remaining provider request/header handling, IPC liveness, completed-state cleanup, initial-post behavior, malformed request handling, BOM state, and simulation UI changes are independent and remain eligible for review. Please rebase onto main@60757b3c and reconcile any llm_client.py or ontology-error overlap with #742; do not reintroduce the superseded ontology path.

This PR stays open for the remaining runtime scope.

[CyrusAuyeung]

Addressed in the force-updated head d426d6b:

• rebased fix: harden runtime API and simulation handling #718 onto main@60757b3c after fix: harden ontology JSON generation across LLM providers #742
• preserved fix: harden ontology JSON generation across LLM providers #742's ontology structured-output parsing, truncation retry, and safe 502 error path without reintroducing the superseded implementation
• reconciled the remaining llm_client.py change so it only adds the provider-safe User-Agent header to fix: harden ontology JSON generation across LLM providers #742's new client constructor
• retained malformed graph request handling as an independent validation-path fix

Validation after the rebase: backend 133 passed, frontend production build passed, git diff --check passed, and origin/main is an ancestor of the updated head.