Harden self-evolve replay repair and artifact lifecycle#933
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive 'self-evolve' framework for AWorld agents, enabling background candidate generation, evaluation, and automated skill improvement. Key additions include a prompt budget mechanism for agents, a robust replay adaptation system for deterministic task execution, and a candidate population executor that handles task batching and repair. The implementation also adds support for model profiles, artifact retention policies, and structured tool-call compaction. The review comments identified several areas where broad exception handling should be tightened to improve debugging and error visibility, which the author should address.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| except Exception: | ||
| pass |
There was a problem hiding this comment.
Silently passing on all exceptions with except Exception: pass is risky, as it can hide critical errors during configuration loading. This makes debugging difficult if something goes wrong. It would be better to catch specific exceptions (like ImportError) and at least log a warning for other unexpected exceptions.
except ImportError:
# The config module may not be available in all contexts.
pass
except Exception as e:
# Log other unexpected errors during config loading.
logger.warning(f"Failed to load global model profiles: {e}")| try: | ||
| reserved_output_tokens = int(raw_value) | ||
| except ValueError as exc: | ||
| raise ValueError( | ||
| "AWORLD_PROMPT_BUDGET_RESERVED_OUTPUT_TOKENS must be a positive integer" | ||
| ) from exc | ||
| if reserved_output_tokens <= 0: | ||
| raise ValueError( | ||
| "AWORLD_PROMPT_BUDGET_RESERVED_OUTPUT_TOKENS must be a positive integer" | ||
| ) |
There was a problem hiding this comment.
The validation logic for AWORLD_PROMPT_BUDGET_RESERVED_OUTPUT_TOKENS can be made more concise. The same ValueError is raised from two separate checks (one for non-integer values and one for non-positive values). Combining these checks would improve readability and reduce code duplication.
try:
reserved_output_tokens = int(raw_value)
if reserved_output_tokens <= 0:
raise ValueError("Value must be a positive integer")
except (ValueError, TypeError) as exc:
raise ValueError(
"AWORLD_PROMPT_BUDGET_RESERVED_OUTPUT_TOKENS must be a positive integer"
) from exc| except Exception as e: | ||
| logger.warning(f"⚠️ Failed to resolve model_profile '{model_profile}' in {md_file_path}: {e}") |
There was a problem hiding this comment.
Catching a broad Exception can hide unexpected errors and make debugging more difficult. It's better to catch more specific exceptions that you anticipate, such as ImportError if the module can't be found, or KeyError and ValueError if the profile is invalid or not found.
| except Exception as e: | |
| logger.warning(f"⚠️ Failed to resolve model_profile '{model_profile}' in {md_file_path}: {e}") | |
| except (ImportError, KeyError, ValueError) as e: |
| try: | ||
| conf.self_evolve_config = self_evolve_config | ||
| except Exception: | ||
| continue |
There was a problem hiding this comment.
Using a broad except Exception can mask underlying issues. If an agent's configuration object is unexpectedly not writeable or has a different structure, this exception would be silently ignored, leading to incorrect behavior that is hard to trace. It's safer to catch more specific exceptions, such as AttributeError or TypeError, if those are the expected failure modes.
| try: | |
| conf.self_evolve_config = self_evolve_config | |
| except Exception: | |
| continue | |
| try: | |
| conf.self_evolve_config = self_evolve_config | |
| except (AttributeError, TypeError): | |
| # Agent config may not support this property; continue silently. | |
| continue |
| text = json.dumps(value, ensure_ascii=False) | ||
| except Exception: | ||
| text = str(value) |
There was a problem hiding this comment.
Catching a broad Exception can hide specific errors that might be useful for debugging. For instance, json.dumps typically raises a TypeError for un-serializable objects. Catching this specific exception would make the error handling more precise.
| text = json.dumps(value, ensure_ascii=False) | |
| except Exception: | |
| text = str(value) | |
| text = json.dumps(value, ensure_ascii=False) | |
| except TypeError: | |
| text = str(value) |
| except Exception: | ||
| if scratch: | ||
| trace_path = Path(scratch) / "protocol_trace.jsonl" | ||
| try: | ||
| terminal_entry = { | ||
| "direction": "emitted", | ||
| "sequence": _next_seq(), | ||
| "kind": "error", | ||
| "fields": ["error"], | ||
| "correlation": {"error": "runtime_exception"}, | ||
| } | ||
| with open(trace_path, "a", encoding="utf-8") as f: | ||
| f.write(json.dumps(terminal_entry, ensure_ascii=False) + "\n") | ||
| except Exception: | ||
| pass |
There was a problem hiding this comment.
Catching a generic Exception can suppress important error information, making it difficult to debug runtime failures. It's better to catch more specific exceptions or, if a broad catch is necessary, to log the exception details before exiting.
| except Exception: | |
| if scratch: | |
| trace_path = Path(scratch) / "protocol_trace.jsonl" | |
| try: | |
| terminal_entry = { | |
| "direction": "emitted", | |
| "sequence": _next_seq(), | |
| "kind": "error", | |
| "fields": ["error"], | |
| "correlation": {"error": "runtime_exception"}, | |
| } | |
| with open(trace_path, "a", encoding="utf-8") as f: | |
| f.write(json.dumps(terminal_entry, ensure_ascii=False) + "\n") | |
| except Exception: | |
| pass | |
| except Exception as exc: | |
| if scratch: | |
| trace_path = Path(scratch) / "protocol_trace.jsonl" | |
| try: | |
| terminal_entry = { | |
| "direction": "emitted", | |
| "sequence": _next_seq(), | |
| "kind": "error", | |
| "fields": ["error"], | |
| "correlation": {"error": "runtime_exception", "type": type(exc).__name__}, | |
| } | |
| with open(trace_path, "a", encoding="utf-8") as f: | |
| f.write(json.dumps(terminal_entry, ensure_ascii=False) + "\n") | |
| except Exception: | |
| pass |
| except Exception as exc: | ||
| raise CandidateGenerationInfrastructureError( | ||
| stage="model_provider", | ||
| error_type=type(exc).__name__, | ||
| ) from None |
There was a problem hiding this comment.
Using a broad except Exception can hide the root cause of failures within the model provider's streaming logic. It would be more robust to catch specific, expected exceptions from the underlying library and re-raise them as CandidateGenerationInfrastructureError while allowing unexpected exceptions to propagate for better debugging.
Summary
docs/featuresanddocs/superpowerscontent from the public documentation surface and redirect example references to the curated public pageWhy
Source-level candidate conformance alone did not guarantee that a generated skill runtime could complete the WebSocket handshake and return the exact task-bound response expected by replay evaluation. The repair path also risked accumulating temporary candidate and replay artifacts over repeated optimization runs. These changes make runtime verification protocol-aware and reusable across targets, fail invalid candidates before the costly rollout phase, and reclaim stale artifacts without disrupting active self-evolve runs.
Impact
Validation
conda run -n aworld_env pytest -q tests/self_evolve— 704 passedgit diff --checkpassed