Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
__pycache__/
*.py[cod]
*$py.class
.pytest_cache/
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
<img src="assets/group_capybaras_flat.png" alt="Qwen-AgentWorld" width="1000px"/>
</div>

Welcome to the GitHub repository of Qwen-AgentWorld. Here, you can find official information about Qwen-AgentWorld, post your questions ([Issues](https://github.com/QwenLM/Qwen-AgentWorld/issues)), and share your ideas with the community ([Discussions](https://github.com/QwenLM/Qwen-AgentWorld/discussions)).
Welcome to the GitHub repository of Qwen-AgentWorld. Here, you can find official information about Qwen-AgentWorld, post your questions ([Issues](https://github.com/QwenLM/Qwen-AgentWorld/issues)), and share your ideas with the community ([Discussions](https://github.com/QwenLM/Qwen-AgentWorld/discussions)). See the [FAQ](docs/FAQ.md) for common questions about benchmark comparisons and evaluation prompts.


## News
Expand Down
17 changes: 17 additions & 0 deletions docs/FAQ.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# FAQ

## Model comparison and training lineage

Qwen-AgentWorld benchmark results can be compared with other published model results under the evaluation setup described in the README, blog, technical report, or model card. This repository contains the AgentWorldBench evaluation tooling and prompt/configuration code. It is not the authoritative source for unpublished third-party model lineage or retraining history.

For model-background questions, please rely on the model card, release notes, or maintainer announcements for the specific checkpoint being evaluated. If a benchmark table compares against another model family, treat it as an evaluation result under the stated benchmark setup rather than a statement about that model's training recipe unless the release explicitly says so.

## Official prompt for SimRL/OpenClaw-style evaluation

The evaluation script does not hard-code one universal prompt for SimRL or OpenClaw-style runs. It builds chat messages from each benchmark sample:

- `system_str`, when present, is sent as the system message.
- `current_prompt`, when present, is sent as the current user message.
- If `current_prompt` is absent, the script falls back to the sample's `prompt` list and `turn_idx`.

Use the prompt fields shipped with the benchmark/evaluation data as the canonical prompt source for reproducing AgentWorldBench runs. Domain-specific response markers and prompt file paths are configured in `eval/lwm_eval_utils/task_configs.py`.
20 changes: 20 additions & 0 deletions tests/test_faq.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import unittest
from pathlib import Path


class FaqDocumentationTest(unittest.TestCase):
def test_faq_documents_model_lineage_scope_and_prompt_source(self):
faq = Path(__file__).resolve().parents[1] / "docs" / "FAQ.md"
text = faq.read_text(encoding="utf-8")

self.assertIn("benchmark results", text)
self.assertIn("not the authoritative source", text)
self.assertIn("third-party model lineage", text)
self.assertIn("SimRL or OpenClaw-style runs", text)
self.assertIn("system_str", text)
self.assertIn("current_prompt", text)
self.assertIn("prompt fields shipped with the benchmark/evaluation data", text)


if __name__ == "__main__":
unittest.main()