Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information (WoT)

📷 This is the code repository for the paper: Wrong-of-Thought: An Integrated Reasoning Framework with Multi-Perspective Verification and Wrong Information. EMNLP 2024 Findings.

The overall workflow of WoT, which consist of Planning and Solving, Multi-Perspective Verification, and Wrong Information Utilization.

Preparation steps: environment installation

(1) Environment installation command:

pip install -r requirements.txt

---

(2) Please fill in the API information in the file: src/brain.py.

openai.api_base = ""
KEY_GROUP = {'a': [""]}

💻 Stage 1: Perform WoT Reasoning.

(1) Please fill in the name of the dataset to be reasoned and the name of the model to be used in the file: scripts/run_reasoning.sh.

DATASET='dataset_name'
MODEL='model_name'

where [dataset_name] in ['gsm', 'gsmhard', 'algebra', 'addsub', 'singleop', 'singleeq', 'multiarith', 'svamp'].

---

(2) Run the following code to start using the WoT framework for reasoning.

bash src/run_reasoning.sh

💻 Stage 2: Verification of WoT reasoning results.

(1) Please fill in the name of the dataset to be validated in the file: scripts/eval.sh.

DATASET='dataset_name'

---

(2) Run the following code to obtain the result of WoT reasoning.

bash src/eval.sh

💯 Model Performance

💬 Contact

Please create Github issues here or email Yongheng Zhang or Qiguang Chen or Libo Qin if you have any questions or suggestions.