Socratic Self-Refine (SSR)

This is the official repo (for research purpose only) for the paper "SSR: Socratic Self-Refine for Large Language Model Reasoning," built upon the codebase of AoT.

Environment Setup

pip install openai ipdb math_verify datasets huggingface

⚙️ API Configuration Setup

You need to set up your API key and URL:

1. If you would like to use the OpenAI model, create an apikey.py file in the project root directory with the following format:

url = "https://api.openai.com/v1"  # Replace with your API endpoint
openai_api_key = [
    "your-api-key-here",  # Replace with your actual API key
    # You can add multiple API keys to improve concurrency performance.
]

2. If you want to use the Gemini model, locate the following code snippet in module_atomic.py and fill in your corresponding information:

# TODO(developer): Update and un-comment below lines
project_id = "<your-project-id>"
location = "<your-location>"

🚀 Quick Start

Running the Experiments

bash scripts/gpt-5-mini.sh

Running the Evaluation (result extraction)

Please refer to eval.ipynb.

📋 Implementation

🤔 Socratic Self-Refine Variants (Ours)

• SSR-Lin
• SSR-Ada
• SSR-Plan

🤖 Baselines

• Chain-of-Thoughts (CoT)
• Self-Refine
• Debate
• Chain-of-Thoughts Self-Consistency (CoT-SC)
• Monte-Carlo Tree Self-refine (MTCSr), to replace ToT (a bit outdated)
• Atom-of-Thoughts (AoT)

📈 Datasets

• MATH-Level-5
• AIME24
• AIME25
• Zebra-Puzzles
• Mini-Sudoku