Skip to content

Commit 523d09b

Browse files
CodexVeritasclaude
andcommitted
Transition to Fall 2026 FutureEval season
- Add FE_FALL_2026_ID (33121) and make it the current AI competition - Add 2026 Q3/Q4 Market Pulse slugs and make Q4 current - Add FallTemplateBot2026 (identical to summer) and point TemplateBot at it - Switch MainBot default to claude-opus-5 with high adaptive-thinking effort - Remove the finished Gulf Breeze tournament from the main-site tournaments - Mark bot modes with no workflow job as TournConfig.NONE - Rename "AI benchmarking" to FutureEval in the README - Bump version to 0.3.1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
1 parent d2a57b1 commit 523d09b

12 files changed

Lines changed: 804 additions & 26 deletions

File tree

README.ipynb

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@
5858
"This repository contains forecasting and research tools built with Python and Streamlit. The project aims to assist users in making predictions, conducting research, and analyzing data related to hard to answer questions (especially those from Metaculus).\n",
5959
"\n",
6060
"Here are the tools most likely to be useful to you:\n",
61-
"- 🎯 **Forecasting Bot:** General forecaster that integrates with the Metaculus AI benchmarking competition and provides a number of utilities. You can forecast with a pre-existing bot or override the class to customize your own (without redoing all the aggregation/API code, etc)\n",
61+
"- 🎯 **Forecasting Bot:** General forecaster that integrates with the Metaculus FutureEval bot tournament and provides a number of utilities. You can forecast with a pre-existing bot or override the class to customize your own (without redoing all the aggregation/API code, etc)\n",
6262
"- 🔌 **Metaculus API Wrapper:** for interacting with questions and tournaments\n",
6363
"- 🤖 **In-House Metaculus Bots**: You can see all the bots that Metaculus is running on their site in `run_bots.py`\n",
6464
"\n",
@@ -69,7 +69,7 @@
6969
"\n",
7070
"All the examples below are in a Jupyter Notebook called `README.ipynb` which you can run locally to test the package (make sure to run the first cell though).\n",
7171
"\n",
72-
"If you decide you want to join the Metaculus AI Benchmarking Tournament, it is recommended that you start [here](https://github.com/Metaculus/metac-bot-template). This repo is easy to start with and has a 30min tutorial for how to set it up.\n",
72+
"If you decide you want to join the Metaculus FutureEval Bot Tournament, it is recommended that you start [here](https://github.com/Metaculus/metac-bot-template). This repo is easy to start with and has a 30min tutorial for how to set it up.\n",
7373
"\n",
7474
"Join the [discord](https://discord.gg/Dtq4JNdXnw) for updates and to give feedback (btw feedback is very appreciated, even just a quick \"I did/didn't decide to use tool X for reason Y, though am busy and don't have time to elaborate\" is helpful to know)\n",
7575
"\n",

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ Demo repo (get a Metaculus bot running in 30min): https://github.com/Metaculus/m
1818
This repository contains forecasting and research tools built with Python and Streamlit. The project aims to assist users in making predictions, conducting research, and analyzing data related to hard to answer questions (especially those from Metaculus).
1919

2020
Here are the tools most likely to be useful to you:
21-
- 🎯 **Forecasting Bot:** General forecaster that integrates with the Metaculus AI benchmarking competition and provides a number of utilities. You can forecast with a pre-existing bot or override the class to customize your own (without redoing all the aggregation/API code, etc)
21+
- 🎯 **Forecasting Bot:** General forecaster that integrates with the Metaculus FutureEval bot tournament and provides a number of utilities. You can forecast with a pre-existing bot or override the class to customize your own (without redoing all the aggregation/API code, etc)
2222
- 🔌 **Metaculus API Wrapper:** for interacting with questions and tournaments
2323
- 🤖 **In-House Metaculus Bots**: You can see all the bots that Metaculus is running on their site in `run_bots.py`
2424

@@ -29,7 +29,7 @@ Here are some other features of the project (not all are documented yet):
2929

3030
All the examples below are in a Jupyter Notebook called `README.ipynb` which you can run locally to test the package (make sure to run the first cell though).
3131

32-
If you decide you want to join the Metaculus AI Benchmarking Tournament, it is recommended that you start [here](https://github.com/Metaculus/metac-bot-template). This repo is easy to start with and has a 30min tutorial for how to set it up.
32+
If you decide you want to join the Metaculus FutureEval Bot Tournament, it is recommended that you start [here](https://github.com/Metaculus/metac-bot-template). This repo is easy to start with and has a 30min tutorial for how to set it up.
3333

3434
Join the [discord](https://discord.gg/Dtq4JNdXnw) for updates and to give feedback (btw feedback is very appreciated, even just a quick "I did/didn't decide to use tool X for reason Y, though am busy and don't have time to elaborate" is helpful to know)
3535

code_tests/unit_tests/test_public_api.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@ def test_public_api_imports() -> None:
7272
"DiscreteReport",
7373
"FallResearchOnlyBot2025",
7474
"FallTemplateBot2025",
75+
"FallTemplateBot2026",
7576
"GPT41OptimizedBot",
7677
"MetaculusClient",
7778
"Percentile",

forecasting_tools/__init__.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,9 @@
150150
from forecasting_tools.forecast_bots.official_bots.template_bot_2025_fall import (
151151
FallTemplateBot2025 as FallTemplateBot2025,
152152
)
153+
from forecasting_tools.forecast_bots.official_bots.template_bot_2026_fall import (
154+
FallTemplateBot2026 as FallTemplateBot2026,
155+
)
153156
from forecasting_tools.forecast_bots.official_bots.template_bot_2026_spring import (
154157
SpringTemplateBot2026 as SpringTemplateBot2026,
155158
)

forecasting_tools/forecast_bots/bot_lists.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,9 @@
3232
from forecasting_tools.forecast_bots.official_bots.template_bot_2025_fall import (
3333
FallTemplateBot2025,
3434
)
35+
from forecasting_tools.forecast_bots.official_bots.template_bot_2026_fall import (
36+
FallTemplateBot2026,
37+
)
3538
from forecasting_tools.forecast_bots.official_bots.template_bot_2026_spring import (
3639
SpringTemplateBot2026,
3740
)
@@ -62,6 +65,7 @@ def get_all_important_bot_classes() -> list[type[ForecastBot]]:
6265
FallResearchOnlyBot2025,
6366
SpringTemplateBot2026,
6467
SummerTemplateBot2026,
68+
FallTemplateBot2026,
6569
GPT41OptimizedBot,
6670
NoResearchOneShotBot,
6771
]

forecasting_tools/forecast_bots/main_bot.py

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -42,10 +42,12 @@ async def run_research(self, question: MetaculusQuestion) -> str:
4242
def _llm_config_defaults(cls) -> dict[str, str | GeneralLlm]:
4343
return {
4444
"default": GeneralLlm(
45-
model="openai/gpt-5",
46-
reasoning_effort="high",
47-
temperature=0.3,
48-
timeout=15 * 60,
45+
model="anthropic/claude-opus-5",
46+
thinking={"type": "adaptive"},
47+
output_config={"effort": "high"},
48+
allowed_openai_params=["output_config"],
49+
max_tokens=64000,
50+
timeout=160,
4951
),
5052
"summarizer": GeneralLlm(model="openai/gpt-4o", temperature=0),
5153
}

0 commit comments

Comments
 (0)