Skip to content

Transition to Fall 2026 FutureEval season - #341

Merged
CodexVeritas merged 1 commit into
mainfrom
claude/new-season-code-review-d07fb4
Sep 8, 2026
Merged

Transition to Fall 2026 FutureEval season#341
CodexVeritas merged 1 commit into
mainfrom
claude/new-season-code-review-d07fb4

Conversation

@CodexVeritas

@CodexVeritas CodexVeritas commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Summary

Season transition from Summer 2026 to the Fall 2026 FutureEval Bot Tournament (opens Sept 28, resolves by Jan 5).

  • Tournament constants (metaculus_client.py): add FE_FALL_2026_ID = 33121 and make it CURRENT_AI_COMPETITION_ID; add Q3_2026_MARKET_PULSE_ID / Q4_2026_MARKET_PULSE_ID and make Q4 current (Q3 ends in ~8 days). The fall Metaculus Cup ID was already set on main.
  • New template bot: FallTemplateBot2026 (byte-identical copy of summer, class name and docstring updated). Exported from the package, added to bot_lists.py, and TemplateBot now subclasses it. test_public_api.py updated accordingly.
  • MainBot default switched from openai/gpt-5 (API shutoff Dec 10, 2026, mid-season) to anthropic/claude-opus-5 with the same adaptive-thinking high-effort settings used by the run_bots Claude high bots.
  • Gulf Breeze removed from AllowedTourn / main_site_tourns / cost CSV map. The contest ended in January 2026.
  • Bot modes with no workflow job marked TournConfig.NONE so run_bots.py and run-bot-aib-tournament.yaml now match in both directions. No bots were added or removed otherwise.
  • README wording updated from "AI benchmarking" to FutureEval.
  • Version bumped to 0.3.1.

Modes marked TournConfig.NONE

Mode Previously
METAC_DEEPSEEK_R1_DEEPNEWS experimental
METAC_DEEPSEEK_R1_O4_MINI_DEEP_RESEARCH experimental
METAC_DEEPSEEK_R1_SONAR_DEEP_RESEARCH experimental
METAC_DEEPSEEK_R1_SONNET_4_SEARCH experimental
METAC_KIMI_K2_VARIANCE_TEST aib_only
METAC_O4_MINI_DEEP_RESEARCH experimental
METAC_ONLY_SONAR_REASONING_PRO aib_only
METAC_SONAR_DEEP_RESEARCH experimental
METAC_SONNET_4_SEARCH experimental

Reviewer notes

  • Switching CURRENT_AI_COMPETITION_ID now means new summer-tournament questions published after this merges will not be picked up by MAIN_AIB. Merge timing is a judgment call.
  • MainBot omits temperature, matching the existing Opus 5 and Fable 5 high bot configs.
  • The sibling metac-bot-template repo still needs the fall template update separately.

Test plan

  • pre-commit passes on all changed files
  • test_run_bots.py, test_public_api.py, test_optional_imports.py pass
  • Full unit suite: 14 failures, all pre-existing local-timezone flakes (test_basic_get_questions at 01:00 UTC and the two is_jsonable tests). Verified the identical 14 fail on an untouched checkout of main.
  • Smoke-tested MainBot() instantiation and that TemplateBot resolves to FallTemplateBot2026

🤖 Generated with Claude Code

- Add FE_FALL_2026_ID (33121) and make it the current AI competition
- Add 2026 Q3/Q4 Market Pulse slugs and make Q4 current
- Add FallTemplateBot2026 (identical to summer) and point TemplateBot at it
- Switch MainBot default to claude-opus-5 with high adaptive-thinking effort
- Remove the finished Gulf Breeze tournament from the main-site tournaments
- Mark bot modes with no workflow job as TournConfig.NONE
- Rename "AI benchmarking" to FutureEval in the README
- Bump version to 0.3.1

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@CodexVeritas
CodexVeritas merged commit 8742dee into main Sep 8, 2026
2 checks passed
@CodexVeritas
CodexVeritas deleted the claude/new-season-code-review-d07fb4 branch September 8, 2026 15:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant