Update dependencies and move agent/front-end deps to optional extras - #339
Merged
Conversation
Update every package to the latest version reachable within (or by widening) its constraint, then split out the dependencies the bots do not need. Dependency updates - Bump ceilings only, never floors: openai <3->4, plotly <7->8, hyperbrowser <1->2, openai-agents <0.20->0.23. - litellm 1.85.0 -> 1.100.0, plotly 7.0.0, hyperbrowser 1.4.2, openai-agents 0.20.0, streamlit 1.63.0, plus ~60 transitive updates. - openai stays on 2.x: every litellm >=1.85 pins openai<3.0.0, so openai 3.x and a current litellm are mutually exclusive. Keeping the floor at 1.51.0 and the ceiling at <4.0.0 lets a downstream project pin openai ^3 itself and get openai 3.8 + openai-agents 0.22 + litellm 1.83 instead. - numpy stays at 2.4.6: numpy 2.5 requires Python >=3.12 and this is ^3.11. Optional extras None of openai-agents, hyperbrowser or streamlit is reachable from the template bot, the forecast bot base, MetaculusClient, GeneralLlm or run_bots.py. They were only pulled in eagerly by forecasting_tools/__init__.py, so they are now optional: agents openai-agents agent SDK tools, Benchmarker, BotOptimizer front-end streamlit the Streamlit app, run_benchmark_streamlit_page computer-use hyperbrowser ComputerUse all everything A base install drops from 229 to 88 packages. The nine affected public names load on first access via a module __getattr__, so importing forecasting_tools never needs an extra, and each gated import raises an error naming the exact install command. Verified against a wheel installed into a clean venv. Local dev and the unit-test workflow now use --all-extras; the bot-running workflows deliberately stay on a plain install so CI keeps proving the bot path needs no extras. Test fix Streamlit now resolves a relative AppTest.from_file() path against the calling test file rather than the CWD, which broke 19 tests. They now pass the module's own absolute path, which is also CWD-independent. Version bumped to 0.3.0 (minor, not patch): the extras move is breaking for anyone using Benchmarker, ComputerUse or the front end from a plain install. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Move the "Optional extras" section out of Quick Start and remove its "Installs" column: the table now just describes what each extra is for, since the concrete package lists are already in pyproject.toml. - Replace the SmartSearcher usage in the bot customization example with AskNewsSearcher, which is a core (non-optional) dependency. Uses get_formatted_news_async since the non-async variant is not awaitable, and drops the two prompt lines that described SmartSearcher-only filtering behaviour. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- __getattr__ now converts a failed optional import into AttributeError instead of letting ImportError escape, so hasattr() on a gated name reports False rather than raising. The install message is preserved, so direct attribute access still says which extra to install. - Restore subpackage attribute access. `forecasting_tools.cp_benchmarking` and friends used to be bound as a side effect of the eager imports that were removed; __getattr__ now falls back to importing the subpackage. - Drop streamlit from the source-archive extra. Only viewer.py needs it and it already points at the front-end extra, so the CLI no longer drags in streamlit and its transitive tree. - Add code_tests/unit_tests/test_optional_imports.py, which hides the three optional packages behind a sys.meta_path blocker in a subprocess and asserts that importing forecasting_tools, run_bots and TemplateBot still work. CI installs every extra, so without this a future eager import in __init__.py would pass CI while breaking every base install. Verified the test fails when such an import is reintroduced. - Fix the front end run command in the README: front_end/Home.py, not front_end/main.py. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ADME - Fold the computer-use extra into agents. ComputerUse needs openai-agents anyway, so a separate extra only made people install two things to get one feature. agents now carries hyperbrowser as well. - The README bot example now researches with GeneralLlm(model="perplexity/sonar-pro"), which takes a prompt. The previous AskNewsSearcher call was wrong for this example: its methods take a search query, not a composed research prompt. Verified a new bot maker's path end to end against a wheel installed into a clean venv outside the repo: `import forecasting_tools` pulls in none of the optional packages, the README example subclass defines and instantiates, every name the README references imports, gated names raise with an install command, and hasattr degrades to False. Installing [agents] then unlocks ComputerUse and the rest while front-end stays gated. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Move the large, rarely used dependencies behind extras and drop one that is not used at all. A plain `pip install forecasting-tools` goes from 88 packages / 781MB to 79 packages / 416MB. - Drop regex. It is declared but never imported anywhere in the codebase (it still arrives transitively via tiktoken, so nothing changes at runtime). - Gate scikit-learn behind a new `calibration` extra. It was the single biggest cost: sklearn itself is 31MB, it is the only thing pulling scipy (114MB), and it imports pandas (40MB) internally, so it accounted for roughly 185MB of every install. - Stop the deduplicator using sklearn. It called cosine_similarity once, on a pair of 1-D vectors. That is now computed with numpy, which is already a core dependency. Checked against sklearn over 2000 random vector pairs (max difference 1.7e-16) and on the zero-vector, identical and opposite edge cases. This matters because Deduplicator is reached from KeyFactorsResearcher, so it has to stay dependency-light; the remaining sklearn users are all in calibration_adjustment. - Gate plotly and pandas behind `front-end`, and faker behind `agents`. Each was already confined to code that needs those extras anyway. Also fixes forecasting_tools/calibration_adjustment/__init__.py, which has been broken since it was added: it imported from "calibration_adjustments" (plural), a package that does not exist, so the whole subpackage raised ModuleNotFoundError. Corrected while adding its guard. __getattr__ now lets ImportError through again instead of converting it to AttributeError. The conversion made hasattr() report False, but Python rewrites an AttributeError raised in __getattr__ into a bare "cannot import name" message, which stripped the install instructions from `from forecasting_tools import Benchmarker` - the documented import style. Keeping the instructions there is worth more than hasattr degrading quietly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Declare scipy explicitly. It was never a declared dependency; it only ever arrived as a scikit-learn transitive, so gating scikit-learn broke util/stats.py, which imports scipy at module level. That in turn broke the benchmark displayer for anyone on [front-end], and test_util/test_stats.py on any install without [calibration]. scipy is now its own optional dependency with a small `stats` extra, and is included in front-end and calibration. util/stats.py is guarded so the failure names the extra. - Guard pandas in source_archive/viewer.py. The bare `import pandas` sat above the streamlit guard, so an install of [source-archive] failed with a raw ImportError naming no extra. - Make front-end a superset of agents. It was missing faker, which the chat page's TopicGenerator tools need at call time, and hyperbrowser, which chat_page imports via ComputerUse at module level - so [front-end] alone could not even import the Streamlit app. Verified Home, chat_page, benchmark_displayer and util.stats all import under [front-end] now, with scikit-learn still absent. - Define __all__ so `from forecasting_tools import *` still exposes the nine lazy names. Without it a star import copied only what was already bound, so those names vanished silently and later blew up as a bare NameError. Defining __all__ activated a dormant assertion in test_public_api.py, which compares __all__ against a curated list. That surfaced two things: TYPE_CHECKING and Any were leaking into __all__ (now excluded), and the curated list was 29 names out of date. Both fixed; __all__ is a strict superset of the old list, so nothing left the public API. Base install is unchanged at 79 packages / 416MB. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Fold `calibration` into `stats`, so `stats = ["scipy", "scikit-learn", "pandas"]` covers both util/stats.py and the calibration adjusters. Four extras remain: agents, front-end, stats, source-archive (plus all). The cost is that someone who only wants ConfidenceIntervalCalculator now also pulls scikit-learn and pandas. - Remove the comments added earlier in this branch from [tool.poetry.extras]. They restated which module pulls which package and quoted install sizes, both of which go stale as soon as anything moves. The pre-existing source-archive and cloakbrowser comments are left alone; the cloakbrowser one records a deliberate version pin. Integration suite run against this state: 183 passed, 17 failed, 36 skipped. All 17 are pre-existing or environmental - 6 HTTP 429s from pytest-xdist running 24 workers against the live Metaculus API, 9 Metaculus responses returning None that reproduce on main, 1 pre-existing cost-tracking failure, and 1 LLM non-determinism. Nothing new from this branch. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every module in calibration_adjustment/ imported from forecasting_tools.calibration_adjustments (plural), a package that does not exist, so the whole subpackage has raised ModuleNotFoundError since it was added in f7579d3. An earlier commit here fixed only __init__.py, which left it half-corrected; this fixes the remaining 10 imports across the 5 adjuster modules. Found by installing the new stats extra from a wheel and importing what it is meant to provide - the extra was pointing at a package that could not load. Note that this subpackage has never been importable, so it has no test coverage and none of its runtime behaviour has ever been exercised. It imports cleanly now, but that is all this commit claims. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updates every package to the latest version reachable within (or by widening) its constraint, then splits out the dependencies the bots don't actually need.
Dependency updates
Only ceilings were raised, never floors:
openai <3→<4,plotly <7→<8,hyperbrowser <1→<2,openai-agents <0.20→<0.23.Resulting versions: litellm 1.85.0 → 1.100.0, plotly 7.0.0, hyperbrowser 1.4.2, openai-agents 0.20.0, streamlit 1.63.0, pandas 3.0.5, scikit-learn 1.9.0, pytest 9.1.1, plus ~60 transitive updates.
Two packages intentionally aren't at their newest:
>=1.85pinsopenai<3.0.0, so openai 3.x and a current litellm are mutually exclusive — forcing openai 3 would drag litellm back 17 minors to 1.83.0. litellm is what every model call routes through, so it wins. Because the floor stays at1.51.0and the ceiling is<4.0.0, a downstream project can still pinopenai = "^3.0.0"itself and get openai 3.8 + openai-agents 0.22 + litellm 1.83. Verified by resolving it.>=3.12; this project is^3.11.cloakbrowserandpymupdf4llmwere left alone (deliberate pin / optional extra).Optional extras
None of
openai-agents,hyperbrowserorstreamlitis reachable fromTemplateBot,ForecastBot,MetaculusClient,GeneralLlmorrun_bots.py. They were only pulled in eagerly byforecasting_tools/__init__.py, so they're now optional:agentsBenchmarker,BenchmarkForBot,BotOptimizer,CustomizableBotfront-endrun_benchmark_streamlit_pagecomputer-useComputerUseallA base install drops from 229 to 88 packages.
The nine affected public names now load on first access via a module-level
__getattr__(PEP 562), soimport forecasting_toolsnever requires an extra.TYPE_CHECKINGimports preserve type-checker/IDE behavior. Each gated import raises an error naming the exact command:forecasting_tools/front_end/__init__.pyis new — the directory was a namespace package, and adding it gives one chokepoint guarding all 12 page modules instead of editing each.Streamlit test fix
Streamlit 1.63 resolves a relative
AppTest.from_file()path against the calling test file rather than the CWD, which broke 19 tests. They now pass the module's own absolute path, which is also CWD-independent (the old code only worked when pytest ran from the repo root).Version
0.2.92→0.3.0— minor rather than patch, because the extras move is breaking for anyone usingBenchmarker,ComputerUseor the front end from a plainpip install forecasting-tools.Reviewer notes
poetry install --all-extras(README updated).unit-tests.yamland the devcontainer were updated; the bot-running workflows deliberately stay on a plainpoetry install, so CI keeps proving the bot path needs no extras.extra ==markers,import forecasting_toolsloads none of them,TemplateBotinstantiates, and each optional name raises the right install command.Testing
TZ=UTC, matching CI).-nautoagainst live Metaculus, 7 pre-existing Metaculus API responses returningNone, 1 pre-existing cost-tracking failure, 1 stale fixture ("Question 27184 is already closed to forecasting!"), 1 LLM non-determinism (passes on re-run).Two pre-existing issues surfaced along the way, not fixed here: the 7 Metaculus API failures look like a live API change, and 14 unit tests are timezone-dependent (they pass under UTC, so CI is green, but fail on a non-UTC dev machine).
🤖 Generated with Claude Code