[Mypy] Type-checking errors in evaluation2.py and run_nhits.py

[mohamedelbahnasawi]

What happened + What you expected to happen

After cloning the current main branch of Nixtla/neuralforecast and running the pre-commit hooks (as configured in the repo), I get multiple mypy type-checking errors in unmodified upstream files.

I expected a clean checkout of main to pass pre-commit without errors, but mypy fails with the following output:

Cells with imports and code..............................................Passed
nbdev_clean..............................................................Passed
nbdev_export.............................................................Passed
ruff.....................................................................Passed
mypy.....................................................................Failed
- hook id: mypy
- exit code: 1

action_files\test_models\src\evaluation2.py:50: error: No overload variant of "__getitem__" of "list" matches argument type "list[str]"  [call-overload]
action_files\test_models\src\evaluation2.py:50: note: Possible overload variants:
action_files\test_models\src\evaluation2.py:50: note:     def __getitem__(self, SupportsIndex, /) -> Any
action_files\test_models\src\evaluation2.py:50: note:     def __getitem__(self, slice, /) -> list[Any]
action_files\test_models\src\evaluation2.py:51: error: No overload variant of "__getitem__" of "list" matches argument type "str"  [call-overload]
action_files\test_models\src\evaluation2.py:51: note: Possible overload variants:
action_files\test_models\src\evaluation2.py:51: note:     def __getitem__(self, SupportsIndex, /) -> Any
action_files\test_models\src\evaluation2.py:51: note:     def __getitem__(self, slice, /) -> list[Any]
action_files\test_models\src\evaluation2.py:51: error: No overload variant of "__setitem__" of "list" matches argument types "str", "Any"  [call-overload]
action_files\test_models\src\evaluation2.py:51: note:     def __setitem__(self, SupportsIndex, Any, /) -> None
action_files\test_models\src\evaluation2.py:51: note:     def __setitem__(self, slice, Iterable[Any], /) -> None
action_files\test_models\src\evaluation2.py:52: error: "list[Any]" has no attribute "set_index"  [attr-defined]
action_files\test_models\src\evaluation2.py:53: error: "list[Any]" has no attribute "columns"  [attr-defined]
action_files\test_models\src\evaluation2.py:54: error: "list[Any]" has no attribute "set_index"  [attr-defined]
action_files\test_models\src\evaluation2.py:55: error: "list[Any]" has no attribute "droplevel"  [attr-defined]
action_files\test_models\src\evaluation2.py:56: error: No overload variant of "__setitem__" of "list" matches argument types "str", "list[float]"  [call-overload]
action_files\test_models\src\evaluation2.py:56: note: Possible overload variants:
action_files\test_models\src\evaluation2.py:56: note:     def __setitem__(self, SupportsIndex, Any, /) -> None
action_files\test_models\src\evaluation2.py:56: note:     def __setitem__(self, slice, Iterable[Any], /) -> None
action_files\test_models\src\evaluation2.py:57: error: "list[Any]" has no attribute "to_csv"  [attr-defined]
action_files\test_models\src\evaluation2.py:58: error: "list[Any]" has no attribute "T"  [attr-defined]
experiments\long_horizon\run_nhits.py:83: error: Argument "n_windows" to "cross_validation" of "NeuralForecast" has incompatible type "None"; expected "int"  [arg-type]

Observed: The above mypy errors appear
Expected: No mypy errors on a clean checkout of main.

Versions / Dependencies

OS: Windows 11
Python 3.11.13
mypy 1.14.1
pre-commit 4.3.0
ruff 0.12.8
black, 25.1.0 (compiled: yes)
conda 25.5.1

Environment created from the repo's longhorizon environment.yml

Reproduction script

git clone https://github.com/Nixtla/neuralforecast.git
cd neuralforecast\experiments\long_horizon
conda env create -f environment.yml
conda activate longhorizon
pre-commit install
pre-commit run --all-files

Issue Severity

Medium: It is a significant difficulty but I can work around it.

[mohamedelbahnasawi]

Update:

The error happens because the type checker thinks evaluation is a list when it's actually a pandas DataFrame. You just need to add a few type hints to solve the issue:

In action_files\test_models\src\evaluation2.py

from typing import List

line: 47-49

evaluation_list: List[pd.DataFrame] = [evaluate(model, dataset, group) for model, group in product(models, groups) for dataset in datasets]
evaluation_list = [eval_ for eval_ in evaluation_list if eval_ is not None]
evaluation: pd.DataFrame = pd.concat(evaluation_list)

In experiments\long_horizon\run_nhits.py:

line 83:
removed

n_windows = None

Terminal Output:

pre-commit run --all-files
Cells with imports and code..............................................Passed
nbdev_clean..............................................................Passed
nbdev_export.............................................................Passed
ruff.....................................................................Passed
mypy.....................................................................Passed