[BUG] np.int_ causes 32-bit Overflow on Windows (NumPy < 2.0)

[Adityakushwaha2006]

Describe the bug

I encountered a critical platform dependency issue where np.int_ is used for indexing and array initialization in core utilities (e.g., aeon/utils/windowing.py).

While np.int_ maps to a 64-bit integer on Linux/macOS, it maps to a 32-bit C long on Windows (even on 64-bit systems). This causes an OverflowError (or silent data corruption if wrapping occurs) when handling indices or strides larger than 2,147,483,647 (approx. 2.14 billion), effectively breaking aeon for large-scale time series on Windows.

Steps/Code to reproduce the bug

Run the following script on a Windows machine vs. a Linux environment (e.g., Colab).

import numpy as np
import sys

print(f"OS Platform: {sys.platform}")
print(f"NumPy Version: {np.__version__}")

# np.int_ corresponds to C 'long'
# Windows: 32-bit (4 bytes)
# Linux/Mac: 64-bit (8 bytes)
int_size = np.dtype(np.int_).itemsize
print(f"Size of np.int_ (C long): {int_size * 8}-bit")

# Attempt to store a value > 2.14 Billion (32-bit limit)
test_value = 3_000_000_000
print(f"Attempting to store: {test_value}")

try:
    # This crashes on Windows but works on Linux
    val = np.array([test_value], dtype=np.int_)
    print(f"Success: {val[0]}")
except Exception as e:
    print(f"FAIL: {e}")

Expected results

The code should handle standard large integers (64-bit) consistently across all supported 64-bit platforms (Windows, Linux, macOS), allowing the storage of values > 2.14 billion without crashing.

Actual results

On Windows (Reproduction Log)[for most of the users' numpy version] :

OS Platform: win32
NumPy Version: 1.26.4
Size of np.int_ (C long): 32-bit
Attempting to store: 3000000000
FAIL: OverflowError: Python int too large to convert to C long

On Linux:

OS Platform: linux
NumPy Version: 2.0.2
Size of np.int_ (C long): 64-bit
Attempting to store: 3000000000
Success: 3000000000

Versions

• OS: Windows 11 (64-bit)
• Python: 3.x (64-bit)
• NumPy: 1.26.4 (Windows) vs 2.0.2 (Linux) -fails
• NumPy: 2.0.2 (Windows) vs 2.0.2 (Linux) -passes
• aeon version: v0.13.0 (dev)

Results: Tested and confirmed , once fixes are applied, the functionality is consistent across all numpy versions.

[Adityakushwaha2006]

@aeon-actions-bot assign @Adityakushwaha2006

[Adityakushwaha2006]

@aeon-actions-bot unassign @Adityakushwaha2006

[SebastianSchmidl]

This is nit-picky. We expect most of our estimators to either run for too long or break on inputs as large as 3 billion time points.

However, for consistency, we should use dtype=int across the codebase anyway.

[bhakunisagar512-ui]

Hi @SebastianSchmidl

I saw that this issue was reclassified as an enhancement.
I’d be happy to help by aligning integer dtypes across the codebase (for example, replacing np.int_ with int or np.intp) in a limited, well-scoped PR.

Please let me know if that would be welcome. Thanks!

[Adityakushwaha2006]

Hey @bhakunisagar512-ui , the related PR #3202 has already been updated with the requested changes.

[SebastianSchmidl]

Hi @bhakunisagar512-ui,
@Adityakushwaha2006 already addressed this issue in #3202 for the critical files aeon/utils/windowing.py and aeon/utils/numba/general.py‎.

You could, however, check the usage of dtype=... within NumPy calls across the rest of the code base for consistency, if you like. Note that we do not necessarily want to enforce consistency for calls to tensorflow/keras.

[MatthewMiddlehurst]

Please don't unassign yourself if you are planning to work on the issue still. Agreed that It's not really an "issue" fine with changes to make everything more consistent.