Getting problem with understanding electricity dataset

[yuvrajiro]

TLDR

electricity_nips appears to contain a misaligned test entry for feat_static_cat == 40. This breaks MultivariateGrouper when the dataset is used as a 370-dimensional multivariate dataset.

The dataset metadata says there are 370 time series:

freq = h
prediction_length = 24
feat_static_cat cardinality = 370

The test split has 2590 entries, which looks like:

370 series x 7 rolling test windows = 2590 entries

However, each 370-entry test block contains 369 normally aligned series plus one misaligned entry for category 40.

Minimal Reproduction

from pathlib import Path
from collections import Counter
from gluonts.dataset.repository import datasets

ds = datasets.get_dataset(
    "electricity_nips",
    path=Path("./gluonts_fresh_data_check"),
    regenerate=True,
)

print(ds.metadata)
entries = list(ds.test)
print("test entries:", len(entries))

cardinality = int(ds.metadata.feat_static_cat[0].cardinality)

for i in range(0, len(entries), cardinality):
    block = entries[i:i + cardinality]
    starts = Counter(str(e["start"]) for e in block)
    lengths = Counter(len(e["target"]) for e in block)
    cat40 = [
        (str(e["start"]), len(e["target"]))
        for e in block
        if e["feat_static_cat"][0] == 40
    ]

    print("block", i // cardinality)
    print("  entries:", len(block))
    print("  starts:", dict(starts))
    print("  lengths:", dict(lengths))
    print("  cat40:", cat40)

Observed Output

Freshly materialized GluonTS data gives:

metadata: freq='h' ... feat_static_cat=[... cardinality='370'] ... prediction_length=24
test entries: 2590
cardinality: 370

block 0
  entries: 370
  starts: {'2014-03-19 09:00': 369, '2014-07-18 00:00': 1}
  lengths: {4000: 369, 1105: 1}
  cat40: [('2014-07-18 00:00', 1105)]

block 1
  entries: 370
  starts: {'2014-03-20 09:00': 369, '2014-07-18 00:00': 1}
  lengths: {4000: 369, 1129: 1}
  cat40: [('2014-07-18 00:00', 1129)]

block 2
  entries: 370
  starts: {'2014-03-21 09:00': 369, '2014-07-18 00:00': 1}
  lengths: {4000: 369, 1153: 1}
  cat40: [('2014-07-18 00:00', 1153)]

block 3
  entries: 370
  starts: {'2014-03-22 09:00': 369, '2014-07-18 00:00': 1}
  lengths: {4000: 369, 1177: 1}
  cat40: [('2014-07-18 00:00', 1177)]

block 4
  entries: 370
  starts: {'2014-03-23 09:00': 369, '2014-07-18 00:00': 1}
  lengths: {4000: 369, 1201: 1}
  cat40: [('2014-07-18 00:00', 1201)]

block 5
  entries: 370
  starts: {'2014-03-24 09:00': 369, '2014-07-18 00:00': 1}
  lengths: {4000: 369, 1225: 1}
  cat40: [('2014-07-18 00:00', 1225)]

block 6
  entries: 370
  starts: {'2014-03-25 09:00': 369, '2014-07-18 00:00': 1}
  lengths: {4000: 369, 1249: 1}
  cat40: [('2014-07-18 00:00', 1249)]

Consequence

When downstream code uses MultivariateGrouper:

from pathlib import Path
from gluonts.dataset.repository import datasets
from gluonts.dataset.multivariate_grouper import MultivariateGrouper

ds = datasets.get_dataset("electricity_nips", path=Path("./data"), regenerate=False)

test_grouper = MultivariateGrouper(
    num_test_dates=int(len(ds.test) / len(ds.train)),
    max_target_dim=370,
)

test_set = test_grouper(ds.test)

it fails with:

ValueError: all the input array dimensions except for the concatenation axis must match exactly,
but along dimension 1, the array at index 0 has size 4000 and the array at index 370 has size 4024

In our direct inspection, the underlying issue is that category 40 is not aligned with the other 369 categories in the seven rolling test windows.

Expected Behavior

For electricity_nips, each rolling test block should have 370 aligned series, e.g. one common start timestamp and compatible target lengths per block:

block k:
  370 entries
  one rolling-window start timestamp
  compatible target lengths
  feat_static_cat values 0..369

Environment

Observed with:

gluonts == 0.15.1
dataset = electricity_nips
regenerate = True

The same issue was also observed with a locally downloaded copy of electricity_nips.

Question

Is this an expected quirk of electricity_nips, or should the repository data / materialization logic align category 40 with the other 369 series? If it is expected, what is the recommended way to use electricity_nips with MultivariateGrouper for multivariate probabilistic forecasting benchmarks?