Trying to create train/val/test DataSets (by date), for multiple TSs with different start dates

[yuvalarbel]

Hi community!

I've been banging my head against the wall for many days, and still can't get this to work. I'd really appreciate any recommendations and help.

I have multiple time series, each beginning at different start times, from the year 2000 up until now (freq='B').
I would like to create a train/val/test split, with training data up until 2017, val up until 2019, and test up to now.
I then need to apply Transformations to these datasets.
Then, I'd like to create a torch DataLoader for each split, that returns randomly-positioned slices of (context_len, prediction_len). I'd like all windows, overlapping (distance/step_size=1).

I've tried many, many combinations of gluonts functions, but nothing has been working. Please give me your advice!
Thanks so much.

[lostella]

Hi @yuvalarbel, this seems like a fairly generic task that should be easy to do, but maybe is not well covered by tooling/documentation currently. Let's take it step by step and see what may be missing: I'm assuming your data looks like the following, correct? (It's 10 series in my example)

import pandas as pd
import numpy as np

from gluonts.dataset.pandas import PandasDataset

series = []
for k in range(10):
    index = pd.period_range(
        pd.Period("2000-01-03", freq="B") + 5 * k,
        pd.Period("2023-01-31", freq="B")
    )
    series.append(pd.Series(np.random.uniform(size=len(index)), index=index))

dataset = PandasDataset(series)

print(dataset)

for entry in dataset:
    print(entry)

Is that accurate? If so, then the first step you can achieve with split:

from gluonts.dataset.split import split

test_dataset = dataset
valid_dataset, _ = split(dataset, date=pd.Period("2019-12-31", freq="B"))
train_dataset, _ = split(dataset, date=pd.Period("2017-12-31", freq="B"))

for a, b, c in zip(train_dataset, valid_dataset, test_dataset):
    print(f"{len(a['target'])} / {len(b['target'])} / {len(c['target'])}")

[lostella]

Hi @yuvalarbel, I think your solution makes perfect sense. In fact, achieving what you need appears otherwise hard with the current tooling:

• split works by date, but 1) only operates on the "input schema" and otherwise fails if you massage your data to add or rename fields; it's mostly designed to make sure one can split the dataset so that train/valid/test data are "consistent" (ie there's no overlap between training data and validation labels). The idea however is that the instance splitter takes care of sampling and slicing windows (for whatever purpose).
• The instance splitter logic is currently by index only. So to achieve what you need, one would make sure the two (split and instance splitter) are used together the right way, which is probably hard to do and error prone.

So, some more direct mechanism to directly express "I want windows between this and that date from each series" (either randomly sampled, or just all of them) may be actually good to have.

[lostella]

Let me open an issue about this, where I can jot down my thoughts on what a good general tool there would be

[yuvalarbel]

Great, thanks @lostella !
I'd love to get updates on this issue :)
Waiting to see which general tool will be added!
Cheers.

[yuvalarbel]

Hello @lostella , has an issue been opened?

[lostella]

@yuvalarbel I gave it some thought, and made a proposal here (draft, feedback is appreciated)