Question: resampling (to_period, t_minutes, ...), how to enforce the start datetime?

[hurak]

I am trying to handle the resampling procedure for time series in Julia. I am using the functionality of the TSFrames.jl package, specifically the to_minutes function. First, I create a test time series with a sampling period of 5 minutes as in the MWE below:

julia> dt0 = DateTime(2022, 10, 25, 10, 0);

julia> dt1 = DateTime(2022, 10, 26, 12, 0);

julia> dt_range = dt0:Minute(5):dt1;

julia> x = 1:length(dt_range);

julia> ts_x = TSFrame(x, dt_range)
313×1 TSFrame with DateTime Index
 Index                x1    
 DateTime             Int64 
────────────────────────────
 2022-10-25T10:00:00      1
 2022-10-25T10:05:00      2
 2022-10-25T10:10:00      3
 2022-10-25T10:15:00      4
 2022-10-25T10:20:00      5
 2022-10-25T10:25:00      6
 2022-10-25T10:30:00      7
 2022-10-25T10:35:00      8
 2022-10-25T10:40:00      9
 2022-10-25T10:45:00     10
          ⋮             ⋮
 2022-10-26T11:20:00    305
 2022-10-26T11:25:00    306
 2022-10-26T11:30:00    307
 2022-10-26T11:35:00    308
 2022-10-26T11:40:00    309
 2022-10-26T11:45:00    310
 2022-10-26T11:50:00    311
 2022-10-26T11:55:00    312
 2022-10-26T12:00:00    313
            294 rows omitted

and then I want to resample it to a period of 15 minutes:

julia> ts_x_resampled = TSFrames.to_minutes(ts_x, 15)
105×1 TSFrame with DateTime Index
 Index                x1    
 DateTime             Int64 
────────────────────────────
 2022-10-25T10:10:00      3
 2022-10-25T10:25:00      6
 2022-10-25T10:40:00      9
 2022-10-25T10:55:00     12
 2022-10-25T11:10:00     15
 2022-10-25T11:25:00     18
 2022-10-25T11:40:00     21
 2022-10-25T11:55:00     24
 2022-10-25T12:10:00     27
 2022-10-25T12:25:00     30
          ⋮             ⋮
 2022-10-26T10:10:00    291
 2022-10-26T10:25:00    294
 2022-10-26T10:40:00    297
 2022-10-26T10:55:00    300
 2022-10-26T11:10:00    303
 2022-10-26T11:25:00    306
 2022-10-26T11:40:00    309
 2022-10-26T11:55:00    312
 2022-10-26T12:00:00    313
             86 rows omitted

While the original time series starts at "something" and 0 minutes and the next sample is at 5 minutes, in the resampled series the first sample appears at 10 minutes – apparently already the second sampling time in the original series. The next sample in the new series is at 25 minutes, i.e., correctly 15 minutes later. After that, the series continues as expected.

How can I make the resampled series start at 0 minutes, with the first sample at 15 minutes, etc.?

[chiraganand]

While the original time series starts at "something" and 0 minutes and the next sample is at 5 minutes, in the resampled series the first sample appears at 10 minutes – apparently already the second sampling time in the original series. The next sample in the new series is at 25 minutes, i.e., correctly 15 minutes later. After that, the series continues as expected. How can I make the resampled series start at 0 minutes, with the first sample at 15 minutes, etc.?

Actually, `to_period()` functions use `endpoints()` function internally which, by definition, takes the _last_ observation of the period. For example, `Minute(15)` will give you the last observation of each 15-minute period, so in your data set, from `10:00` till `10:15` it will give the observation _just_ before `10:15:00` which occurs at `10:10:00`. So, if your series starts from `10:01` on a `Minute(1)` frequency then there will be 15 observations exactly and if you resample with `Minute(15)` then you will get: ``` julia 2022-10-25T10:15:00 15 2022-10-25T10:30:00 30 2022-10-25T10:45:00 45 ``` A workaround to this is to use `apply` function wherein you take the `first` observation instead of the `last` within a period: ``` julia julia> apply(tsx, Minute(15), y -> first(y), first) 9×1 TSFrame with DateTime Index Index x1_function DateTime Int64 ────────────────────────────────── 2022-10-25T10:00:00 1 2022-10-25T10:15:00 4 2022-10-25T10:30:00 7 ``` You can change `first(y)` to `sum(y)` or `mean(y)` if you need to aggregate data within each period. Hope this helps. -- Chirag Anand

[hurak]

Thanks.

As a matter of fact, I was originally attracted to the to_period and alike because my true need is resampling. Now I understand that it is not supported. But is there any recommendable (idiomatic) way of doing resampling of time series represented in TSFrames? I am sure that it is a functionality that is needed every now and then.