Best Practice for Missing Timestamps in discontinuous data

[pauladkisson]

Currently, the best practices section on breaks in continuity specifies what to do if you have gaps in your data, but you still have all the timestamps (or starting times/rates) in a common time basis. But, it doesn't give any recommendation for what to do if you have discontinuous data with no record of the timing relationship between segments.

This recently came up with a user (@ElpyDE) on slack and then subsequently on penalab/ndx-auditory#1, in which there is a series of auditory stimuli whose order is known and within-stimulus timing information is known, but the between-stimulus interval was not recorded.

My initial reaction is that we should strongly encourage experimentalists to record the global timing information of all the data in their experiments, since such info can be critical for data reuse. Of course, if that didn't happen, maybe we can recommend estimating the starting times of each discontinuous block (as accurately as possible), and adding a note in the description that starting_time is approximate.

Curious to hear other ideas/opinions. @rly @bendichter

[ElpyDE]

Other sections on the best practices page make references to gaps/discontinous data as well:

• Timestamps vs. Start & Rate - suggests either multiple TimeSeries with appropriate starting_time values or a single TimeSeries with a timestamps vector that indicates gaps
• Timestamps without NaNs basically suggests the same, adding that the timestamps vector should never contain NaN values

Adding some backstory to this: This is mostly for legacy data or data from legacy setups where a definitive common time base isn't available. Each segment, e.g. a trial, is recorded with well defined internal timing. But the actual gaps between segments are lost and their duration might be unknown or even unknowable afterwards, e.g. because operations between trials such as downloading recorded data from the hardware, preparing the next trial and initiating the trials start are controlled by software running on a normal PC (which inherently is not a real-time system).

In such cases, there may be some information like a minimum inter-trial interval, and there could be an educated guess on the range of how much the gap duration varies.

In my case, the software has an inter-trial interval setting which effectively works as "minimum time between trial initiations". I would be okay with adding the segments of data I have in a common time base that uses this inter-trial interval to space them out. However, this feels wrong if I can't indicated unambiguously that this spacing is an assumption/approximation and does not reflect accurate gap durations.

It's always an uncomfortable situation wanting/needing to represent "unknown" data in a data format. But since the common time base per file is such a strong (and for many reasons good) design choice in NWB, it could be a useful idea to represent a (necessary) break of this principle in a consistent manner.

[ElpyDE]

I could see some possible solutions. All of them assume that data was "forced" into a common time base first.

Add a comment

Add a comment somewhere that can be read by humans.

PRO: Does not need any additional implementation

CON: It will likely be missed and tools can't interpret or know about it, thus leading to potentially wrong interpretations when data is re-used.

Global "flag of discontinuity"

Make a flag/boolean option available on a per-file bases that indicates that (any) gaps timestamps vectors are supposed to be interpreted as arbitrarily set and unknown.

PRO: This provides one location to be checked before reusing data with a wrongful assumption of a single time base across breaks in continuity.

CON: If there is another use of non-continuous timestamps vectors where the intervals are well-known, those would be lost. This could be mitigated by some human-readable clarification, but it would probably break things in some circumstances. There could also be a reference to say something like "gaps in these TimeSeries objects are indicating unknown gaps in the time base", but that seems to become a nightmare to work with.

Conversely, if there is no TimeSeries spanning the entirety of the time base, but instead separated TimeSeries per segment with their individual starting_time, this mechanism would not work.

A global flag could of course also mean something like: Interpret all gaps between non-overlapping TimeSeries as unknown in duration. Logically, that could be rather clear - and close to where the problem is originating. But it seems like another complexity nightmare to compute whether even two time points in the common time base are "connected" by any combination of overlapping TimeSeries or not. And what happens if something gets removed later (to make files smaller) which is otherwise not needed? Or worse, something like a single SpikeEventSeries is added across the entire file and later (wrongfully) interpreted as providing a trustworthy continuous time?

Local "flags of discontinuity"

Add a similar flag to individual TimeSeries members.

I think this is a bad idea from the start, as it would make interpretations across data structures in the same file very difficult. One would potentially have to crawl through all TimeSeries and similar objects in a file to ensure/determine a trustworthy continuous time.

Save times of breaks in continuity explicitly (my favorite)

A dedicated ContinuityBreaks object could be used to indicate all individual breaks in continuity in the (forced) common time base. This would be somewhat similar to a TimeSeries, but maybe with only the timestamps vector and the data being ignored. This would come closest to the idea of "forcing" everything into the same time base (which hopefully allows existing standard tools to keep working) but adds a machine-readable information about where breaks in that time base should be expected and taken care of.

This object could include meta information (if known) such as the min/max range of gap durations, comments about their origin (hardware limitations as described above, user-initiated pauses of unkown duration could be another example).

This extra type could be part of an ndx extension (ndx-continuitybreaks?) first which would raise awareness in users (hopefully questioning why it's a requirement for a certain dataset) and could provide additional helpers/tools to deal with it.

PRO: This provides an explicit, machine-readable, single point of truth. Ignoring it should not lead to disruptive errors when using the data, only when it's wrongfully interpreted across those break points. It should be rather simple to check whether any time span of interest is disrupted by a break point or not.

CON: It's an extra object that might be ignored. It will/should be ignored by many tools as well, which can lead to wrongful interpretations of their results. But really, that happens when you don't pay attention...

The logical inverse of a ContinuityBreak which indicates where the common time base is "broken" could be to use an IntervalSeries (or similar type) indicating instead all of the segments that are internally trustworthy - implicating that the temporal relationship in between these intervals is not guaranteed. While in some way identical, I think it's much harder to work with.