Skip to content

Considerations around label-based indexing for time-series data #232

Open
@SimonHeybrock

Description

@SimonHeybrock

ScippNexus has supported label-based indexing for groups such as NXlog and NXevent_data for a while. When trying to use this, I have come across two problems:

  1. Sometimes data is not perfectly ordered, e.g., if the data from one pulse is slightly delayed. This fails to load with the current mechanism, since Scipp label-based indexing required ordered coords. See also Better handling of unsorted coords with label-based indexing? scipp#3527
  2. Given an interval, Scipp will load all data points in between. This is not what we commonly need from NXlogs — we are more interested in the previous value (or at least need to include it). Otherwise we might end up loading an empty range.

It is not clear if 2.) should simply be solved after loading, e.g., by loading from the start. Or is there a conceptually better mechanism that could be implemented in ScippNexus?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    • Status

      Triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions