Skip to content

Read mode: Distinguish random access from streaming/linear access #1234

Open
@franzpoeschel

Description

@franzpoeschel

Is your feature request related to a problem? If so, please describe.

Two angles to this:

  1. Calling Series("data.bp", Access::READ) will implicitly parse the entire Series in random-access mode. If using series.readIterations() afterwards, this is unnecessary. Using a streaming read mode allows us to skip this, possibly simplifying the control flow in the ADIOS2 backend.
  2. The upcoming BP5 engine in ADIOS2 will also distinguish between these two kinds of access modes. Random-accessing a series written via the streaming API will then only be possible by explicitly using the random access read mode.

Describe the solution you'd like
The existing Access::READ_ONLY keeps its semantics. a new mode READ_LINEAR or READ_STREAMING is added.

Describe alternatives you've considered
Deferred iteration parsing does similar things and will probably share some logic. It's not the same though:

  1. A READ_LINEAR mode affects the moment when BeginStep() is first called during reading.
  2. A READ_LINEAR mode will affect the mode that will be used to open the ADIOS2 engine.
  3. Deferred iteration parsing does not change the behavior of the backend at all, it just does not descend into the single iterations while parsing initially.

Additional context
Further questions:

  • What should be the behavior of this mode in file-based iteration encoding, group-based iteration encoding without steps, or other backends?
    -> Probably similar to deferred iteration parsing
  • As a perspective, we might merge deferred iteration parsing with this feature. But this will probably require an API addition for explicit random access (series.jumpToStep() or sth). For now, I would keep them separate.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions