Skip to content

[Feature] Extract snapshot-reader logic into a standalone library #1457

@kanatti

Description

@kanatti

Currently, RFS has two core responsibilities:

  1. Reading from snapshots
  2. Writing to the target cluster

There are other potential use cases where only reading from snapshots is valuable.

For example, embedding the snapshot-reading logic into a Spark/EMR application could enable:

  1. Running parity/data quality checks across clusters by taking snapshots from both and comparing them directly in Spark.
  2. During production data issues, taking a snapshot of the affected indices before mitigation, and then using Spark to assess the impact by reading directly from the snapshot.

Extracting the snapshot-reader logic into a standalone library would unlock these types of use cases outside of RFS.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions