[Experimental] Frontend interface for Migration Assistant accessibility, focused on Backfill

[peternied]

Migrating data between search clusters can be a straight forward process or deviously complex. In both cases the current Migration Assistant experience requires considerable knowhow to navigate command line interface along with few positive signals of progress and success.

Adding a frontend interface would lower the barrier to entry by making it clear what actions have been taken, the current state of the migration, and what actions to take next.

To scope this project, we will be working on a subset of the many Migration Assistant scenarios - in this case by working with Snapshot, Metadata, and Backfill it provides a mix of asynchronous activity and synchronous work. The complexity of the Metadata migration output is also a well suited challenge for this initial draft. Once this scenario is available we can get this into customer's hands for feedback while iterating on additional scenarios.

Supported Workflow graph

flowchart TD
    Start@{ shape: dbl-circ, label: "Start" }
    subgraph Migration_Types
        A1[Full Migration]
        A2[[Backfill Only]]
        A3[Capture and Replay Only]
        A4[Bring Your Own Snapshot]
    end
    Start --> |Pick Workflow| Migration_Types
    subgraph "Capture and Replay"
        C1[Request Capture]
        C2[Replay Captured Traffic]
    end
    subgraph "Snapshot-Based Migration"
        S1[[Create Snapshot]]
        S2[[Metadata Migration]]
        S3[[Backfill<br/>Reindex from Snapshot]]
    end
    Stop@{ shape: dbl-circ, label: "Stop" }
    subgraph Legend
        L1@{ shape: rect, label: "Initial Implementation"}
        L2@{ shape: rect, label: "Future Revisions"}
    end

    %% Full Migration Flow
    A1 --> C1
    C1 --> |Backfill before replay| S1
    S1 --> S2
    S2 --> S3
    S3 --> C2
    C2--> Stop

    %% Bring Your Own Snapshot Flow
    A2 --> S1
    S3 --> Stop


    %% Capture and Replay Only Flow
    A3 --> C1
    C1 --> |No backfill| C2
    C2 --> Stop

    %% Bring Your Own Snapshot Flow
    A4 --> S2
    S3 --> Stop

   Stop ~~~ Legend

    linkStyle 3,4,7,8 stroke:#3a3,stroke-width:3px
    style A2 fill:#dfd,stroke:#3a3,stroke-width:3px
    style S1 fill:#dfd,stroke:#3a3,stroke-width:3px
    style S2 fill:#dfd,stroke:#3a3,stroke-width:3px
    style S3 fill:#dfd,stroke:#3a3,stroke-width:3px
    style L1 fill:#dfd,stroke:#3a3,stroke-width:3px

Loading

Technical details

There is considerable work between picking out how to build a frontend system, integrating it with the existing deployment, and coming up with the kinds of conventions we will use for our UX patterns - these details will be captured in high fidelity within our Jira ecosystem [1].

See also

• [1] EPIC System Design - Migration Assistant Frontend https://opensearch.atlassian.net/browse/MIGRATIONS-2195
• [2] EPIC Migration Frontend UX - Phase 1: Fundamentals https://opensearch.atlassian.net/browse/MIGRATIONS-2537

[peternied]

@sumobrian How do you think we should differiate this framing vs the existing issue on our project board [1]?

• [1] Centralized Migration Management UI for Metadata Migrations, Live Capture, and Backfill #1093

[peternied]

@sumobrian I've updated this item to highlight the critical path workflow that will be supported before its release and scheduled it on the project board for within 3 month timeframe.

[sumobrian]

@sumobrian How do you think we should differiate this framing vs the existing issue on our project board [1]?

• [1] Centralized Migration Management UI for Metadata Migrations, Live Capture, and Backfill #1093

#1093 is and end-2-end deliverable intensions of GA. This issue is tracking an incremental deliverable that will focus on the migrating index settings and backfill.