Skip to content

GCS mirror sync does not cover torch-index-stacks.json (or anything published outside the env build) #10

Description

@Kosinkadink

The desktop app falls back to a public GCS mirror when R2 is unreachable (regions where the R2 edge is throttled, e.g. China):

  • Primary: https://desktop-assets.comfy.org/standalone-environments/
  • Mirror: https://storage.googleapis.com/comfy-desktop-public/standalone-environments/

The app's r2Mirror.ts assumes the mirror is a 1:1 copy of the R2 namespace, "kept in sync at each release". However, nothing in this repo performs that sync - both workflows (build-standalone-env.yml and the new publish-torch-index-stacks.yml from #9) only write to R2. The mirror is presumably synced by an outside process.

Problem

Once #9 lands, torch-index-stacks.json will be replaced in-place on R2 whenever the manifest changes - decoupled from env releases. If the mirror sync only runs at env-release time (or doesn't know about this object at all), mirror-dependent users get a stale PyTorch stack manifest indefinitely. That's worse than it sounds for two cases:

  • Withdrawals: pulling a known-bad stack via explicit removal would not reach mirror users.
  • New stacks: the whole point of the remote manifest is shipping stacks without an app release; mirror users would not receive them until the next unrelated env release (if ever).

Options

  1. Add a GCS upload step to publish-torch-index-stacks.yml - upload the same file to the mirror bucket right after the R2 upload. Needs GCS credentials as repo secrets. Most direct; keeps the two objects atomically close in time.
  2. Fold the manifest into the existing external mirror-sync process - whoever owns it adds this object (or switches to a full-prefix rsync of standalone-environments/, which also removes the per-object bookkeeping).
  3. Scheduled sync workflow in this repo - an rsync-style job mirroring the whole prefix on a cron. Covers all current and future objects but adds propagation delay.

Option 1 (plus keeping the archive sync wherever it lives today) is probably the smallest correct fix; whoever owns the current mirror-sync process should weigh in.

Acceptance

  • A change to torch-index-stacks.json on main reaches both R2 and the GCS mirror without manual steps.
  • Documented (README or mirrors.md) who/what syncs the mirror, so the next published object doesn't repeat this gap.

Refs: #9, Comfy-Org/Comfy-Desktop#1248

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions