Skip to content

Automate upload/sync of dependencies to pytorch s3 index  #6097

Open
@atalman

Description

@atalman

We would like to solve the issue of adding and synchronising new dependencies to download pytoch.org

We do have following script: https://github.com/pytorch/test-infra/blob/main/s3_management/update_dependencies.py that run manually, which upload latest available binaries to pytorch nightly and test index.

I would like to create workflow that would automatically update required dependencies on push changes to this script, so that we can easily add new Cuda or Xpu dependencies.

Rather then syncing latest we should have {package, version} tuple that we want to sync.

We should perhaps do this in stages:

  1. Enable certain packages to be synced with pinned version on push changes
  2. Enable automatic sync of latest packages to the index on daily basis

Nvidia. Dependencies : https://github.com/pytorch/pytorch/blob/main/.github/scripts/generate_binary_build_matrix.py#L47

cc @malfet @ptrblck @tinglvv @nWEIdia

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

Cold Storage

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions