Skip to content

[Proposal] Artifact registry: gather artifacts across projects #607

Description

@adrien-grl

Why

My team is migrating off W&B following their recent ToS changes. We were convinced by Trackio's approach and we would like to fully migrate. Artifacts support (#586, #601, and hopefully references via #602) already covers most of what we used. The last missing piece for a full migration is the artifact registry. We're prepared to build it in a fork if it's out of scope for Trackio, but we'd much rather design it with you and contribute it upstream.

What an artifact registry is

A registry is a curated, cross-project catalog of artifact versions, organized into collections. Where a project's artifact list answers "what did my experiments produce?", a registry answers "what is the model running in production right now?"

A collection represents one asset as a product — e.g. the churn model, the golden eval set — and aggregates the versions of it worth publishing, regardless of which project or retraining produced them. Versions are linked into a collection (pointers, no copy) and promoted through lifecycle stages by moving aliases like staging and production.

The primitive is generic: the same mechanism gives you a model registry, a dataset registry, a prompt registry — any curated family of artifacts.

Why it's nice

  • One source of truth for consumers. Deployment, evaluation, and CI resolve model:production without knowing anything about experiment projects. Promoting a new version breaks nothing downstream — the alias moves, consumers don't change.

  • The research → production hand-off point. Training publishes into the registry; everything after consumes from it. This is the boundary between "tracker" and "system of record".

  • Auditability. Which version is deployed, since when, what it replaced, and what run produced it — connected to the lineage that Add artifact tracking: versioned file bundles with log_artifact / use_artifact #586 / feat: browse artifacts from the UI #601 already track.

  • Automation hook. "A version was promoted to production" is the natural event for CI/CD to react to (webhook or polling; mechanism TBD, but the registry is where the event lives).

Features we need

  • Create registries and typed collections within them (a collection accepts one artifact type).
  • Link an existing artifact version into a collection (pointer, no copy).
  • Collection-side version numbering (v0, v1, ... per collection), since linked versions come from different source artifacts whose own numbering neither aligns nor stays unique.
  • Alias-based promotion, one version per alias per collection, same semantics as the existing per-artifact aliases.
  • Resolution and lineage through the registry: consuming model:production records the run ← registry ← source-artifact chain.
  • An audit history of link/promote operations.
  • CLI support (and UI eventually, as a follow-up in the spirit of Add artifact tracking: versioned file bundles with log_artifact / use_artifact #586feat: browse artifacts from the UI #601).

API sketch

# Publish: link an already-logged artifact version into a collection
run.link_artifact(artifact, "models/churn-model", aliases=["staging"])

# Consume: resolves through the registry, records lineage
artifact = run.use_artifact("churn-model:production", registry="models") # note: path-string vs registry= kwarg is an open question below
trackio registry link models/churn-model my-project/resnet-run:v4 --alias staging
trackio registry promote models/churn-model production v1

TBD

  • Storage layout: one registry database containing all registries / one database per registry (possibly reusing the per-project DB machinery) / something else?
  • Resolution syntax: registry= kwarg / path string e.g. registry/model / reusing project=?
  • Digest should linking require a manifest digest, or tolerate digest-less reference entries as well (feat: add Artifact.add_reference to reference external objects without copy #602)?
  • roles/RBAC for v1, we'd suggest none as it would probably be part of a bigger work

If this is in scope, we're happy to implement it and iterate on the design here.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions