crane: registry needs to hold full layers in memory

[malt3]

Describe the bug

When using crane registry serve and pushing large layers with PUT, I observe excessive memory usage.
I traced this to the PUT handler function:

go-containerregistry/pkg/registry/blobs.go

Line 485 in 59a4b85

in := io.NopCloser(io.MultiReader(bytes.NewBuffer(b.uploads[target]), req.Body))

The in-flight blob is cached in full in a buffer until all chunks have been uploaded, and then Put is invoked after the transfer is complete.
This creates two problems:

• Excessive memory usage: with a typical CUDA base image, I can reliably produce OOM situations, since individual layers are gigabytes in size.
• The CLI side looks broken to the end-user: after every byte was sent via HTTP, the progress bar is at 100%, but the server still has to call Put. For a slow BlobHandler (one that has to upload data to a remote storage backend), this results in minutes of waiting where the progress bar is full and the user thinks that the upload may be stuck.

To Reproduce

Terminal 1:

crane registry serve --address 127.0.0.1:1234

crane copy nvidia/cuda@sha256:f353ffca86e0cd93ab2470fe274ecf766519c24c37ed58cc2f91d915f7ebe53c localhost:1234/nvidia/cuda:latest

Expected behavior

I want the memory usage to be reasonable, even for large blobs.

Additional context

I'd like to provide a fix in the form of a custom io.Reader with an internal fixed-size buffer that allows a BlobHandler to read data in chunks.
This should make uploads faster, less memory intensive, and provide better feedback to users.
Is this an acceptable solution?

[malt3]

I noticed that my proposed solution doesn't work in practice with the chunked upload (POST, PATCH, then PUT) from the spec, since the digest is only part of the final PUT.
To limit memory usage, we could add an option to use temporary files instead of a buffer.
If we want to truly solve this (and call bph.Put early), we would need to change the interface of BlobHandler.Put so that it doesn't know the digest in advance.
That's not a great solution either.