sync stops with AWS error "NoSuchKey"

[asyrique]

Hi,

We're experiencing an error with Drone artifact uploads to S3 via S3 sync.

The drone build will randomly fail on certain files with the following error:

ERROR: failed to upload /foo/bar/..../baz.svg to foo/img/login.svg: NoSuchKey: The specified key does not exist.
    status code: 404, request id: 123456
[info] build failed (exit code 1)

There are no discernible patterns in the filename, filetype or filesize that we causes the error, nor is there a discernible pattern in the number of files before the upload fails.

Please let me know if I can provide any extra info to help contextualize this issue, and thanks in advance.

[SAprelov]

+1 Annoying bug

[bradrydzewski]

I came across this as well. It looks like it is related to the delete: true parameter and some sort of path mismatching. I haven't quite figured it out, but this is what I see in my debugging

here is my yaml

  upload:
    image: plugins/s3-sync:1.0
    access: public-read
    region: us-east-1
    bucket: readme.drone.io
    source: public/
    target: /0.5/
    delete: true

I can see it skipping uploads (expected):

Skipping "/Users/bradrydzewski/code/src/github.com/drone/docs/public/404.html" because hashes and metadata match
Skipping "/Users/bradrydzewski/code/src/github.com/drone/docs/public/.gitkeep" because hashes and metadata match

but then trying to delete those same files:

Removing remote file "0.5/.gitkeep"
Removing remote file "0.5/404.html"

I think it has to do with this line of code and path mismatches:

found := false
for _, l := range local {
    if l == r {
        found = true
        break
    }
}

Specifically in my example the base directory locally doesn't match the base directory to which I am uploading. This is probably part of the problem ..

    source: public/
    target: /0.5/

I've temporarily disabled delete: false and now it is working fine. I will try to debug further and see if I can come up with a solution in my spare time.

[astlock]

Any updates? What should I do if I still need to delete nonexistent files?

[bradrydzewski]

There are no updates. The individual that created this plugin is no longer able to dedicate time toward maintaining it. If this is an important issue for you, please consider sending a patch.