Skip to content

[upload] the -continue flag does not work when recursively encrypting and uploading folders #541

@nanjiangshu

Description

@nanjiangshu

Describe the bug
When there exist c4gh files in the folder, the -continue flag does not work when recursively encrypting and uploading folder(s). That is, instead of ignoring already uploaded files and continue, the command is aborted with the following error when it encounters an existing c4gh file.

ERRO[0000] Skipping input file test_dataset_small/dummy_1.txt. Reason: outfile test_dataset_small/dummy_1.txt.c4gh already exists. 
Error: aborting

Steps to reproduce
Start the SDA integration test and download s3cmd-inbox.conf and run docker cp verify:/shared /tmp/

Upload a folder containing some files by sda-cli, for example

sda-cli -config s3cmd-inbox.conf upload -r -encrypt-with-key  /tmp/shared/c4gh.pub.pem test_dir1

and break in the middle

Then run

sda-cli -config s3cmd-inbox.conf upload -r -encrypt-with-key  /tmp/shared/c4gh.pub.pem test_dir1 -continue

Note that the flag -continue works when uploading without the -encrypt-with-key flag.

Expected behavior

  • The command should ignore already uploaded files and continue the process when the flag -continue is used, either with or without the -encrypt-with-key flag.
  • Tests verifying the fix are added

Additional context

Estimation of size: small/medium/big

Estimation of priority: low/medium/high

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions