ERROR -- : Cannot rename partial file

[patrickwolf]

I'm getting this error repeatedly with v4.22 and v4.23

E, [2025-09-16T21:36:14.253106 #1] ERROR -- : Cannot rename partial file

It's from ascli running in a docker container on a mac accessing a synology via a smb share.

What are the best ways to troubleshoot this please @laurent-martin ?

Permission wise I can create, rename/move files from the docker container

[laurent-martin]

This is an error coming from ascp when it is configured to use a partial file name, and at the end of the transfer, the partial file, now complete, does not exist anymore.

This can happen if two transfers start in parallel for the same file:

• session 1 starts for file1, it creates file: file1.partial and fills it
• session 2 starts for file1 (same), it creates file: file1.partial and fills it
• session 1 finishes, and renames file1.partial to file1
• session 2 finishes, and tries to rename file1.partial to file1, but it fails as it does not exist anymore...

It is one scenario.

By default, ascli creates a config file: ~/.aspera/sdk/aspera.conf like this:

<?xml version='1.0' encoding='UTF-8'?>
<CONF version="2">
<default>
    <file_system>
        <resume_suffix>.aspera-ckpt</resume_suffix>
        <partial_file_suffix>.partial</partial_file_suffix>
    </file_system>
</default>
</CONF>

In container, this is located in /ibm_aspera

One possibility to avoid that error is to disable partial filename suffix... But that probably only hides the problem.

For example, when using the container, override that file with a volume and remove the line for extension.
Another possibility is to add this: --transfer-info==@json:'{"ascp_args":["--partial-file-suffix="]}' : this overrides the value in config file.

[patrickwolf]

@laurent-martin thank you for the explanation and I get how that would happen in a scenario with concurrent transfers.

and when it happened yesterday i can confirm that there was only one session happening (ie one execution of the ascli (this time on MacOS as I was troubleshooting outside the container)

here is the command line

[patrickwolf]

also can --lock-port=12345 be used in a scenario with multiple docker containers?

[laurent-martin]

No, --lock-port=12345 will not work as expected, because ascli will lock this port locally in the container, but it's isolated from other containers...

So, to have a lock, you might rather use flock utility where containers are started, as described in doc:

https://github.com/IBM/aspera-cli?tab=readme-ov-file#locking-for-exclusive-execution

[patrickwolf]

So, to have a lock, you might rather use flock

got it thanks.

and when it happened yesterday i can confirm that there was only one session happening (ie one execution of the ascli (this time on MacOS as I was troubleshooting outside the container)
here is the command line

do you have thoughts on what this might be please @laurent-martin ?

[patrickwolf]

So are those downloads still valid and okay when this happens? @laurent-martin

Destination /Users/patrick/p/VFX/IO/_staging_AutoAoC
creating folder: /Users/patrick/.aspera/ascli
latest: Pulling from martinlaurent/ascli
Digest: sha256:b5b57b3e233bc9a6dcd0e3df588ed040890792be07404535d74c335a2172c442
Status: Image is up to date for martinlaurent/ascli:latest
docker.io/martinlaurent/ascli:latest
Workspace: Shared packages
Found 1 package(s).
Downloading package: [qG3dZw6EYg] "io_250917_001" to []
E, [2025-09-17T14:07:02.939788 #1] ERROR -- : Cannot rename partial file
E, [2025-09-17T14:07:11.289117 #1] ERROR -- : Cannot rename partial file
E, [2025-09-17T14:07:11.306542 #1] ERROR -- : Cannot rename partial file
E, [2025-09-17T14:07:22.801549 #1] ERROR -- : Cannot rename partial file
E, [2025-09-17T14:07:26.981452 #1] ERROR -- : Cannot rename partial file
E, [2025-09-17T14:07:26.982254 #1] ERROR -- : Cannot rename partial file
E, [2025-09-17T14:07:26.982922 #1] ERROR -- : Cannot rename partial file
E, [2025-09-17T14:07:26.984035 #1] ERROR -- : Cannot rename partial file
E, [2025-09-17T14:07:31.822799 #1] ERROR -- : Cannot rename partial file
E, [2025-09-17T14:07:42.280597 #1] ERROR -- : Cannot rename partial file
E, [2025-09-17T14:07:42.321500 #1] ERROR -- : Cannot rename partial file

[patrickwolf]

Cannot rename partial file is continuously happening on production even with only one session running @laurent-martin . any idea what we can do? and more importantly do the files still get 100% correctly downloaded with the error happening?

thanks

[patrickwolf]

Another possibility is to add this: --transfer-info==@json:'{"ascp_args":["--partial-file-suffix="]}' : this overrides the value in config file.

what would happen if there are two sessions and there isn't a partial file suffix? also would an incomplete file look like a complete file without the suffix?

[patrickwolf]

@laurent-martin i saw your label of not ascli and the issue is with ascli

here is the command line

[laurent-martin]

Well, I mean that ascli is a front end to ascp and other Aspera transfer agents (node, httpgw, etc...).
ascli does not actually transfer files, it generates an ascp command line (if agent is direct, the default), that actually performs the transfer.

Effectively, the error is reported by ascli, but it gets it from ascp.

At the end of the transfer of a file, if there is a partial file suffix configured (default), and if that file cannot be renamed, this error is displayed.
Reasons can be:

• The file with partial extension does not exist, because it was deleted (by another process, ascp, or not)
• the folder where that file is became not writable.

In any case, not much that ascli can do for that.
The usual reason is that the file got transferred twice, in two separate sessions, or in the same...

If partial extension is not used, even if two sessions transfer the same file, the eventual file will be complete, but it will just have been written twice.

[laurent-martin]

By default, on Linux, ascp logs go to syslog facility local2, which goes by default to /var/log/messages.

You should get error messages like:

ERR _rename_in_progress_file: could not rename in-progress file 100MB.partial to 100MB, err=ENOENT
ERR __func__, cannot rename path='100MB.partial' to '100MB', err=ENOENT

For example, here you can see the reason for error: ENOENT, i.e. the partial filename is not present.

I can get that error easily when starting two transfers in //, or even by transferring the same file twice.

ASPERA_SCP_PASS=demoaspera ascp -L- --mode recv --host eudemo.asperademo.com --user aspera -P 33001 --partial-file-suffix=.part /aspera-test-dir-large/100MB /aspera-test-dir-large/100MB .

[patrickwolf]

If partial extension is not used, even if two sessions transfer the same file, the eventual file will be complete, but it will just have been written twice.

thank you. the biggest question @laurent-martin is if we can ignore the error since eventually all files will be written correctly and everything will be cleaned up correctly? thanks for your help