Possible bug in version v2.5.7 when using upload_only option

[jrpelegrina]

Using version 2.5.6 of the client (installed from the openSUSE deb), I configured an instance of it to only upload files to my OneDrive account. Specifically, I configured it with the options: upload_only = true, disable_upload_validation = true, and remove_source_files = "true" and --single-directory

The instance worked correctly without any issues. However, when updating the client to version 2.5.7, also using the openSUSE deb, I found that only files that don't exist in the cloud are sent to OneDrive. If I try to upload an existing file, the client tells me it can't upload it due to lack of space (I have over 4 GB of free space). I've tried configuring a similar instance with another Outlook account, and the behavior is the same. I've attached a screenshot of what the client returns.

Sync with Microsoft OneDrive is complete
[M] New local file added: xxxxxx/nuevo_222.txt
New items to upload to Microsoft OneDrive: 1
Total New Data to Upload:        2 Bytes
Uploading new file: xxxxx/nuevo_222.txt ... done
WARNING: Skipping upload integrity check for: xxxx/nuevo_222.txt
Attempting removal of local file as --upload-only & --remove-source-files configured
Removed local file: xxxx/nuevo_222.txt
[M] Total number of local file(s) added or changed: 1
[M] Local item deleted: xxxxx/nuevo_222.txt
The operating system sent a deletion notification. Trying to delete this item as requested: xxxx/nuevo_222.txt
Item cannot be deleted from Microsoft OneDrive because it was not found in the local database
[M] New local file added: xxxxx/nuevo_222.txt
New items to upload to Microsoft OneDrive: 1
Total New Data to Upload:        2 Bytes
The file we are attempting to upload as a new file already exists on Microsoft OneDrive: xxxxx/nuevo_222.txt
Skipping uploading this item as a new file, will upload as a modified file (online file already exists): xxxxx/nuevo_222.txt

[abraunegg]

@jrpelegrina
Curious situation given that nothing within the changes from v2.5.6 -> v2.5.7 sticks out as a possible culprit.

Also - your logs do not show what you are claiming.

To take this further, use a script similar to the following to create a specific local version of v2.5.6 and v2.5.7

#!/bin/bash
  
BRANCH=v2.5.6

# Remove the existing 'onedrive' directory
rm -rf ./onedrive-${BRANCH}

# Clone the repository and checkout the specific tag
git clone --branch ${BRANCH} https://github.com/abraunegg/onedrive.git onedrive-${BRANCH}
cd onedrive-${BRANCH}

# Activate DMD
source ~/dlang/dmd-2.100.2/activate

# Build the application
./configure --enable-debug --enable-notifications
make clean
make

# Deactivate DMD
deactivate

# Show the version of the built application
./onedrive --version

This will create a local folder called:

• onedrive-v2.5.6
• onedrive-v2.5.7

Please replicate the issue and have correct reproduction so that your application output matches your claim.

[abraunegg]

Please also learn to use 3 x backticks at the start and end of your logs for readability

[abraunegg]

@jrpelegrina
Looking at the 'logging' output only .. it potentially appears that the file is actually being uploaded .. just an error in the logging output that is potentially being triggered .. can you confirm that the file is actually being uploaded the second time as a modified file ?

[jrpelegrina]

This is the result with version 2.5.6:

xxxx@xxxx-virtualbox /tmp/onedrive-v2.5.6((sin rama)) $ ./onedrive --monitor --verbose --confdir="/home/xxxx/.config/onedrive/onedrivexxxxx" --single-directory xxxxxxx
Reading configuration file: /home/xxxx/.config/onedrive/onedrivexxxx/config
Configuration file successfully loaded
Using 'user' configuration path for application config and state data: /home/xxxxx/.config/onedrive/xxxxx
WARNING: Configured 'threads = 8' exceeds available CPU cores (2). Capping to 'threads' to 2.
D-Bus message bus daemon is available; GUI notifications are now enabled

WARNING: Your cURL/libcurl version (7.81.0) has known operational bugs that impact the use of this client.
         Please report this to your distribution, requesting an update to a newer cURL version, or consider upgrading it yourself for optimal stability.

Using the following path to store the runtime application log: /home/xxxx/.config/onedrive/xxxx/log/
Forcing client to use IPv4 connections only
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Checking Application Version ...

INFO: A new onedrive client version is available. Please upgrade your client version when possible.
Current Application Version: 2.5.6
Version Available:           2.5.7

Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
WARNING: Application has been configured NOT to cleanup remote files that are deleted locally.
User Configured Rate Limit: 104857600
Downgrading all HTTP operations to HTTP/1.1 due to user configuration
Application Version:   onedrive v2.5.6
Account Type:          personal
Default Drive ID:      xxxxx
Default Root ID:       xxxxxx
Microsoft Data Centre: France Central
Remaining Free Space:  4.60 GB (4939212390 bytes)
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /home/xxxx/xxxxx
The OneDrive Client was asked to search for this directory online and create it if it's not located: ./xxxxx
Syncing changes from this selected path: xxxxx
OneDrive synchronisation interval (seconds): 300
Maximum allowed open files:                  9223372036854775807
Maximum allowed inotify user watches:        29031
Initialising filesystem inotify monitoring ...
Monitoring directory: ./xxxxx
Performing initial synchronisation to ensure consistent local state ...
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Performing a database consistency and integrity check on locally stored data
Processing DB entries for this Drive ID: xxxxx
Scanning the local file system 'xxxxx' for new data to upload
Skipping file - excluded by skip_file config: xxxxx/.directory
Sync with Microsoft OneDrive is complete
[M] New local file added: xxxxx/test_nuevo_444.txt
New items to upload to Microsoft OneDrive: 1
Total New Data to Upload:        2 Bytes
Uploading new file: xxxx/test_nuevo_444.txt ... done
WARNING: Skipping upload integrity check for: xxxx/test_nuevo_444.txt
Attempting removal of local file as --upload-only & --remove-source-files configured
Removed local file:  xxxxx/test_nuevo_444.txt
[M] Total number of local file(s) added or changed: 1
[M] Local item deleted:xxxx/test_nuevo_444.txt
The operating system sent a deletion notification. Trying to delete this item as requested: xxxxx/test_nuevo_444.txt
Item cannot be deleted from Microsoft OneDrive because it was not found in the local database
[M] New local file added: xxxxx/test_nuevo_444.txt
New items to upload to Microsoft OneDrive: 1
Total New Data to Upload:        2 Bytes
The file we are attempting to upload as a new file already exists on Microsoft OneDrive:xxxxx/test_nuevo_444.txt
Skipping uploading this item as a new file, will upload as a modified file (online file already exists): xxxxx/test_nuevo_444.txt
Uploading modified file: xxxxx/test_nuevo_444.txt ... done
WARNING: Skipping upload integrity check for:xxxxx/test_nuevo_444.txt
Attempting removal of local file as --upload-only & --remove-source-files configured
Removed local file:  xxxxx/test_nuevo_444.txt
[M] Total number of local file(s) added or changed: 1
[M] Local item deleted: xxxxx/test_nuevo_444.txt
The operating system sent a deletion notification. Trying to delete this item as requested: xxxxx/test_nuevo_444.txt
Item cannot be deleted from Microsoft OneDrive because it was not found in the local database

And this the result with 2.5.7:

xxxx@xxxxx-virtualbox /tmp/onedrive-v2.5.7((sin rama)) $ ./onedrive --monitor --verbose --confdir="/home/xxxx/.config/onedrive/xxxx" --single-directory xxxx
Reading configuration file: /home/xxxx/.config/onedrive/xxxx/config
Configuration file successfully loaded
Using 'user' configuration path for application config and state data: /home/xxxx/.config/onedrive/xxxxx
WARNING: Configured 'threads = 8' exceeds available CPU cores (2).
         This may lead to reduced performance, CPU contention, and instability. For best results, set 'threads' no higher than the number of physical CPU cores.
D-Bus message bus daemon is available; GUI notifications are now enabled

WARNING: Your cURL/libcurl version (7.81.0) has known operational bugs that impact the use of this client.
         Please report this to your distribution, requesting an update to a newer cURL version, or consider upgrading it yourself for optimal stability.

Using the following path to store the runtime application log: /home/xxxxx/.config/onedrive/xxxxx/log/
Forcing client to use IPv4 connections only
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Checking Application Version ...
Attempting to initialise the OneDrive API ...
Configuring Global Azure AD Endpoints
The OneDrive API was initialised successfully
Opening the item database ...
The item database is incompatible, re-creating database table structures
WARNING: Application has been configured NOT to cleanup remote files that are deleted locally.
User Configured Rate Limit: 104857600
Downgrading all HTTP operations to HTTP/1.1 due to user configuration
Application Version:   onedrive v2.5.7
Account Type:          personal
Default Drive ID:      xxxx
Default Root ID:       xxxxxx
Microsoft Data Centre: France Central
Remaining Free Space:  4.60 GB (4939212390 bytes)
Sync Engine Initialised with new Onedrive API instance
All application operations will be performed in the configured local 'sync_dir' directory: /home/xxxx/xxxx
The OneDrive Client was asked to search for this directory online and create it if it's not located: ./xxxxx
Syncing changes from this selected path: xxxxx
OneDrive synchronisation interval (seconds): 300
Maximum allowed open files:                  9223372036854775807
Maximum allowed inotify user watches:        29031
Initialising filesystem inotify monitoring ...
Monitoring directory: ./xxxxx
Performing initial synchronisation to ensure consistent local state ...
Attempting to contact Microsoft OneDrive Login Service
Successfully reached Microsoft OneDrive Login Service
Starting a sync with Microsoft OneDrive
Performing a database consistency and integrity check on locally stored data
Processing DB entries for this Drive ID: xxxxx
Scanning the local file system 'xxxx' for new data to upload
Skipping file - excluded by skip_file config: xxxxx/.directory
Sync with Microsoft OneDrive is complete
[M] New local file added: xxxxx/test_nuevo_555.txt
New items to upload to Microsoft OneDrive: 1
Total New Data to Upload:        2 Bytes
Uploading new file: xxxxx/test_nuevo_555.txt ... done
WARNING: Skipping upload integrity check for:xxxx/test_nuevo_555.txt
Attempting removal of local file as --upload-only & --remove-source-files configured
Removed local file: xxxxx/test_nuevo_555.txt
[M] Total number of local file(s) added or changed: 1
[M] Local item deleted: xxxx/test_nuevo_555.txt
The operating system sent a deletion notification. Trying to delete this item as requested: xxxxx/test_nuevo_555.txt
Item cannot be deleted from Microsoft OneDrive because it was not found in the local database
[M] New local file added:xxxx/test_nuevo_555.txt
New items to upload to Microsoft OneDrive: 1
Total New Data to Upload:        2 Bytes
The file we are attempting to upload as a new file already exists on Microsoft OneDrive:xxxx/test_nuevo_555.txt
Skipping uploading this item as a new file, will upload as a modified file (online file already exists): xxxx/test_nuevo_555.txt
Skipping uploading modified file:xxxx/test_nuevo_555.txt due to insufficient free space available on Microsoft OneDrive
[M] Total number of local file(s) added or changed: 1

Regarding your question, in both tests the file I try to upload the second time is exactly the same, unmodified. In the case of v2.5.7, if I upload the modified file, the error is the same:

[M] New local file added: xxxxx/test_nuevo_555.txt
New items to upload to Microsoft OneDrive: 1
Total New Data to Upload:        11 Bytes
The file we are attempting to upload as a new file already exists on Microsoft OneDrive: xxxxx/test_nuevo_555.txt
Skipping uploading this item as a new file, will upload as a modified file (online file already exists):xxxx/test_nuevo_555.txt
Skipping uploading modified file:xxx/test_nuevo_555.txt due to insufficient free space available on Microsoft OneDrive
[M] Total number of local file(s) added or changed: 1

Accessing my OneDrive account I see that the file uploaded with version v2.5.7 does not reflect the last change made.

[abraunegg]

@jrpelegrina
Please raise a new bug report. Unfortunately it will be a few days before I can look into this with some detail.

You should also update your curl/libcurl version from 7.81.0. Please read: https://github.com/abraunegg/onedrive/blob/master/docs/usage.md#compatibility-with-curl

If using Ubuntu / Debian please read #2997 for resolution.

When the next 'release' of the client occurs, this will include websocket notification support which will require you to be running a way more modern curl/libcurl version. Your choice here will then to be update those components or use a better distribution.

[jrpelegrina]

OK will open a bug report

Thank you so much for your interest