[source-sftp-bulk] SFTP bulk connector - Incremental Append (sync) troubles. (Downloading same files each run)

[Bazger]

Hi,

I’m using an SFTP Bulk → S3 connection. The goal is to download files from SFTP every 2-3 hours and only fetch new files, without re-downloading files from previous runs.

However, it looks like Incremental Append / Incremental sync is not working as expected.

For example, in the previous run, the last file was:
2025_12_30_1767076584669_145.csv

The first row contains:

_ab_source_file_last_modified
2025-08-28T19:48:57.000000Z

In the most recent run, the first file is:
2025_12_30_1767106251539_0.csv

And its first row contains:

_ab_source_file_last_modified
2025-08-28T11:17:48.000000Z

This timestamp is older than the one from the previous run, which suggests that files that were already downloaded are being fetched again. Because of this, I’m unclear on how the cursor is being tracked internally.

My current configuration (via Terraform) is:

sync_mode    = "incremental_append"
cursor_field = ["_ab_source_file_last_modified"]

Could you clarify how the cursor is determined in this case, or what might be causing previously downloaded files to be included again?