dvc push not updating Push %

[danshome]

Bug Report

Issue name

dvc push -v -j 4: Doesn't update Pushing %, B/s transferred, or transfer times.

Description

When I run dvc push to an S3 bucket the % always reports 0%. I think it's pushing because I can see my outbound network traffic spike, but the percentage never changes.

dvc push -v -j 4
2024-09-13 12:06:26,854 DEBUG: v3.55.2 (conda), CPython 3.11.10 on macOS-14.6.1-arm64-arm-64bit
2024-09-13 12:06:26,854 DEBUG: command: /Users/myid/workspace/myproject/.venv/bin/dvc push -v -j 4
Collecting |0.00 [00:00, ?entry/s]
2024-09-13 12:06:27,058 DEBUG: Preparing to transfer data from /Users/myid/workspace/myproject/.dvc/cache/files/md5' to 's3://my-dvc-data-bucket/data/files/md5'
2024-09-13 12:06:27,058 DEBUG: Preparing to collect status from 'my-dvc-data-bucket/data/files/md5'
2024-09-13 12:06:27,058 DEBUG: Collecting status from 'my-dvc-data-bucket/data/files/md5'
2024-09-13 12:06:29,488 DEBUG: Estimated remote size: 4096 files
2024-09-13 12:06:29,489 DEBUG: Large remote (3 oids < 4.096 traverse weight), using object_exists for remaining oids
2024-09-13 12:06:29,490 DEBUG: Querying 3 oids via object_exists
2024-09-13 12:06:30,456 DEBUG: Preparing to collect status from '/Users/myid/.dvc/cache/files/md5'
2024-09-13 12:06:30,457 DEBUG: Collecting status from '/Users/myid/.dvc/cache/files/md5'
Pushing
0%| |Pushing to s3 0/3 [00:00<?, ?file/s]
0%| |/Users/myid/.dvc/cache/files/md5/9b/f40a981a0f663131cd5df87949dd2e 0.00/73.8M [00:00<?, ?B/s]

Reproduce

1. dvc init
2. dvc remote add -d myremote s3://my-dvc-data-bucket/data
3. Copy 3 encrypted zip files dataset1.zip, dataset2.zip, dataset3.zip to ./data
4. dvc add data/*
5. dvc push -v -j 4

Expected

I would expect the Pushing %, B/s transferred, or transfer times to update as it's uploading to S3.

Environment information

MacOS Macbook M3 Max

Output of dvc doctor:

dvc doctor
DVC version: 3.55.2 (conda)

Platform: Python 3.11.10 on macOS-14.6.1-arm64-arm-64bit
Subprojects:
dvc_data = 3.16.5
dvc_objects = 5.1.0
dvc_render = 1.0.2
dvc_task = 0.4.0
scmrepo = 3.3.7
Supports:
http (aiohttp = 3.10.5, aiohttp-retry = 2.8.3),
https (aiohttp = 3.10.5, aiohttp-retry = 2.8.3),
s3 (s3fs = 2024.6.1, boto3 = 1.34.154)
Config:
Global: /Users/myid/Library/Application Support/dvc
System: /Library/Application Support/dvc
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk3s1s1
Caches: local
Remotes: s3
Workspace directory: apfs on /dev/disk3s1s1
Repo: dvc, git
Repo.site_cache_dir: /Library/Caches/dvc/repo/a463a6711ebfd4b1cff05ad93828b21b

[danshome]

As I was opening this issue I had a dvc push running and it eventually did do something, but as you can see it still looks messed up. The % changed to 33% after it uploaded one file, but the % and B/s for the files themselves never updates...

Pushing
33%|███▎ |Pushing to s3 1/3 [12:54<25:47, 773.91s/file]
0%| |/Users/myid/workspace/myproject/.dvc/cache/files/md5/a5/d2f3c1cfc9bd18decfe5c243af638b 0.00/198M [00:00<?, ?B/s]
0%| |/Users/myid/workspace/myproject/.dvc/cache/files/md5/6b/2a1e56628a17a38cf775807b667aa4 0.00/13.5M [00:00<?, ?B/s]

[danshome]

Here's what it looks like now...

Pushing
67%|██████▋ |Pushing to s3 2/3 [22:52<11:10, 670.61s/file]
0%| |/Users/myid/workspace/myproject/.dvc/cache/files/md5/a5/d2f3c1cfc9bd18decfe5c243af638b 0.00/198M [00:00<?, ?B/s]

[danshome]

On the last file it finally updated a %, but its not actively changing in realtime...
Pushing
67%|██████▋ |Pushing to s3 2/3 [22:52<11:10, 670.61s/file]
25%|██▌ |/Users/myid/workspace/myproject/.dvc/cache/files/md5/a5/d2f3c1cfc9bd18decfe5c243af638b 50.0M/198M [17:35<52:06, 49.7kB/s]

If you didn't know any better you'd think it was hanging because nothing is updating, but if you look at the network traffic you can see it is actually sending data.

[danshome]

If it's not supposed to update in realtime that would explain the behavior I'm seeing, but my expectation is that it would behave the same as any other file transfer application I've ever used in the past 20 years...and usually during a file transfer the % and B/s will updated realtime.