Skip to content

recsplit: support progress reporting#19536

Merged
AskAlexSharov merged 6 commits intomainfrom
alex/rs_progress_34
Mar 1, 2026
Merged

recsplit: support progress reporting#19536
AskAlexSharov merged 6 commits intomainfrom
alex/rs_progress_34

Conversation

@AskAlexSharov
Copy link
Collaborator

@AskAlexSharov AskAlexSharov commented Mar 1, 2026

from

[INFO] [02-28|06:26:16.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=12m0s alloc=4.3GB sys=11.2GB
[INFO] [02-28|06:26:36.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=12m20s alloc=4.5GB sys=11.2GB
[INFO] [02-28|06:26:56.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=12m40s alloc=4.7GB sys=11.2GB
[INFO] [02-28|06:27:16.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=13m0s alloc=4.5GB sys=11.2GB
[INFO] [02-28|06:27:36.791] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=13m20s alloc=4.3GB sys=11.2GB
[INFO] [02-28|06:27:56.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=13m40s alloc=7.8GB sys=11.2GB

to

INFO[03-01|03:40:03.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=40%, v1.1-storage.2304-2560.vi=6%" total-indexing-time=20s alloc=3.7GB sys=3.9GB
INFO[03-01|03:40:23.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=64%, v1.1-storage.2304-2560.vi=15%" total-indexing-time=40s alloc=3.8GB sys=4.0GB
INFO[03-01|03:40:43.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=83%, v1.1-storage.2304-2560.vi=23%" total-indexing-time=1m0s alloc=3.9GB sys=4.0GB
INFO[03-01|03:41:03.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=99%, v1.1-storage.2304-2560.vi=32%" total-indexing-time=1m20s alloc=5.0GB sys=5.2GB
INFO[03-01|03:41:23.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=41%" total-indexing-time=1m40s alloc=5.1GB sys=5.2GB
INFO[03-01|03:41:43.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=50%" total-indexing-time=2m0s alloc=3.6GB sys=5.4GB
INFO[03-01|03:42:03.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=53%" total-indexing-time=2m20s alloc=3.6GB sys=5.4GB
INFO[03-01|03:42:23.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=56%" total-indexing-time=2m40s alloc=3.6GB sys=5.4GB
INFO[03-01|03:42:43.226] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=60%" total-indexing-time=3m0s alloc=3.6GB sys=5.4GB
INFO[03-01|03:43:03.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=63%" total-indexing-time=3m20s alloc=3.6GB sys=5.4GB
INFO[03-01|03:43:23.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=67%" total-indexing-time=3m40s alloc=3.7GB sys=5.4GB
INFO[03-01|03:43:43.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=70%" total-indexing-time=4m0s alloc=3.7GB sys=5.4GB
INFO[03-01|03:44:03.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=74%" total-indexing-time=4m20s alloc=3.6GB sys=5.4GB
INFO[03-01|03:44:23.226] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=77%" total-indexing-time=4m40s alloc=3.6GB sys=5.4GB
INFO[03-01|03:44:43.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=80%" total-indexing-time=5m0s alloc=3.6GB sys=5.4GB

@AskAlexSharov AskAlexSharov marked this pull request as draft March 1, 2026 02:22
@AskAlexSharov AskAlexSharov marked this pull request as ready for review March 1, 2026 03:44
@AskAlexSharov AskAlexSharov enabled auto-merge (squash) March 1, 2026 03:52
Copy link
Collaborator

@Giulio2002 Giulio2002 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — clean refactor centralizing progress tracking into RecSplit.SetProgress(), removing scattered duplicate progress code across call sites.

@AskAlexSharov AskAlexSharov merged commit fe17d1f into main Mar 1, 2026
25 checks passed
@AskAlexSharov AskAlexSharov deleted the alex/rs_progress_34 branch March 1, 2026 11:00
sudeepdino008 pushed a commit that referenced this pull request Mar 4, 2026
from
```
[INFO] [02-28|06:26:16.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=12m0s alloc=4.3GB sys=11.2GB
[INFO] [02-28|06:26:36.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=12m20s alloc=4.5GB sys=11.2GB
[INFO] [02-28|06:26:56.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=12m40s alloc=4.7GB sys=11.2GB
[INFO] [02-28|06:27:16.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=13m0s alloc=4.5GB sys=11.2GB
[INFO] [02-28|06:27:36.791] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=13m20s alloc=4.3GB sys=11.2GB
[INFO] [02-28|06:27:56.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=13m40s alloc=7.8GB sys=11.2GB
```
to
```
INFO[03-01|03:40:03.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=40%, v1.1-storage.2304-2560.vi=6%" total-indexing-time=20s alloc=3.7GB sys=3.9GB
INFO[03-01|03:40:23.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=64%, v1.1-storage.2304-2560.vi=15%" total-indexing-time=40s alloc=3.8GB sys=4.0GB
INFO[03-01|03:40:43.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=83%, v1.1-storage.2304-2560.vi=23%" total-indexing-time=1m0s alloc=3.9GB sys=4.0GB
INFO[03-01|03:41:03.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=99%, v1.1-storage.2304-2560.vi=32%" total-indexing-time=1m20s alloc=5.0GB sys=5.2GB
INFO[03-01|03:41:23.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=41%" total-indexing-time=1m40s alloc=5.1GB sys=5.2GB
INFO[03-01|03:41:43.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=50%" total-indexing-time=2m0s alloc=3.6GB sys=5.4GB
INFO[03-01|03:42:03.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=53%" total-indexing-time=2m20s alloc=3.6GB sys=5.4GB
INFO[03-01|03:42:23.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=56%" total-indexing-time=2m40s alloc=3.6GB sys=5.4GB
INFO[03-01|03:42:43.226] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=60%" total-indexing-time=3m0s alloc=3.6GB sys=5.4GB
INFO[03-01|03:43:03.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=63%" total-indexing-time=3m20s alloc=3.6GB sys=5.4GB
INFO[03-01|03:43:23.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=67%" total-indexing-time=3m40s alloc=3.7GB sys=5.4GB
INFO[03-01|03:43:43.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=70%" total-indexing-time=4m0s alloc=3.7GB sys=5.4GB
INFO[03-01|03:44:03.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=74%" total-indexing-time=4m20s alloc=3.6GB sys=5.4GB
INFO[03-01|03:44:23.226] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=77%" total-indexing-time=4m40s alloc=3.6GB sys=5.4GB
INFO[03-01|03:44:43.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=80%" total-indexing-time=5m0s alloc=3.6GB sys=5.4GB
```
sudeepdino008 pushed a commit that referenced this pull request Mar 4, 2026
from
```
[INFO] [02-28|06:26:16.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=12m0s alloc=4.3GB sys=11.2GB
[INFO] [02-28|06:26:36.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=12m20s alloc=4.5GB sys=11.2GB
[INFO] [02-28|06:26:56.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=12m40s alloc=4.7GB sys=11.2GB
[INFO] [02-28|06:27:16.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=13m0s alloc=4.5GB sys=11.2GB
[INFO] [02-28|06:27:36.791] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=13m20s alloc=4.3GB sys=11.2GB
[INFO] [02-28|06:27:56.790] [snapshots] Indexing                     progress="v1.1-commitment.2304-2560.vi=100%" total-indexing-time=13m40s alloc=7.8GB sys=11.2GB
```
to
```
INFO[03-01|03:40:03.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=40%, v1.1-storage.2304-2560.vi=6%" total-indexing-time=20s alloc=3.7GB sys=3.9GB
INFO[03-01|03:40:23.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=64%, v1.1-storage.2304-2560.vi=15%" total-indexing-time=40s alloc=3.8GB sys=4.0GB
INFO[03-01|03:40:43.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=83%, v1.1-storage.2304-2560.vi=23%" total-indexing-time=1m0s alloc=3.9GB sys=4.0GB
INFO[03-01|03:41:03.227] [snapshots] Indexing                     progress="v2.0-storage.2304-2560.efi=99%, v1.1-storage.2304-2560.vi=32%" total-indexing-time=1m20s alloc=5.0GB sys=5.2GB
INFO[03-01|03:41:23.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=41%" total-indexing-time=1m40s alloc=5.1GB sys=5.2GB
INFO[03-01|03:41:43.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=50%" total-indexing-time=2m0s alloc=3.6GB sys=5.4GB
INFO[03-01|03:42:03.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=53%" total-indexing-time=2m20s alloc=3.6GB sys=5.4GB
INFO[03-01|03:42:23.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=56%" total-indexing-time=2m40s alloc=3.6GB sys=5.4GB
INFO[03-01|03:42:43.226] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=60%" total-indexing-time=3m0s alloc=3.6GB sys=5.4GB
INFO[03-01|03:43:03.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=63%" total-indexing-time=3m20s alloc=3.6GB sys=5.4GB
INFO[03-01|03:43:23.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=67%" total-indexing-time=3m40s alloc=3.7GB sys=5.4GB
INFO[03-01|03:43:43.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=70%" total-indexing-time=4m0s alloc=3.7GB sys=5.4GB
INFO[03-01|03:44:03.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=74%" total-indexing-time=4m20s alloc=3.6GB sys=5.4GB
INFO[03-01|03:44:23.226] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=77%" total-indexing-time=4m40s alloc=3.6GB sys=5.4GB
INFO[03-01|03:44:43.227] [snapshots] Indexing                     progress="v1.1-storage.2304-2560.vi=80%" total-indexing-time=5m0s alloc=3.6GB sys=5.4GB
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants