Skip to content

check_flexbuff_usage takes extremely long #2

@jwagnerhki

Description

@jwagnerhki

In shared.py check_flexbuff_usage() the step of listing everything found on all disk, i.e., "find /mnt/disk*/ -mindepth 1 -maxdepth 1 -type d | xargs du -b -s --exclude lost+found/" is relatively slow.

For example on Onsala flexbuff of 36 disks it produces about 19 MByte worth of output and takes about 10 minutes.

bonn@kare ~ $ cat try.sh
find /mnt/disk*/ -mindepth 1 -maxdepth 1 -type d | xargs du -b -s --exclude lost+found > /tmp/lst

bonn@kare ~ $ /usr/bin/time ./try.sh
du: cannot read directory '/mnt/disk31/Data/aips': Permission denied
Command exited with non-zero status 123
29.28user 39.10system 10:04.66elapsed 11%CPU (0avgtext+0avgdata 34976maxresident)k
3350472inputs+0outputs (0major+106015minor)pagefaults 0swaps

bonn@kare ~ $ ls -alh /tmp/lst
-rw-rw-r-- 1 bonn bonn 19M Feb 8 12:38 /tmp/lst

With "xargs -P 8" the time drops to 5 minutes. But still excessively long.

Maybe there is a VSI-S query to replace SSH+find+xargs+du...?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions