wget2 2.1.0 excessive CPU usage: all cores at 100%

[mrdc]

Hello,

I'm experiencing a bug in wget2 2.1.0: during site mirroring it occupies all cores at 100%. It's reproducible.

Command line I use:
wget2 --max-threads 15 -r -l 10 --reject-regex='wp-login.php?redirect_to=' --mirror --convert-links --adjust-extension --page-requisites --no-parent https://thetechnicgear.com

Tech specs:
Apple M1 Pro
macOS 14.5
wget2 installed via:
brew install wget2

wget2 --version

GNU Wget2 2.1.0 - multithreaded metalink/file/website downloader

+digest +https +ssl/gnutls +ipv6 +iri +large-file +nls -ntlm -opie +psl -hsts
+iconv +idn2 +zlib +lzma +brotlidec +zstd +bzip2 +lzip +http2 +gpgme

Hope it helps.

[rockdaboot]

Hm, here on Linux, it still shows 10% CPU usage, which is quite high.
Can you try with --progress=none and report back?

[mrdc]

Hm, here on Linux, it still shows 10% CPU usage, which is quite high. Can you try with --progress=none and report back?

Now it shows 20% with the command line I've posted in the issue.
Using --progress=none drops CPU utilization to 7-12%, with some rare spikes up to 18%.

[rockdaboot]

Using --progress=none drops CPU utilization to 7-12%, with some rare spikes up to 18%.

That sounds reasonable downloads via HTTPS and also depends on the utilized network bandwidth and your CPU, of course.

I think, the progress bar issue is identified and can be fixed by limiting the number of bar updates per second. Just needs to be implemented =)