perf: resolve each uid once per process_iter() call

[Dipet]

Summary

• OS: Linux (Ubuntu 22.04, x86_64), but the change is platform-independent POSIX code
• Bug fix: no
• Type: performance

Description

Process.username() calls pwd.getpwuid() on every invocation. That goes through NSS, and on a host that resolves users over a directory service each call costs a fraction of a millisecond. A system has orders of magnitude fewer users than processes, so a full process_iter(attrs=["username", ...]) resolves the same handful of uids thousands of times.

On a box with ~15 600 processes and 80 distinct uids:

process_iter(attrs=["username", "pid", "name"])	median
master	10250.4 ms
this PR	2365.6 ms

4.33x. Resolved names are identical to the uncached ones for all 15 196 processes present in both runs.

Scope of the reuse

Reuse is deliberately limited to a single process_iter() call rather than being process-wide. A snapshot is already a snapshot, so reusing an answer inside one is free of consequences; keeping it across calls would mean a long-running monitor reports a renamed user under the old name until restart. That is a behaviour change I did not want to make on your behalf.

Outside process_iter() a bare username() resolves the uid every time, exactly as today. Nesting keeps the outer scope's cache, and concurrent use from several threads is safe: the mapping does not depend on the caller, so the worst case is that one scan resolves a uid another one had already looked up.

If you would rather have a wider cache, this becomes a one-liner — functools.lru_cache on the resolver, plus a cache_clear hook alongside the existing process_iter.cache_clear. I am happy to switch to that if you consider the staleness acceptable; I just did not want to decide it myself. Note lru_cache evicts by recency rather than age, so hot uids would effectively never be refreshed.

Update: uids the system cannot resolve

The first commit cached successful lookups only. pwd.getpwuid() raises KeyError for a uid with no entry behind it, and resolve() let that propagate to username(), which turned it into str(uid) without recording anything — so every process owned by a deleted account still paid a full NSS round trip.

Moving that fallback out of username() and into resolve() lets a miss be cached like a hit, and removes the try/except at the call site.

Measured on the same host, now at 21 153 processes / 79 distinct uids, two of which resolve to nothing and are shared by 607 processes (passwd: files <network module> in nsswitch.conf, nscd and sssd both inactive, ~800 µs per failing lookup):

getpwuid() calls per process_iter()
master	21 140
first commit	688
with this fix	79

The 609 avoidable calls are worth ~486 ms per scan. The username attribute alone goes from 12 155 ms to 1 638 ms (7.4x) on this host.

For a real consumer: a full glances refresh that includes its cached attributes drops from 20.2 s to 8.6 s with this PR plus #2955.

Tests

Two tests added to TestProcessIter, both of which fail without the change:

	on master	on the first commit	with the fix
test_username_resolves_each_uid_once	21089 <= 79 ❌	689 <= 79 ❌	✅
test_username_resolves_each_unknown_uid_once	21214 <= 79 ❌	21234 <= 79 ❌	✅

The first guards the original claim; the second isolates the miss path by mocking every lookup to fail.

tests/test_system.py tests/test_process.py → 151 passed, 9 skipped, 3 failed, plus 3 passed in the isolated pass.

The three failures — test_long_name, test_memory_maps_lists_lib and test_users — fail identically on unmodified upstream/master in this environment, verified by running exactly those three on both.

test_pid_exists_2 was deselected. A profile shows it spends its time in the except AssertionError branch, where each process that died between psutil.pids() and pid_exists() costs time.sleep(0.1) plus a full re-walk of /proc. This machine has ~21 000 processes and 88 logged-in users, so that branch fires constantly and the test runs for minutes — and since it is a single test it pins one xdist worker while the other 15 idle. Unrelated to this change, which does not touch pids(), but it looks worth fixing separately: comparing against one snapshot taken after the loop would do the same job.