Umbrella issue: psutil C Extension Analysis Report

[devdanzin]

Summary

• OS: Linux
• Architecture: 64bit
• Psutil version: 7.2.2
• Python version: 3.14.3+
• Type: core

Description

psutil C Extension Analysis Report

Extension: psutil/_psutil_linux

Scope: ~/projects/laruche/repositories/psutil/psutil/ (Linux build: 17 C files, deep analysis + cross-platform git history)

Agents Run: refcount-auditor, error-path-analyzer, null-safety-scanner, gil-discipline-checker, resource-lifecycle-checker, module-state-checker, pyerr-clear-auditor, version-compat-scanner, c-complexity-analyzer, git-history-analyzer (10/10)

Executive Summary

psutil's Linux C extension is an exceptionally well-structured codebase — zero complexity hotspots, clean GIL discipline, zero PyErr_Clear() calls, and consistent use of safe helper wrappers (pylist_append_fmt, pydict_add). The in-scope Linux code has relatively few bugs: a sizeof(ptr) buffer size error in MAC address formatting, NULL exception objects passed to PyErr_SetObject, a py_retlist leak, and some PyModule_AddObject error handling gaps. However, the cross-platform git history analysis revealed high-value correctness bugs in other platform implementations: saved_gid returns the wrong value on all three BSDs (copy-paste of svuid instead of svgid), OpenBSD shared memory is missing a * pagesize multiplier, and SunOS has a NULL-check-on-wrong-variable segfault. These cross-platform findings demonstrate the power of the git-history-analyzer for multi-platform codebases.

Extension Profile

• Module: _psutil_linux (17 C files, ~1.9k LOC hand-written C, Linux scope)
• Full extension: 90 C files across 12 platforms, ~15k LOC total
• Init style: single-phase (PyInit__psutil_linux, m_size=-1)
• Python targets: >=3.7
• Limited API: no
• Types defined: 0 (pure function module)
• Free-threading: Py_MOD_GIL_NOT_USED declared, utxent_lock mutex for non-thread-safe APIs
• External tools: 517 clang-tidy (127 errors, 390 warnings)

Key Metrics

Dimension	Status	FIX	CONSIDER	Top Finding
Refcount Safety	Y	2	3	py_retlist leak on PyArg_ParseTuple failure in disk.c
Error Handling	R	6	7	sizeof(ptr) buffer overflow; NULL exception to PyErr_SetObject x3
NULL Safety	Y	3	5	sizeof(ptr) confirmed; PyErr_SetObject(OSError, NULL) corrupts exception state
GIL Discipline	G	0	10	No bugs; 10 opportunities to release GIL around syscalls
Resource Lifecycle	Y	1	2	py_retlist leak; socket init to 0 instead of -1
Module State	Y	2	2	PyModule_AddObject ref leaks for ZombieProcessError and RLIM_INFINITY
PyErr_Clear Safety	G	0	0	Zero PyErr_Clear() calls in Linux scope (13 in other platforms)
Version Compat	Y	2	6	HAVE_LONG_LONG dead on 3.12+; PY_LONG_LONG removed in 3.15
Complexity	G	0	1	Zero hotspots; max score 4.6; avg 22 lines/function
Git History	R	7	1	BSD saved_gid returns UID; OpenBSD shared missing pagesize; SunOS NULL check bug

G = No FIX findings | Y = 1-3 FIX findings | R = 4+ FIX findings

Findings by Priority

Must Fix (FIX)

Cross-Platform (from git-history-analyzer)

1. BSD saved_gid returns saved UID instead of saved GID

• Location: arch/bsd/proc.c:136,158
• Agent: git-history-analyzer
• Impact: Process.gids().saved returns wrong value on FreeBSD, OpenBSD, NetBSD. Copy-paste: ki_svuid/p_svuid should be ki_svgid/p_svgid. Masked when UID == GID.

2. OpenBSD shared memory missing * pagesize multiplier

• Location: arch/openbsd/mem.c:53
• Agent: git-history-analyzer
• Impact: shared reported ~4096x too small. FreeBSD and NetBSD correctly multiply. Same bug class as the recently-fixed memory_maps pages-to-bytes issue.

3. SunOS psutil_proc_environ checks wrong variable after decode

• Location: arch/sunos/proc.c:224
• Agent: git-history-analyzer
• Impact: Checks !py_envname (already verified non-NULL) instead of !py_envval. NULL py_envval passed to PyDict_SetItem -> segfault.

4. SunOS psutil_proc_environ leaks py_retdict on parse failure

• Location: arch/sunos/proc.c:188-194
• Agent: git-history-analyzer

5. SunOS psutil_disk_io_counters leaks py_retdict on kstat_read failure

• Location: arch/sunos/disk.c:37
• Agent: git-history-analyzer

Linux Scope

6. sizeof(ptr) instead of remaining buffer size in MAC address formatting

• Location: arch/posix/net.c:110
• Agents: null-safety-scanner, error-path-analyzer, complexity-analyzer
• Impact: sizeof(ptr) is 8 (pointer size on 64-bit), not the remaining buffer space. Latent buffer overflow with unusual hardware address lengths. Works by coincidence for standard 6-byte MACs.

7. NULL exception objects passed to PyErr_SetObject (3 error helper functions)

• Location: arch/all/errors.c:47-48,64-65,83-84
• Agents: null-safety-scanner, error-path-analyzer
• Impact: If PyObject_CallFunction(PyExc_OSError, ...) fails (OOM), NULL is passed to PyErr_SetObject, clobbering the MemoryError with a bare OSError.

8. py_retlist leaked on PyArg_ParseTuple failure in disk_partitions

• Location: arch/linux/disk.c:22-28
• Agents: refcount-auditor, error-path-analyzer, resource-lifecycle-checker, git-history-analyzer

9. HAVE_LONG_LONG dead on Python 3.12+ — RLIM_INFINITY truncated on 32-bit

• Location: arch/posix/init.c:211-219
• Agent: version-compat-scanner
• Impact: PY_LONG_LONG will fail to compile on Python 3.15. On 32-bit Linux with Python 3.12+, RLIM_INFINITY is truncated.

10. RLIM_INFINITY silently swallowed on PyLong_From* failure

• Location: arch/posix/init.c:211-223
• Agents: error-path-analyzer, null-safety-scanner
• Impact: If allocation fails, v is NULL, the if (v) guard silently skips adding the constant, and the active MemoryError leaks to the next call site.

11. psutil_pid_exists returns -1 without setting Python exception

• Location: arch/posix/pids.c:58
• Agent: error-path-analyzer
• Impact: Violates the function's own API contract (documented: -1 means "error with exception set").

12. sock initialized to 0 instead of -1 in psutil_net_if_duplex_speed

• Location: arch/linux/net.c:59
• Agents: error-path-analyzer, resource-lifecycle-checker
• Impact: Latent bug — if error path reached before socket(), close(0) would close stdin. All POSIX counterparts correctly use -1.

Should Consider (CONSIDER)

• GIL release opportunities: mallinfo2() (known-slow), network ioctl() calls (4 sites), getifaddrs(), getutxent() loop, getmntent() loop — all hold GIL during I/O
• PyModule_AddObject ref leaks on error for ZombieProcessError and RLIM_INFINITY
• Module object leaked on all init error paths (common pattern, low impact)
• ifaddr uninitialized before getifaddrs error path
• close() may clobber errno before psutil_oserror() in net functions
• PyLong_AsLong error check uses || instead of && (works by coincidence)
• BSD children time reports identical user/system values (documented limitation)
• Py_MOD_GIL_NOT_USED declared but global int flags lack atomics

Policy Decisions (POLICY)

• Single-phase init: Keep — only 1 global PyObject*, zero static types, LOW migration difficulty
• pythoncapi-compat: Would enable PyModule_AddObjectRef migration
• Free-threading: Partially ready (utxent_lock, Py_MOD_GIL_NOT_USED), minor gaps in PSUTIL_DEBUG flag atomics

Strengths

• Exceptionally well-structured: Zero complexity hotspots, avg 22 lines/function, max nesting 3
• Clean PyErr_Clear usage: Zero calls in Linux scope (unprecedented in our analyses)
• Good helper abstractions: pylist_append_fmt, pylist_append_obj, pydict_add handle refcounting correctly
• Consistent goto error cleanup: All resource-heavy functions use single-label cleanup
• Free-threading awareness: PyMutex for utxent, Py_MOD_GIL_NOT_USED declared
• LOW migration difficulty: Only 1 global PyObject* to move to module state
• Cross-platform consistency: The pydict_add refactoring standardized patterns across 12 platforms
• Good resource cleanup: CPU_ALLOC/CPU_FREE, setmntent/endmntent, dlopen/dlclose all correctly paired

Recommended Action Plan

Immediate (cross-platform correctness bugs)

1. Fix saved_gid in arch/bsd/proc.c:136,158 — change ki_svuid/p_svuid to ki_svgid/p_svgid
2. Fix OpenBSD shared in arch/openbsd/mem.c:53 — add * pagesize
3. Fix SunOS NULL check in arch/sunos/proc.c:224 — change !py_envname to !py_envval
4. Fix sizeof(ptr) in arch/posix/net.c:110 — use sizeof(buf) - (ptr - buf)
5. Fix NULL to PyErr_SetObject in arch/all/errors.c — add NULL check, propagate original exception
6. Fix HAVE_LONG_LONG/PY_LONG_LONG in arch/posix/init.c — use sizeof check + long long

Short-term (Linux scope)

1. Move PyArg_ParseTuple before PyList_New in disk.c (and audit other platforms for same pattern)
2. Fix RLIM_INFINITY silent swallow — return -1 when PyLong_From* fails
3. Initialize sock = -1 in arch/linux/net.c:59
4. Fix psutil_pid_exists — set OSError before returning -1
5. Release GIL around mallinfo2() and network ioctl() calls

Longer-term

1. Audit all 12 platforms for the "allocate then parse" leak pattern (systematic)
2. Migrate PyModule_AddObject to PyModule_AddObjectRef (or adopt pythoncapi-compat)
3. Add POSIX resource pairs to resource_pairs.json (socket/close, setmntent/endmntent, etc.)

psutil Analysis Report — Reproducer Appendix

psutil 7.2.2, Python 3.14.3+, Linux x86_64

Summary

1 bug reproduced on Linux with measurable impact. The cross-platform correctness bugs (BSD saved_gid, OpenBSD shared memory, SunOS NULL check) are confirmed by code review but require their respective platforms to reproduce at runtime.

Reproduced Bugs

Finding	Bug	Evidence
8	disk_partitions leaks py_retlist on bad args	56 bytes leaked per call

Confirmed by Code Review Only (require other platforms)

Finding	Bug	Platform	Evidence
1	saved_gid returns saved UID	FreeBSD, OpenBSD, NetBSD	ki_svuid/p_svuid used instead of ki_svgid/p_svgid
2	shared memory missing * pagesize	OpenBSD	All other fields multiply by pagesize; shared does not
3	psutil_proc_environ NULL check on wrong variable	SunOS	!py_envname (already non-NULL) instead of !py_envval
6	sizeof(ptr) in MAC formatting	All (latent)	8 bytes on 64-bit, format needs 4 — works by coincidence

Not Reproducible from Pure Python on Linux

Finding	Bug	Reason
7	NULL to PyErr_SetObject	Requires OOM during error-formatting PyObject_CallFunction
9	HAVE_LONG_LONG truncation	64-bit: sizeof(long) == sizeof(long long), no truncation
10	RLIM_INFINITY silent swallow	Requires OOM during PyLong_From* at module init
11	psutil_pid_exists missing exception	Returns -1 for unexpected errno, which is extremely rare
12	sock = 0 instead of -1	No current error path reaches error: before socket()

---

Reproducer 1: disk_partitions Leaks List on Bad Arguments

Bug: PyList_New(0) is called before PyArg_ParseTuple. When PyArg_ParseTuple fails (wrong argument type), the function returns NULL directly instead of going to the error label, leaking the list object.

File: arch/linux/disk.c:22-28

import tracemalloc, gc
import psutil._psutil_linux as _psutil

tracemalloc.start()
gc.collect()
before = tracemalloc.get_traced_memory()[0]

for _ in range(10000):
    try:
        _psutil.disk_partitions(123)  # int instead of str
    except TypeError:
        pass

gc.collect()
after = tracemalloc.get_traced_memory()[0]
print(f"Leaked {after - before} bytes over 10000 calls")
assert after - before > 100000
# Leaked 562008 bytes over 10000 calls (~56 bytes/call)

Output:

Leaked 562008 bytes over 10000 calls

---

Code-Review-Only Findings (Cross-Platform)

The following bugs cannot be reproduced on Linux but are unambiguous from the source code.

BSD saved_gid Returns Saved UID

File: arch/bsd/proc.c:136 (FreeBSD) and :158 (OpenBSD/NetBSD)

The macOS implementation at arch/osx/proc.c:104 correctly uses p_svgid:

// macOS (CORRECT):
if (!pydict_add(dict, "saved_gid", "l", (long)kp.kp_eproc.e_pcred.p_svgid)) goto error;

The BSD implementation uses svuid for both:

// FreeBSD (BUG):
if (!pydict_add(dict, "saved_uid", "l", (long)kp.ki_svuid)) goto error;  // correct
if (!pydict_add(dict, "saved_gid", "l", (long)kp.ki_svuid)) goto error;  // BUG: ki_svuid should be ki_svgid

// OpenBSD/NetBSD (same BUG):
if (!pydict_add(dict, "saved_uid", "l", (long)kp.p_svuid)) goto error;   // correct
if (!pydict_add(dict, "saved_gid", "l", (long)kp.p_svuid)) goto error;   // BUG: p_svuid should be p_svgid

Process.gids().saved returns the saved UID, not the saved GID, on FreeBSD, OpenBSD, and NetBSD. Masked when they're equal (common for regular processes).

OpenBSD shared Memory Missing Pagesize

File: arch/openbsd/mem.c:53

// OpenBSD (BUG — missing * pagesize):
shared = (unsigned long long)vmdata.t_vmshr + vmdata.t_rmshr;

// FreeBSD (CORRECT):
shared = (unsigned long long)(vm.t_vmshr + vm.t_rmshr) * pagesize;

// NetBSD (CORRECT):
shared = (unsigned long long)(vmdata.t_vmshr + vmdata.t_rmshr) * pagesize;

Reports ~4096x too small on systems with 4KB pages.

SunOS psutil_proc_environ Checks Wrong Variable

File: arch/sunos/proc.c:220-224

py_envname = PyUnicode_DecodeFSDefault(env[i]);
if (!py_envname)       // line 220: correct
    goto error;

py_envval = PyUnicode_DecodeFSDefault(dm + 1);
if (!py_envname)       // line 224: BUG — should be !py_envval
    goto error;
// NULL py_envval flows to PyDict_SetItem → segfault

Full report also available at: https://gist.github.com/devdanzin/c050bdb008c0f5351d3e78afd565f8f1