|
| 1 | +# Performance |
| 2 | + |
| 3 | +`autoread-dotenv` hooks into `sitecustomize`, which runs on the start of *every* Python |
| 4 | +process in the venv - short-lived CLI invocations, `pytest`, `pip`/`uv` themselves, and so |
| 5 | +on. That makes its startup-time cost worth measuring and tracking explicitly, rather than |
| 6 | +assuming it away. |
| 7 | + |
| 8 | +## Summary |
| 9 | + |
| 10 | +Measured with `hyperfine` (see "Methodology" section below), comparing four scenarios in |
| 11 | +throwaway venvs: |
| 12 | + |
| 13 | +| Scenario | Mean | vs. bare venv | |
| 14 | +| ------------------------------------------------------------ | ---------------: | ------------: | |
| 15 | +| Bare venv, no sitecustomize hooks at all | 12.3 ms ± 0.5 ms | 1.00x | |
| 16 | +| `sitecustomize-entrypoints` installed, no entries registered | 36.5 ms ± 1.8 ms | 2.97x | |
| 17 | +| `autoread-dotenv` installed, no `.env` found | 38.7 ms ± 3.7 ms | 3.14x | |
| 18 | +| `autoread-dotenv` installed, `.env` loaded | 58.0 ms ± 2.5 ms | 4.71x | |
| 19 | + |
| 20 | +Measured 2026-08-16, uv 0.12.3, hyperfine 1.20.0, Python 3.14.7, Linux x86_64. Each venv had |
| 21 | +only the packages named above installed (see "Methodology" section for why that matters). |
| 22 | +Re-run `just benchmark-startup` to reproduce/update these numbers - they are a point-in-time |
| 23 | +snapshot, not a guarantee. |
| 24 | + |
| 25 | +**Reading these numbers:** |
| 26 | + |
| 27 | +- The jump from "bare venv" to "`sitecustomize-entrypoints` only" (+24.2 ms) is the cost of |
| 28 | + the entry-point *discovery* mechanism itself - scanning every installed distribution's |
| 29 | + metadata for a registered `sitecustomize` entry point - and happens regardless of whether |
| 30 | + autoread-dotenv is one of the packages found. This is not something autoread-dotenv |
| 31 | + controls or can optimize away; it is the fixed cost of opting into |
| 32 | + `sitecustomize-entrypoints` at all. |
| 33 | +- **"No `.env` found" now costs essentially nothing beyond that floor** (38.7 ms vs. 36.5 ms |
| 34 | + for the bare hook mechanism - within noise). This is the result of the fix described |
| 35 | + below; it used to cost 60.4 ms, +~24 ms over the same floor. See "What was fixed". |
| 36 | +- "`.env` loaded" still costs +~21 ms over the floor (58.0 ms), because actually loading a |
| 37 | + `.env` requires importing `python-dotenv` - that part of the cost is real, unavoidable |
| 38 | + work, not overhead. |
| 39 | +- All of these numbers scale with **how many packages are installed in the venv**, because |
| 40 | + the entry-point discovery scan has to check all of them, not just autoread-dotenv's own |
| 41 | + dependencies. A real project's venv (with its own dependencies, dev tools, etc.) will see |
| 42 | + a larger absolute number than the minimal venvs used here. `just benchmark-importtime` |
| 43 | + (below) uses this project's own ~95-package dev venv for that reason, and shows |
| 44 | + proportionally larger absolute numbers for the same relative breakdown. |
| 45 | + |
| 46 | +## What was fixed |
| 47 | + |
| 48 | +Measuring this surfaced two things that were costing every process a lookup or an import it |
| 49 | +usually didn't need, regardless of whether there was a `.env` to load: |
| 50 | + |
| 51 | +1. **`import dotenv` was eager.** [`src/autoread_dotenv/__init__.py`](../src/autoread_dotenv/__init__.py) |
| 52 | + used to `import dotenv` (python-dotenv) unconditionally at module level, before |
| 53 | + `entrypoint()` even checked whether a `.env` file exists. Fixed: `entrypoint()` now |
| 54 | + checks `get_dotenv_path()` *first*, and only imports `dotenv` once it knows there's an |
| 55 | + actual file to hand to `dotenv.load_dotenv()`. No `.env` found -> `dotenv` is never |
| 56 | + imported. |
| 57 | + |
| 58 | +1. **`autoread_dotenv.about`'s metadata lookup was eager, and pulled in unconditionally.** |
| 59 | + `about.py` called `importlib.metadata.metadata()` at module level purely to populate |
| 60 | + `__version__`/`__author__`/`__license__` on the `autoread_dotenv` package - and |
| 61 | + `__init__.py` re-exported those three names, which forced `about.py` to be imported (and |
| 62 | + its metadata lookup to run) on every process start via the sitecustomize hook, whether or |
| 63 | + not anything ever read them. Fixed by removing that re-export: `__init__.py` no longer |
| 64 | + imports `about.py` at all, so its cost is paid only if something explicitly does |
| 65 | + `from autoread_dotenv.about import version` (as the test-suite does) - never as a side |
| 66 | + effect of the sitecustomize hook firing. |
| 67 | + |
| 68 | + **This is a breaking change to the public API:** `autoread_dotenv.__version__`, |
| 69 | + `__author__`, and `__license__` no longer exist. Use |
| 70 | + `autoread_dotenv.about.version`/`.license_`/`.authors` directly if you need them (see |
| 71 | + `about.py` - unchanged, still eager, but now only runs when you actually import it). |
| 72 | + |
| 73 | +Net effect: the "no `.env` found" scenario dropped from 60.4 ms to 38.7 ms - it now sits |
| 74 | +right at the `sitecustomize-entrypoints`-only floor, meaning autoread-dotenv itself adds |
| 75 | +essentially nothing in that case anymore. The "`.env` loaded" scenario only dropped slightly |
| 76 | +(59.6 ms -> 58.0 ms), because it still needs to import `dotenv` to actually do its job - see |
| 77 | +"Where the remaining cost goes" below for confirmation neither `dotenv` nor |
| 78 | +`autoread_dotenv.about` show up in the "no `.env`" import tree anymore. |
| 79 | + |
| 80 | +## Where the remaining cost goes |
| 81 | + |
| 82 | +`python -X importtime` breaks down import cost per module. Filtered to the relevant lines |
| 83 | +(via `just benchmark-importtime`, run against this project's own dev venv, which has a real |
| 84 | +`.env`): |
| 85 | + |
| 86 | +```text |
| 87 | +import time: 2223 | 38934 | sitecustomize._vendor.importlib_metadata |
| 88 | +import time: 358 | 358 | autoread_dotenv.utils |
| 89 | +import time: 464 | 464 | autoread_dotenv.warnings |
| 90 | +import time: 1149 | 1149 | dotenv.parser |
| 91 | +import time: 585 | 585 | dotenv.variables |
| 92 | +import time: 1063 | 23467 | dotenv.main |
| 93 | +import time: 508 | 23975 | dotenv |
| 94 | +import time: 12760 | 81876 | sitecustomize |
| 95 | +import time: 2745 | 94071 | site |
| 96 | +``` |
| 97 | + |
| 98 | +(First column is self-time in µs, second is cumulative including sub-imports, both in |
| 99 | +`site`'s subtree.) Note `autoread_dotenv.about` no longer appears in this tree at all - only |
| 100 | +`utils` and `warnings`, both negligible. `dotenv` still does, because a `.env` was actually |
| 101 | +found and loaded here; re-running against a venv with no `.env` drops the `dotenv` lines too, |
| 102 | +leaving only `autoread_dotenv.utils`/`warnings` under `sitecustomize`. |
| 103 | + |
| 104 | +What's left is `sitecustomize-entrypoints`'s own entry-point discovery scan (not |
| 105 | +autoread-dotenv's to optimize) and, when a `.env` is actually found, the real cost of |
| 106 | +importing `python-dotenv` to load it. |
| 107 | + |
| 108 | +## Conclusions |
| 109 | + |
| 110 | +1. **autoread-dotenv no longer meaningfully adds to the hook-mechanism tax when there's no |
| 111 | + `.env` to load.** Before the fix, autoread-dotenv doubled the cost of opting into |
| 112 | + `sitecustomize-entrypoints` at all (+~24 ms on top of the +~24 ms discovery-scan floor). |
| 113 | + After: the "no `.env`" case sits within noise of that floor. |
| 114 | + |
| 115 | +1. **The remaining cost, when a `.env` *is* found, is real work - not overhead.** Loading a |
| 116 | + `.env` requires importing `python-dotenv` (~21 ms of the +~21 ms over the floor); that's |
| 117 | + the package doing its actual job, not something left on the table. **Caveat:** the `.env` |
| 118 | + fixture used here is a single line. This isolates the import-cost floor for that path, not |
| 119 | + necessarily the ceiling for a much larger, real-world `.env` (dozens of vars, `${VAR}` |
| 120 | + interpolation) - that's an open question this benchmark doesn't answer yet. |
| 121 | + |
| 122 | +1. **The cost scales with venv size, so these numbers aren't fixed constants.** The discovery |
| 123 | + scan checks every installed distribution, not just autoread-dotenv's own dependencies - a |
| 124 | + lean production venv pays less than a kitchen-sink dev venv. This project's own |
| 125 | + ~95-package dev venv (used for the `importtime` breakdown above) shows proportionally |
| 126 | + larger absolute numbers for the same relative shape. |
| 127 | + |
| 128 | +1. **Practical takeaway:** the "no `.env`" case - which covers `pip`/`uv`/utility scripts run |
| 129 | + from outside a project root, or any venv where autoread-dotenv is installed but unused - |
| 130 | + now costs essentially nothing beyond opting into `sitecustomize-entrypoints` itself. The |
| 131 | + "`.env` found and loaded" case still carries `python-dotenv`'s own import cost, which |
| 132 | + compounds for anything that spawns Python repeatedly (CI matrices, per-test subprocess |
| 133 | + isolation, cold-start-sensitive environments) - but that's now the honest floor for doing |
| 134 | + the job, not accidental overhead. |
| 135 | + |
| 136 | +## Methodology |
| 137 | + |
| 138 | +Two complementary tools, both wrapped in `justfile` recipes so they're reproducible rather |
| 139 | +than one-off measurements: |
| 140 | + |
| 141 | +- **[`hyperfine`](https://github.com/sharkdp/hyperfine)** (`just benchmark-startup`) for |
| 142 | + end-to-end wall-clock comparisons, with proper warmup runs and statistical variance - a |
| 143 | + single-shot `time python -c pass` is not trustworthy enough to publish. Builds throwaway |
| 144 | + venvs in a tempdir (no side-effects on this project's own `.venv`) for four scenarios: |
| 145 | + bare / `sitecustomize-entrypoints`-only / autoread-dotenv-without-`.env` / |
| 146 | + autoread-dotenv-with-`.env`. |
| 147 | + |
| 148 | + > `hyperfine` is a Rust CLI binary, install via your system package manager (`apt install hyperfine`, `brew install hyperfine`, `cargo install hyperfine`, ...). **Not** `pip install hyperfine` / `uv add hyperfine` - that installs an unrelated PyPI package (a |
| 149 | + > scientific curve-fitting library depending on `numpy`, `jax`, `scipy`, `pandas`, |
| 150 | + > `iminuit`) that happens to share the name. |
| 151 | +
|
| 152 | +- **`python -X importtime`** (stdlib, `just benchmark-importtime`) for *attributing* cost to |
| 153 | + specific imports, to see where the wall-clock difference actually goes rather than just |
| 154 | + how big it is. |
| 155 | + |
| 156 | +Tracking approach: doc-only, re-run manually (e.g. before releases, or when touching |
| 157 | +`entrypoint()`/`about.py`/dependencies). No CI job or third-party benchmarking service for |
| 158 | +now - revisit if regressions start slipping through unnoticed. |
0 commit comments