Skip to content

Commit c18f567

Browse files
committed
Document and fix autoread-dotenv's startup-time impact
autoread-dotenv runs on every Python process start in the venv via the sitecustomize hook, so its startup-time cost is worth measuring and tracking explicitly. Measured with hyperfine (throwaway venvs, 50+ runs each): bare venv 12.3ms +- 0.5ms (1.00x) sitecustomize-entrypoints, no entries 36.5ms +- 1.8ms (2.97x) autoread-dotenv, no .env found before: 60.4ms (4.88x) after: 38.7ms +- 3.7ms (3.14x) autoread-dotenv, .env loaded 58.0ms +- 2.5ms (4.71x) The entry-point discovery mechanism (sitecustomize-entrypoints scanning every installed distribution's metadata) accounts for most of the jump from bare to "hooks installed" - not autoread-dotenv's to optimize. autoread-dotenv itself was adding another ~24-25ms on top of that, entirely import cost (import dotenv + about.py's eager importlib.metadata lookup), not actual .env-parsing work. Fixed both: - entrypoint() now checks whether a .env exists via get_dotenv_path() before importing dotenv at all - `import dotenv` moved inside the function, deferred until there's an actual file to load. Removed the now-unneeded DOTENV_INSTALLED module-level sentinel along with it. - __init__.py no longer re-exports about.py's version/license_/authors as __version__/__author__/__license__, so about.py's importlib.metadata lookup is never triggered by the sitecustomize hook - only if something explicitly imports autoread_dotenv.about (as the test-suite does). BREAKING: autoread_dotenv.__version__, __author__, and __license__ no longer exist. Use autoread_dotenv.about.version/.license_/.authors directly - unchanged, just no longer re-exported from the package root. tests/test_init.py (only tested those two dunders) deleted; conftest.py's importlib.reload(autoread_dotenv.about) removed since nothing imports it by default anymore. Net effect: the common "no .env found" case (pip/uv/utility scripts run outside a project root) now costs essentially nothing beyond opting into sitecustomize-entrypoints itself. Adds: - docs/performance.md: methodology, before/after numbers, what was fixed and why, linked from docs/readme.md and docs/index.rst. - .just/benchmark.justfile (benchmark-startup, benchmark-importtime) so the numbers are reproducible rather than a one-off snapshot. Flags that PyPI's "hyperfine" package is an unrelated scientific-fitting library, not the real Rust CLI tool. Tracking approach: doc-only, re-run manually. No CI job or third-party benchmarking service for now.
1 parent 47599b4 commit c18f567

10 files changed

Lines changed: 262 additions & 58 deletions

File tree

.just/benchmark.justfile

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# See ../makefile
2+
#
3+
# Benchmarks for the startup-time cost autoread-dotenv adds via the sitecustomize hook.
4+
# See docs/performance.md for the methodology, current numbers, and how to read them.
5+
#
6+
# Requires `hyperfine` (https://github.com/sharkdp/hyperfine) on PATH for benchmark-startup.
7+
# That is a Rust CLI binary, NOT the PyPI package of the same name (`pip install hyperfine` /
8+
# `uv add hyperfine` installs an unrelated scientific-fitting library that pulls in numpy,
9+
# jax, scipy, pandas, iminuit). Install hyperfine via your system package manager instead,
10+
# e.g. `apt install hyperfine`, `brew install hyperfine`, or `cargo install hyperfine`.
11+
12+
13+
# check that the real (Rust) hyperfine is on PATH, not the unrelated PyPI package
14+
[group: 'benchmark']
15+
benchmark-check-hyperfine:
16+
@command -v hyperfine >/dev/null || { \
17+
echo "hyperfine not found on PATH."; \
18+
echo "Install: https://github.com/sharkdp/hyperfine#installation"; \
19+
echo "(NOT 'pip install hyperfine' / 'uv add hyperfine' - that installs an unrelated PyPI package)."; \
20+
exit 1; \
21+
}
22+
23+
24+
# compare python startup time: bare venv vs sitecustomize-entrypoints vs autoread-dotenv
25+
[group: 'benchmark']
26+
benchmark-startup: benchmark-check-hyperfine
27+
#!/usr/bin/env bash
28+
# Builds throwaway venvs in a tempdir (no side-effects on this project's own .venv):
29+
# bare / sitecustomize-entrypoints-only / autoread-dotenv-without-.env /
30+
# autoread-dotenv-with-.env. Prints a markdown table - copy the numbers you want to
31+
# keep into docs/performance.md, noting the date, `uv --version`, `python --version`,
32+
# and how many packages were installed in the compared venvs.
33+
set -euo pipefail
34+
workdir="$(mktemp -d)"
35+
trap 'rm -rf "$workdir"' EXIT
36+
37+
echo "Building throwaway venvs in $workdir ..." >&2
38+
uv venv --python 3.14 "$workdir/bare" -q
39+
uv venv --python 3.14 "$workdir/ste-only" -q
40+
uv pip install --python "$workdir/ste-only/bin/python" sitecustomize-entrypoints -q
41+
uv venv --python 3.14 "$workdir/autoread" -q
42+
uv pip install --python "$workdir/autoread/bin/python" . -q
43+
echo "FOO=bar" > "$workdir/dotenv-fixture.env"
44+
45+
hyperfine --warmup 10 --min-runs 50 \
46+
-n "bare venv (no sitecustomize hooks)" \
47+
"$workdir/bare/bin/python -c pass" \
48+
-n "sitecustomize-entrypoints only (no entries registered)" \
49+
"$workdir/ste-only/bin/python -c pass" \
50+
-n "autoread-dotenv installed, no .env found" \
51+
"$workdir/autoread/bin/python -c pass" \
52+
-n "autoread-dotenv installed, .env loaded" \
53+
"env AUTOREAD_DOTENV_PATH=$workdir/dotenv-fixture.env $workdir/autoread/bin/python -c pass"
54+
55+
56+
# show which imports the sitecustomize hook pulls in and what they cost
57+
[group: 'benchmark']
58+
benchmark-importtime:
59+
# Complements benchmark-startup (total end-to-end cost) with an attribution breakdown:
60+
# which specific imports the hook triggers. Uses this project's own dev .venv, so the
61+
# numbers include however many dev-dependencies happen to be installed there - see
62+
# docs/performance.md for why that matters (entry-point discovery scans every
63+
# installed distribution, not just autoread-dotenv's own dependencies).
64+
uv run python -X importtime -c pass 2>&1 | grep -E \
65+
"dotenv|autoread_dotenv|importlib_metadata$|importlib\.metadata$|sitecustomize$|\\| site$"

docs/changes.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,25 @@ All notable changes to this project will be documented in this file.
6262
no stable 3.15 release yet, so `uv` treats the rc as the match), and the full test-suite
6363
passes against it unmodified.
6464

65+
- Document and fix autoread-dotenv's startup-time cost in the new `docs/performance.md`, since it
66+
runs on every Python process start in the venv. Measured with `hyperfine`: opting into
67+
`sitecustomize-entrypoints`'s entry-point discovery costs ~2.9x a bare venv's startup time
68+
regardless of which entrypoints are registered (not autoread-dotenv's to fix); autoread-dotenv
69+
itself was adding another ~24-25ms on top of that, entirely import cost (`import dotenv` +
70+
`about.py`'s eager `importlib.metadata` lookup), not actual `.env`-parsing work. Fixed both:
71+
`entrypoint()` now checks whether a `.env` exists before importing `dotenv` at all, and
72+
`__init__.py` no longer imports `about.py` (see BREAKING note below). The "no `.env` found" case
73+
dropped from 60.4ms to 38.7ms - now within noise of the bare hook-mechanism floor. Added
74+
`.just/benchmark.justfile` (`just benchmark-startup`, `just benchmark-importtime`) so the
75+
numbers are reproducible rather than a one-off snapshot. Tracking is doc-only / re-run manually
76+
for now, no CI job or third-party benchmarking service.
77+
78+
- **BREAKING:** Remove `autoread_dotenv.__version__`, `__author__`, and `__license__`. They forced
79+
`about.py`'s `importlib.metadata` lookup to run on every process start via the sitecustomize
80+
hook, whether or not anything read them (see above). Use
81+
`autoread_dotenv.about.version`/`.license_`/`.authors` directly instead - unchanged, just no
82+
longer re-exported from the package root.
83+
6584
## 1.0.5 (2026-08-16)
6685

6786
- Add codespell as dev-dependency for spellchecking.

docs/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ autoread_dotenv
55
:maxdepth: 2
66

77
readme
8+
performance
89
changes
910
security
1011
license

docs/modules/autoread_dotenv/index.rst

Lines changed: 0 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -52,16 +52,6 @@ Submodules
5252
/modules/autoread_dotenv/warnings/index
5353

5454

55-
Attributes
56-
----------
57-
58-
.. autoapisummary::
59-
60-
autoread_dotenv.__author__
61-
autoread_dotenv.__license__
62-
autoread_dotenv.__version__
63-
64-
6555
Functions
6656
---------
6757

@@ -76,15 +66,6 @@ Functions
7666
Package Contents
7767
----------------
7868

79-
.. py:data:: __author__
80-
:type: str
81-
82-
.. py:data:: __license__
83-
:type: str
84-
85-
.. py:data:: __version__
86-
:type: str
87-
8869
.. py:function:: get_dotenv_path()
8970
9071
Return the location of the .env for in-project virtualenvs.

docs/performance.md

Lines changed: 158 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,158 @@
1+
# Performance
2+
3+
`autoread-dotenv` hooks into `sitecustomize`, which runs on the start of *every* Python
4+
process in the venv - short-lived CLI invocations, `pytest`, `pip`/`uv` themselves, and so
5+
on. That makes its startup-time cost worth measuring and tracking explicitly, rather than
6+
assuming it away.
7+
8+
## Summary
9+
10+
Measured with `hyperfine` (see "Methodology" section below), comparing four scenarios in
11+
throwaway venvs:
12+
13+
| Scenario | Mean | vs. bare venv |
14+
| ------------------------------------------------------------ | ---------------: | ------------: |
15+
| Bare venv, no sitecustomize hooks at all | 12.3 ms ± 0.5 ms | 1.00x |
16+
| `sitecustomize-entrypoints` installed, no entries registered | 36.5 ms ± 1.8 ms | 2.97x |
17+
| `autoread-dotenv` installed, no `.env` found | 38.7 ms ± 3.7 ms | 3.14x |
18+
| `autoread-dotenv` installed, `.env` loaded | 58.0 ms ± 2.5 ms | 4.71x |
19+
20+
Measured 2026-08-16, uv 0.12.3, hyperfine 1.20.0, Python 3.14.7, Linux x86_64. Each venv had
21+
only the packages named above installed (see "Methodology" section for why that matters).
22+
Re-run `just benchmark-startup` to reproduce/update these numbers - they are a point-in-time
23+
snapshot, not a guarantee.
24+
25+
**Reading these numbers:**
26+
27+
- The jump from "bare venv" to "`sitecustomize-entrypoints` only" (+24.2 ms) is the cost of
28+
the entry-point *discovery* mechanism itself - scanning every installed distribution's
29+
metadata for a registered `sitecustomize` entry point - and happens regardless of whether
30+
autoread-dotenv is one of the packages found. This is not something autoread-dotenv
31+
controls or can optimize away; it is the fixed cost of opting into
32+
`sitecustomize-entrypoints` at all.
33+
- **"No `.env` found" now costs essentially nothing beyond that floor** (38.7 ms vs. 36.5 ms
34+
for the bare hook mechanism - within noise). This is the result of the fix described
35+
below; it used to cost 60.4 ms, +~24 ms over the same floor. See "What was fixed".
36+
- "`.env` loaded" still costs +~21 ms over the floor (58.0 ms), because actually loading a
37+
`.env` requires importing `python-dotenv` - that part of the cost is real, unavoidable
38+
work, not overhead.
39+
- All of these numbers scale with **how many packages are installed in the venv**, because
40+
the entry-point discovery scan has to check all of them, not just autoread-dotenv's own
41+
dependencies. A real project's venv (with its own dependencies, dev tools, etc.) will see
42+
a larger absolute number than the minimal venvs used here. `just benchmark-importtime`
43+
(below) uses this project's own ~95-package dev venv for that reason, and shows
44+
proportionally larger absolute numbers for the same relative breakdown.
45+
46+
## What was fixed
47+
48+
Measuring this surfaced two things that were costing every process a lookup or an import it
49+
usually didn't need, regardless of whether there was a `.env` to load:
50+
51+
1. **`import dotenv` was eager.** [`src/autoread_dotenv/__init__.py`](../src/autoread_dotenv/__init__.py)
52+
used to `import dotenv` (python-dotenv) unconditionally at module level, before
53+
`entrypoint()` even checked whether a `.env` file exists. Fixed: `entrypoint()` now
54+
checks `get_dotenv_path()` *first*, and only imports `dotenv` once it knows there's an
55+
actual file to hand to `dotenv.load_dotenv()`. No `.env` found -> `dotenv` is never
56+
imported.
57+
58+
1. **`autoread_dotenv.about`'s metadata lookup was eager, and pulled in unconditionally.**
59+
`about.py` called `importlib.metadata.metadata()` at module level purely to populate
60+
`__version__`/`__author__`/`__license__` on the `autoread_dotenv` package - and
61+
`__init__.py` re-exported those three names, which forced `about.py` to be imported (and
62+
its metadata lookup to run) on every process start via the sitecustomize hook, whether or
63+
not anything ever read them. Fixed by removing that re-export: `__init__.py` no longer
64+
imports `about.py` at all, so its cost is paid only if something explicitly does
65+
`from autoread_dotenv.about import version` (as the test-suite does) - never as a side
66+
effect of the sitecustomize hook firing.
67+
68+
**This is a breaking change to the public API:** `autoread_dotenv.__version__`,
69+
`__author__`, and `__license__` no longer exist. Use
70+
`autoread_dotenv.about.version`/`.license_`/`.authors` directly if you need them (see
71+
`about.py` - unchanged, still eager, but now only runs when you actually import it).
72+
73+
Net effect: the "no `.env` found" scenario dropped from 60.4 ms to 38.7 ms - it now sits
74+
right at the `sitecustomize-entrypoints`-only floor, meaning autoread-dotenv itself adds
75+
essentially nothing in that case anymore. The "`.env` loaded" scenario only dropped slightly
76+
(59.6 ms -> 58.0 ms), because it still needs to import `dotenv` to actually do its job - see
77+
"Where the remaining cost goes" below for confirmation neither `dotenv` nor
78+
`autoread_dotenv.about` show up in the "no `.env`" import tree anymore.
79+
80+
## Where the remaining cost goes
81+
82+
`python -X importtime` breaks down import cost per module. Filtered to the relevant lines
83+
(via `just benchmark-importtime`, run against this project's own dev venv, which has a real
84+
`.env`):
85+
86+
```text
87+
import time: 2223 | 38934 | sitecustomize._vendor.importlib_metadata
88+
import time: 358 | 358 | autoread_dotenv.utils
89+
import time: 464 | 464 | autoread_dotenv.warnings
90+
import time: 1149 | 1149 | dotenv.parser
91+
import time: 585 | 585 | dotenv.variables
92+
import time: 1063 | 23467 | dotenv.main
93+
import time: 508 | 23975 | dotenv
94+
import time: 12760 | 81876 | sitecustomize
95+
import time: 2745 | 94071 | site
96+
```
97+
98+
(First column is self-time in µs, second is cumulative including sub-imports, both in
99+
`site`'s subtree.) Note `autoread_dotenv.about` no longer appears in this tree at all - only
100+
`utils` and `warnings`, both negligible. `dotenv` still does, because a `.env` was actually
101+
found and loaded here; re-running against a venv with no `.env` drops the `dotenv` lines too,
102+
leaving only `autoread_dotenv.utils`/`warnings` under `sitecustomize`.
103+
104+
What's left is `sitecustomize-entrypoints`'s own entry-point discovery scan (not
105+
autoread-dotenv's to optimize) and, when a `.env` is actually found, the real cost of
106+
importing `python-dotenv` to load it.
107+
108+
## Conclusions
109+
110+
1. **autoread-dotenv no longer meaningfully adds to the hook-mechanism tax when there's no
111+
`.env` to load.** Before the fix, autoread-dotenv doubled the cost of opting into
112+
`sitecustomize-entrypoints` at all (+~24 ms on top of the +~24 ms discovery-scan floor).
113+
After: the "no `.env`" case sits within noise of that floor.
114+
115+
1. **The remaining cost, when a `.env` *is* found, is real work - not overhead.** Loading a
116+
`.env` requires importing `python-dotenv` (~21 ms of the +~21 ms over the floor); that's
117+
the package doing its actual job, not something left on the table. **Caveat:** the `.env`
118+
fixture used here is a single line. This isolates the import-cost floor for that path, not
119+
necessarily the ceiling for a much larger, real-world `.env` (dozens of vars, `${VAR}`
120+
interpolation) - that's an open question this benchmark doesn't answer yet.
121+
122+
1. **The cost scales with venv size, so these numbers aren't fixed constants.** The discovery
123+
scan checks every installed distribution, not just autoread-dotenv's own dependencies - a
124+
lean production venv pays less than a kitchen-sink dev venv. This project's own
125+
~95-package dev venv (used for the `importtime` breakdown above) shows proportionally
126+
larger absolute numbers for the same relative shape.
127+
128+
1. **Practical takeaway:** the "no `.env`" case - which covers `pip`/`uv`/utility scripts run
129+
from outside a project root, or any venv where autoread-dotenv is installed but unused -
130+
now costs essentially nothing beyond opting into `sitecustomize-entrypoints` itself. The
131+
"`.env` found and loaded" case still carries `python-dotenv`'s own import cost, which
132+
compounds for anything that spawns Python repeatedly (CI matrices, per-test subprocess
133+
isolation, cold-start-sensitive environments) - but that's now the honest floor for doing
134+
the job, not accidental overhead.
135+
136+
## Methodology
137+
138+
Two complementary tools, both wrapped in `justfile` recipes so they're reproducible rather
139+
than one-off measurements:
140+
141+
- **[`hyperfine`](https://github.com/sharkdp/hyperfine)** (`just benchmark-startup`) for
142+
end-to-end wall-clock comparisons, with proper warmup runs and statistical variance - a
143+
single-shot `time python -c pass` is not trustworthy enough to publish. Builds throwaway
144+
venvs in a tempdir (no side-effects on this project's own `.venv`) for four scenarios:
145+
bare / `sitecustomize-entrypoints`-only / autoread-dotenv-without-`.env` /
146+
autoread-dotenv-with-`.env`.
147+
148+
> `hyperfine` is a Rust CLI binary, install via your system package manager (`apt install hyperfine`, `brew install hyperfine`, `cargo install hyperfine`, ...). **Not** `pip install hyperfine` / `uv add hyperfine` - that installs an unrelated PyPI package (a
149+
> scientific curve-fitting library depending on `numpy`, `jax`, `scipy`, `pandas`,
150+
> `iminuit`) that happens to share the name.
151+
152+
- **`python -X importtime`** (stdlib, `just benchmark-importtime`) for *attributing* cost to
153+
specific imports, to see where the wall-clock difference actually goes rather than just
154+
how big it is.
155+
156+
Tracking approach: doc-only, re-run manually (e.g. before releases, or when touching
157+
`entrypoint()`/`about.py`/dependencies). No CI job or third-party benchmarking service for
158+
now - revisit if regressions start slipping through unnoticed.

docs/readme.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,12 @@ entirely:
7171
> export AUTOREAD_DOTENV_PATH=/etc/myapp/production.env
7272
```
7373

74+
## Performance
75+
76+
`autoread-dotenv` runs on the start of every Python process in the venv, so its startup-time
77+
cost is measured and tracked explicitly - see [docs/performance.md](performance.md) for the
78+
current numbers and how to reproduce them.
79+
7480
## Compatibility
7581

7682
[![Python Version](https://img.shields.io/pypi/pyversions/autoread-dotenv?:alt:PyPI-PythonVersion)](https://pypi.org/project/autoread-dotenv/)

justfile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ set shell := ["bash", "-uc"]
3434
# - https://github.com/casey/just/issues/1885
3535
# - https://github.com/casey/just/pull/2376
3636
import? '.just/bandit.justfile'
37+
import? '.just/benchmark.justfile'
3738
import? '.just/dir-structure.justfile'
3839
import? '.just/dotenv.justfile'
3940
import? '.just/gh.justfile'

src/autoread_dotenv/__init__.py

Lines changed: 12 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -43,25 +43,10 @@
4343
if tp.TYPE_CHECKING: # pragma: no cover
4444
import pathlib as pl
4545

46-
try:
47-
import dotenv
48-
49-
DOTENV_INSTALLED = True
50-
except ImportError: # pragma: no cover
51-
DOTENV_INSTALLED = False
52-
53-
from autoread_dotenv.about import (
54-
authors as __author__,
55-
license_ as __license__,
56-
version as __version__,
57-
)
5846
from autoread_dotenv.utils import get_dotenv_path, get_expected_dotenv_path, str_to_bool
5947
from autoread_dotenv.warnings import simple_warning
6048

6149
__all__: list[str] = [
62-
"__author__",
63-
"__license__",
64-
"__version__",
6550
"entrypoint",
6651
"get_dotenv_path",
6752
"simple_warning",
@@ -72,19 +57,25 @@
7257
def entrypoint() -> None:
7358
"""Set environment-variable from the in-project .env-file."""
7459
dotenv_file: pl.Path | None = get_dotenv_path()
75-
enforce_dotenv: bool = str_to_bool(os.getenv("AUTOREAD_ENFORCE_DOTENV", "1"))
76-
77-
if not DOTENV_INSTALLED: # pragma: no cover
78-
with simple_warning():
79-
stdlib_warnings.warn("Module 'dotenv' not found. Please pip install 'python-dotenv'.", stacklevel=2)
80-
return
8160

8261
if not dotenv_file: # pragma: no cover
8362
with simple_warning():
8463
expected_path = get_expected_dotenv_path()
8564
stdlib_warnings.warn(f"{expected_path} does not exist, please create it.", stacklevel=2)
8665
return
8766

67+
try:
68+
# deferred on purpose: avoids paying python-dotenv's import cost on every Python
69+
# process start via the sitecustomize entrypoint when there's no .env to load
70+
# anyway. See docs/performance.md.
71+
import dotenv # noqa: PLC0415
72+
except ImportError: # pragma: no cover
73+
with simple_warning():
74+
stdlib_warnings.warn("Module 'dotenv' not found. Please pip install 'python-dotenv'.", stacklevel=2)
75+
return
76+
77+
enforce_dotenv: bool = str_to_bool(os.getenv("AUTOREAD_ENFORCE_DOTENV", "1"))
78+
8879
try:
8980
dotenv.load_dotenv(dotenv_file, override=enforce_dotenv, interpolate=True, verbose=True)
9081
except AttributeError: # pragma: no cover

tests/conftest.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,6 @@
5757

5858
# reload each module to fix coverage report
5959
importlib.reload(autoread_dotenv)
60-
importlib.reload(autoread_dotenv.about)
6160
importlib.reload(autoread_dotenv.utils)
6261
importlib.reload(autoread_dotenv.warnings)
6362

tests/test_init.py

Lines changed: 0 additions & 17 deletions
This file was deleted.

0 commit comments

Comments
 (0)