Skip to content

feat: detect CUDA image bloat in dockerfile validator - #140

Merged
mitulgarg merged 1 commit into
mainfrom
feat/dockerfile-image-bloat-detection
Aug 29, 2026
Merged

feat: detect CUDA image bloat in dockerfile validator#140
mitulgarg merged 1 commit into
mainfrom
feat/dockerfile-image-bloat-detection

Conversation

@mitulgarg

@mitulgarg mitulgarg commented Aug 29, 2026

Copy link
Copy Markdown
Owner

Adds size-awareness to env-doctor dockerfile, which previously validated CUDA correctness but said nothing about image size.

Fixes a pre-existing false positive first. _validate_runtime_devel_mismatch scanned every line for compilation keywords while _validate_base_image only recorded the final stage's flavor, so a correct multi-stage build (compile in -devel, ship in -runtime) was reported as a hard ERROR. Lines are now partitioned by build stage and compilation detection is scoped to the final stage. Without this, the new multi-stage suggestion would have recommended a layout the tool then condemned.

New checks, all conservative:

  • devel image with no compilation detected -> suggest a slimmer variant. Inverting the runtime/devel heuristic is riskier than it reads: a missed keyword would tell the user to change base images and break their build, so this is INFO only and stays quiet when dependencies are opaque (requirements.txt, local project installs, git URLs).
  • single-stage devel that must compile -> suggest a multi-stage split
  • apt/conda CUDA installs on a CUDA base image -> flag as redundant. Folded into the existing _validate_cuda_toolkit_installation rather than added as a separate method, which would have double-reported on the same line.

Size data is keyed on variant, not tag. The nvidia/cuda tag space (CUDA patch x variant x cuDNN x distro) is combinatorial and grows monthly, so an exact-tag map would go stale silently. Recommendations quote the delta ("roughly 3.3 GB smaller") rather than absolute sizes, since the ratio between variants is stable across releases while the megabytes are not.

Also dedupes the compilation keyword list that was copy-pasted in two places.

No CLI changes: _print_validation_result renders issue text generically, and the MCP dockerfile_validate path picks the new findings up unchanged.

Tests: 39 pass in the dockerfile validator (24 existing + 15 new), including a regression test asserting the multi-stage pattern produces no error.

Adds size-awareness to `env-doctor dockerfile`, which previously validated
CUDA correctness but said nothing about image size.

Fixes a pre-existing false positive first. `_validate_runtime_devel_mismatch`
scanned every line for compilation keywords while `_validate_base_image` only
recorded the final stage's flavor, so a correct multi-stage build (compile in
-devel, ship in -runtime) was reported as a hard ERROR. Lines are now
partitioned by build stage and compilation detection is scoped to the final
stage. Without this, the new multi-stage suggestion would have recommended a
layout the tool then condemned.

New checks, all conservative:
- devel image with no compilation detected -> suggest a slimmer variant.
  Inverting the runtime/devel heuristic is riskier than it reads: a missed
  keyword would tell the user to change base images and break their build, so
  this is INFO only and stays quiet when dependencies are opaque
  (requirements.txt, local project installs, git URLs).
- single-stage devel that must compile -> suggest a multi-stage split
- apt/conda CUDA installs on a CUDA base image -> flag as redundant. Folded
  into the existing _validate_cuda_toolkit_installation rather than added as a
  separate method, which would have double-reported on the same line.

Size data is keyed on variant, not tag. The nvidia/cuda tag space (CUDA patch
x variant x cuDNN x distro) is combinatorial and grows monthly, so an
exact-tag map would go stale silently. Recommendations quote the delta
("roughly 3.3 GB smaller") rather than absolute sizes, since the ratio between
variants is stable across releases while the megabytes are not.

Also dedupes the compilation keyword list that was copy-pasted in two places.

No CLI changes: _print_validation_result renders issue text generically, and
the MCP dockerfile_validate path picks the new findings up unchanged.

Tests: 39 pass in the dockerfile validator (24 existing + 15 new), including a
regression test asserting the multi-stage pattern produces no error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UFJDqdhBShSdESfPBKpubz
@mitulgarg

Copy link
Copy Markdown
Owner Author

Summary

Adds image size and bloat detection to env-doctor dockerfile. The command validated CUDA correctness but said nothing about size — and CUDA images are notoriously bloated. A -devel variant runs ~4.5 GB against ~1.2 GB for -runtime, and users routinely ship the toolkit to production without needing nvcc at runtime.

Along the way this fixes a pre-existing false positive on main.

The bug this fixes first

_validate_runtime_devel_mismatch scanned every line for compilation keywords, but _validate_base_image only recorded the final stage's flavor. So a correct multi-stage build — compile in -devel, ship in -runtime — was reported as a hard ERROR:

$ env-doctor dockerfile      # builder: devel + flash-attn, final: runtime
[error] line 1: CUDA compilation required (detected compilation keywords at line 2)
                but base image is -runtime

That is exactly the layout this PR's new multi-stage suggestion recommends. Without the fix, the tool would emit a fix it then condemns. Lines are now partitioned by build stage, and compilation detection is scoped to the final stage.

Verified by round-tripping the tool's own suggested output back through the validator: 0 errors, 0 warnings.

New checks

Check Severity Fires when
Unnecessary -devel INFO devel base, no compilation detected anywhere
Multi-stage suggestion INFO compilation needed, single-stage devel build
Redundant CUDA install WARNING apt/conda CUDA install on a CUDA base image

Example output:

Line 2:
  Issue: Base image is -devel but no CUDA compilation was detected
  Fix:   No CUDA compilation was detected, so the full toolkit is not needed at
         runtime. A -runtime image provides the same CUDA runtime libraries and
         is roughly 3.3 GB smaller.

  Suggested fix:
    FROM nvidia/cuda:12.4.0-runtime-ubuntu22.04

Design decisions worth reviewing

Size data is keyed on variant, not tag. The nvidia/cuda tag space is combinatorial — CUDA patch × variant × optional cuDNN × distro — thousands of live tags growing monthly. An exact-tag map would go stale immediately and fail silently (unknown tag → no check → user assumes clean). cuda_image_sizes.json holds five variant buckets instead.

Recommendations quote the delta, never absolute sizes. "roughly 3.3 GB smaller" rather than "4500 MB → 1200 MB". The ratio between variants has been stable for years; the absolute megabytes drift every release and would make the tool look unmaintained.

The -devel downgrade check is deliberately timid. It inverts an existing heuristic, and the inversion is riskier than it reads: today a missed keyword costs a missed warning, but inverted, a missed keyword tells the user to change base images and breaks their build. So it is INFO only, and stays silent whenever dependencies are opaque — requirements.txt, local project installs, git URLs.

Redundancy detection was folded into the existing _validate_cuda_toolkit_installation, not added as a new method. That method already detected apt-get install cuda-toolkit and already said "adds 2-5GB"; a second method would have double-reported on the same line. What was genuinely missing was the base-image awareness, which is what this adds.

Dropped from the original proposal: generic Dockerfile lint (--no-cache-dir, --no-install-recommends, apt cleanup). hadolint already covers those as DL3009/DL3013/DL3015, and env-doctor's edge is the CUDA-specific reasoning.

Test plan

  • 39 tests pass in the dockerfile validator: 24 pre-existing + 15 new, including a regression test asserting the multi-stage pattern produces no error
  • 433 pass across tests/unit/ and tests/test_basic.py
  • Fixture sweep confirms Dockerfile.valid and Dockerfile.multistage gain no new findings, and Dockerfile.runtime_needs_compilation still errors correctly — the genuine single-stage case is not weakened
  • MCP dockerfile_validate surfaces the new findings with no changes to that path
  • No CLI changes needed: _print_validation_result renders issue text generically

Note on the full suite: 16 tests fail in test_model_cli.py, test_cli_json.py, and test_mcp_tools.py (async fixture config). These are pre-existing and unrelated — confirmed by stashing this branch's work and re-running on a clean tree, which produces the identical 16 failures.

Files

  • src/env_doctor/validators/dockerfile_validator.py — stage partitioning, three new checks, deduped the compilation keyword list that was copy-pasted in two places
  • src/env_doctor/data/cuda_image_sizes.json — new; pyproject.toml already globs data/*.json, so no packaging change
  • tests/unit/validators/test_dockerfile_validator.py — new TestDockerfileBloatDetection class
  • 5 new fixtures under tests/fixtures/

@mitulgarg
mitulgarg merged commit 822046c into main Aug 29, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant