Skip to content

perf(helm): reuse the v4 post-renderer plugin and helm version across a run - #10153

Draft
kvey wants to merge 2 commits into
GoogleContainerTools:mainfrom
kvey:upstream-helm-per-process-reuse
Draft

perf(helm): reuse the v4 post-renderer plugin and helm version across a run#10153
kvey wants to merge 2 commits into
GoogleContainerTools:mainfrom
kvey:upstream-helm-per-process-reuse

Conversation

@kvey

@kvey kvey commented Aug 7, 2026

Copy link
Copy Markdown

Draft — opening early for direction before I invest further. Happy to split,
reshape, or drop either commit.

Summary

Two pieces of per-call work in the Helm renderer/deployer that are invariant for
the lifetime of the process, but are paid once per release on the deploy path
and once per config on the render path.

Two independent commits, reviewable separately:

  1. perf(helm): install the v4 post-renderer plugin once per run, not per release

    Since 4.0.0-beta.1, PreparePostRenderer installs Skaffold's post-renderer
    as a Helm plugin, and does so on every call: create a temp dir, write a
    manifest, helm plugin install, then the caller's defer runs
    helm plugin uninstall.

    The manifest depends only on the Skaffold binary path. Everything that varies
    per release — labels, image replacements, debug settings, kubeconfig —
    travels in SKAFFOLD_CMDLINE at helm template / helm install invocation
    time, not in the manifest. So the plugin installed for release N is
    byte-identical to the one for release N+1 apart from its randomly generated
    name.

    Now installed once, keyed on the Skaffold binary, and removed at process
    shutdown via CleanupSharedPostRenderer. A different binary still forces a
    reinstall.

    Helm v3 is unaffected — that path passes --post-renderer <executable>
    and installs no plugin at all, which is why this is invisible to most users
    today.

  2. perf(helm): memoize helm version``

    BinVer shells out per deployer and per renderer. The helm binary on PATH
    cannot change mid-run.

Impact

Measured on a repository with 21 Helm releases across 21 configs (skaffold render, warm caches, alternating A/B against the same commit, 6 runs each):

before after
helm subprocesses 107 45
plugin install/uninstall 42 2
helm version 23 1
wall clock (mean) 33.63s 19.61s

The distributions do not overlap (before 32.01–35.21s, after 19.03–20.70s), and
rendered output is byte-identical before and after (179,484 bytes).

The saving scales with release count, so it is negligible for a handful of
releases and material for a large monorepo. It also compounds with
#8363 /
#5417 (serial
render/deploy of Helm releases) — this doesn't address that, it just stops each
serialized release from paying avoidable subprocess overhead.

Risks / things worth reviewer attention

  • Both caches are process-wide. That is the point, but it means tests that
    assert on the executed command sequence observe each other. I added
    ResetSharedPostRendererForTest and ResetBinVerCacheForTest and wired them
    into the affected tests. If you'd prefer a different mechanism (a TestMain
    reset, or threading state through a struct rather than package-level), say so —
    package-level state was chosen to keep the diff small, not because it's the
    nicest shape.
  • CleanupSharedPostRenderer is called from main(), after app.Run. That
    is a new shutdown responsibility; if there's a more idiomatic teardown hook I
    missed, point me at it.
  • It captures the Client used at install time to uninstall with, because
    generateHelmCommand dereferences the Client for GlobalFlags() regardless
    of subcommand, so passing nil there would panic.
  • Behaviour change on the error path: BinVer now memoizes failures too, so a
    broken helm reports the same error per call instead of re-executing. That
    seemed right, but it is a change.

Testing

  • New tests: plugin installed once across repeated calls, reinstall on a changed
    binary, Helm v3 installs no plugin, cleanup safe when nothing installed.
  • go test ./pkg/skaffold/helm/... ./pkg/skaffold/deploy/helm/... ./pkg/skaffold/render/renderer/helm/... passes.
  • go build ./... and go vet clean.

I have not run the integration suite — happy to if that's expected before this
leaves draft.

CLA

Not yet signed. I'll get that sorted before this is ready for review.

🤖 Generated with Claude Code

Colton Pierson added 2 commits August 6, 2026 20:09
… release

Since Helm 4.0.0-beta.1, `PreparePostRenderer` installs Skaffold's post-renderer
as a Helm plugin, and it does so on every call: once per release on the deploy
path, once per config on the render path. Each call creates a temp dir, writes a
manifest, shells out to `helm plugin install`, and the caller's defer shells out
again to `helm plugin uninstall`.

The manifest depends only on the Skaffold binary path. Everything that varies per
release — labels, image replacements, debug settings, kubeconfig — travels in
SKAFFOLD_CMDLINE at `helm template` / `helm install` invocation time, not in the
manifest. So the plugin installed for release N is byte-identical to the one for
release N+1 apart from its randomly generated name, and a single installed plugin
serves the whole run.

It is now installed once, keyed on the Skaffold binary, and removed at process
shutdown by CleanupSharedPostRenderer. A different binary still forces a
reinstall.

On a repository with 21 Helm releases across 21 configs, `skaffold render` went
from 42 plugin install/uninstall invocations to 2. Rendered output is unchanged
(byte-identical).

Helm v3 is unaffected: that path passes `--post-renderer <executable>` and
installs no plugin at all.

The cache is process-wide, so tests asserting on the executed command sequence
must clear it between cases; ResetSharedPostRendererForTest does that and is
wired into the affected tests.
BinVer shells out to `helm version` on every call, and it is called once per
deployer and once per renderer. On a repository with 21 configs that is 23
invocations of a command whose answer cannot change: the helm binary on PATH is
fixed for the lifetime of the process.

Memoized, including the error result so a genuinely broken helm still reports on
every call rather than being retried per config.

The cache is process-wide, so tests asserting on the executed command sequence
must clear it between cases; ResetBinVerCacheForTest does that and is wired into
the affected tests.
@google-cla

google-cla Bot commented Aug 7, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant