perf(e2e): load the operator images in one kind invocation - #2300
Merged
Conversation
LoggingOperator looped over the six images calling LoadImages once each, so every cluster ran six kind load docker-image invocations: 102 across a full run where 17 would do. LoadImages was already variadic and LoadDockerImage already appends every image to one command, so only the caller had to change. docker save writes a shared layer once per invocation rather than once per image, and kube-logging#2287 measured the bottleneck as roughly 13 GB of tar-and-import per run with fluentd-full unpacked into every cluster. Batching cuts that volume without serialising anything, so it does not contradict the finding there that concurrent loads beat serial ones. Measured on three layer-sharing images locally: three separate tars came to 88,659,456 bytes against 59,109,376 for one batched tar, 33 percent less. The six e2e images are not built here, so their real overlap is unmeasured. TestInvocations now pins that several images produce a single command line. The caller side has no unit test: LoggingOperator needs a live cluster, so only a real e2e run exercises it. Signed-off-by: Vyncint Ng <115854244+vyncint@users.noreply.github.com>
csatib02
approved these changes
Jul 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LoggingOperatorlooped over the six images callingLoadImagesonce each, so every cluster ran sixkind load docker-imageinvocations — 102 across a full run where 17 would do.LoadImageswas already variadic andLoadDockerImagealready appends every image to one command, so only the caller changed.docker savewrites a shared layer once per invocation rather than once per image. Measured on three layer-sharing images locally: 88,659,456 bytes as three separate tars against 59,109,376 batched, 33% less. The six e2e images are not built locally, so their real overlap is unmeasured.This reduces I/O volume without serialising anything, so it does not contradict the measurement in #2287 that concurrent loads beat serial ones. It is one contributor to that flake, not the whole fix — the concurrency cap is still open there.
TestInvocationsnow pins that several images produce a single command line. The caller side has no unit test:LoggingOperatorneeds a live cluster, so only a real e2e run exercises it.