fix: stop duplicating block-scalar lines that look like comments - #6226
fix: stop duplicating block-scalar lines that look like comments#6226pujitha24 wants to merge 1 commit into
Conversation
Motivation: kustomize/commands/internal/kustfile/kustomizationfile.go preserves comments across `kustomize edit` calls with a hand-written, line-based scan (parseCommentedFields) rather than a YAML-aware parser. It treats any line that, after trimming leading spaces, starts with '#' as a comment to reattach to the next recognized top-level field. That heuristic doesn't know about YAML block scalars: when a field's value is a literal/folded block scalar (e.g. `transformers: |-`) and its content happens to contain lines starting with '#' at the scalar's indentation, those lines are literal string content, not comments. They already appear once in the correctly marshaled scalar value, but the parser also mis-captures them as a "comment" and reattaches that copy ahead of the next field, duplicating them. Because the duplicate copy is itself re-captured as a "comment" on the next parse, repeated `kustomize edit set image` invocations double the duplicated lines each time (1 -> 2 -> 4 -> 8 -> 16 ...), growing the file without bound. Approach: Track block-scalar state while scanning: when a line ends with a block scalar header (|, |-, |+, >, >-, >+, and their explicit-indentation variants like |2 or |-2), remember that line's indentation. While inside the block, any non-blank line indented more than the header line is treated as opaque scalar content and bypasses the comment/field-match logic entirely, so '#'-prefixed content lines are no longer mistaken for comments. The block ends at the first non-blank line whose indentation drops to the header's level or below, matching how YAML determines block scalar extent. Blank-line handling is otherwise unchanged. Known limitation, disclosed rather than fixed here: a block scalar containing a genuinely blank interior line can still cause the number of blank lines before the next field to grow by one on each repeated write. This is a distinct, pre-existing bug (confirmed present with this change reverted too, growing at the same linear rate) about blank-line accounting, not the exponential content-duplication bug reported in the issue below. Fixing it would require a larger rework of this file's ad hoc comment-preservation model, so it's left out of scope for this minimal fix. Validation: - go test ./commands/internal/kustfile/... -run TestPreserve -v (in the kustomize module): passes, including a new regression test, TestPreserveBlockScalarCommentLikeContent, which performs 3 simulated `kustomize edit set image` invocations (fresh NewKustomizationFile / Read / Write per iteration, matching RunSetImage's real usage) against a kustomization.yaml with a '#'-prefixed line inside a transformers block scalar, and asserts the line appears exactly once afterward. Confirmed this test fails (line duplicated) on the pre-fix code and passes after the fix. - go build ./... and go test ./... in the kustomize module: all pass. - Built the kustomize binary and reran the exact reproduction from the issue (transformers block scalar with two '#'-prefixed marker lines, 4 repeated `kustomize edit set image` calls): marker count stays at 1 throughout, versus exponential 1/2/4/8/16 before the fix. - gofmt -l and go vet ./commands/internal/kustfile/...: clean. - make lint (this module's golangci-lint target, matching this repo's CI Lint job for the kustomize module): exits 0 with no findings against the changed files. Report: kubernetes-sigs#6225 Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
|
Welcome @pujitha24! |
|
Hi @pujitha24. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Tip We noticed you've done this a few times! Consider joining the org to skip this step and gain Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
saitejabandaru-in
left a comment
There was a problem hiding this comment.
LGTM! This fixes a very annoying and hard-to-debug issue with comment duplication in kustomize edit set image. The approach of tracking when we are inside a literal block scalar and respecting the indentation level to determine scalar boundaries is extremely sound. Thanks for catching this!
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: pujitha24, saitejabandaru-in The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Motivation:
kustomize/commands/internal/kustfile/kustomizationfile.go preserves
comments across
kustomize editcalls with a hand-written, line-basedscan (parseCommentedFields) rather than a YAML-aware parser. It treats
any line that, after trimming leading spaces, starts with '#' as a
comment to reattach to the next recognized top-level field. That
heuristic doesn't know about YAML block scalars: when a field's value
is a literal/folded block scalar (e.g.
transformers: |-) and itscontent happens to contain lines starting with '#' at the scalar's
indentation, those lines are literal string content, not comments.
They already appear once in the correctly marshaled scalar value, but
the parser also mis-captures them as a "comment" and reattaches that
copy ahead of the next field, duplicating them. Because the duplicate
copy is itself re-captured as a "comment" on the next parse, repeated
kustomize edit set imageinvocations double the duplicated lineseach time (1 -> 2 -> 4 -> 8 -> 16 ...), growing the file without
bound.
Approach:
Track block-scalar state while scanning: when a line ends with a block
scalar header (|, |-, |+, >, >-, >+, and their explicit-indentation
variants like |2 or |-2), remember that line's indentation. While
inside the block, any non-blank line indented more than the header
line is treated as opaque scalar content and bypasses the
comment/field-match logic entirely, so '#'-prefixed content lines are
no longer mistaken for comments. The block ends at the first non-blank
line whose indentation drops to the header's level or below, matching
how YAML determines block scalar extent. Blank-line handling is
otherwise unchanged.
Known limitation, disclosed rather than fixed here: a block scalar
containing a genuinely blank interior line can still cause the number
of blank lines before the next field to grow by one on each repeated
write. This is a distinct, pre-existing bug (confirmed present with
this change reverted too, growing at the same linear rate) about
blank-line accounting, not the exponential content-duplication bug
reported in the issue below. Fixing it would require a larger rework
of this file's ad hoc comment-preservation model, so it's left out of
scope for this minimal fix.
Validation:
(in the kustomize module): passes, including a new regression test,
TestPreserveBlockScalarCommentLikeContent, which performs 3 simulated
kustomize edit set imageinvocations (fresh NewKustomizationFile /Read / Write per iteration, matching RunSetImage's real usage) against
a kustomization.yaml with a '#'-prefixed line inside a transformers
block scalar, and asserts the line appears exactly once afterward.
Confirmed this test fails (line duplicated) on the pre-fix code and
passes after the fix.
issue (transformers block scalar with two '#'-prefixed marker lines,
4 repeated
kustomize edit set imagecalls): marker count stays at 1throughout, versus exponential 1/2/4/8/16 before the fix.
CI Lint job for the kustomize module): exits 0 with no findings
against the changed files.
Report: #6225
Signed-off-by: Pujitha Paladugu 10557236+pujitha24@users.noreply.github.com
Fixes #6225