fix(overlay): keep folded block scalars stable across repeated applies

[AshGodfrey]

Why

gopkg.in/yaml.v3 emits an extra line break immediately before a more-indented line inside a folded (>-) scalar. Applying an overlay is a decode/mutate/encode round trip, so every apply grew such a scalar by one blank line. The break lands inside the scalar, so it becomes part of the decoded string rather than cosmetic whitespace.

A source configured with many overlays accumulated roughly one blank line per overlay in descriptions that no overlay targeted, and those inflated descriptions reached generated SDK and provider doc strings.

Minimal reproduction, no overlays needed:

cur := "d: >-\n  normal line\n   more-indented line\n  another normal\n"
for i := 0; i < 4; i++ {
    var n yaml.Node
    yaml.Unmarshal([]byte(cur), &n)
    out, _ := yaml.Marshal(&n)
    cur = string(out) // grows one blank line each pass
}

Folded scalars without a more-indented line, and literal (|-) blocks, are already stable.

What changed

stabilizeFoldedScalars re-styles folded scalars containing a more-indented line as literal blocks, called in apply() before RenderDocument. Literal blocks reproduce their value verbatim and round trip unchanged.

This removes the trigger rather than patching the emitter, so it needs no fork or replace directive. The decoded value is unchanged — the node already holds the folded value — so only the on-disk representation differs.

apply() is the shared path for speakeasy run, speakeasy overlay apply, CI document generation, and registry application, so all four are covered.

Testing

• go test ./pkg/overlay/... passes, including the pre-existing expected-output fixtures.
• End-to-end: a spec containing an affected description chained through 26 distinct overlays. Before this change the description grew by 26 characters, one injected newline per overlay. After, it is byte-identical to the input.
• TestFoldedScalarGrowsWithoutStabilizer asserts the unpatched emitter still misbehaves, so it fails if yaml.v3 is ever fixed or bumped and the workaround can be removed.

Note for reviewers

Affected descriptions change representation from >- to | on their first regeneration after this lands. That is a one-time diff; output is stable afterwards.

Two call sites still invoke ApplyTo directly and bypass this path: internal/suggest/suggest.go and pkg/transform/filterOperations.go. Neither is on the regeneration path, so they are left out of this change.

🤖 Generated with Claude Code

---

Summary by cubic

Stabilizes YAML folded (>-) scalars across repeated overlay applies. Previously, each apply added a blank line before any more‑indented line due to gopkg.in/yaml.v3; now we re-style those scalars to literal (|) before render so values round-trip unchanged. Decoded strings stay the same; only on-disk representation changes.

• Coverage: runs in apply() via stabilizeFoldedScalars, so speakeasy run, speakeasy overlay apply, CI doc generation, and registry application all benefit.
• One-time diff: affected scalars switch from >- to | on first regeneration; subsequent outputs are stable.
• No migration required: decoded text is identical; generated SDK/provider docs stop accumulating blank lines.
• Notes: two call sites (internal/suggest/suggest.go, pkg/transform/filterOperations.go) still call ApplyTo directly and are not on the regeneration path. Tests assert current yaml.v3 behavior and stability to flag removal if the emitter is fixed.

^{Written for commit 1a2b3b1. Summary will update on new commits.}

[cubic-dev-ai]

2 issues found across 3 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pkg/overlay/overlay.go">

<violation number="1" location="pkg/overlay/overlay.go:123">
P2: When a folded scalar starts with a blank line and also contains a more-indented line, this call changes its decoded value on the first apply. Preserve folded style for leading-newline values or update the stabilizer to emit a literal block that retains the initial blank line.</violation>
</file>

<file name="pkg/overlay/foldedscalar.go">

<violation number="1" location="pkg/overlay/foldedscalar.go:29">
P2: Explicitly tagged folded scalars bypass stabilization because `Style` is a bitmask and includes `TaggedStyle`. Test the folded bit and replace only that bit while preserving the tag, so tagged descriptions cannot keep gaining blank lines.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P2: When a folded scalar starts with a blank line and also contains a more-indented line, this call changes its decoded value on the first apply. Preserve folded style for leading-newline values or update the stabilizer to emit a literal block that retains the initial blank line.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At pkg/overlay/overlay.go, line 123:

<comment>When a folded scalar starts with a blank line and also contains a more-indented line, this call changes its decoded value on the first apply. Preserve folded style for leading-newline values or update the stabilizer to emit a literal block that retains the initial blank line.</comment>

<file context>
@@ -120,6 +120,8 @@ func apply(document *yaml.Node, o *overlay.Overlay, sourceLocation string, yamlI
 		}
 	}
 
+	stabilizeFoldedScalars(document)
+
 	bytes, err := schemas.RenderDocument(document, sourceLocation, yamlIn, yamlOut)
</file context>

[cubic-dev-ai]

P2: Explicitly tagged folded scalars bypass stabilization because Style is a bitmask and includes TaggedStyle. Test the folded bit and replace only that bit while preserving the tag, so tagged descriptions cannot keep gaining blank lines.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At pkg/overlay/foldedscalar.go, line 29:

<comment>Explicitly tagged folded scalars bypass stabilization because `Style` is a bitmask and includes `TaggedStyle`. Test the folded bit and replace only that bit while preserving the tag, so tagged descriptions cannot keep gaining blank lines.</comment>

<file context>
@@ -0,0 +1,36 @@
+		return
+	}
+
+	if node.Kind == yaml.ScalarNode && node.Style == yaml.FoldedStyle && hasMoreIndentedLine(node.Value) {
+		node.Style = yaml.LiteralStyle
+	}
</file context>

Suggested change

	if node.Kind == yaml.ScalarNode && node.Style == yaml.FoldedStyle && hasMoreIndentedLine(node.Value) {
	node.Style = yaml.LiteralStyle
	if node.Kind == yaml.ScalarNode && node.Style&yaml.FoldedStyle != 0 && hasMoreIndentedLine(node.Value) {
	node.Style = (node.Style &^ yaml.FoldedStyle) | yaml.LiteralStyle

[linear-code]

GEN-3127

[ThomasRooney]

There are a few places we use the overlay package directly. Could we apply this to that package instead? Feels like this would be better to upstream

[AshGodfrey]

Closing in favor of speakeasy-api/openapi#244, which applies the same fix in the overlay package itself. pkg/overlay here is a thin wrapper around that package, so fixing it upstream covers this repository once the dependency is bumped, and also covers direct consumers of the package.