Skip to content

Commit 4714bcf

Browse files
feat: add field-level custom_sync generator config (#732)
## Description Some fields are managed by an AWS API separate from the resource's `Update` operation — tags via `CreateOrUpdateTags`, a logging configuration via its own `Put` call. Controllers reconcile these today by hand-writing an `sdk_update_pre_build_request` hook that invokes a sync function and then short-circuits out of `sdkUpdate` when nothing else differs. Every controller doing this duplicates the same block. Worse, adding a second such field means widening the existing `DifferentExcept` call by hand, and forgetting to makes the resource silently stop applying legitimate updates, with no error surfaced anywhere. This PR adds `custom_sync` as a field-level config that generates the boilerplate. ```yaml resources: AutoScalingGroup: fields: Tags: custom_sync: {} ``` The sync function itself stays hand-written. It's a method on `resourceManager`, matching every other seam where generated code calls user-supplied code (`custom_implementation`, `set_output_custom_method_name`, and so on): ```go func (rm *resourceManager) syncTags( ctx context.Context, desired *resource, latest *resource, ) error ``` The method name is always `sync<FieldName>` and is not configurable, so it's identical across every controller and can be found from the field name alone. `CustomSyncConfig` is an empty struct rather than a bool, so options can be added later without breaking the `generator.yaml` files that adopt `custom_sync: {}` now — promoting a bool to a struct would break under the strict unmarshalling `config.New` performs. ## Generated code Into `sdkUpdate`: ```go updatedDesired := desired.DeepCopy() updatedDesired.SetStatus(latest) if delta.DifferentAt("Spec.Tags") { err = rm.syncTags(ctx, desired, latest) if err != nil { return nil, err } } if !delta.DifferentExcept("Spec.Tags") { return rm.concreteResource(updatedDesired), nil } ``` Into `sdkCreate`: ```go if ko.Spec.Tags != nil { msg := "Secondary sync required; resource will be requeued" ackcondition.SetSynced(&resource{ko}, corev1.ConditionFalse, &msg, nil) } ``` A `custom_sync` field is only ever applied in the update path, so immediately after create it exists in Spec but has not been pushed to AWS. Marking the resource unsynced makes the runtime requeue after `requeue.DefaultRequeueAfterDuration` rather than waiting for the full resync period, and the condition message tells the user a further sync is coming without their involvement. The message is a generic constant rather than a list of the pending field paths. The nil checks guarding it are evaluated at runtime, but a generated message can only be assembled from every configured field, so naming them would over-report whenever a user populates only a subset. With multiple `custom_sync` fields, the create-path guard collapses into one block, since `SetSynced` overwrites rather than accumulates: ```go if ko.Spec.LogDeliveryConfigurations != nil || ko.Spec.Tags != nil { msg := "Secondary sync required; resource will be requeued" ackcondition.SetSynced(&resource{ko}, corev1.ConditionFalse, &msg, nil) } ``` On the update side, all paths land in a single `DifferentExcept` call: ```go if !delta.DifferentExcept("Spec.LogDeliveryConfigurations", "Spec.Tags") { ``` That auto-widening is the main argument for generating this rather than documenting a snippet. ## Placement Position in `sdk_update.go.tpl` is load-bearing in both directions, and is recorded in a comment on `CustomSyncUpdate`: - **After** the `updateable.when` guard, so a resource in a state where mutations aren't allowed is requeued before any out-of-band API call is made, and so the short-circuit can't return success past the guard. - **Before** the Update operation's `custom_implementation`, which returns from `sdkUpdate` directly and would otherwise skip the sync entirely. ## Validation Rejected at generation time, since each otherwise produces a controller that compiles but misbehaves at runtime: | Config | Why it's rejected | | --- | --- | | `custom_sync` on a nested field | Emitted code builds a `Spec.<Field>` delta path and a nil check off `ko.Spec` | | `custom_sync` with `is_read_only` | Field lands in Status, where there's no desired value to sync toward | | `custom_sync` with `compare.is_ignored` | Field never enters the delta, so the sync never runs and the short-circuit fires every reconcile | | `custom_sync` with no Update operation | Generated code lives in `sdkUpdate`, so the sync would never be called | ## Testing `make test` passes — 15 packages, no failures. New unit tests cover the emitters (single field, two fields, and the no-config inert case) and the model accessors plus all validation errors. Fixtures are added under `pkg/testdata/models/apis/elasticache/0000-00-00/`. Verified end to end by regenerating `autoscaling-controller`, which has exactly this pattern hand-written today: - With no `custom_sync` configured, generated `sdk.go` is **byte-identical** to before, confirming the emitters are inert for resources that don't use the feature. - With `custom_sync` on `Tags` and both hand-written hook templates deleted, the generated code matches what those hooks produced. The controller builds and its tests pass. - Placement relative to `updateable.when` was confirmed by temporarily adding a guard to `AutoScalingGroup` and inspecting the generated output. ## Notes for reviewers - Ordering across multiple `custom_sync` fields is alphabetical, which keeps generated output stable. Config-driven ordering is deferred until a resource needs it; note that YAML document order isn't recoverable here, since `ResourceConfig.Fields` is a map and the loader routes through `sigs.k8s.io/yaml`. An explicit `order:` key would be the way to add it. - `custom_sync` is restricted to top-level Spec fields. Beyond the codegen reasons, `DifferentExcept` matches with `Path.Contains`, so allowing nested paths would let one diff match two except-paths and skew the count. - No test asserts template placement, since the emitters are unit-tested in isolation. Happy to add a rendered-output assertion if you'd like that ordering pinned. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
1 parent 65d45b2 commit 4714bcf

14 files changed

Lines changed: 731 additions & 0 deletions

pkg/config/field.go

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -275,6 +275,50 @@ type CompareFieldConfig struct {
275275
PreDeleteInclude bool `json:"pre_delete_include"`
276276
}
277277

278+
// CustomSyncConfig instructs the code generator that the field is not
279+
// reconciled by the resource's normal Update operation, but instead by a
280+
// hand-written sync function that the controller author implements.
281+
//
282+
// The field's value is typically managed by a separate AWS API (tags via
283+
// CreateTags/DeleteTags, a scaling configuration via its own Put* call, and so
284+
// on), so it cannot be set through the resource's Update input shape. Setting
285+
// this config makes the code generator emit the boilerplate that invokes the
286+
// sync function from sdkUpdate, and that marks the resource unsynced after
287+
// create so a follow-up reconcile applies the field.
288+
//
289+
// The code generator does NOT generate the sync function itself. The author
290+
// implements it as a method on the resource manager in the resource's hooks.go,
291+
// which is how every other hand-written seam in the generated code is shaped:
292+
//
293+
// func (rm *resourceManager) sync<Field>(
294+
// ctx context.Context,
295+
// desired *resource,
296+
// latest *resource,
297+
// ) error
298+
//
299+
// The receiver gives the implementation access to rm.sdkapi and rm.metrics, so
300+
// no additional plumbing is needed to make API calls or record them.
301+
//
302+
// Given a Tags field, the generator emits a call to `syncTags` and expects
303+
// `syncTags` to exist. For example:
304+
//
305+
// resources:
306+
//
307+
// AutoScalingGroup:
308+
// fields:
309+
// Tags:
310+
// custom_sync: {}
311+
//
312+
// The struct is intentionally empty. Presence of the `custom_sync` key is the
313+
// entire configuration, and the sync method name is always derived from the
314+
// field name so that it is identical across every controller.
315+
//
316+
// It is declared as a struct rather than a bool so that options can be added
317+
// later without breaking the generator.yaml files that adopt it now: promoting a
318+
// bool to a struct would be a breaking change under the strict unmarshalling
319+
// config.New performs.
320+
type CustomSyncConfig struct{}
321+
278322
// PrintFieldConfig instructs the code generator how to handle kubebuilder:printcolumn
279323
// comment marker generation. If this struct is not nil, the field will be added to the
280324
// columns of `kubectl get` response.
@@ -459,6 +503,11 @@ type FieldConfig struct {
459503
// References instructs the code generator how to refer this field from
460504
// other custom resource
461505
References *ReferencesConfig `json:"references,omitempty"`
506+
// CustomSync instructs the code generator that this field is reconciled by
507+
// a hand-written sync function rather than by the resource's Update
508+
// operation, and that the boilerplate invoking that function should be
509+
// generated into sdkUpdate.
510+
CustomSync *CustomSyncConfig `json:"custom_sync,omitempty"`
462511
// Type *overrides* the inferred Go type of the field. This is required for
463512
// custom fields that are not inferred either as a Create Input/Output
464513
// shape or via the SourceFieldConfig attribute.

pkg/generate/ack/controller.go

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,12 @@ var (
164164
"GoCodeResourceIsDeletable": func(r *ackmodel.CRD, resVarName string, indentLevel int) (string, error) {
165165
return code.ResourceIsDeletable(r.Config(), r, resVarName, indentLevel)
166166
},
167+
"GoCodeCustomSyncUpdate": func(r *ackmodel.CRD, desiredVarName string, latestVarName string, deltaVarName string, indentLevel int) string {
168+
return code.CustomSyncUpdate(r, desiredVarName, latestVarName, deltaVarName, indentLevel)
169+
},
170+
"GoCodeCustomSyncCreate": func(r *ackmodel.CRD, koVarName string, indentLevel int) string {
171+
return code.CustomSyncCreate(r, koVarName, indentLevel)
172+
},
167173
"GoCodeCompareStruct": func(r *ackmodel.CRD, shape *awssdkmodel.Shape, deltaVarName string, sourceVarName string, targetVarName string, fieldPath string, indentLevel int) (string, error) {
168174
return code.CompareStruct(r.Config(), r, nil, shape, deltaVarName, sourceVarName, targetVarName, fieldPath, indentLevel)
169175
},

pkg/generate/code/custom_sync.go

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License"). You may
4+
// not use this file except in compliance with the License. A copy of the
5+
// License is located at
6+
//
7+
// http://aws.amazon.com/apache2.0/
8+
//
9+
// or in the "license" file accompanying this file. This file is distributed
10+
// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
11+
// express or implied. See the License for the specific language governing
12+
// permissions and limitations under the License.
13+
14+
package code
15+
16+
import (
17+
"fmt"
18+
"strings"
19+
20+
"github.com/aws-controllers-k8s/code-generator/pkg/model"
21+
)
22+
23+
// customSyncPendingMessage is the Synced condition message set after create when
24+
// a resource has `custom_sync` fields still to be applied.
25+
//
26+
// It is deliberately generic rather than naming the pending fields. The nil
27+
// checks that guard it are evaluated at runtime, but a generated message can
28+
// only be built from every configured field, so naming them would over-report
29+
// whenever a user populates only some of them.
30+
const customSyncPendingMessage = "Secondary sync required; resource will be requeued"
31+
32+
// CustomSyncUpdate returns Go code that invokes the hand-written sync function
33+
// for each of the resource's `custom_sync` Spec fields, and then short-circuits
34+
// out of sdkUpdate when those fields are the only ones that differ.
35+
//
36+
// Fields configured with `custom_sync` are managed by a separate AWS API rather
37+
// than by the resource's Update operation, so when nothing else has changed
38+
// there is no Update call left to make. The generated code returns the desired
39+
// state with the observed status grafted on, which is what the reconciler
40+
// expects back from Update.
41+
//
42+
// All `custom_sync` fields on the resource are collected into a single
43+
// DifferentExcept call. That is the main reason this is generated rather than
44+
// hand-written: adding a second out-of-band field to a hand-written hook
45+
// requires widening the existing DifferentExcept, and forgetting to do so makes
46+
// the resource silently short-circuit out of legitimate updates.
47+
//
48+
// Placement in sdk_update.go.tpl is load-bearing. This block must come AFTER
49+
// the `updateable.when` guard, so that a resource in a state where mutations
50+
// are not allowed is requeued before any out-of-band API call is made, and so
51+
// that the short-circuit below cannot return success past the guard. It must
52+
// come BEFORE the Update operation's `custom_implementation`, which returns
53+
// from sdkUpdate directly and would otherwise skip the sync entirely.
54+
//
55+
// The empty string is returned when the resource has no `custom_sync` fields.
56+
//
57+
// Sample output:
58+
//
59+
// updatedDesired := desired.DeepCopy()
60+
// updatedDesired.SetStatus(latest)
61+
// if delta.DifferentAt("Spec.Tags") {
62+
// err = rm.syncTags(ctx, desired, latest)
63+
// if err != nil {
64+
// return nil, err
65+
// }
66+
// }
67+
// if !delta.DifferentExcept("Spec.Tags") {
68+
// return rm.concreteResource(updatedDesired), nil
69+
// }
70+
func CustomSyncUpdate(
71+
r *model.CRD,
72+
// desired resource variable name — "desired" for sdkUpdate
73+
desiredVarName string,
74+
// latest resource variable name — "latest" for sdkUpdate
75+
latestVarName string,
76+
// delta variable name — "delta" for sdkUpdate
77+
deltaVarName string,
78+
// Number of levels of indentation to use
79+
indentLevel int,
80+
) string {
81+
fields := r.CustomSyncFields()
82+
if len(fields) == 0 {
83+
return ""
84+
}
85+
indent := strings.Repeat("\t", indentLevel)
86+
87+
fieldPaths := customSyncFieldPaths(r, fields)
88+
89+
out := "\n"
90+
// The reconciler expects Update to hand back the desired state carrying the
91+
// observed status. Build it before syncing so the short-circuit below has
92+
// something to return.
93+
out += fmt.Sprintf(
94+
"%supdatedDesired := %s.DeepCopy()\n", indent, desiredVarName,
95+
)
96+
out += fmt.Sprintf(
97+
"%supdatedDesired.SetStatus(%s)\n", indent, latestVarName,
98+
)
99+
for i, f := range fields {
100+
out += fmt.Sprintf(
101+
"%sif %s.DifferentAt(%q) {\n", indent, deltaVarName, fieldPaths[i],
102+
)
103+
out += fmt.Sprintf(
104+
"%s\terr = rm.%s(ctx, %s, %s)\n",
105+
indent, f.CustomSyncMethodName(), desiredVarName, latestVarName,
106+
)
107+
out += fmt.Sprintf("%s\tif err != nil {\n", indent)
108+
out += fmt.Sprintf("%s\t\treturn nil, err\n", indent)
109+
out += fmt.Sprintf("%s\t}\n", indent)
110+
out += fmt.Sprintf("%s}\n", indent)
111+
}
112+
// Every custom_sync field has now been reconciled through its own API. If
113+
// nothing outside that set differs, there is no Update call to make.
114+
quoted := make([]string, 0, len(fieldPaths))
115+
for _, fp := range fieldPaths {
116+
quoted = append(quoted, fmt.Sprintf("%q", fp))
117+
}
118+
out += fmt.Sprintf(
119+
"%sif !%s.DifferentExcept(%s) {\n",
120+
indent, deltaVarName, strings.Join(quoted, ", "),
121+
)
122+
out += fmt.Sprintf(
123+
"%s\treturn rm.concreteResource(updatedDesired), nil\n", indent,
124+
)
125+
out += fmt.Sprintf("%s}\n", indent)
126+
return out
127+
}
128+
129+
// CustomSyncCreate returns Go code that marks the resource unsynced after a
130+
// successful create when any of its `custom_sync` fields is set.
131+
//
132+
// A `custom_sync` field is only ever applied in the update path, so immediately
133+
// after create the field is present in the resource's Spec but has not been
134+
// pushed to AWS. Setting the Synced condition to false makes the runtime requeue
135+
// after requeue.DefaultRequeueAfterDuration (30 seconds), which lands in
136+
// sdkUpdate and runs the sync. Without this, the resource would report itself
137+
// synced while the field was still unapplied, and the correction would wait for
138+
// the full resync period.
139+
//
140+
// The Synced condition carries a message so that a user running
141+
// `kubectl describe` sees why the resource is not synced yet and that the
142+
// controller intends to sync again on its own.
143+
//
144+
// The empty string is returned when the resource has no `custom_sync` fields.
145+
//
146+
// Sample output:
147+
//
148+
// if ko.Spec.Tags != nil {
149+
// msg := "Secondary sync required; resource will be requeued"
150+
// ackcondition.SetSynced(&resource{ko}, corev1.ConditionFalse, &msg, nil)
151+
// }
152+
func CustomSyncCreate(
153+
r *model.CRD,
154+
// the variable name of the resource's Kubernetes object — "ko" for sdkCreate
155+
koVarName string,
156+
// Number of levels of indentation to use
157+
indentLevel int,
158+
) string {
159+
fields := r.CustomSyncFields()
160+
if len(fields) == 0 {
161+
return ""
162+
}
163+
indent := strings.Repeat("\t", indentLevel)
164+
specPrefix := customSyncSpecPrefix(r)
165+
166+
// Only mark unsynced when there is actually something to sync. A resource
167+
// created without any of these fields set has nothing for the follow-up
168+
// reconcile to do.
169+
conditions := make([]string, 0, len(fields))
170+
for _, f := range fields {
171+
conditions = append(conditions, fmt.Sprintf(
172+
"%s.%s.%s != nil", koVarName, specPrefix, f.Names.Camel,
173+
))
174+
}
175+
176+
out := "\n"
177+
out += fmt.Sprintf(
178+
"%sif %s {\n", indent, strings.Join(conditions, " || "),
179+
)
180+
out += fmt.Sprintf("%s\tmsg := %q\n", indent, customSyncPendingMessage)
181+
out += fmt.Sprintf(
182+
"%s\tackcondition.SetSynced(&resource{%s}, corev1.ConditionFalse, &msg, nil)\n",
183+
indent, koVarName,
184+
)
185+
out += fmt.Sprintf("%s}\n", indent)
186+
return out
187+
}
188+
189+
// customSyncFieldPaths returns the delta field path for each supplied field,
190+
// e.g. "Spec.Tags". These are the paths that the generated delta.go passes to
191+
// delta.Add, so they must be constructed the same way — see the fieldPath
192+
// calculation in CompareResource.
193+
func customSyncFieldPaths(r *model.CRD, fields []*model.Field) []string {
194+
specPrefix := customSyncSpecPrefix(r)
195+
paths := make([]string, 0, len(fields))
196+
for _, f := range fields {
197+
paths = append(paths, fmt.Sprintf("%s.%s", specPrefix, f.Names.Camel))
198+
}
199+
return paths
200+
}
201+
202+
// customSyncSpecPrefix returns the Spec prefix without its leading dot, i.e.
203+
// "Spec" for the default prefix config.
204+
func customSyncSpecPrefix(r *model.CRD) string {
205+
return strings.TrimPrefix(r.Config().PrefixConfig.SpecField, ".")
206+
}
Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
// Copyright Amazon.com Inc. or its affiliates. All Rights Reserved.
2+
//
3+
// Licensed under the Apache License, Version 2.0 (the "License"). You may
4+
// not use this file except in compliance with the License. A copy of the
5+
// License is located at
6+
//
7+
// http://aws.amazon.com/apache2.0/
8+
//
9+
// or in the "license" file accompanying this file. This file is distributed
10+
// on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
11+
// express or implied. See the License for the specific language governing
12+
// permissions and limitations under the License.
13+
14+
package code_test
15+
16+
import (
17+
"strings"
18+
"testing"
19+
20+
"github.com/stretchr/testify/assert"
21+
"github.com/stretchr/testify/require"
22+
23+
"github.com/aws-controllers-k8s/code-generator/pkg/generate/code"
24+
"github.com/aws-controllers-k8s/code-generator/pkg/testutil"
25+
)
26+
27+
// TestCustomSyncUpdate verifies the sdkUpdate boilerplate for a resource with
28+
// two custom_sync fields. Both are collected into a single DifferentExcept
29+
// call, which is the property that makes generating this worthwhile: a
30+
// hand-written hook has to be widened by hand every time a field is added, and
31+
// forgetting to do so silently short-circuits legitimate updates.
32+
//
33+
// Each method name is derived from its field name, so the calls are rm.syncTags
34+
// and rm.syncLogDeliveryConfigurations.
35+
func TestCustomSyncUpdate(t *testing.T) {
36+
assert := assert.New(t)
37+
require := require.New(t)
38+
39+
g := testutil.NewModelForServiceWithOptions(t, "elasticache",
40+
&testutil.TestingModelOptions{
41+
GeneratorConfigFile: "generator-with-custom-sync.yaml",
42+
})
43+
44+
crd := testutil.GetCRDByName(t, g, "ReplicationGroup")
45+
require.NotNil(crd)
46+
47+
// Fields are emitted in sorted order, so LogDeliveryConfigurations precedes
48+
// Tags regardless of the order they appear in generator.yaml.
49+
expected := `
50+
updatedDesired := desired.DeepCopy()
51+
updatedDesired.SetStatus(latest)
52+
if delta.DifferentAt("Spec.LogDeliveryConfigurations") {
53+
err = rm.syncLogDeliveryConfigurations(ctx, desired, latest)
54+
if err != nil {
55+
return nil, err
56+
}
57+
}
58+
if delta.DifferentAt("Spec.Tags") {
59+
err = rm.syncTags(ctx, desired, latest)
60+
if err != nil {
61+
return nil, err
62+
}
63+
}
64+
if !delta.DifferentExcept("Spec.LogDeliveryConfigurations", "Spec.Tags") {
65+
return rm.concreteResource(updatedDesired), nil
66+
}
67+
`
68+
assert.Equal(
69+
strings.TrimSpace(expected),
70+
strings.TrimSpace(code.CustomSyncUpdate(crd, "desired", "latest", "delta", 1)),
71+
)
72+
}
73+
74+
// TestCustomSyncCreate verifies the post-create marker. The resource is marked
75+
// unsynced only when at least one custom_sync field is actually set, since a
76+
// resource created without any of them has nothing for the follow-up reconcile
77+
// to do. The condition message is generic, so it stays accurate no matter which
78+
// subset of the fields the user populated.
79+
func TestCustomSyncCreate(t *testing.T) {
80+
assert := assert.New(t)
81+
require := require.New(t)
82+
83+
g := testutil.NewModelForServiceWithOptions(t, "elasticache",
84+
&testutil.TestingModelOptions{
85+
GeneratorConfigFile: "generator-with-custom-sync.yaml",
86+
})
87+
88+
crd := testutil.GetCRDByName(t, g, "ReplicationGroup")
89+
require.NotNil(crd)
90+
91+
expected := `
92+
if ko.Spec.LogDeliveryConfigurations != nil || ko.Spec.Tags != nil {
93+
msg := "Secondary sync required; resource will be requeued"
94+
ackcondition.SetSynced(&resource{ko}, corev1.ConditionFalse, &msg, nil)
95+
}
96+
`
97+
assert.Equal(
98+
strings.TrimSpace(expected),
99+
strings.TrimSpace(code.CustomSyncCreate(crd, "ko", 1)),
100+
)
101+
}
102+
103+
// TestCustomSyncNoFields confirms both emitters are inert for the overwhelming
104+
// majority of resources, which have no custom_sync fields at all.
105+
func TestCustomSyncNoFields(t *testing.T) {
106+
assert := assert.New(t)
107+
require := require.New(t)
108+
109+
g := testutil.NewModelForService(t, "elasticache")
110+
111+
crd := testutil.GetCRDByName(t, g, "ReplicationGroup")
112+
require.NotNil(crd)
113+
114+
assert.Equal("", code.CustomSyncUpdate(crd, "desired", "latest", "delta", 1))
115+
assert.Equal("", code.CustomSyncCreate(crd, "ko", 1))
116+
}

0 commit comments

Comments
 (0)