Skip to content

Commit 54f8577

Browse files
authored
feat(config): validate jobs, not just the global section (#778)
Closes #773. Depends on nothing; complements #777, which fixed the daemon half. ## The validator never saw a job It walks structs and skipped maps — and every job lives in one. So an unparsable schedule passed `validate` with the strict flag on or off, and the only later sign was a warning while the daemon started and the job never fired. Two things were needed to reach a job's fields: - **Job sections are maps**, so the walk descends into them. - **Every job type embeds `core.<Kind>Job`, which embeds `BareJob`** — where `schedule` and `command` live. Squashed structs were skipped too, so even after traversing the maps the schedule stayed invisible. They are now descended into **only inside a job**, so the global section keeps the behaviour it had. ## Strict, but not the wrong kind of strict Applying the existing rule to jobs would have rejected configurations that work today. *"A field with no `default` tag is required"* is a heuristic tuned to the global section; on a job it demands nearly every key a job can carry. So that rule is suppressed inside jobs, and what a job needs is stated explicitly — taken from what the runtime already demands, not invented here: | section | required | source | |---|---|---| | `job-exec` | `container`, `command` | `RunExec(ctx, j.Container, …)` | | `job-run` | `image` **or** `container` | `core.RunJob.Validate` → `ErrImageOrContainer` | | `job-service-run` | `image` | `core.RunServiceJob.Validate` → `ErrImageRequired` | | `job-local` | `command` | `args.GetArgs(j.Command)` | | `job-compose` | `file`, `service` | the `docker compose -f … run` it builds | | all | `schedule` | nothing to register the job under otherwise | Errors name the section and the job the user wrote: ``` config validation error for field 'job-exec "foo": container': is required (the container to exec in) ``` ## Impact measured, not assumed Every config in the repo still validates clean: `example/ofelia.ini`, `test/test-config.ini`, `test/run-job/ofelia.ini`. There is an e2e test pinning the shipped example specifically, so adding job checks cannot start rejecting the file new users are handed. **One existing fixture had to change rather than the rule.** `TestValidateExecuteValidFile` declared a `job-exec` with no `container` and asserted the config was valid. It is not — verified against the running daemon, that job fails on *every tick*: ``` error: job run: exec run: run_exec container "": invalid container name or ID: value is empty ``` The test was pinning a config that cannot work. This is precisely the class of failure the change exists to catch, so the fixture gained a container. ## Note on the constants `command`, `image`, `container` and `schedule` are now constants. They were literals in the format switch, which carried a comment arguing that extracting them only relocates duplication — true while they appeared once. The requirements table makes them appear in two independent lists with nothing tying them together, which is what goconst was pointing at. ## Test plan - [x] `go test ./...` — green - [x] `go test -race -tags=e2e ./e2e/...` — green - [x] `golangci-lint run` incl. `--build-tags="e2e unix"` — 0 issues - [x] `lefthook run pre-push` — exit 0 - [x] All three repo configs validate clean, before and after - [x] Each requirement verified to fire, and each valid form verified to pass (including `job-run` with only a container)
2 parents c743097 + ff2ee7b commit 54f8577

6 files changed

Lines changed: 473 additions & 32 deletions

File tree

cli/validate_test.go

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,9 +20,14 @@ func TestValidateExecuteValidFile(t *testing.T) {
2020
// Not parallel: modifies global os.Stdout which races with other tests.
2121

2222
configFile := filepath.Join(t.TempDir(), "config.ini")
23+
// container is part of a runnable job-exec: without it every run fails
24+
// with `run_exec container "": invalid container name or ID`. The fixture
25+
// omitted it and this test asserted the config was valid, which is what
26+
// job validation now catches.
2327
content := `
2428
[job-exec "foo"]
2529
schedule = @every 10s
30+
container = some-container
2631
command = echo "foo"
2732
`
2833
err := os.WriteFile(configFile, []byte(content), 0o644)

config/validator.go

Lines changed: 217 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -208,6 +208,13 @@ func (cv *Validator2) Validate() error {
208208

209209
// validateStruct recursively validates struct fields based on tags
210210
func (cv *Validator2) validateStruct(v *Validator, obj any, path string) {
211+
cv.validateStructIn(v, obj, path, false)
212+
}
213+
214+
// validateStructIn is validateStruct with the job flag, which travels down
215+
// through nested and squashed structs so an embedded job field is still
216+
// recognized as being inside a job.
217+
func (cv *Validator2) validateStructIn(v *Validator, obj any, path string, inJob bool) {
211218
val, ok := derefToStruct(obj)
212219
if !ok {
213220
return
@@ -226,14 +233,35 @@ func (cv *Validator2) validateStruct(v *Validator, obj any, path string) {
226233
fieldPath := resolveFieldPath(path, fieldType.Name, fieldType.Tag.Get("gcfg"), mapstructureTag)
227234

228235
// Handle nested structs
229-
if field.Kind() == reflect.Struct && mapstructureTag != ",squash" {
230-
cv.validateStruct(v, field.Interface(), fieldPath)
236+
if field.Kind() == reflect.Struct {
237+
squashed := mapstructureTag == ",squash"
238+
if !squashed {
239+
cv.validateStructIn(v, field.Interface(), fieldPath, inJob)
240+
continue
241+
}
242+
// A squashed struct contributes its fields at the parent's level,
243+
// so it is walked with the parent's path. Only inside a job:
244+
// every job type embeds core.<Kind>Job and, through it, BareJob,
245+
// which is where schedule and command live — skipping squashed
246+
// structs is why a job's schedule was never checked. Outside a job
247+
// the old behavior stands, so the global section is untouched by
248+
// this change.
249+
if inJob {
250+
cv.validateStructIn(v, field.Interface(), path, inJob)
251+
}
231252
continue
232253
}
233254

234-
// Validate based on field type and value. The enclosing struct travels
235-
// with it so a field can be required conditionally on a sibling.
236-
cv.validateField(v, val, field, fieldPath, defaultTag)
255+
// Job sections are maps keyed by the name the user wrote; walk them so
256+
// a job's fields are validated at all. Skipping maps meant no job was
257+
// ever reachable by the validator.
258+
if field.Kind() == reflect.Map {
259+
cv.validateJobMap(v, field, fieldPath)
260+
continue
261+
}
262+
263+
// Validate based on field type and value
264+
cv.validateField(v, field, fieldCtx{parent: val, path: fieldPath, defaultTag: defaultTag, inJob: inJob})
237265
}
238266
}
239267

@@ -274,17 +302,29 @@ func resolveFieldPath(parentPath, fieldName, gcfgTag, mapstructureTag string) st
274302
return fieldPath
275303
}
276304

305+
// fieldCtx carries what a field needs beyond its own value: the struct it
306+
// belongs to (so a requirement can be conditional on a sibling), its config
307+
// key, its default tag, and whether it sits inside a job section.
308+
type fieldCtx struct {
309+
parent reflect.Value
310+
path string
311+
defaultTag string
312+
// inJob suppresses the "a field with no default is required" rule. That
313+
// rule is a heuristic tuned to the global section; applied to a job it
314+
// would demand nearly every key a job can carry. What a job genuinely
315+
// needs is stated in jobRequirements and checked separately.
316+
inJob bool
317+
}
318+
277319
// validateField validates individual fields based on their type and tags
278-
func (cv *Validator2) validateField(
279-
v *Validator, parent, field reflect.Value, path string, defaultTag string,
280-
) {
320+
func (cv *Validator2) validateField(v *Validator, field reflect.Value, ctx fieldCtx) {
281321
switch field.Kind() {
282322
case reflect.String:
283-
cv.validateStringField(v, parent, field, path, defaultTag)
323+
cv.validateStringField(v, field, ctx)
284324
case reflect.Int, reflect.Int64:
285-
cv.validateIntField(v, field, path)
325+
cv.validateIntField(v, field, ctx.path)
286326
case reflect.Slice:
287-
cv.validateSliceField(v, field, path)
327+
cv.validateSliceField(v, field, ctx.path)
288328
case reflect.Invalid, reflect.Bool, reflect.Int8, reflect.Int16, reflect.Int32,
289329
reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr,
290330
reflect.Float32, reflect.Float64, reflect.Complex64, reflect.Complex128,
@@ -299,50 +339,63 @@ func (cv *Validator2) validateField(
299339
}
300340

301341
// validateStringField validates string type fields
302-
func (cv *Validator2) validateStringField(
303-
v *Validator, parent, field reflect.Value, path string, defaultTag string,
304-
) {
342+
func (cv *Validator2) validateStringField(v *Validator, field reflect.Value, ctx fieldCtx) {
305343
str := field.String()
306344

307345
// Skip validation for fields with defaults when they're empty
308-
if defaultTag != "" && str == "" {
346+
if ctx.defaultTag != "" && str == "" {
309347
return
310348
}
311349

312350
// Check for required fields
313-
if defaultTag == "" && str == "" && !cv.isOptionalField(path) && cv.gateIsOpen(parent, path) {
314-
v.ValidateRequired(path, str)
351+
if ctx.defaultTag == "" && str == "" && !ctx.inJob &&
352+
!cv.isOptionalField(ctx.path) && cv.gateIsOpen(ctx.parent, ctx.path) {
353+
v.ValidateRequired(ctx.path, str)
315354
}
316355

317356
// Validate specific string fields
318357
if str != "" {
319-
cv.validateSpecificStringField(v, path, str)
358+
cv.validateSpecificStringField(v, ctx.path, str)
320359
}
321360
}
322361

362+
// fieldKey returns the config key a path ends in, dropping any section
363+
// qualifier the path carries for reporting.
364+
func fieldKey(path string) string {
365+
if i := strings.LastIndex(path, "."); i >= 0 {
366+
return strings.ToLower(path[i+1:])
367+
}
368+
return strings.ToLower(path)
369+
}
370+
323371
// validateSpecificStringField validates specific string field formats
324372
func (cv *Validator2) validateSpecificStringField(v *Validator, path string, str string) {
325373
// First perform general security validation
326374
if !cv.performSecurityValidation(v, path, str) {
327375
return // Stop validation if security check fails
328376
}
329377

330-
// Validate based on field type. The case strings are user-facing INI key
331-
// names; Go struct tags (gcfg:"…" / mapstructure:"…") on Config require
332-
// them as literals there too, so extracting constants only relocates the
333-
// duplication. Suppressing keeps the switch readable.
334-
switch path {
335-
case "schedule", "cron": //nolint:goconst // see comment above switch
378+
// Match on the last segment of the path. Inside a job the path is
379+
// qualified with its section ("job-local \"backup\".schedule") so the error
380+
// says which job is at fault, while the key that selects the check is the
381+
// bare field name. Without this the schedule of a job was never checked:
382+
// the qualified path matched no case and fell through silently.
383+
//
384+
// The case strings are user-facing INI key names; Go struct tags
385+
// (gcfg:"…" / mapstructure:"…") on Config require them as literals there
386+
// too, so extracting constants only relocates the duplication.
387+
switch fieldKey(path) {
388+
case keySchedule, "cron":
336389
cv.validateCronField(v, path, str)
337390
case "email-to", "email-from": //nolint:goconst // see comment above switch
338391
cv.validateEmailField(v, path, str)
339392
case "web-address", "pprof-address":
340393
cv.validateAddressField(v, path, str)
341394
case "log-level": //nolint:goconst // see comment above switch
342395
cv.validateLogLevelField(v, path, str)
343-
case "command", "cmd":
396+
case keyCommand, "cmd":
344397
cv.validateCommandField(v, path, str)
345-
case "image":
398+
case keyImage:
346399
cv.validateImageField(v, path, str)
347400
case "save-folder", "working_dir":
348401
cv.validatePathField(v, path, str)
@@ -572,3 +625,141 @@ func (cv *Validator2) isValidLogLevel(level string) bool {
572625
level = strings.ToLower(level)
573626
return slices.Contains(validLevels, level)
574627
}
628+
629+
// INI keys named in more than one place here: the switch that picks a format
630+
// check, and the table of what each job kind needs. They were literals in both
631+
// until the table arrived, at which point the same key existed twice with
632+
// nothing tying the two together.
633+
const (
634+
keySchedule = "schedule"
635+
keyCommand = "command"
636+
keyImage = "image"
637+
keyContainer = "container"
638+
)
639+
640+
// jobRequirement is one thing a job of a given kind must carry. Several field
641+
// names mean "at least one of these", which is how job-run accepts either an
642+
// image to start or an existing container to reuse.
643+
type jobRequirement struct {
644+
fields []string
645+
// why is appended to the error so the message says what the field is for
646+
// rather than only that it is missing.
647+
why string
648+
}
649+
650+
// jobRequirements states what each job section needs in order to run.
651+
//
652+
// The entries are taken from what the runtime already demands, not invented
653+
// here: core.RunJob.Validate returns ErrImageOrContainer, core.RunServiceJob
654+
// .Validate returns ErrImageRequired, ExecJob execs by container id and
655+
// command, LocalJob runs a command, and ComposeJob shells out with a file and
656+
// a service. Checking them here moves those failures from "the job silently
657+
// never runs" to "validate says so".
658+
var jobRequirements = map[string][]jobRequirement{
659+
"job-exec": {
660+
{fields: []string{keyContainer}, why: "the container to exec in"},
661+
{fields: []string{keyCommand}, why: "the command to run"},
662+
},
663+
"job-run": {
664+
{fields: []string{keyImage, keyContainer}, why: "an image to start or an existing container to reuse"},
665+
},
666+
"job-local": {
667+
{fields: []string{keyCommand}, why: "the command to run"},
668+
},
669+
"job-service-run": {
670+
{fields: []string{keyImage}, why: "the image to create the swarm service from"},
671+
},
672+
"job-compose": {
673+
{fields: []string{"file"}, why: "the compose file"},
674+
{fields: []string{"service"}, why: "the service to run"},
675+
},
676+
}
677+
678+
// scheduleRequirement applies to every job kind: without a schedule there is
679+
// nothing to register the job under.
680+
var scheduleRequirement = jobRequirement{fields: []string{keySchedule}, why: "when to run"}
681+
682+
// validateJobMap walks the jobs of one section. Each entry is validated like
683+
// any other struct — so formats are checked — and then against the
684+
// requirements for its kind.
685+
func (cv *Validator2) validateJobMap(v *Validator, m reflect.Value, section string) {
686+
if m.Kind() != reflect.Map || m.IsNil() {
687+
return
688+
}
689+
690+
reqs, known := jobRequirements[section]
691+
for _, key := range m.MapKeys() {
692+
entry := m.MapIndex(key)
693+
val, ok := derefToStruct(entry.Interface())
694+
if !ok {
695+
continue
696+
}
697+
698+
// Field-level checks (formats, ranges) for everything the job carries.
699+
// The path carries the section and the job name so an error names the
700+
// job the user wrote rather than a bare key.
701+
cv.validateStructIn(v, entry.Interface(), fmt.Sprintf("%s %q", section, key.String()), true)
702+
703+
if !known {
704+
continue
705+
}
706+
jobName := key.String()
707+
for _, req := range append([]jobRequirement{scheduleRequirement}, reqs...) {
708+
cv.checkJobRequirement(v, val, section, jobName, req)
709+
}
710+
}
711+
}
712+
713+
// checkJobRequirement reports a requirement that no field satisfies.
714+
func (cv *Validator2) checkJobRequirement(
715+
v *Validator, job reflect.Value, section, jobName string, req jobRequirement,
716+
) {
717+
for _, name := range req.fields {
718+
if strings.TrimSpace(fieldValueByKey(job, name)) != "" {
719+
return
720+
}
721+
}
722+
723+
v.AddError(
724+
fmt.Sprintf("%s %q: %s", section, jobName, strings.Join(req.fields, " or ")),
725+
"",
726+
fmt.Sprintf("is required (%s)", req.why),
727+
)
728+
}
729+
730+
// fieldValueByKey returns the string value of the field carrying the given
731+
// config key, searching embedded structs because a job's schedule and command
732+
// live on the BareJob it embeds rather than on the job type itself.
733+
func fieldValueByKey(val reflect.Value, key string) string {
734+
if val.Kind() != reflect.Struct {
735+
return ""
736+
}
737+
738+
typ := val.Type()
739+
for fieldType := range typ.Fields() {
740+
field := val.FieldByIndex(fieldType.Index)
741+
742+
// Embedded structs are descended into even when the embedded type is
743+
// unexported: the job types embed exported ones today, but the values
744+
// are only read here, never handed out, so there is no reason for the
745+
// lookup to depend on that staying true.
746+
if fieldType.Anonymous && field.Kind() == reflect.Struct {
747+
if found := fieldValueByKey(field, key); found != "" {
748+
return found
749+
}
750+
continue
751+
}
752+
753+
if !fieldType.IsExported() {
754+
continue
755+
}
756+
if resolveFieldPath("", fieldType.Name, fieldType.Tag.Get("gcfg"), fieldType.Tag.Get("mapstructure")) != key &&
757+
!strings.EqualFold(fieldType.Name, key) {
758+
continue
759+
}
760+
if field.Kind() == reflect.String {
761+
return field.String()
762+
}
763+
}
764+
return ""
765+
}

config/validator_boundary_test.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -432,7 +432,7 @@ func TestValidator2ValidateStringFieldDefaults(t *testing.T) {
432432
v := NewValidator()
433433

434434
field := reflect.ValueOf(tt.value)
435-
cv.validateStringField(v, reflect.Value{}, field, tt.path, tt.defaultTag)
435+
cv.validateStringField(v, field, fieldCtx{path: tt.path, defaultTag: tt.defaultTag})
436436

437437
if v.HasErrors() != tt.wantError {
438438
t.Errorf("validateStringField(%q, %q, %q) hasError = %v, want %v",

0 commit comments

Comments
 (0)