Skip to content

--zap-log-level flag accepted but ignored (Development: true hardcoded overrides level) #763

@giampy-am

Description

@giampy-am

Description

When running workload-variant-autoscaler v0.5.0 on Kubernetes, the --zap-log-level flag
is accepted by the binary (no startup error), but the configured log level is completely
ignored at runtime — INFO and DEBUG messages continue to be emitted regardless of the value passed.

Steps to reproduce

Patch the controller deployment to add --zap-log-level=error to the manager args:

kubectl patch deployment workload-variant-autoscaler-controller-manager -n <namespace> \
  --type='json' \
  -p='[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--zap-log-level=error"}]'

Observe that after rollout, pods start correctly (no crash), but logs still show INFO and DEBUG entries continuously:

2026-02-19T16:07:03Z  INFO  saturation/engine.go:190  Grouped VAs by model  ...
2026-02-19T16:07:03Z  DEBUG saturation/engine.go:357  BuildVariantStates map lookup  ...

Root cause (suspected)

Based on the source code, main.go initializes the zap options with Development: true hardcoded:

opts := ctrlzap.Options{
    Development: true,
}

In controller-runtime's zap, Development: true forces zapcore.DebugLevel as the base level.
The custom logging.InitLogging(&opts, &loggerVerbosity) function appears to not apply the
Level field from opts (set via --zap-log-level), resulting in the flag being parsed but never applied.

Additionally, note that --zap-log-level=warn causes CrashLoopBackOff because warn is not
a valid value for the flag (only debug, info, error, panic and integers are accepted) —
the error message should ideally be clearer.

Expected behavior

--zap-log-level=error should suppress INFO and DEBUG logs, emitting only ERROR-level entries.

Environment

- Version: v0.5.0 (ghcr.io/llm-d/workload-variant-autoscaler:v0.5.0)
- Kubernetes: AKS
- controller-runtime version: (as bundled in v0.5.0)

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-triageIndicates an issue or PR lacks a triage label and requires one.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions