Skip to content

fix(gitops-engine): fix nil pointer dereference error in removeWebookMutation()#27749

Open
peikk0 wants to merge 1 commit intoargoproj:masterfrom
peikk0:pg/fix-remove-webhook-mutation-nil-pointer-deference
Open

fix(gitops-engine): fix nil pointer dereference error in removeWebookMutation()#27749
peikk0 wants to merge 1 commit intoargoproj:masterfrom
peikk0:pg/fix-remove-webhook-mutation-nil-pointer-deference

Conversation

@peikk0
Copy link
Copy Markdown

@peikk0 peikk0 commented May 8, 2026

If typedLive.Merge(typedPredictedLive) fails for any reason typedPredictedLive becomes nil, resulting in an invalid memory address or nil pointer dereference error on the next line because err was not checked first:

Recovered from panic: runtime error: invalid memory address or nil pointer dereference
goroutine 357 [running]:
runtime/debug.Stack()
	/usr/local/go/src/runtime/debug/stack.go:26 +0x5e
github.com/argoproj/argo-cd/v3/controller.(*ApplicationController).processAppRefreshQueueItem.func1()
	/go/src/github.com/argoproj/argo-cd/controller/appcontroller.go:1704 +0x94
panic({0x48ef0c0?, 0x98e27d0?})
	/usr/local/go/src/runtime/panic.go:860 +0x13a
github.com/argoproj/argo-cd/gitops-engine/pkg/diff.removeWebhookMutation(0x211d47b05130, 0x211d7ab72788, 0x211d1dabb900, {0x52b62ea, 0x11})
	/go/src/github.com/argoproj/argo-cd/gitops-engine/pkg/diff/diff.go:283 +0xd39
github.com/argoproj/argo-cd/gitops-engine/pkg/diff.serverSideDiff(0x211d7ab72778, 0x211d7ab72788, {0x211d21caabc8, 0x8, 0x8})
	/go/src/github.com/argoproj/argo-cd/gitops-engine/pkg/diff/diff.go:180 +0x2fd
github.com/argoproj/argo-cd/gitops-engine/pkg/diff.ServerSideDiff(0x211d7ab72788?, 0x211d13ba8480?, {0x211d36512bc8?, 0x10?, 0x63312a0?})
	/go/src/github.com/argoproj/argo-cd/gitops-engine/pkg/diff/diff.go:142 +0x37
github.com/argoproj/argo-cd/gitops-engine/pkg/diff.Diff(0x211d7ab726f8, 0x211d7ab72700, {0x211d36512bc8, 0x8, 0x8})
	/go/src/github.com/argoproj/argo-cd/gitops-engine/pkg/diff/diff.go:94 +0x3bf
github.com/argoproj/argo-cd/gitops-engine/pkg/diff.DiffArray({0x211d2e2c9650, 0x2, 0x211d1028be00?}, {0x211d2e2c9640, 0x2?, 0x58?}, {0x211d21caabc8, 0x8, 0x8})
	/go/src/github.com/argoproj/argo-cd/gitops-engine/pkg/diff/diff.go:856 +0x12c
github.com/argoproj/argo-cd/v3/util/argo/diff.StateDiffs({0x211d2e2c8620?, 0x211d31de2908?, 0x531149d?}, {0x211d2e2c8610?, 0x211d39fd1e30?, 0xc?}, {0x6359548, 0x211d0ce74480})
	/go/src/github.com/argoproj/argo-cd/util/argo/diff/diff.go:334 +0x7bf
github.com/argoproj/argo-cd/v3/controller.(*appStateManager).CompareAppState(0x211d0a438780, 0x211d0de88808, 0x211d388c5d48, {0x211d40052f00, 0x3, 0x4}, {0x211d1edd7800, 0x3, 0x4}, 0x0, ...)
	/go/src/github.com/argoproj/argo-cd/controller/state.go:864 +0x3b65
github.com/argoproj/argo-cd/v3/controller.(*ApplicationController).processAppRefreshQueueItem(0x211d0a242600)
	/go/src/github.com/argoproj/argo-cd/controller/appcontroller.go:1841 +0x1c25
github.com/argoproj/argo-cd/v3/controller.(*ApplicationController).Run.func3()
	/go/src/github.com/argoproj/argo-cd/controller/appcontroller.go:932 +0x25
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1({0x0?, 0x0?})
	/go/pkg/mod/k8s.io/apimachinery@v0.34.0/pkg/util/wait/backoff.go:233 +0x13
k8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext.func1({0x631ea30?, 0x211d09aafe30?}, 0x425554?)
	/go/pkg/mod/k8s.io/apimachinery@v0.34.0/pkg/util/wait/backoff.go:255 +0x51
k8s.io/apimachinery/pkg/util/wait.BackoffUntilWithContext({0x631ea30, 0x211d09aafe30}, 0x211d21cadf50, {0x62cc300, 0x211d12d4a180}, 0x1)
	/go/pkg/mod/k8s.io/apimachinery@v0.34.0/pkg/util/wait/backoff.go:256 +0xe5
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x0?, {0x62cc300?, 0x211d12d4a180?}, 0x0?, 0x0?)
	/go/pkg/mod/k8s.io/apimachinery@v0.34.0/pkg/util/wait/backoff.go:233 +0x46
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0x211d12999780, 0x3b9aca00, 0x0, 0x1, 0x211d09aafe30)
	/go/pkg/mod/k8s.io/apimachinery@v0.34.0/pkg/util/wait/backoff.go:210 +0x7f
k8s.io/apimachinery/pkg/util/wait.Until(...)
	/go/pkg/mod/k8s.io/apimachinery@v0.34.0/pkg/util/wait/backoff.go:163
created by github.com/argoproj/argo-cd/v3/controller.(*ApplicationController).Run in goroutine 30
	/go/src/github.com/argoproj/argo-cd/controller/appcontroller.go:931 +0x906

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • The title of the PR conforms to the Title of the PR
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).
  • My new feature complies with the feature status guidelines.
  • I have added a brief description of why this PR is necessary and/or what this PR solves.
  • Optional. My organization is added to USERS.md.
  • Optional. For bug fixes, I've indicated what older releases this fix should be cherry-picked into (this may or may not happen depending on risk/complexity).

…kMutation()

Signed-off-by: Pierre Guinoiseau <pierre@guinoiseau.nz>
@peikk0 peikk0 requested a review from a team as a code owner May 8, 2026 05:23
@bunnyshell
Copy link
Copy Markdown

bunnyshell Bot commented May 8, 2026

✅ Preview Environment deployed on Bunnyshell

Component Endpoints
argocd https://argocd-u26ntb.bunnyenv.com/
argocd-ttyd https://argocd-web-cli-u26ntb.bunnyenv.com/

See: Environment Details | Pipeline Logs

Available commands (reply to this comment):

  • 🔴 /bns:stop to stop the environment
  • 🚀 /bns:deploy to redeploy the environment
  • /bns:delete to remove the environment

@codecov
Copy link
Copy Markdown

codecov Bot commented May 8, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 63.88%. Comparing base (97e31db) to head (3fa49df).
⚠️ Report is 9 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #27749      +/-   ##
==========================================
+ Coverage   63.87%   63.88%   +0.01%     
==========================================
  Files         421      421              
  Lines       57657    57657              
==========================================
+ Hits        36828    36836       +8     
+ Misses      17370    17360      -10     
- Partials     3459     3461       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Copy Markdown
Contributor

@ppapapetrou76 ppapapetrou76 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching and fixing this
Do you think a test that fails before the change and passes now worth the effort?

@peikk0
Copy link
Copy Markdown
Author

peikk0 commented May 8, 2026

@ppapapetrou76 I'm not sure it's worth the effort for this specific fix which was just a misplaced error check, the real error is somewhere in typedLive.Merge(typedPredictedLive) but I don't know exactly what yet, some edge case with a webhook probably, so I don't know how to reproduce it in a test for this. We've simply temporarily disabled server-side diffs for now for the failing app when we saw this error, without looking further. But once this fix is released we will be able to try again and catch the real error, and will then submit another fix for it, which would be accompagnied with a test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants