fix: eliminate data races in fake IAM and SSM API test doubles - #9428
Open
Archong-Liu wants to merge 2 commits into
Open
fix: eliminate data races in fake IAM and SSM API test doubles#9428Archong-Liu wants to merge 2 commits into
Archong-Liu wants to merge 2 commits into
Conversation
The fake IAMAPI and SSMAPI used by unit tests had three data races: - IAMAPI.Reset() reassigned the InstanceProfiles/Roles maps without holding the embedded mutex, racing with the API methods that access those maps under lock. - GetInstanceProfile and CreateInstanceProfile returned the pointer stored in the internal map, so callers could mutate fake-internal state after the lock was released (the instanceprofile provider's Create does exactly this). ListInstanceProfiles similarly aliased the stored Roles/Tags slices. - SSMAPI had no mutex at all; GetParameter performed an unsynchronized check-then-act on the shared defaultParameters map while Reset reassigned it, which can trigger a concurrent-map-read-and-write fatal runtime panic. Reset() now acquires the lock like the other methods; the IAM getters return independent copies (struct plus cloned Roles/Tags slices), which also mirrors how a real AWS API returns a fresh object per response; and SSMAPI gains a mutex with GetParameter switched to a pointer receiver so the lock actually protects the shared map. Adds -race regression tests in pkg/fake for all three.
Archong-Liu
force-pushed
the
fix/fake-client-data-races
branch
from
July 22, 2026 10:30
a2b8fcd to
36b3edb
Compare
Archong-Liu
marked this pull request as ready for review
July 22, 2026 14:10
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related to #7969
Description
While looking into #7969 (data races surfaced when running tests with the race detector), I found three concrete, reproducible data races in the fake AWS API test doubles under
pkg/fake. These are test-harness-only bugs (the production code paths deserialize a fresh AWS response per call and never alias state), but they can cause flaky failures and even hard panics when tests run concurrently under-race.1.
IAMAPI.Reset()doesn't hold the lock (pkg/fake/iamapi.go)IAMAPIembedssync.Mutex, and every other method (GetInstanceProfile,CreateInstanceProfile,DeleteInstanceProfile,TagInstanceProfile,AddRoleToInstanceProfile,RemoveRoleFromInstanceProfile,ListInstanceProfiles) takess.Lock()before touchings.InstanceProfiles/s.Roles.Reset()alone reassigned both maps with no lock, racing against any concurrent locked method. Fixed by acquiring the lock inReset()like every other method.2. Pointer escape from
IAMAPIgetters (pkg/fake/iamapi.go)GetInstanceProfileandCreateInstanceProfilereturned the same*iamtypes.InstanceProfilepointer stored in the internal map. The instanceprofile provider caches that pointer (instanceProfileCache.SetDefault(...)) and then mutates it directly (instanceProfile.Roles = []iamtypes.Role{...}) outside the fake's lock, so the mutex protected nothing once the pointer escaped. A real AWS API returns a freshly deserialized, independent object per response, so these methods now return a copy (shallow struct copy plus clonedRoles/Tagsslices).ListInstanceProfileshad the same slice-aliasing issue (it copied the struct by value but shared theRoles/Tagsbacking arrays) and now returns copies too.3.
SSMAPIhas no mutex (pkg/fake/ssmapi.go)SSMAPIhad no synchronization at all.GetParameterdid an unsynchronized check-then-act on the shareddefaultParametersmap (read, check, write) whileReset()reassigned it. Since Go maps are not safe for concurrent read/write, this can trigger afatal error: concurrent map read and map writepanic, not just wrong data. NoteGetParameterused a value receiver, so even adding a field mutex wouldn't have helped — a value receiver copies the mutex. Fixed by adding an embeddedsync.Mutex, switchingGetParameterto a pointer receiver, and locking in bothGetParameterandReset().I also checked the other fakes in
pkg/fake/*.gofor the same two anti-patterns. The rest are already safe (ec2api.gousessync.Map;eksapi.go/sqsapi.go/arczonalshiftapi.gouse the atomic helpers inatomic.go/types.go;pricingapi.gobuilds fresh maps per call), so this change is intentionally scoped to justiamapi.goandssmapi.go.Note this does not fully resolve #7969 — a separate integration/E2E-test race described in that issue is being investigated independently and is out of scope here. This PR only addresses the fake-client races above.
How was this change tested?
-raceregression tests inpkg/fake(iamapi_test.go,ssmapi_test.go): the pointer-escape tests fail on the unfixed code (mutation leaks into internal state), and the concurrency tests reportWARNING: DATA RACEon the unfixed code for bothIAMAPI.Reset()andSSMAPI.GetParameter()/Reset(). All pass after the fix.go test -race ./pkg/fake/...— pass (repeated).go test -race ./pkg/providers/instanceprofile/...and./pkg/providers/amifamily/...(heavy consumers of these fakes) — pass under-race.gofmt -l,go vet ./..., andgo build ./...— clean.Does this change impact docs?
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.