[Bazel CI] Integration Failure: Multi-shard Regression in bazel_query_test on macOS (altered locale environment is causing problems on the macOS CI) #28726
Description
Platform: MacOs
Logs:
(10:52:08) FAIL: //src/test/shell/integration:bazel_query_test (shard 5 of 5) (Exit 1) (see /Users/buildkite/Library/Caches/bazel/_bazel_buildkite/11ba0ca4c3c5cc50ed930720f996a13b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/src/test/shell/integration/bazel_query_test/shard_5_of_5/test_attempts/attempt_1.log)
--
(10:52:08) [9,474 / 9,567] 140 / 182 tests; 93 actions, 4 running; last test: //src/test/shell/bazel:empty_package_test
Testing //src/test/shell/integration:bazel_query_test (shard 4 of 5); 430s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 5 of 5); 17s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:toolchain_test (shard 2 of 10); 8s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:build_event_stream_test (shard 1 of 10); 0s remote-cache, darwin-sandbox
[Sched] Testing //src/test/shell/bazel:bazel_test_test (shard 2 of 3); 626s
[Sched] Testing //src/test/shell/integration:configured_query_test (shard 2 of 5); 625s
[Sched] Testing //src/test/py/bazel:bazel_fetch_test (shard 4 of 5); 623s
(10:55:15) FAIL: //src/test/shell/integration:bazel_query_test (shard 1 of 5) (Exit 1) (see /Users/buildkite/Library/Caches/bazel/_bazel_buildkite/11ba0ca4c3c5cc50ed930720f996a13b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/src/test/shell/integration/bazel_query_test/shard_1_of_5/test_attempts/attempt_1.log)
(10:55:15) [9,490 / 9,567] 145 / 182 tests; 77 actions, 6 running; last test: //src/test/shell/integration:starlark_configurations_test
Testing //src/test/shell/integration:bazel_query_test (shard 4 of 5); 618s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 5 of 5); 205s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 1 of 5); 27s remote-cache, darwin-sandbox
Testing //src/test/py/bazel:bazel_fetch_test (shard 5 of 5); 8s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:build_event_stream_test (shard 9 of 10); 0s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:exec_group_test (shard 3 of 5); 0s remote-cache, darwin-sandbox
[Sched] Testing //src/test/shell/bazel:bazel_test_test (shard 1 of 3); 787s
(11:01:44) FAIL: //src/test/shell/integration:bazel_query_test (shard 4 of 5) (Exit 1) (see /Users/buildkite/Library/Caches/bazel/_bazel_buildkite/11ba0ca4c3c5cc50ed930720f996a13b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/src/test/shell/integration/bazel_query_test/shard_4_of_5/test_attempts/attempt_2.log)
(11:01:44) [9,534 / 9,567] 168 / 182 tests; 33 actions, 5 running; last test: .../devtools/build/lib/buildtool:CorruptedActionCacheTest
Testing //src/test/shell/integration:bazel_query_test (shard 4 of 5); 1006s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 5 of 5); 593s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 1 of 5); 416s remote-cache, darwin-sandbox
Testing //src/test/java/com/google/devtools/build/lib/rules/objc:ObjcRulesTests (shard 19 of 20); 3s remote-cache, darwin-sandbox
Testing //src/test/java/com/google/devtools/build/lib/blackbox/tests/workspace:GitRepositoryBlackBoxTest; 0s remote-cache, darwin-sandbox
[Sched] Testing //src/test/java/com/google/devtools/build/lib/rules/objc:ObjcRulesTests (shard 13 of 20); 937s
[Sched] Testing //src/test/java/com/google/devtools/build/lib/rules/objc:ObjcRulesTests (shard 17 of 20); 936s
(11:03:49) FAIL: //src/test/shell/integration:bazel_query_test (shard 5 of 5) (Exit 1) (see /Users/buildkite/Library/Caches/bazel/_bazel_buildkite/11ba0ca4c3c5cc50ed930720f996a13b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/src/test/shell/integration/bazel_query_test/shard_5_of_5/test_attempts/attempt_2.log)
(11:03:49) [9,564 / 9,567] 180 / 182 tests; 3 actions running; last test: .../devtools/build/lib/remote:BuildWithoutTheBytesIntegrationTest
Testing //src/test/shell/integration:bazel_query_test (shard 4 of 5); 1132s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 5 of 5); 719s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 1 of 5); 541s remote-cache, darwin-sandbox
buildkite-agent artifact upload --content-type text/plain;charset=utf-8 src/test/shell/integration/bazel_query_test/shard_5_of_5/test_attempts/attempt_2.log
2026-02-20 11:03:50 INFO Found 1 files that match "src/test/shell/integration/bazel_query_test/shard_5_of_5/test_attempts/attempt_2.log"
2026-02-20 11:03:50 INFO Uploading to Google Cloud Storage ("gs://bazel-untrusted-buildkite-artifacts/019c7a67-4e54-46ba-bc10-04b548bd8a51"), using your agent configuration
(11:04:04) FAIL: //src/test/shell/integration:bazel_query_test (shard 4 of 5) (Exit 1) (see /Users/buildkite/Library/Caches/bazel/_bazel_buildkite/11ba0ca4c3c5cc50ed930720f996a13b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/src/test/shell/integration/bazel_query_test/shard_4_of_5/test.log)
(11:04:04) [9,564 / 9,567] 180 / 182 tests; 3 actions running; last test: .../devtools/build/lib/remote:BuildWithoutTheBytesIntegrationTest
Testing //src/test/shell/integration:bazel_query_test (shard 4 of 5); 1147s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 5 of 5); 734s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 1 of 5); 557s remote-cache, darwin-sandbox
buildkite-agent artifact upload --content-type text/plain;charset=utf-8 src/test/shell/integration/bazel_query_test/shard_4_of_5/test.log
2026-02-20 11:04:05 INFO Found 1 files that match "src/test/shell/integration/bazel_query_test/shard_4_of_5/test.log"
2026-02-20 11:04:05 INFO Uploading to Google Cloud Storage ("gs://bazel-untrusted-buildkite-artifacts/019c7a67-4e54-46ba-bc10-04b548bd8a51"), using your agent(11:04:09) FAIL: //src/test/shell/integration:bazel_query_test (shard 1 of 5) (Exit 1) (see /Users/buildkite/Library/Caches/bazel/_bazel_buildkite/11ba0ca4c3c5cc50ed930720f996a13b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/src/test/shell/integration/bazel_query_test/shard_1_of_5/test_attempts/attempt_2.log)
(11:04:09) [9,565 / 9,567] 180 / 182 tests; 2 actions running; last test: .../devtools/build/lib/remote:BuildWithoutTheBytesIntegrationTest
Testing //src/test/shell/integration:bazel_query_test (shard 5 of 5); 738s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 1 of 5); 561s remote-cache, darwin-sandbox
buildkite-agent artifact upload --content-type text/plain;charset=utf-8 src/test/shell/integration/bazel_query_test/shard_1_of_5/test_attempts/attempt_2.log
2026-02-20 11:04:09 INFO Found 1 files that match "src/test/shell/integration/bazel_query_test/shard_1_of_5/test_attempts/attempt_2.log"
2026-02-20 11:04:09 INFO Uploading to Google Cloud Storage ("gs://bazel-untrusted-buildkite-artifacts/019c7a67-4e54-46ba-bc10-04b548bd8a51"), using your agent configuration
2026-02-20 11:04:09 INFO Creating (0-1)/1 artifacts
(11:04:21) FAIL: //src/test/shell/integration:bazel_query_test (shard 5 of 5) (Exit 1) (see /Users/buildkite/Library/Caches/bazel/_bazel_buildkite/11ba0ca4c3c5cc50ed930720f996a13b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/src/test/shell/integration/bazel_query_test/shard_5_of_5/test.log)
(11:04:21) [9,565 / 9,567] 180 / 182 tests; 2 actions running; last test: .../devtools/build/lib/remote:BuildWithoutTheBytesIntegrationTest
Testing //src/test/shell/integration:bazel_query_test (shard 5 of 5); 750s remote-cache, darwin-sandbox
Testing //src/test/shell/integration:bazel_query_test (shard 1 of 5); 573s remote-cache, darwin-sandbox
buildkite-agent artifact upload --content-type text/plain;charset=utf-8 src/test/shell/integration/bazel_query_test/shard_5_of_5/test.log
2026-02-20 11:04:21 INFO Found 1 files that match "src/test/shell/integration/bazel_query_test/shard_5_of_5/test.log"
2026-02-20 11:04:21 INFO Uploading to Google Cloud Storage ("gs://bazel-untrusted-buildkite-artifacts/019c7a67-4e54-46ba-bc10-04b548bd8a51"), using your agent configuration
(11:04:35) FAIL: //src/test/shell/integration:bazel_query_test (shard 1 of 5) (Exit 1) (see /Users/buildkite/Library/Caches/bazel/_bazel_buildkite/11ba0ca4c3c5cc50ed930720f996a13b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/src/test/shell/integration/bazel_query_test/shard_1_of_5/test.log)
FAILED: //src/test/shell/integration:bazel_query_test (Summary)
/Users/buildkite/Library/Caches/bazel/_bazel_buildkite/11ba0ca4c3c5cc50ed930720f996a13b/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/src/test/shell/integration/bazel_query_test/shard_4_of_5/test.log
Culprit: f613b87
Steps:
git clone https://github.com/bazelbuild/bazel.git
git reset d2a1bed --hard
export USE_BAZEL_VERSION=d2a1bedb3a87c5bdb5fa2664ec73dfa8462d0b63
bazel build --config=ci-macos --jobs=6 --remote_cache=https://storage.googleapis.com/bazel-untrusted-build-cache --google_default_credentials --lockfile_mode=off -- //src:bazel //src:bazel_jdk_minimal //src:test_repos //...
CC Greenteam @meteorcloudy