Skip to content

fix 10s logging data race#60

Merged
RaduBerinde merged 1 commit intomasterfrom
fix-10s-race
Oct 6, 2025
Merged

fix 10s logging data race#60
RaduBerinde merged 1 commit intomasterfrom
fix-10s-race

Conversation

@RaduBerinde
Copy link
Member

@RaduBerinde RaduBerinde commented Oct 6, 2025

The goroutine that logs every 10 seconds can race with the function exiting, which can lead to a data race or even a panic because of t.Log used after the test exits.

The fix is to block until the goroutine is exiting, by writing to the unbuffered channel instead of closing it.

I reproduced and verified the fix, but it required temporary code changes and the test is too slow to check in.


This change is Reviewable

The goroutine that logs every 10 seconds can race with the function
exiting, which can lead to a data race or even a panic because of
t.Log used after the test exits.

The fix is to block until the goroutine is exiting, by writing to the
unbuffered channel instead of closing it.
@RaduBerinde RaduBerinde requested a review from kev-cao October 6, 2025 15:49
Copy link

@kev-cao kev-cao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for getting the fix out

@RaduBerinde
Copy link
Member Author

TFTR!

@RaduBerinde RaduBerinde merged commit f84f9e5 into master Oct 6, 2025
12 checks passed
@RaduBerinde RaduBerinde deleted the fix-10s-race branch October 6, 2025 15:59
RaduBerinde added a commit to RaduBerinde/cockroach that referenced this pull request Oct 6, 2025
Bump datadriven to incorporate a fix
(cockroachdb/datadriven#60).

Epic: none
Release note: None
RaduBerinde added a commit to RaduBerinde/cockroach that referenced this pull request Oct 6, 2025
Bump datadriven to incorporate a fix
(cockroachdb/datadriven#60).

Epic: none
Release note: None
craig bot pushed a commit to cockroachdb/cockroach that referenced this pull request Oct 6, 2025
154541: unsafesql: avoid panicking during query formatting r=angles-n-daemons a=angles-n-daemons

Part of the effort to guard access to the crdb_internal and system namespaces includes auditing override access (and denied access) to these unsafe internals. Included in this audit is the offending query which attempted to pry into these namespaces.

In multiple locations however, this auditing caused the system to panic, for different reasons. In one case, an incorrect number of annotations on the query caused a panic. Another included a plan builder which had no associated statement.

We see the process of going from plan -> query as a difficult one, and thus guard this attempt to audit these accesses in a blanket panic catcher, as it's not common that this will happen, and when it does we don't want the system to wholesale fail the query.

Fixes: #153590 Epic: CRDB-24527

Release note: none

154740: go.mod: bump Pebble to 0ac45a74e10a r=RaduBerinde a=RaduBerinde

Changes:

 * [`0ac45a74`](cockroachdb/pebble@0ac45a74) metrics: fix TestMetrics flake
 * [`98c989b1`](cockroachdb/pebble@98c989b1) metamorphic: fix bug in suffix generation when prefix == startPrefix
 * [`d37d2f4b`](cockroachdb/pebble@d37d2f4b) pebble: materialize virtual tables only if backing contains >= 30% garbage

Release note: none.
Epic: none.

154867: go.mod: bump datadriven r=RaduBerinde a=RaduBerinde

Bump datadriven to incorporate a fix
(cockroachdb/datadriven#60).

Epic: none
Release note: None

Co-authored-by: Brian Dillmann <brian.dillmann@cockroachlabs.com>
Co-authored-by: Radu Berinde <radu@cockroachlabs.com>
craig bot pushed a commit to cockroachdb/cockroach that referenced this pull request Oct 6, 2025
154684: backup: split up the multiregion datadriven test r=jeffswenson a=jeffswenson

This splits up the multiregion datadriven test so that each test has at most 2 clusters in it. We've been seeing some stuck server shutdowns and this should make them easier to troubleshoot.

Release note: none
Informs: #145079

154687: backup: improve datadriven test cleanup r=jeffswenson a=jeffswenson

Previously, the datadriven test harness would tear down clusters in order. This makes it difficult to troubleshoot stuck tear downs because there are goroutines for a running server mixed in with goroutines for a server with a stuck shutdown.

Release note: none
Informs: #145079

154752: sql/schemachanger: fix incorrect filter for pk index swaps r=fqazi a=fqazi

Previously, we adjusted the schema changer rules to ensure that
old secondary indexes are only dropped when the new secondary index is
usable. Unfortunately, one of the rules had an incorrect filter. To
address this, this patch fixes the filter to expect the new secondary
index, which will have the new flag. Additionally, the index recreation
logic for ALTER PRIMARY KEY was delaying when the new secondary indexes
could be made public.

Fixes: #154751

Release note: None

154867: go.mod: bump datadriven r=RaduBerinde a=RaduBerinde

Bump datadriven to incorporate a fix
(cockroachdb/datadriven#60).

Epic: none
Release note: None

154870: changefeedccl: make bulk delivery of rangefeed events optional r=aerfrei a=asg0451

This is a temporary opt-out until we can properly test the performance
impact of bulk delivery.

Epic: none

Release note (general change): The changefeed bulk
delivery setting was made optional.


Co-authored-by: Jeff Swenson <jeffswenson@betterthannull.com>
Co-authored-by: Faizan Qazi <faizan@cockroachlabs.com>
Co-authored-by: Radu Berinde <radu@cockroachlabs.com>
Co-authored-by: Miles Frankel <miles.frankel@cockroachlabs.com>
RaduBerinde added a commit to RaduBerinde/cockroach that referenced this pull request Oct 7, 2025
Bump datadriven to incorporate a fix
(cockroachdb/datadriven#60).

Epic: none
Release note: None
craig bot pushed a commit to cockroachdb/cockroach that referenced this pull request Oct 7, 2025
154867: go.mod: bump datadriven r=RaduBerinde a=RaduBerinde

Bump datadriven to incorporate a fix
(cockroachdb/datadriven#60).

Epic: none
Release note: None

Co-authored-by: Radu Berinde <radu@cockroachlabs.com>
kev-cao pushed a commit to kev-cao/cockroach that referenced this pull request Oct 7, 2025
Bump datadriven to incorporate a fix
(cockroachdb/datadriven#60).

Epic: none
Release note: None
kev-cao pushed a commit to kev-cao/cockroach that referenced this pull request Oct 14, 2025
Bump datadriven to incorporate a fix
(cockroachdb/datadriven#60).

Epic: none
Release note: None
kev-cao pushed a commit to kev-cao/cockroach that referenced this pull request Oct 14, 2025
Bump datadriven to incorporate a fix
(cockroachdb/datadriven#60).

Epic: none
Release note: None
kev-cao pushed a commit to kev-cao/cockroach that referenced this pull request Oct 14, 2025
Bump datadriven to incorporate a fix
(cockroachdb/datadriven#60).

Epic: none
Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants