pkg/ccl/kvccl/kvtenantccl/upgradeccl/upgradeccl_test: TestTenantUpgradeFailure failed

[cockroach-teamcity]

pkg/ccl/kvccl/kvtenantccl/upgradeccl/upgradeccl_test.TestTenantUpgradeFailure failed on release-25.2 @ 15a7aacba93b7dc379f1b0b52b3ed8368a9c587d:

F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	GOROOT/src/runtime/sema.go:587 +0x159
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !sync.(*Cond).Wait(0xc003dc6098)
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	GOROOT/src/sync/cond.go:71 +0x75
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !github.com/cockroachdb/pebble/record.(*flusherCond).Wait(...)
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	external/com_github_cockroachdb_pebble/record/log_writer.go:371
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !github.com/cockroachdb/pebble/record.(*LogWriter).flushLoop(0xc003dc6000, {0xc0070bc460?, 0xa91b200?})
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	external/com_github_cockroachdb_pebble/record/log_writer.go:671 +0xc07
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !runtime/pprof.Do({0xcb934f0, 0x12b350c0}, {{0xc001193c40?, 0x1f51d70?, 0xc00a30b188?}}, 0xc003b53fc0)
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	GOROOT/src/runtime/pprof/runtime.go:51 +0x118
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !github.com/cockroachdb/pebble/record.NewLogWriter.func2()
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	external/com_github_cockroachdb_pebble/record/log_writer.go:585 +0x92
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !created by github.com/cockroachdb/pebble/record.NewLogWriter in goroutine 503
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	external/com_github_cockroachdb_pebble/record/log_writer.go:584 +0xa6e
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !goroutine 20002 [select]:
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).startTicker.func1()
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	external/com_github_cockroachdb_pebble/vfs/disk_health.go:254 +0x192
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !created by github.com/cockroachdb/pebble/vfs.(*diskHealthCheckingFile).startTicker in goroutine 503
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	external/com_github_cockroachdb_pebble/vfs/disk_health.go:249 +0xad
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !goroutine 12875 [select]:
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !github.com/cockroachdb/cockroach/pkg/sql.(*runnerCoordinator).init.func1({0x0?, 0xcb934f0?})
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	pkg/sql/distsql_running.go:164 +0x126
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx.func2({0xb4c2a0b, 0xe})
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	pkg/util/stop/stopper.go:500 +0x343
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTaskEx in goroutine 12884
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !	pkg/util/stop/stopper.go:489 +0x730
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !For more context, check log files in: /var/lib/engflow/worker/work/3/exec/bazel-out/k8-fastbuild/testlogs/pkg/ccl/kvccl/kvtenantccl/upgradeccl/upgradeccl_test/shard_1_of_4_run_3_of_3/test.outputs/logTestTenantUpgradeFailure4058289970
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !****************************************************************************
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !This node experienced a fatal error (printed above), and as a result the
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !process is terminating.
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !Fatal errors can occur due to faulty hardware (disks, memory, clocks) or a
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !problem in CockroachDB. With your help, the support team at Cockroach Labs
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !will try to determine the root cause, recommend next steps, and we can
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !improve CockroachDB based on your report.
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !Please submit a crash report by following the instructions here:
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !    https://github.com/cockroachdb/cockroach/issues/new/choose
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !If you would rather not post publicly, please contact us directly at:
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !    support@cockroachlabs.com
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !
F250512 11:59:37.970258 24015 1@util/log/file.go:270  [-] 1 !The Cockroach Labs team appreciates your feedback.

Parameters:

• attempt=1
• race=true
• run=3
• shard=1

Help

See also: How To Investigate a Go Test Failure (internal)

/cc @cockroachdb/server

This test on roachdash | Improve this report!

Jira issue: CRDB-50581