Skip to content

test hung after CockroachDB died during initialization #1224

Open
@davepacheco

Description

@davepacheco

In #1223, I found the test hung after CockroachDB died on SIGSEGV.

The context was I was running the try_repro.sh script to try to repro #1130. This runs (among other things) ./target/debug/deps/test_all-d586ea57740e3382 test_disk_create_disk_that_already_exists_fails in a loop. This was on commit 095dfb6, which is not from "main", but is pretty close to 236f2ac from "main".

When I found the problem, the test was hung, having emitted only this at the end:

ATTEMPT 246

running 1 test
test integration_tests::disks::test_disk_create_disk_that_already_exists_fails has been running for over 60 seconds

The punchline is that we were stuck during test setup waiting for Oximeter to register with Nexus. This was in an infinite backoff (that is, infinite tries, not an infinite delay) because these requests were failing because the database was down.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Test FlakeTests that work. Wait, no. Actually yes. Hang on. Something is broken.databaseRelated to database access

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions