CNDB-17110: Fix possible deadlock if flush fails

[pkolaczk]

JVMStabilityInspector.inspectThrowable(t) can throw an exception.
In that case the control flow never reaches
postFlush.latch.countDown() and threads waiting for flush to finish
never unlock and never get a chance to propagate the exception
up the call chain. This ends up in a bad lockup with no error logged
anywhere, system appearing to be performing a pending flush,
but no flush actually running. This scenario has been observed in tests
when the system hit the limit of open files.

This commit moves latch.countDown() to a finally block
so it can never be skipped.

[github-actions]

Checklist before you submit for review

• This PR adheres to the Definition of Done
• Make sure there is a PR in the CNDB project updating the Converged Cassandra version
• Use NoSpamLogger for log lines that may appear frequently in the logs
• Verify test results on Butler
• Test coverage for new/modified code is > 80%
• Proper code formatting
• Proper title for each commit staring with the project-issue number, like CNDB-1234
• Each commit has a meaningful description
• Each commit is not very long and contains related changes
• Renames, moves and reformatting are in distinct commits
• All new files should contain the DataStax copyright header instead of the Apache License one

[pkolaczk]

A short explanation why there are no unit tests for this:
This is hard to reproduce in an isolated way; but it was triggered by #2269 tests before they were fixed to cleanup tables properly after each test. Triggering the deadlock required bringing the system to an unstable state by running out of file descriptors. I confirmed that moving the latch countdown to a finally block resolved the deadlock and logged the flush failure reason properly (which was: "Too many open files"). Considering the triviality of the fix, trying to reproduce the same thing in a unit test is a bad investment of our time.

[pkolaczk]

The tests that failed in CI pass locally.
The failures in CI unrelated.

[jasonstack]

LGTM

[pkolaczk]

test this please

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[cassci-bot]

❌ Build ds-cassandra-pr-gate/PR-2275 rejected by Butler

---

6 regressions found
See build details here

---

Found 6 new test failures

Test	Explanation	Runs	Upstream
o.a.c.index.sai.cql.VectorCompaction100dTest.testOneToManyCompaction[version=ec enableNVQ=true]	NEW	🔴⚪	0 / 29
o.a.c.index.sai.cql.VectorCompaction2dTest.testZeroOrOneToManyCompaction[version=ca enableNVQ=false] (compression)	NEW	🔴⚪	0 / 29
o.a.c.index.sai.cql.VectorKeyRestrictedOnPartitionTest.partitionRestrictedWidePartitionBqCompressedTest[version=ca enableNVQ=false enableFused=false]	REGRESSION	🔴⚪	0 / 29
o.a.c.index.sai.cql.VectorSiftSmallTest.testSiftSmall[version=fa enableNVQ=false enableFused=false]	REGRESSION	🔴⚪	0 / 29
o.a.c.index.sai.cql.VectorTypeTest.testIndexingMultipleVectorColumns[version=ca enableNVQ=false enableFused=false]	REGRESSION	🔴⚪	0 / 29
o.a.c.index.sai.disk.v1.SegmentFlushTest.testFlushBetweenRowIds[aa]	REGRESSION	🔴⚪	2 / 29

---

Found 7 known test failures