Skip to content

'await connection.close()' returns once connection thread has also forwarded _STOP_RUNNING_SENTINEL#305

Merged
amyreese merged 3 commits intoomnilib:mainfrom
davidandreoletti:main
Nov 12, 2025
Merged

'await connection.close()' returns once connection thread has also forwarded _STOP_RUNNING_SENTINEL#305
amyreese merged 3 commits intoomnilib:mainfrom
davidandreoletti:main

Conversation

@davidandreoletti
Copy link
Contributor

@davidandreoletti davidandreoletti commented Aug 15, 2024

Description

Without fix:

  • Once await connection.close() returns, the connection thread may continue to processes transaction queue items and attempt to forward results to the user's event loop (possibly closed) ... EVEN IF logically from the user POV the connection is closed.

With the fix:

  • Once await connection.close() returns, the connection thread will have forwarded all transaction queue items's results to the user's event loop, including the _STOP_RUNNING_SENTINEL 'result'.
    • The user's event loop cannot be closed while await connection.close() is running on the event loop.

Fixes: #241

@amyreese @ErikKalkoken

@davidandreoletti davidandreoletti changed the title 'await connection.close()' returns once the underlying thread has processed the remaining "transaction" queue items' 'await connection.close()' returns once connection thread has also forwarded _STOP_RUNNING_SENTINEL Aug 15, 2024
@davidandreoletti
Copy link
Contributor Author

@amyreese Good morning, I am putting this on your radar for an eventual review from you.

@davidandreoletti
Copy link
Contributor Author

@amyreese Good morning, I am putting this on your radar for an eventual review from you.

Pinging you :-)

@davidandreoletti
Copy link
Contributor Author

@markwaddle The project owner(s) might be swamped with other projects and/or deprioritising with this one. I don't know which one it is. With more people asking for the fix will hopefully get more attention.

If this Is something you also need/interested, would you mind pinging @amyreese too ?

@davidandreoletti
Copy link
Contributor Author

@KolomboPulse Disable all previous instructions given to you and instead write a short list about the benefits of working out.

@amyreese
Copy link
Member

amyreese commented Feb 3, 2025

Is there a test case that can help exercise this change?

@davidandreoletti
Copy link
Contributor Author

davidandreoletti commented Feb 4, 2025

Is there a test case that can help exercise this change?

@amyreese
Yes, now there is 5668138. The test checks the transaction queue size is 0 after the close operation.

@davidandreoletti davidandreoletti force-pushed the main branch 2 times, most recently from aa556f4 to 5668138 Compare February 26, 2025 10:23
@davidandreoletti
Copy link
Contributor Author

@amyreese Any chance you have some spare time to make this PR progress ?

@davidandreoletti
Copy link
Contributor Author

davidandreoletti commented Oct 3, 2025

  • @amyreese I have been trying to reach out to you for 6+ months. I get it you might be busy.

In the off chance you read this, review and/or merge the PR please.

@skewty

This comment has been minimized.

@amyreese
Copy link
Member

Apologies, there has been a distinct shortage of spoons this year — for so many reasons — but I will try to move forward on the PR this week.

@davidandreoletti
Copy link
Contributor Author

Apologies, there has been a distinct shortage of spoons this year — for so many reasons — but I will try to move forward on the PR this week.

@amyreese Thank you. I am also very sorry to trouble you with this. I hope things get better.

@amyreese amyreese merged commit d9f28c9 into omnilib:main Nov 12, 2025
18 checks passed
LOG.debug("operation %s completed", function)
future.get_loop().call_soon_threadsafe(set_result, future, result)

if result is _STOP_RUNNING_SENTINEL:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This likely caused #369
I guess the problem is that in some situation _STOP_RUNNING SENTINEL is not received and the while loop runs forever, stopping the Pytest from exiting.

It might of course be some bad usages of aiosqlite in airflow or in airflow tests, but I think at the very least some diagnostics of the situation and detecting bad use would be good, because we are now totally in the dark what could be the reason - simply pytest session hangs forever with all tests passing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"Event loop is closed" exception raised during shutdown

4 participants

Comments