estuary-cdk: make Task.checkpoint async to improve capture concurrency

[Alex-Bair]

Description:

When multiple tasks run concurrently in a CDK-based capture, the synchronous Task.checkpoint() method blocks the entire asyncio event loop while writing buffered documents to stdout. This means when one task flushes a large buffer, all other tasks are blocked from fetching data, processing rows, or building up their own document buffers.

We can make Task.checkpoint async in order to improve concurrency. Blocking I/O when flushing a buffer is run with asyncio.to_thread, freeing the main event loop to run other coroutines while a buffer is being flushed. A module-level asyncio.Lock is used to serialize emissions to stdout. This prevents documents and checkpoints from being interleaved or corrupted between tasks trying to flush their buffers.

With this change, while one task holds the lock and emits its buffer, other tasks can continue fetching data from the API, process/parse responses, and capture documents to build up their own buffers. When the emitting task releases the lock, another waiting task can begin flushing its buffer.

This commit makes Task.checkpoint() async, which is a breaking change. All connectors calling Task.checkpoint must now await it. The connectors in the connectors repo affected by this are updated in this commit as well.

Workflow steps:

(How does one use this feature, and how has it changed)

Documentation links affected:

(list any documentation links that you created, or existing ones that you've identified as needing updates, along with a brief description)

Notes for reviewers:

Tested on a local stack. Confirmed Task.checkpoint no longer blocks the asyncio event loop.

Note: I also made BaseCaptureConnector.checkpoint and BaseCaptureConnector._emit async as well and made them use the same asyncio.Lock to ensure they don't emit to stdout at the same time as any Tasks. Right now, that won't ever happen because BaseCaptureConnector._emit is only called before Task creation, but it's a good future-proofing idea to ensure stdout is behind a lock no matter who's writing to it.

[JustinASmith]

This looks good to me and aligns with the discussions on Slack! 🚀