bleikcp: Define graceful shutdown and final ACK semantics

[idy]

Background

This issue supersedes h2vivi/firmwares#555 after the public libs/bleikcp implementation and its source-of-truth guide moved to GizOS.

The current worker exits its main loop as soon as closing becomes true. Client disconnect, explicit close, server handler return and TX-subscription loss can therefore race with KCP datagrams already admitted to the input queue, final RX publication, waitsnd snapshot and the ACK that would complete the peer's final send. Existing tests cover ordinary disconnect, flush-result precedence and retryable cleanup, but do not define the terminal worker-publication boundary or final-ACK ordering.

Graceful local shutdown and physical output loss are different states: a local close may still use the live BLE connection and TX subscription to finish admitted work, while a disconnect or unsubscribe must suppress every later BLE output attempt.

Goal

Define and implement deterministic graceful shutdown for GizOS libs/bleikcp:

• reject new session input and writes after closing begins while retaining ownership of frames already admitted to the worker queue;
• drain admitted input, publish complete received bytes and snapshot terminal KCP state before terminal I/O results are exposed;
• emit the required final ACK only while the BLE connection and TX subscription remain usable;
• preserve already-published RX bytes for read() after close;
• make flush(), final output failure, disconnect events, client cleanup and server handler-return cleanup deterministic;
• keep each reconnect on a new KCP stream and state generation.

Non-goals

• Do not change public BLE iKCP headers, GATT UUIDs, ATT MTU policy, KCP window/congestion configuration or retry tuning.
• Do not add automatic reconnect, replay interrupted application commands or reuse KCP state across BLE connections.
• Do not change BLE PAL, H2Loader command/package behavior, board policy or controller-specific link tuning.
• Do not combine shutdown correctness with throughput optimization, source migration or unrelated cleanup.
• Do not require real-device acceptance unless host tests expose behavior that cannot be represented by the fake BLE/PAL contract.

Code Changes Tree

libs/
└── bleikcp/
    ├── src/
    │   ├── h2_bleikcp.c                            # add terminal worker publication, admitted-input draining, final ACK and deterministic read/flush/close ordering
    │   ├── h2_bleikcp_internal.h                   # define private output-availability and terminal-publication state owned across worker and BLE event paths
    │   ├── h2_bleikcp_client.c                     # keep client event observation and TX subscription alive through graceful finalization while suppressing output after disconnect
    │   └── h2_bleikcp_server.c                     # order handler return, stream finalization, disconnect cleanup and return-to-idle without reusing connection state
    └── tests/
        └── test_bleikcp.c                          # cover final ACK, admitted RX, output-loss races, flush precedence, cleanup retry and independent reconnect state
guides/
└── zh/
    └── developing/
        └── bleikcp.md                              # document the final graceful-close, disconnect, read, flush, output-availability and reconnect contract

The existing public headers under libs/bleikcp/include/ already express read, write, flush, close and event behavior; this Issue does not change that API surface. If implementation proves a public-contract change unavoidable, stop and require a separate design decision before modifying a header.

Design

Terminal state model

Separate three facts that the current closing flag conflates:

• admission is closed, so callbacks and callers cannot enqueue new input or writes;
• output is available, so the worker may still emit KCP ACK data over the current connection and TX subscription;
• terminal state is published, so readers and flushers may consume the final RX, waitsnd and fatal result and cleanup may release KCP state.

Only the worker owns and mutates KCP. BLE event paths update bounded connection/output state under the existing synchronization boundary, enqueue or reject frames, and wake the worker; they never call KCP directly.

Graceful close and physical loss

For graceful local close while output remains available, the worker drains every frame admitted before admission closed, moves complete payload into the RX byte buffer, performs the final KCP update/flush, emits at most the required final ACK, snapshots final waitsnd and fatal status, then publishes terminal state and wakes blocked I/O.

For physical disconnect or TX unsubscribe, the event path marks output unavailable before waking terminal processing. The worker still drains and publishes locally admitted RX, but must not call notification/write output after availability is lost. An output failure during the final ACK marks output unavailable, preserves published RX, records the output error as terminal status and does not retry through a dead link.

I/O result ordering

read() may drain bytes already published in the RX buffer after admission closes. Once RX is empty and terminal state is published, it returns the terminal fatal status or H2_PAL_ERR_CLOSED.

flush() waits for terminal publication when shutdown is in progress. Result precedence is:

1. terminal fatal/output error;
2. H2_PAL_OK when both the local TX queue and final KCP waitsnd are empty;
3. H2_PAL_ERR_CLOSED when pending data can no longer be acknowledged;
4. timeout while the stream is still able to make progress and the caller's deadline expires.

No path may report success merely because closing became true before the worker published its final state.

Client and server lifecycle

Client close keeps BLE event observation and TX subscription registered until the worker publishes terminal state and joins. A racing disconnect first disables output, produces at most one disconnect event and prevents later writes or notification attempts. Cleanup remains retryable when task join or PAL release returns an error; state is freed only after all callbacks and worker access have ended.

Server handler return closes the borrowed stream. If the same connection and subscription are still live, terminal processing may emit the final ACK before server-owned disconnect cleanup. If the peer already disconnected or unsubscribed, output is suppressed. The server then clears all connection-scoped state, returns to idle and creates a new stream/KCP instance for a later connection.

Bounds and failure behavior

Shutdown processing remains bounded by the frames admitted before close and existing fixed-capacity queues. It must not wait for new peer traffic after physical output loss, hold a mutex across a PAL BLE output call that reacquires the same mutex, emit duplicate disconnect events, access released KCP state or carry terminal RX/events into a reconnect.

Test And Acceptance Criteria

Acceptance Criteria

• Graceful client and server shutdown process every input frame admitted before close and expose all complete received bytes through read().
• A graceful session with a usable BLE output path emits exactly the required final ACK before client unsubscribe or server disconnect cleanup.
• Physical disconnect, TX unsubscribe and final output failure suppress every subsequent BLE output attempt.
• Final ACK output failure preserves received payload and becomes the deterministic terminal result.
• flush() follows the documented fatal/success/closed/timeout precedence using worker-published final queue and waitsnd state.
• Close, disconnect-during-final-ACK, disconnect-during-worker-start, handler return and retryable cleanup do not deadlock, use freed state or emit duplicate terminal events.
• Server teardown returns to idle and a subsequent connection receives an independent stream, KCP state, buffer and event generation.
• Public headers, GATT profile, MTU, KCP tuning, reconnect ownership and H2Loader behavior remain unchanged.
• guides/zh/developing/bleikcp.md is synchronized with the implemented terminal-state and lifecycle contract.

Validation

bazel test //libs/bleikcp:all --test_output=errors --cache_test_results=no
make guides-build
git diff --check

• Repeat //libs/bleikcp:bleikcp_test 30 times with --cache_test_results=no to exercise race boundaries.
• Deterministic fake-PAL coverage must include final ACK success, final ACK output failure, output already unavailable at shutdown, disconnect during final ACK, disconnect during worker start, admitted payload readable before terminal status, zero/non-zero final waitsnd, client unsubscribe ordering, server handler return, return-to-idle and reconnect isolation.
• Real-device BLE validation is SKIP by default because this Issue does not change controller/backend policy. If deterministic tests reveal a controller-dependent boundary, record the missing device matrix and residual risk and keep the Issue open until that boundary is verified or removed from the implementation.