Merge pull request #105 from dasomji/feat/issues-97-104

Harden Postbox protocol and recovery flows

Author: dasomji

README.md

@@ -8,7 +8,7 @@ The product requirements document is in [`docs/prd/pi-postbox.md`](docs/prd/pi-p
 
 ## Current status
 
-Version 0.2.10 coordinates protocol 0.1.9 with Android 0.4.3 (build 7), keeps the current-owner footer count synchronized with Question lifecycle changes, makes configured development and production API ports strict, and adds a privacy-preserving Android notification-backed launcher badge for the full pending queue. It retains generated cross-language conformance fixtures, exact-match version gating, hard-block mismatch UX, durable Answer-driven agent auto-wake, and answer-ready widget cleanup after reads.
+Version 0.2.14 coordinates protocol 0.1.10 with Android 0.4.7 (build 11), distinguishes transport failures from protocol mismatches, opens cached saved-server workflows offline, restores stamped Question Chat transport state, makes Answer auto-wake recovery queue-aware, preserves valid config fields, reconciles resolved-push badges offline and idempotently (including after process restarts and opened child notifications), handles stale Question revisions as refreshable conflicts, and derives Android contract inputs from their sources of truth.
 
 Issues #1-#11 provide the v1 implementation: runnable TypeScript workspace, `pi-postbox-server` CLI, Pi extension with `write_question`, WebSocket session registration, SSE browser state, SQLite persistence/history, structured Questions and options, semantic working/blocked/idle state, reconnect/idempotency/expiry, local terminal fallback commands, editable presentation metadata, and packaging/deployment docs plus a release smoke script. Version 0.2.6 updates Android 0.4.1 to include the server-required Question revision when submitting an Answer. Version 0.2.5 keeps the footer and status surfaces from undercounting a locally tracked open Question when the durable owner snapshot is briefly stale, makes published package updates robust on npm 11, and prebuilds the shared protocol before clean-checkout test runs. Version 0.2.3 adds validated npm Trusted Publishing from pushes to the main branch through GitHub Actions OIDC. Version 0.2.2 adds complete npm license and source metadata, fixes the published CLI bin path, and excludes test source from the package tarball. Version 0.2.1 bounds and paginates every model-facing bulk read, uses compact stateless cursors, hides inactive historical owners by default, trims repeated list fields, and preserves checkout development ports across restarts. Version 0.2.0 replaces the separate model-facing create/update tools with one explicit-action `write_question` surface and returns reusable current Question handles from creation. Version 0.1.9 requires a concise ambiguity for new Questions, simplifies parent and expiry inputs, renames option `meaning` to `impact`, removes top-level handoff context and per-option context, removes reconstructed Question Chat, and renders single/multi choice with accessible ballot controls. Version 0.1.8 exposed single-Question create/idempotent receipt disposition and safely required a full Pi restart when `/reload` retained an incompatible shared protocol dependency. Version 0.1.7 reduced model-facing tool schemas, strictly described exact-owner filters, and made recovery reads compact by default with an explicit full view while preserving strict server-side action validation and internal provenance/expiry compatibility.
 

apps/android/README.md

@@ -51,7 +51,7 @@ https://postbox.your-tailnet.ts.net/
 
 The app verifies the server with `GET /healthz` before saving the URL. Keep the existing Tailnet-private trust model: do not expose the Postbox server publicly for this prototype.
 
-Android 0.4.3 (build 7) supports Postbox protocol 0.1.9 exactly. The app displays this identity before and after connection, rechecks saved endpoints, and blocks state, Answer, cancel, refresh, Question Chat, and push registration when the active endpoint reports missing or different protocol evidence. Regenerate the shared contract with `npm run generate:android-protocol-contract`; CI uses `npm run check:android-protocol-contract` plus Android unit/lint/assembly gates to reject drift.
+Android 0.4.7 (build 11) supports Postbox protocol 0.1.10 exactly. The app displays this identity before and after connection, rechecks saved endpoints, and blocks state, Answer, cancel, refresh, Question Chat, and push registration when the active endpoint reports missing or different protocol evidence. Regenerate the shared contract with `npm run generate:android-protocol-contract`; CI uses `npm run check:android-protocol-contract` plus Android unit/lint/assembly gates to reject drift.
 
 ## Emulator localhost fallback
 

apps/android/app/build.gradle.kts

@@ -15,8 +15,8 @@ android {
         applicationId = "dev.pi.postbox"
         minSdk = 26
         targetSdk = 36
-        versionCode = 7
-        versionName = "0.4.3"
+        versionCode = 11
+        versionName = "0.4.7"
 
         testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner"
     }

apps/android/app/src/main/java/dev/pi/postbox/notification/AndroidPendingQuestionNotifier.kt

@@ -11,6 +11,7 @@ import android.content.Intent
 import android.content.pm.PackageManager
 import android.net.Uri
 import android.os.Build
+import android.os.Bundle
 import androidx.core.content.ContextCompat
 import java.lang.SecurityException
 import dev.pi.postbox.MainActivity
@@ -126,11 +127,56 @@ class AndroidPendingQuestionNotifier(
         } catch (_: SecurityException) {
             // Notification access can change while the app is running; reconciliation is best-effort.
         }
-        reconcilePendingSummary(pendingRequestIds.size)
+        reconcilePendingSummary(pendingRequestIds)
+    }
+
+    /** Recompute the badge after a resolved push when no authoritative cached queue is available. */
+    fun reconcileResolvedPendingSummary(requestId: String) {
+        val resolvedNotificationId = requestId.hashCode()
+        val resolution = try {
+            val activeNotifications = notificationManager.activeNotifications
+            val summary = activeNotifications.firstOrNull { it.id == PENDING_SUMMARY_NOTIFICATION_ID }?.notification
+            val authoritativePendingIds = summary?.extras
+                ?.getStringArray(EXTRA_PENDING_REQUEST_IDS)
+                ?.toSet()
+            if (authoritativePendingIds != null) {
+                reconcilePendingSummary(authoritativePendingIds - requestId)
+                return
+            }
+            pendingSummaryAfterResolvedPush(
+                summaryCount = summary?.number?.takeIf { it > 0 },
+                activePendingNotificationIds = activeNotifications
+                    .filter { isPendingQuestionNotification(it.notification.channelId, it.id) }
+                    .mapTo(hashSetOf()) { it.id },
+                resolvedNotificationId = resolvedNotificationId,
+                resolvedRequestId = requestId,
+                alreadyResolvedRequestIds = summary?.extras
+                    ?.getStringArray(EXTRA_RESOLVED_REQUEST_IDS)
+                    ?.toSet()
+                    .orEmpty()
+            )
+        } catch (_: SecurityException) {
+            return
+        }
+        reconcilePendingSummary(
+            pendingCount = resolution.pendingCount,
+            resolvedRequestIds = resolution.resolvedRequestIds
+        )
+    }
+
+    private fun reconcilePendingSummary(pendingRequestIds: Set<String>) {
+        reconcilePendingSummary(
+            pendingCount = pendingRequestIds.size,
+            pendingRequestIds = pendingRequestIds
+        )
     }
 
     @SuppressLint("MissingPermission")
-    private fun reconcilePendingSummary(pendingCount: Int) {
+    private fun reconcilePendingSummary(
+        pendingCount: Int,
+        pendingRequestIds: Set<String>? = null,
+        resolvedRequestIds: Set<String> = emptySet()
+    ) {
         if (pendingCount == 0) {
             notificationManager.cancel(PENDING_SUMMARY_NOTIFICATION_ID)
             return
@@ -151,6 +197,12 @@ class AndroidPendingQuestionNotifier(
                     .setGroup(PENDING_QUESTIONS_GROUP_KEY)
                     .setGroupSummary(true)
                     .setNumber(pendingCount)
+                    .setExtras(Bundle().apply {
+                        pendingRequestIds?.let { putStringArray(EXTRA_PENDING_REQUEST_IDS, it.sorted().toTypedArray()) }
+                        if (resolvedRequestIds.isNotEmpty()) {
+                            putStringArray(EXTRA_RESOLVED_REQUEST_IDS, resolvedRequestIds.sorted().toTypedArray())
+                        }
+                    })
                     .setOngoing(true)
                     .setOnlyAlertOnce(true)
                     .setShowWhen(false)
@@ -233,6 +285,8 @@ class AndroidPendingQuestionNotifier(
         const val PENDING_SUMMARY_CHANNEL_DESCRIPTION: String = "Quiet pending-question total used for the launcher badge."
         const val PRIVATE_NOTIFICATION_TEXT: String = "Open Postbox to review and answer."
         const val EXTRA_REQUEST_ID: String = "dev.pi.postbox.extra.REQUEST_ID"
+        internal const val EXTRA_PENDING_REQUEST_IDS: String = "dev.pi.postbox.extra.PENDING_REQUEST_IDS"
+        internal const val EXTRA_RESOLVED_REQUEST_IDS: String = "dev.pi.postbox.extra.RESOLVED_REQUEST_IDS"
         const val ACTION_OPEN_PROTOCOL_MISMATCH: String = "dev.pi.postbox.OPEN_PROTOCOL_MISMATCH"
         const val PROTOCOL_MISMATCH_NOTIFICATION_ID: Int = 0x50524f54
         const val PENDING_SUMMARY_NOTIFICATION_ID: Int = 0x50424f58
@@ -244,11 +298,36 @@ internal fun shouldCancelDuringPendingReconciliation(
     channelId: String?,
     notificationId: Int,
     pendingNotificationIds: Set<Int>
-): Boolean = channelId == AndroidPendingQuestionNotifier.CHANNEL_ID &&
-    notificationId != AndroidPendingQuestionNotifier.PROTOCOL_MISMATCH_NOTIFICATION_ID &&
-    notificationId != AndroidPendingQuestionNotifier.PENDING_SUMMARY_NOTIFICATION_ID &&
+): Boolean = isPendingQuestionNotification(channelId, notificationId) &&
     notificationId !in pendingNotificationIds
 
+internal fun isPendingQuestionNotification(channelId: String?, notificationId: Int): Boolean =
+    channelId == AndroidPendingQuestionNotifier.CHANNEL_ID &&
+        notificationId != AndroidPendingQuestionNotifier.PROTOCOL_MISMATCH_NOTIFICATION_ID &&
+        notificationId != AndroidPendingQuestionNotifier.PENDING_SUMMARY_NOTIFICATION_ID
+
+internal data class PendingSummaryResolution(
+    val pendingCount: Int,
+    val resolvedRequestIds: Set<String>
+)
+
+internal fun pendingSummaryAfterResolvedPush(
+    summaryCount: Int?,
+    activePendingNotificationIds: Set<Int>,
+    resolvedNotificationId: Int,
+    resolvedRequestId: String,
+    alreadyResolvedRequestIds: Set<String>
+): PendingSummaryResolution {
+    val currentCount = summaryCount ?: activePendingNotificationIds.count { it != resolvedNotificationId }
+    if (resolvedRequestId in alreadyResolvedRequestIds) {
+        return PendingSummaryResolution(currentCount, alreadyResolvedRequestIds)
+    }
+    return PendingSummaryResolution(
+        pendingCount = summaryCount?.let { (it - 1).coerceAtLeast(0) } ?: currentCount,
+        resolvedRequestIds = alreadyResolvedRequestIds + resolvedRequestId
+    )
+}
+
 fun Intent.postboxNotificationRequestId(): String? {
     if (action != NotificationTapTarget.ACTION_OPEN_QUESTION) return null
     return getStringExtra(AndroidPendingQuestionNotifier.EXTRA_REQUEST_ID)

apps/android/app/src/main/java/dev/pi/postbox/onboarding/PostboxHealthVerifier.kt

@@ -10,6 +10,7 @@ import java.util.concurrent.TimeUnit
 import kotlinx.serialization.Serializable
 import kotlinx.serialization.SerializationException
 import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.decodeFromJsonElement
 import okhttp3.OkHttpClient
 import okhttp3.Request
 import okhttp3.HttpUrl.Companion.toHttpUrlOrNull
@@ -50,15 +51,18 @@ class OkHttpPostboxHealthVerifier(
                 val health = try {
                     compatibilityGate.decodeHttpResponse(
                         rawMessage = body,
+                        statusCode = response.code,
                         responseProtocolVersion = response.header("X-Postbox-Protocol-Version"),
                         source = ProtocolMessageSource.HEALTH
                     ) {
-                        json.decodeFromString<PostboxHealthResponse>(it)
+                        json.decodeFromJsonElement<PostboxHealthResponse>(it)
                     }
                 } catch (exception: PostboxProtocolMismatchException) {
                     return HealthVerificationResult.IncompatibleProtocol(exception.mismatch)
                 } catch (_: SerializationException) {
                     return HealthVerificationResult.Rejected(HealthRejectionReason.MALFORMED_HEALTH_RESPONSE)
+                } catch (_: dev.pi.postbox.protocol.PostboxProtocolHttpException) {
+                    return HealthVerificationResult.Rejected(HealthRejectionReason.MALFORMED_HEALTH_RESPONSE)
                 }
 
                 if (!response.isSuccessful) {

apps/android/app/src/main/java/dev/pi/postbox/onboarding/ServerOnboardingViewModel.kt

@@ -22,11 +22,10 @@ class ServerOnboardingViewModel(
         val savedBaseUrl = store.loadVerifiedServerUrl()
         if (savedBaseUrl != null) {
             serverUrl = savedBaseUrl
-            state = ServerOnboardingState.Verifying(baseUrl = savedBaseUrl)
-            applyVerificationResultIfCurrent(
+            state = ServerOnboardingState.Ready(baseUrl = savedBaseUrl)
+            applySavedVerificationResultIfCurrent(
                 generation = generation,
-                normalizedBaseUrl = savedBaseUrl,
-                warning = null,
+                savedBaseUrl = savedBaseUrl,
                 result = verifier.verify(savedBaseUrl)
             )
         }
@@ -36,14 +35,13 @@ class ServerOnboardingViewModel(
         val generation = ++verificationGeneration
         val savedBaseUrl = store.loadVerifiedServerUrl() ?: return
         serverUrl = savedBaseUrl
-        state = ServerOnboardingState.Verifying(baseUrl = savedBaseUrl)
+        state = ServerOnboardingState.Ready(baseUrl = savedBaseUrl)
         val result = withContext(Dispatchers.IO) {
             verifier.verify(savedBaseUrl)
         }
-        applyVerificationResultIfCurrent(
+        applySavedVerificationResultIfCurrent(
             generation = generation,
-            normalizedBaseUrl = savedBaseUrl,
-            warning = null,
+            savedBaseUrl = savedBaseUrl,
             result = result
         )
     }
@@ -117,6 +115,35 @@ class ServerOnboardingViewModel(
         handleVerificationResult(normalizedBaseUrl, warning, result)
     }
 
+    private fun applySavedVerificationResultIfCurrent(
+        generation: Long,
+        savedBaseUrl: String,
+        result: HealthVerificationResult
+    ) {
+        if (generation != verificationGeneration) return
+        when (result) {
+            is HealthVerificationResult.IncompatibleProtocol -> {
+                state = ServerOnboardingState.IncompatibleProtocol(
+                    baseUrl = savedBaseUrl,
+                    mismatch = result.mismatch
+                )
+            }
+            is HealthVerificationResult.Valid -> {
+                state = ServerOnboardingState.Ready(
+                    baseUrl = savedBaseUrl,
+                    health = VerifiedPostboxHealth(
+                        service = result.service,
+                        version = result.version,
+                        protocolVersion = result.protocolVersion,
+                        buildId = result.buildId
+                    )
+                )
+            }
+            is HealthVerificationResult.Unreachable,
+            is HealthVerificationResult.Rejected -> Unit
+        }
+    }
+
     private fun handleVerificationResult(
         normalizedBaseUrl: String,
         warning: ServerUrlWarning?,

apps/android/app/src/main/java/dev/pi/postbox/protocol/GeneratedPostboxProtocolContract.kt

@@ -2,8 +2,8 @@ package dev.pi.postbox.protocol
 
 /** Generated by scripts/generate-android-protocol-contract.mjs. Do not edit. */
 object GeneratedPostboxProtocolContract {
-    const val SUPPORTED_PROTOCOL_VERSION: String = "0.1.9"
-    const val CONTRACT_FINGERPRINT: String = "ad5e1496494a3a09bdc06fc33a078468b1d8d721600bfff55b1ca733080d7c07"
+    const val SUPPORTED_PROTOCOL_VERSION: String = "0.1.10"
+    const val CONTRACT_FINGERPRINT: String = "714b443f32821ea63c645ea241207a87dbfeaf1e261639305f0f50b70b8109c6"
     val SEMANTIC_STATES: List<String> = listOf("working", "blocked", "waiting_for_postbox", "idle", "unknown")
     val PRESENCE_STATES: List<String> = listOf("live", "stale", "offline")
     val ASK_MODES: List<String> = listOf("single", "multi")

apps/android/app/src/main/java/dev/pi/postbox/protocol/PostboxProtocol.kt

@@ -4,6 +4,7 @@ import kotlinx.serialization.SerialName
 import kotlinx.serialization.Serializable
 import kotlinx.serialization.SerializationException
 import kotlinx.serialization.json.Json
+import kotlinx.serialization.json.JsonObject
 
 /**
  * Synthetic answer value the server accepts alongside a question's own options,
@@ -19,6 +20,8 @@ object PostboxProtocolJson {
 
     fun decodeStateSnapshot(value: String): StateSnapshot = json.decodeFromString(value)
 
+    fun decodeStateSnapshot(value: JsonObject): StateSnapshot = json.decodeFromJsonElement(StateSnapshot.serializer(), value)
+
     fun encodeAnswerPayload(payload: AskAnswerPayload): String = json.encodeToString(AskAnswerPayload.serializer(), payload)
 
     fun encodeCancelPayload(payload: AskCancelPayload): String = json.encodeToString(AskCancelPayload.serializer(), payload)
@@ -248,5 +251,11 @@ internal data class PostboxErrorResponse(
         } catch (_: SerializationException) {
             null
         }
+
+        fun parse(body: JsonObject): PostboxErrorResponse? = try {
+            PostboxProtocolJson.json.decodeFromJsonElement(serializer(), body)
+        } catch (_: SerializationException) {
+            null
+        }
     }
 }