Skip to content

Add notification subscription API for watchapps#172

Open
killdano wants to merge 6 commits into
coredevices:masterfrom
killdano:watchapp-notification-subscription-api
Open

Add notification subscription API for watchapps#172
killdano wants to merge 6 commits into
coredevices:masterfrom
killdano:watchapp-notification-subscription-api

Conversation

@killdano
Copy link
Copy Markdown

@killdano killdano commented May 2, 2026

Note: This PR is built on top of #171; the first three commits are prerequisites that will disappear from this PR's view once #171 lands. Commits 4–6 are the new work.


This PR introduces a notification subscription API for watchapps, designed around per-watchapp field opt-in. A watchapp's package.json notificationFilter declares both which source apps it wants notifications from and which fields of those notifications it wants extracted on the phone and forwarded to PKJS.

Why opt-in fields

Bluetooth bandwidth to the watch is the binding constraint for this whole feature. A naive design that forwards every available notification field to every subscribing watchapp would force every consumer to pay the price of every field — adding album art for one watchapp would slow down navigation updates for another. With per-watchapp opt-in:

  • Cheap, broadly useful fields (text, category, standard icons, action presence flags, primitive extras) ship in the default field set and watchapps that just want "notification arrived from package X" get them automatically.
  • Bandwidth- or CPU-heavy fields (full bitmaps, structured media metadata, message arrays, Android 14+ Ongoing Activity icons) are explicit opt-ins. A watchapp pays for what it asks for.
  • Forward-compatible: future fields can be added without renegotiating the default payload. Existing watchapps continue working unchanged; new ones declare what they want.

Schema

notificationFilter accepts both shapes; they can be mixed in the same array.

Bare-string — package name only, watchapp gets the default field set:

"notificationFilter": ["com.example.app"]

Object form — explicit field opt-in:

"notificationFilter": [
  {
    "package": "com.google.android.apps.maps",
    "fields": ["title", "text", "category", "iconExtras"]
  }
]

The polymorphic deserializer (NotificationSubscription.Serializer) accepts either; encoded form is always the object form.

Field catalog

Field In default set? Notes
title, text, subText, infoText Standard text slots
category Notification.category
smallIconBase64, largeIconBase64 32×32 PNG, base64
actions {title, hasIntent} per action
extras Primitive entries from extras bundle
bigPictureBase64 BigPictureStyle photo, downsampled to fit byte cap
mediaMetadata MediaSession state: {title, artist, album, durationMs, positionMs, playbackState, albumArtBase64}
messagingMessages MessagingStyle messages: [{sender, timestamp, text}, …]
inboxLines InboxStyle lines: […]
iconExtras Icon-typed entries from extras (Android 14+ Ongoing Activity glyphs)

Per-field extraction is best-effort. Each field is wrapped in try/catch; per-field exceptions are logged at trace and the field surfaces as null/empty in the JSON, but other fields continue to be processed.

iconExtras and Maps' Ongoing Activity glyphs

Worth a section of its own — it's the most novel piece in this PR.

Android 14 introduced the Ongoing Activity API for rich live notifications. Google Maps' nav notification is the canonical example: turn-direction arrow, lane guidance, arrival flag, ETA chip glyph all stored as Icon parcelables in the notification's extras bundle, under keys named android.ongoingActivityNoti.chipIcon / nowbarIcon / secondIcon.

These don't surface via Notification.smallIcon / largeIcon (those carry the brand glyph), and they don't surface via Notification.contentView / bigContentView either — Maps' Ongoing Activity notifications don't populate those at all. The extras bundle is where the dynamic glyphs live.

iconExtras walks the extras bundle, finds every Icon-typed entry, rasterizes each to a 32×32 ARGB PNG, and emits a JSON object map keyed by extras key:

"iconExtras": {
  "android.ongoingActivityNoti.chipIcon": {
    "base64": "iVBORw0KGgo…",
    "hash": "9589c01a",
    "intrinsicW": 84,
    "intrinsicH": 84
  },
  "android.ongoingActivityNoti.nowbarIcon": { },
  "android.largeIcon": { }
}

Empirical validation

Live drives with the extractor confirmed Maps publishes a stable set of distinct glyphs across turn states. From two test trips:

State chipIcon hash largeIcon hash
Continue / straight 9589c01a 8fa71c8d
Turn left 1ad8de36 28f6365a
Turn right 5af1cf48 9d427851
U-turn 87d3a7aa 487ecffa
Arrive 35b76267 c4939096

The U-turn hash was confirmed by correlation with Maps' own voice-prompt timing — _MAKE_A_UTURN.mp3 fired 12 ms before the icon transition. The icon channel reliably encodes turn state even when the text channel doesn't (during the U-turn, Maps' notification text was "toward Kelvin Ave", which a text-only parser would have read as "continue straight").

Compatibility

  • The bare-string form is the simple case for watchapps that just want "notify me when something arrives from package X" — they get a sensible default field set without extra ceremony.
  • The schema is forward-compatible: field names not in the running libpebble3's catalog don't cause parse rejection. A watchapp targeting a future libpebble3 version can declare new fields and still load on older phones (it just won't get those fields in payloads).

Out of scope (intentionally)

  • RemoteViews ImageView extraction. Investigated as an alternative path to dynamic glyphs; for Google Maps it returns nothing (contentView and bigContentView are both null on Ongoing Activity notifications). The iconExtras extractor covers the same use case via a stable Android API surface. If a future use case for RemoteViews extraction emerges, it can be added as another opt-in field without disturbing this design.
  • Action invocation. actions field surfaces {title, hasIntent} per declared action button; PendingIntent dispatch from the watch is left for v2.

Diagnostic logging

WatchappNotificationSerializer emits a trace-level line per dispatched notification showing the parsed field set:

serialize <pkg> posted=<bool> fields=<set>

Useful for confirming per-watchapp opt-in is reaching the serializer correctly when integrating a new watchapp; trace-level only so it's silent at production logging levels.

Commits

  1. Per-watchapp field opt-in for notification subscriptions — schema, dispatcher, listener plumbing, and the cheap default fields wired up.
  2. Heavyweight notification extractors (BigPicture / Media / Messaging / Inbox) — opt-in fields for the standard rich-notification styles.
  3. Extract Icon-typed entries from Notification.extras (iconExtras) — Android 14+ Ongoing Activity glyph extraction, motivated by the empirical findings above.

killdano added 2 commits May 2, 2026 15:22
Lets watchapps register as Android share targets so users can "Share to
<watchapp>" from any other app on their phone. The shared payload (text,
URL, or both) routes to the watchapp's PKJS via a new `'shareintent'`
event listener.

This is a platform capability that benefits any watchapp accepting
external input — navigation apps receiving Maps URLs, music apps
receiving Spotify links, note-taking apps receiving selected text, etc.
The motivating use case is MirrorMap (a Pebble nav watchapp), where
sharing a Google Maps URL into the watchapp is the primary user flow,
but nothing in this PR is Maps-specific.

## What's new

### Watchapp side

Watchapps opt in via two new fields in `package.json`:

```json
{
  "shareTarget": {
    "mimeTypes": ["text/plain"]
  }
}
```

PKJS receives shared payloads via:

```javascript
Pebble.addEventListener('shareintent', function(e) {
  // e.text — the shared text/URL
});
```

### Android app side

Three Android share-sheet surfacing strategies, layered for compatibility:

1. **Static manifest entry** ("Pebble") — universal fallback, works on every
   share-sheet implementation
2. **Sharing Shortcuts** with Direct Share metadata — surfaces individual
   watchapps as named entries on Android 11+ where supported
3. **ChooserTargetService** compat — same goal for older / Samsung-style
   share sheets where Sharing Shortcuts don't surface

When the static "Pebble" path is taken and multiple watchapps subscribe to
the shared MIME type, the user is presented with a chooser dialog to pick
the destination watchapp. When only one watchapp subscribes, dispatch is
direct (no extra tap).

### Per-watchapp icons in the share sheet

Each watchapp's share-target shortcut renders with the watchapp's own icon
extracted from its installed PBW. This required a small `.pbpack` resource
pack parser (`PbwResourcePack.kt`) since Pebble watchapps use a custom
binary format that bundles all resources into a single file. The parser
walks the manifest and resource table to extract the menu icon (declared
with `menuIcon: true` in `package.json`'s resources array) as a raw PNG,
then falls back to scanning the PBW zip root for any PNG if the resource
pack lookup fails.

The extracted icons are tinted via ColorMatrix to a brand-cohesive
appearance for share-target presentation while preserving alpha for
anti-aliasing on rounded backgrounds.

### Java-side short URL resolver

Many sharing flows generate short URLs that redirect to the actual content
URL (Google Maps' `maps.app.goo.gl`, Twitter's `t.co`, etc.). PKJS can in
principle resolve these via XHR, but in practice many short URL services
employ aggressive anti-bot measures that block XMLHttpRequest's User-Agent.

`ShareUrlResolver` runs the redirect resolution on the Android side using
the existing OkHttp/Ktor stack with a standard browser-like User-Agent.
The resolved long URL is what gets delivered to PKJS via the `shareintent`
event, so watchapp authors don't have to re-implement HTTP redirect
following in JavaScript or work around bot-detection headers.

The resolver:
- Has structured concurrency (cancels cleanly with the share dispatch)
- Retries once on transient network failure
- Uses a dedicated HttpClient instance so its configuration doesn't leak
  into other libpebble3 HTTP usage
- Times out at 10s — beyond that the share fails gracefully

Currently configured for Google Maps short URL hosts; extending to other
short URL services is a one-line addition.

## Cold-start handling

Share intents commonly arrive while the target watchapp is not currently
running on the watch. The dispatch path:

1. Receives the share intent via the new manifest entry / shortcut
2. Looks up the subscribed watchapp(s) by MIME type
3. Asks the libpebble3 connection to launch the watchapp on the watch
4. Waits for PKJS to signal "ready" (CMD_INIT round-trip complete)
5. Delivers the share event

The launch + ready-signal phase has a 30s timeout for cold-starts where
the watchapp PBW needs to be transferred fresh, with a sub-timeout on the
producer flow for the initial state read.

## Testing notes

End-to-end validated on a Samsung Galaxy + Pebble Time Steel with MirrorMap
as the share target. Tested:

- Cold-start (watchapp not running, app not in foreground) — share intent
  triggers watchapp launch, share is delivered after ready signal
- Warm-start (watchapp foreground) — share is delivered without relaunch
- Multi-watchapp scenario — chooser dialog surfaces correctly
- Per-watchapp icons render with their actual PBW menu icon
- Short URL resolution — `maps.app.goo.gl` URLs resolve to the long Maps
  directions URL before PKJS sees them

## Files

New:
- `composeApp/src/androidMain/kotlin/coredevices/coreapp/sharing/ShareTargetActivity.kt`
- `composeApp/src/androidMain/res/xml/share_targets.xml`
- `libpebble3/src/androidMain/kotlin/io/rebble/libpebblecommon/shareintent/ShareTargetSync.kt`
- `libpebble3/src/commonMain/kotlin/io/rebble/libpebblecommon/di/ShareIntentModule.kt`
- `libpebble3/src/commonMain/kotlin/io/rebble/libpebblecommon/disk/pbw/PbwResourcePack.kt`
- `libpebble3/src/commonMain/kotlin/io/rebble/libpebblecommon/shareintent/ShareIntentDispatcher.kt`
- `libpebble3/src/commonMain/kotlin/io/rebble/libpebblecommon/shareintent/ShareTargetEntry.kt`
- `libpebble3/src/commonMain/kotlin/io/rebble/libpebblecommon/shareintent/ShareTargetsProducer.kt`
- `libpebble3/src/commonMain/kotlin/io/rebble/libpebblecommon/shareintent/ShareUrlResolver.kt`

Modified:
- `composeApp/build.gradle.kts` — Direct Share / shortcut metadata deps
- `composeApp/src/androidMain/AndroidManifest.xml` — share-target manifest entry
- `composeApp/src/androidMain/kotlin/coredevices/coreapp/MainApplication.kt` — bootstrap
- `composeApp/src/androidMain/kotlin/coredevices/coreapp/di/androidDefaultModule.kt` — koin
- `gradle/libs.versions.toml` — version catalog updates
- `libpebble3/src/androidMain/assets/startup.js` — PKJS event registration
- `libpebble3/src/androidMain/kotlin/.../js/WebViewJsRunner.kt` — event dispatch
- `libpebble3/src/commonMain/kotlin/.../di/LibPebbleModule.kt` — koin
- `libpebble3/src/commonMain/kotlin/.../disk/pbw/DiskUtil.kt` — resource pack helpers
- `libpebble3/src/commonMain/kotlin/.../js/JsRunner.kt` — event interface
- `libpebble3/src/commonMain/kotlin/.../js/PKJSApp.kt` — ready signal observation
- `libpebble3/src/commonMain/kotlin/.../metadata/pbw/appinfo/PbwAppInfo.kt` — shareTarget field
- `libpebble3/src/iosMain/kotlin/.../js/JavascriptCoreJsRunner.kt` — iOS no-op stub
Lets watchapps receive structured notification events from packages they
declare interest in. Designed for live mirroring use cases where a
watchapp wants to surface a phone-side app's ongoing-activity state on
the watch face — turn-by-turn from Google Maps, now-playing from
Spotify, fitness session from Strava, ride status from Uber, etc.

The motivating use case is MirrorMap, which uses Maps' navigation
notifications (turn text + distance + ETA + arrow icon) to drive a
live HUD overlay on the watch independent of any MirrorMap-side route
state. But nothing in this PR is Maps-specific — any package can be
declared in the watchapp's `notificationFilter` array.

## What's new

### Watchapp side

Watchapps opt in via a new field in `package.json`:

```json
{
  "notificationFilter": ["com.google.android.apps.maps", "com.spotify.music"]
}
```

PKJS receives notifications via:

```javascript
Pebble.addEventListener('appnotification', function(e) {
  // e.package         — source package name
  // e.posted          — true on post, false on removal
  // e.title, e.text   — convenience-surfaced text fields
  // e.subText, e.infoText
  // e.smallIconBase64 — base64 PNG, 32×32 (see "Icon forwarding" below)
  // e.largeIconBase64 — base64 PNG, 32×32
  // e.extras          — full extras Bundle as a JSON object
});
```

Watchapps that need to catch up on existing notification state when
spinning up (e.g. user launched watchapp mid-trip with Maps already
navigating) call:

```javascript
Pebble.getActiveNotifications(['com.google.android.apps.maps'])
  // returns array of notification objects in the same shape
```

### Consent model

A watchapp's declaration in `package.json` is the consent signal. The
user explicitly installed a watchapp that announced its notification
interests; no separate per-watchapp consent prompt is required. The
Pebble app's existing notification access grant — a single OS-level
permission already granted by the user — is the only OS-level gate.

This keeps the UX simple while remaining responsible: watchapps can't
silently expand their notification access after install (the filter
list is read from the PBW manifest at install time and fixed for the
lifetime of that PBW version).

### Foreground-only dispatch

PKJS only exists for the watchapp currently running on the watch, so
notifications only flow while the user has the watchapp open. This is
a deliberate choice — it bounds the privacy surface area and avoids
the watchapp-as-background-listener antipattern. Watchapps that need
to bootstrap state on launch use `getActiveNotifications` for catch-up.

### Icon forwarding

`smallIconBase64` and `largeIconBase64` ship the notification's icons
as base64-encoded 32×32 PNGs. This is included because notification
icons are often the most semantically rich visual signal — Maps' turn
arrows (left, right, straight, U-turn, exit, roundabout, etc.) live
in the smallIcon, music apps put album art in largeIcon, fitness apps
put activity glyphs in smallIcon. Without forwarding the icons,
watchapps would have to bake a separate icon set into firmware for
every source app they want to mirror — which doesn't scale.

`NotificationIconExtractor` rasterizes the icons via Drawable.draw()
to a fixed 32×32 ARGB bitmap and PNG-encodes them. Failures are
best-effort (logged + skipped) — they don't affect the rest of the
payload. Icons are skipped on removal events (the icon adds no signal
when the notification is being cleared).

Cost: ~280-820 bytes per icon base64-encoded; trivial overhead at
typical notification rates.

### Architecture

Three new classes split by platform:

- `WatchappNotificationDispatcher` (commonMain) — fans incoming
  notifications out to the currently-running PKJS watchapps that
  subscribe to the source package. Cheap to call on every
  notification (typically O(connected_watches) with usually 1 watch
  and 1 running PKJS).

- `WatchappNotificationSerializer` (androidMain) — converts an
  Android `StatusBarNotification` to JSON for PKJS consumption.
  Surfaces well-known fields at top level for convenience plus a raw
  `extras` map for app-specific keys (e.g. Maps's
  `ongoingActivityNoti.next_step_message`). Non-primitive extras
  (Parcelables, RemoteInput) are skipped — JSON isn't the right
  format for those.

- `NotificationIconExtractor` (androidMain) — Drawable → 32×32
  bitmap → PNG → base64. Fast path when the Drawable is already a
  bitmap of the target size; otherwise standard Canvas.draw()
  rasterize.

`LibPebbleNotificationListener` already had the platform-level
notification access; this PR adds a sibling fan-out path to the
existing notification-relay pipeline. The original watch-UI relay is
unchanged; failures in the new watchapp dispatch path don't affect it.

## Builds on PR 1

Stacked on the share-intent target API PR. The two PRs share PKJS
infrastructure — both add new event types via the same registration
pattern in `startup.js` and dispatch through the same `JsRunner`
mechanism, and both read `package.json` fields via the same
`PbwAppInfo` parser. PR 1 should land first, PR 2 second; the GitHub
PR base is set accordingly so reviewers see only the incremental diff.

## Testing notes

End-to-end validated on a Samsung Galaxy + Pebble Time Steel with
MirrorMap as the subscriber. Tested:

- Live notification streaming during Google Maps active navigation
  (notifications arriving every ~10s, parsed for turn text + distance
  + ETA, displayed on watch via MirrorMap's HUD overlay)
- `getActiveNotifications` cold-start retrieval (user launches
  MirrorMap mid-trip with Maps already navigating; gets immediate
  state without waiting for next push)
- Notification removal events firing the `posted: false` event when
  Maps clears its persistent ongoing-activity notification
- Icon forwarding verified end-to-end via PKJS console log:
  smallIcon ~968 bytes base64, largeIcon ~596 bytes base64, both
  null on removal events

## Files

New (commonMain):
- `libpebble3/src/commonMain/kotlin/.../notification/WatchappNotificationDispatcher.kt`
- `libpebble3/src/commonMain/kotlin/.../di/WatchappNotificationModule.kt`

New (androidMain):
- `libpebble3/src/androidMain/kotlin/.../notification/WatchappNotificationSerializer.kt`
- `libpebble3/src/androidMain/kotlin/.../notification/NotificationIconExtractor.kt`

Modified:
- `libpebble3/src/commonMain/kotlin/.../metadata/pbw/appinfo/PbwAppInfo.kt` — notificationFilter field
- `libpebble3/src/commonMain/kotlin/.../js/PKJSApp.kt` — onAppNotification entry point
- `libpebble3/src/commonMain/kotlin/.../js/JsRunner.kt` — event interface
- `libpebble3/src/commonMain/kotlin/.../js/PKJSInterface.kt` — getActiveNotifications binding
- `libpebble3/src/commonMain/kotlin/.../di/LibPebbleModule.kt` — koin
- `libpebble3/src/androidMain/kotlin/.../js/WebViewJsRunner.kt` — Android event dispatch
- `libpebble3/src/androidMain/kotlin/.../js/WebViewPKJSInterface.kt` — Android getActiveNotifications
- `libpebble3/src/androidMain/kotlin/.../notification/LibPebbleNotificationListener.kt` — fan-out hook
- `libpebble3/src/androidMain/assets/startup.js` — PKJS event registration
- `libpebble3/src/iosMain/kotlin/.../js/JavascriptCoreJsRunner.kt` — iOS no-op stub
- `libpebble3/src/iosMain/kotlin/.../js/JSCPKJSInterface.kt` — iOS no-op stub
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented May 2, 2026

CLA assistant check
All committers have signed the CLA.

@killdano killdano force-pushed the watchapp-notification-subscription-api branch from 603c4f6 to 1533fd2 Compare May 3, 2026 03:30
Replaces the 2-attempt 300ms-fixed-delay retry in ShareUrlResolver
with a 4-attempt exponential schedule (0, 2s, 4s, 6s) with +/-200ms
symmetric jitter.

Motivation: field testing showed Google's short-URL service
returning 404 stochastically -- both attempts in the previous
2-attempt loop hit 404 within ~150ms of each other, falling open
to PKJS's XHR fallback which then hits 403 (Google's anti-bot
hits XHR harder than OkHttp).

Spreading attempts across ~13s with jitter avoids the regular-
interval polling pattern that contributes to the bot heuristic,
while still bounded by a 16s overall timeout. Happy path stays
snappy (attempt 1 immediate +/-200ms jitter).
@killdano killdano force-pushed the watchapp-notification-subscription-api branch from 1533fd2 to b85bc0f Compare May 3, 2026 03:46
@ericmigi
Copy link
Copy Markdown
Contributor

ericmigi commented May 3, 2026

@sjp4 is working on a permissioning system for apps that will be a prerequisite for this PR. He can also weigh in on whether this API makes sense as written.

The idea is awesome, though. It would be great for apps to be able to show custom, more detailed UIs for notifications!

killdano added 3 commits May 3, 2026 02:47
Watchapps can now declare which fields of an Android notification
they want delivered to PKJS via an optional `fields` array in
`package.json`'s `notificationFilter` entries:

```
"notificationFilter": [
  { "package": "com.example.app",
    "fields": ["title", "text", "category", "smallIconBase64"] }
]
```

The bare-string form is preserved for backward compat:

```
"notificationFilter": ["com.example.app"]
```

A bare-string entry resolves to a default field set chosen to match
the payload shape libpebble3 delivered before this change, so existing
watchapps continue working unchanged.

Why
===

The previous notification path delivered a fixed payload to every
subscribing watchapp with no opportunity to opt out of fields they
didn't need or in to fields they did. That made every new field a
breaking-bandwidth decision: adding it cost every subscriber even if
only one wanted it. With per-watchapp opt-in, future fields are
strictly opt-in for new consumers without affecting anyone else.

It also lets bandwidth-heavy fields (large bitmaps, structured
arrays, etc.) live in the API as opt-in additions in follow-up
commits without bloating the default payload — more on those in
later commits in this PR.

Schema
======

[NotificationSubscription] is a polymorphic Kotlinx serializable
that accepts both shapes in JSON. Encoded form is always the object
form; decoder accepts either. [Field] holds the v1 catalog of
recognized field names; field names not in [Field.ALL_NAMES] are
not silently filtered (allows watchapps targeting future libpebble3
versions to declare new fields without parse-rejection on older
phones).

This commit ships the cheap fields:

  title, text, subText, infoText      Notification.extras text slots
  category                            notification.category
  smallIconBase64, largeIconBase64    32x32 PNG of standard icon slots
  actions                             { title, hasIntent } per Action
  extras                              primitive entries from extras

Heavyweight fields (BigPicture, MediaSession metadata, MessagingStyle
messages, InboxStyle lines) and Android-14+ Ongoing-Activity icon
extras land in subsequent commits in this PR.

Diagnostic
==========

[WatchappNotificationSerializer] emits a trace-level log line per
dispatched notification showing the parsed field set:

  serialize <pkg> posted=<bool> fields=<set>

Useful for confirming per-watchapp opt-in is reaching the serializer
correctly when integrating a new watchapp; trace-level only so it's
silent at production logging levels.

Failure mode
============

Per-field extraction is best-effort. Each field is wrapped in
try/catch; per-field exceptions are logged at trace and the field
is omitted (or null) in the output, but other fields continue to
be processed. The dispatcher catches per-watchapp serializer
exceptions and continues to the next subscribing watchapp.

Plumbing
========

[WatchappNotificationDispatcher] is rewritten around a
SerializerCallback functional interface so the per-watchapp
subscription state is passed back to platform-side code that does
the actual extras extraction. Platform code (`androidMain`) supplies
[WatchappNotificationSerializer] as the callback;
[LibPebbleNotificationListener] and [WebViewPKJSInterface] are
updated to the new dispatch shape.
… Inbox)

Adds four opt-in field names to the notification subscription API for
the standard heavyweight Android notification styles:

  bigPictureBase64    BigPictureStyle photo, downsampled to fit a byte cap
  mediaMetadata       Structured MediaSession state (title/artist/album/
                      durationMs/positionMs/playbackState/albumArtBase64)
  messagingMessages   Array of MessagingStyle messages
                      (sender/timestamp/text)
  inboxLines          Array of InboxStyle lines

All four are bandwidth- or CPU-heavy by their nature (Bitmap decoding,
MediaSession lookup, structured array assembly). They're explicitly
NOT in the default field set — watchapps that need them opt in
through subscription.fields.

bigPictureBase64
================

[BigPictureExtractor] reads the picture from
`Notification.EXTRA_PICTURE` (legacy Bitmap-typed slot) or
`Notification.EXTRA_PICTURE_ICON` (API 31+ Icon-typed slot, where
modern apps using BigPictureStyle.bigPicture(Icon) deposit it).
Source bitmaps are typically 1080×1080 or larger; we downsample
adaptively through a 144 → 96 → 64 px ladder, picking the largest
that fits under the configured byte cap (default 8KB). Aspect ratio
preserved.

mediaMetadata
=============

[MediaSessionExtractor] follows the MediaSession.Token deposited
in `Notification.EXTRA_MEDIA_SESSION` by MediaStyle notifications,
opens a MediaController, and reads structured fields. Album art
follows the same downsample ladder and byte cap as bigPictureBase64.

`playbackState` is surfaced as a string ("playing" / "paused" /
"stopped" / "buffering" / etc.) rather than the raw PlaybackState int
so JS consumers can do simple equality checks without dealing with
magic numbers.

messagingMessages / inboxLines
==============================

Both inline in [WatchappNotificationSerializer]. Implemented by
direct parsing of the underlying extras keys (EXTRA_MESSAGES /
EXTRA_TEXT_LINES) using only stable Bundle APIs, rather than via
Notification.MessagingStyle.extractMessagingStyleFromNotification
(which has had inconsistent visibility across compileSdk levels in
practice).

For messaging, `sender` is read from `sender_person.name` on API 28+
(matching what MessagingStyle.Message.senderPerson would return)
falling back to the legacy CharSequence `sender` key on API 24-27.
Both can legitimately be null per MessagingStyle convention (null
sender = "this message is from the user themselves").

Failure mode
============

Per-extractor best-effort. Any failure (missing extras keys, no
MediaSession, drawable load fails, PNG encode fails, etc.) is logged
at trace and the field surfaces as null / empty in the JSON
payload. Throwing extractors are caught by the serializer wrapper.
Never breaks the dispatch pipeline.

Removal events (`posted=false`) skip all heavyweight extraction —
a removed notification has no bitmaps to read and no MediaSession
state worth surfacing.
New opt-in field: `iconExtras`. Walks the notification's extras
Bundle, finds every [android.graphics.drawable.Icon]-typed entry,
rasterizes each to a 32×32 ARGB PNG, base64-encodes it, and emits
a JSON object map keyed by the extras-bundle key.

Output shape:

```
"iconExtras": {
  "android.ongoingActivityNoti.chipIcon": {
    "base64":     "<base64 PNG>",
    "hash":       "<hex>",
    "intrinsicW": <int>,
    "intrinsicH": <int>
  },
  "android.ongoingActivityNoti.nowbarIcon": { ... },
  ...
}
```

Why
===

Android 14 introduced the Ongoing Activity API, which lets apps
render rich live notifications with custom dynamic glyphs. Google
Maps' nav notification is the canonical example — the turn-direction
arrow, lane guidance, and arrival flag are all stored as Icon
parcelables in the notification's extras bundle under keys named
`android.ongoingActivityNoti.chipIcon`,
`android.ongoingActivityNoti.nowbarIcon`,
`android.ongoingActivityNoti.secondIcon`.

These don't surface via the standard Notification.smallIcon /
largeIcon slots, which carry the app's brand glyph rather than
turn-specific iconography. They don't surface via
Notification.contentView / bigContentView RemoteViews either —
Maps' Ongoing Activity notifications don't populate those at all
(verified empirically: contentView=null, bigContentView=null on
every nav notification observed across multiple test trips).

The extras bundle is where these dynamic glyphs live, full stop.
This extractor provides watchapp-side access to them through stable
Android APIs (Bundle.get(key) followed by Icon.loadDrawable(context))
— no reflection, no layout-tree walking, no version-fragile
RemoteViews introspection.

Empirical validation
====================

Live drives with the extractor emitting per-icon hashes confirmed
that Maps publishes a consistent set of distinct glyphs across
turn states. From two separate test trips:

  Continue/straight:   chipIcon hash = 9589c01a   (largeIcon = 8fa71c8d)
  Turn left:           chipIcon hash = 1ad8de36   (largeIcon = 28f6365a)
  Turn right:          chipIcon hash = 5af1cf48   (largeIcon = 9d427851)
  Arrive:              chipIcon hash = 35b76267   (largeIcon = c4939096)
  U-turn:              chipIcon hash = 87d3a7aa   (largeIcon = 487ecffa)

The U-turn hash was confirmed by correlation with Maps' own
voice-prompt timing — `_MAKE_A_UTURN.mp3` fired 12 ms before the
hash transition. So the icon channel reliably encodes turn state
even when the text channel doesn't (during the U-turn maneuver Maps'
turnText was "toward Kelvin Ave", a destination-style phrase that
text-only parsing would have read as "continue straight").

Maps populates the chipIcon, nowbarIcon, and secondIcon slots with
the same glyph at 84×84 source resolution; android.largeIcon is also
populated with the same glyph at 56×56. Watchapps can pick whichever
size suits their target device best — all four hashes change in
lockstep across turn states.

API
===

Field: `iconExtras` (constant ICON_EXTRAS in NotificationSubscription.Field).

Per-icon shape: `{ base64, hash, intrinsicW, intrinsicH }`.

Cost
====

One Drawable load + 32×32 raster + PNG encode per Icon entry. Maps'
nav notifications carry 4 such entries totalling ~3 KB of base64
per dispatch. Negligible vs. the rest of the pipeline.

API gating
==========

Icon class is API 23+; on older platforms the extractor returns an
empty JsonObject (no Icon-typed extras can exist).

Failure mode
============

Per-icon best-effort. Load failures, rasterize failures, encode
failures are logged at trace and that key is skipped in the output;
processing continues for other Icon entries. Never throws.
@killdano killdano changed the title Add notification subscription API for watchapps Per-watchapp field opt-in for notifications May 3, 2026
@killdano killdano changed the title Per-watchapp field opt-in for notifications Add notification subscription API for watchapps May 3, 2026
@sjp4
Copy link
Copy Markdown
Member

sjp4 commented May 22, 2026

meta: this isn't really possible to review because it's based on another branch - you can choose the target branch at the top of the PR to fix that.

We probably don't want a dedicated API for this, IMO. Also this is not possible on iOS, so can never be a cross-platform API.

We are working on a broad set of generic APIs for sharing data sources, on top of which notifications could be exposed natively on Android.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants