Complete mobile playback repairs and Android player UI

Author: YangHYue

ffmpeg/DECISIONS.md

@@ -221,10 +221,12 @@ Patch the OHCodec surface path to expose an opaque `AVOHCodecBuffer` through
 installed `libavcodec/ohcodec_surface.h`. The public functions
 `av_ohcodec_release_buffer()` and `av_ohcodec_render_buffer_at_time()` make one
 native render/drop decision. An atomic flag guarantees that the native output
-is decided at most once. If the last frame reference is released without an
-explicit decision, the fallback unconditionally frees/drops the output rather
-than rendering it. The opaque token retains FFmpeg's decoder reference until
-the frame buffer is released.
+is decided at most once. A repeated decision through another retained frame
+view returns `AVERROR(EALREADY)` rather than reporting a second successful
+queue operation. If the last frame reference is released without an explicit
+decision, the fallback unconditionally frees/drops the output rather than
+rendering it. The opaque token retains FFmpeg's decoder reference until the
+frame buffer is released.
 
 The policy is implemented by
 [`0055-ohcodec-explicit-surface-release.patch`](ports/ffmpeg/0055-ohcodec-explicit-surface-release.patch).
@@ -246,13 +248,18 @@ The installed-package verifier requires the header and both symbols.
 The OHOS package carries a small public overlay ABI that must stay paired with
 the QtAVCore OHCodec backend. This API controls direct surface output only and
 makes no native-buffer interop or zero-copy texture claim. Android and Windows
-packages are unchanged.
+packages are unchanged. QtAVCore maps `AVERROR(EALREADY)` to a stale import;
+the interop must not install a new pending producer association because no
+second native buffer or frame-available callback will be produced.
 
 Validation requires `scripts/build-ohos.ps1`, `cmake/verify-install.cmake`,
 QtAVCore OHOS shared/static target compilation, an installed
 `QtAV::HWOHCodec` consumer, and connected-device HAP coverage that observes
 successful timed presentations and explicit drops. The 2026-08-05 Mate 60 Pro
-run completed 30 timed H.264 presentations and three explicit drops.
+run completed 30 timed H.264 presentations and three explicit drops. The
+2026-08-13 player regression additionally held `legend.mkv` at 0.5x through
+repeated full-screen swapchain recreation with balanced native release and
+callback counts.
 
 ### Retirement condition
 
@@ -535,3 +542,64 @@ Revisit the target-specific override when FFmpeg changes the semantics of
 `--enable-small`, when binary-size limits require a separate dependency
 profile, or when repeatable device measurements show that `-O2` or another
 portable optimization level provides better sustained throughput.
+
+## FD-009: Report a suppressed Android MediaCodec output release as stale
+
+- Date: 2026-08-13
+- Status: Accepted
+- Scope: Android FFmpeg MediaCodec output ownership and the installed public
+  release-result contract
+
+### Context
+
+FFmpeg 8.1.2 shares one atomic release flag across all retained
+`AVMediaCodecBuffer` views. Its MediaCodec helpers correctly invoke the native
+release only once, and also suppress a release after decoder flush changes the
+buffer serial. Both suppressed cases nevertheless returned success. A caller
+could therefore register an AImageReader timestamp association and wait for a
+frame-available callback even though no native output had been queued.
+
+This becomes reachable when Android republishes the same Surface during a
+size/fullscreen refresh. QtAVCore invalidates the old presentation epoch, then
+may redraw the retained latest frame in the new epoch. Epoch/timestamp tracking
+cannot distinguish a real new producer release from FFmpeg's successful no-op;
+the missing callback is an ownership fact, not a slow-producer timeout or an
+unknown coded-size problem.
+
+### Decision
+
+Patch both `av_mediacodec_release_buffer()` and
+`av_mediacodec_render_buffer_at_time()` so zero means the current call actually
+invoked the native output release. Return `AVERROR(EALREADY)` when another view
+already released the shared buffer or decoder flush retired it. Keep all other
+native error values unchanged.
+
+Document the result in installed `libavcodec/mediacodec.h` and require the
+contract from the Android branch of `cmake/verify-install.cmake`. QtAVCore owns
+the higher-level mapping to applied, stale, and failed; the dependency layer
+does not know about renderer epochs or presentation policy.
+
+### Rejected alternatives
+
+- Suppressing redraws by timestamp/frame identity would be an application
+  workaround and could discard a valid frame after a real generation change.
+- Waiting less or treating a timeout as success would retain the false pending
+  association and hide the ownership violation.
+- Reopening MediaCodec or probing the stream dimensions on every surface-size
+  change does not make an already-released output emit another callback.
+- Keeping the result only inside QtAVCore is impossible because the public
+  FFmpeg helper previously erased whether it performed the native release.
+
+### Consequences and validation
+
+Existing callers that check only for negative failure now observe a truthful
+stale result. QtAVCore's bool convenience functions remain source compatible
+but return true only for a real applied release; result-returning APIs preserve
+the three-way distinction.
+
+Any change to this contract must run the directly affected Android dependency
+build and `cmake/verify-install.cmake`, followed by QtAVCore Android
+shared/static builds and installed-package consumers. Connected-device
+validation must exercise a Surface/fullscreen refresh while MediaCodec
+hardware output remains active and confirm that presentation continues without
+an unbounded pending image wait.

ffmpeg/README.md

@@ -79,11 +79,24 @@ vcpkg's current platform model.
 The OHOS FFmpeg overlay installs `libavcodec/ohcodec_surface.h`. Its opaque
 token permits exactly one immediate render, monotonic timed render, or drop of
 an OHCodec surface output without exposing FFmpeg's private decoder structure.
+Repeating a decision through another retained view returns
+`AVERROR(EALREADY)`; callers can therefore reject an old-frame redraw without
+waiting for a surface callback that the already-consumed output cannot emit.
 Releasing the last frame reference without an explicit decision unconditionally
 drops the output; it never implicitly renders an abandoned output.
 It does not expose `OH_AVBuffer`/`OH_NativeBuffer` texture interop. The API and
 retirement criteria are recorded in [FD-004](DECISIONS.md).
 
+The Android FFmpeg overlay also makes MediaCodec output release observable.
+`av_mediacodec_release_buffer()` and
+`av_mediacodec_render_buffer_at_time()` return `0` only when that call actually
+releases the native output. A repeated retained-frame decision, or the first
+decision after decoder flush already retired the output, returns
+`AVERROR(EALREADY)`. This prevents a surface redraw from waiting for an
+`AImageReader` callback that MediaCodec cannot emit. The Android package
+verifier requires this header contract; rationale is recorded in
+[FD-009](DECISIONS.md).
+
 The OHOS overlay also registers `vvc_ohcodec`, maps `AV_CODEC_ID_VVC` to the
 platform VVC MIME, and selects `vvc_mp4toannexb` for MP4 `vvc1` input. The
 wrapper asks `OH_AVCodec_GetCapabilityByCategory(..., false, HARDWARE)` before

ffmpeg/cmake/verify-install.cmake

@@ -81,6 +81,17 @@ if(TRIPLET MATCHES "ohos")
     endforeach()
 endif()
 
+if(TRIPLET MATCHES "android")
+    file(READ "${PREFIX}/include/libavcodec/mediacodec.h"
+         FFMPEG_MEDIACODEC_HEADER)
+    if(NOT FFMPEG_MEDIACODEC_HEADER MATCHES
+       "AVERROR\\(EALREADY\\).*retired by a decoder flush")
+        message(FATAL_ERROR
+            "Android FFmpeg is missing explicit MediaCodec repeat-release status"
+        )
+    endif()
+endif()
+
 file(READ "${PREFIX}/include/libplacebo/config.h" LIBPLACEBO_CONFIG)
 if(NOT LIBPLACEBO_CONFIG MATCHES "#define PL_HAVE_VULKAN 1")
     message(FATAL_ERROR "libplacebo was built without Vulkan support")

ffmpeg/ports/ffmpeg/0055-ohcodec-explicit-surface-release.patch

@@ -15,7 +15,7 @@ new file mode 100644
 index 0000000000..ec0ced37cc
 --- /dev/null
 +++ b/libavcodec/ohcodec_surface.h
-@@ -0,0 +1,55 @@
+@@ -0,0 +1,56 @@
 +/*
 + * OpenHarmony OHCodec surface-output public API
 + *
@@ -52,7 +52,8 @@ index 0000000000..ec0ced37cc
 + *
 + * @param buffer the surface output from AVFrame.data[3]
 + * @param render nonzero to render to the configured surface, zero to drop
-+ * @return 0 on success, a negative AVERROR value on failure
++ * @return 0 on success, AVERROR(EALREADY) when another view already made
++ *         the one permitted decision, or another negative AVERROR on failure
 + */
 +int av_ohcodec_release_buffer(AVOHCodecBuffer *buffer, int render);
 +
@@ -110,7 +111,7 @@ index 7f24d9a943..fb66163e52 100644
 +        return AVERROR(EINVAL);
 +
 +    if (atomic_exchange(&buffer->released, 1))
-+        return 0;
++        return AVERROR(EALREADY);
 +
 +    OH_AVCodec *dec = *buffer->dec_ref;
 +    OH_AVErrCode err = render
@@ -125,7 +126,7 @@ index 7f24d9a943..fb66163e52 100644
 +        return AVERROR(EINVAL);
 +
 +    if (atomic_exchange(&buffer->released, 1))
-+        return 0;
++        return AVERROR(EALREADY);
 +
 +    OH_AVCodec *dec = *buffer->dec_ref;
 +    return ff_oh_err_to_ff_err(

ffmpeg/ports/ffmpeg/0059-mediacodec-repeat-release-status.patch

@@ -0,0 +1,49 @@
+diff --git a/libavcodec/mediacodec.c b/libavcodec/mediacodec.c
+index 4d95dbd0c9..7bc20cb0f9 100644
+--- a/libavcodec/mediacodec.c
++++ b/libavcodec/mediacodec.c
+@@ -100,7 +100,7 @@ int av_mediacodec_release_buffer(AVMediaCodecBuffer *buffer, int render)
+         return ff_AMediaCodec_releaseOutputBuffer(ctx->codec, buffer->index, render);
+     }
+ 
+-    return 0;
++    return AVERROR(EALREADY);
+ }
+ 
+ int av_mediacodec_render_buffer_at_time(AVMediaCodecBuffer *buffer, int64_t time)
+@@ -117,7 +117,7 @@ int av_mediacodec_render_buffer_at_time(AVMediaCodecBuffer *buffer, int64_t time
+         return ff_AMediaCodec_releaseOutputBufferAtTime(ctx->codec, buffer->index, time);
+     }
+ 
+-    return 0;
++    return AVERROR(EALREADY);
+ }
+ 
+ #else
+diff --git a/libavcodec/mediacodec.h b/libavcodec/mediacodec.h
+index 2eb851fc40..87cf9f9d73 100644
+--- a/libavcodec/mediacodec.h
++++ b/libavcodec/mediacodec.h
+@@ -81,8 +81,10 @@ typedef struct MediaCodecBuffer AVMediaCodecBuffer;
+  *
+  * @param buffer the buffer to render
+  * @param render 1 to release and render the buffer to the surface or 0 to
+  * discard the buffer
+- * @return 0 on success, < 0 otherwise
++ * @return 0 when the native output was released, AVERROR(EALREADY) when the
++ * output was already released or retired by a decoder flush, or another
++ * negative AVERROR value on failure
+  */
+ int av_mediacodec_release_buffer(AVMediaCodecBuffer *buffer, int render);
+ 
+@@ -96,7 +98,9 @@ int av_mediacodec_release_buffer(AVMediaCodecBuffer *buffer, int render);
+  *
+  * @param buffer the buffer to render
+  * @param time timestamp in nanoseconds of when to render the buffer
+- * @return 0 on success, < 0 otherwise
++ * @return 0 when the native output was released, AVERROR(EALREADY) when the
++ * output was already released or retired by a decoder flush, or another
++ * negative AVERROR value on failure
+  *
+  * [0]: https://developer.android.com/reference/android/media/MediaCodec#releaseOutputBuffer(int,%20long)
+  */

ffmpeg/ports/ffmpeg/portfile.cmake

@@ -27,6 +27,7 @@ vcpkg_from_github(
         0056-ohcodec-dovi-metadata.patch
         0057-d3d11va-reuse-compatible-decoder.patch
         0058-ohcodec-vvc-decoder.patch
+        0059-mediacodec-repeat-release-status.patch
 )
 
 file(COPY

modern/ARCHITECTURE.md

@@ -337,9 +337,10 @@ MediaCodec + parsed RPU
        │
        ▼
 private AImageReader / AHardwareBuffer
-       ├── Vulkan external image ──► raw Y/Cb/Cr ──┐
+       ├── Vulkan identity (Cr/Y/Cb) ──► .gbr ─────┐
        └── EGLImage + GL_EXT_YUV_target ──────────┤
                                                   ▼
+                                      raw Y/Cb/Cr + parsed RPU
                                       libplacebo DOVI reshape
                                       + color/tone/gamut pipeline
                                                   │
@@ -351,7 +352,10 @@ private AImageReader / AHardwareBuffer
 
 An Android import that cannot prove the raw-component contract is rejected for
 Dolby Vision. Implicit `SurfaceTexture`/external-OES conversion is not treated
-as a valid Profile 5 source.
+as a valid Profile 5 source. Android and OHOS opaque Vulkan identity samplers
+both retain the driver-provided component mapping. The shared normalizer alone
+changes Vulkan's sampled `(Cr,Y,Cb)` to libplacebo's `(Y,Cb,Cr)`; a platform
+interop must not preapply the same rotation.
 
 The Windows path applies the same ordering through D3D11 only:
 
@@ -451,6 +455,16 @@ producer/consumer fence order. Native-buffer queues remain bounded. Cache
 entries may reuse GPU imports by AHardwareBuffer identity, but a codec output
 and its AImage remain retained until the GPU has finished consuming them.
 
+The FFmpeg MediaCodec decision is a three-way ownership boundary: `Applied`
+means the current call released a native output, `AlreadyReleased` means a
+retained peer already decided it or decoder flush retired it, and `Failed`
+means the native operation failed. An AImageReader interop begins its producer
+epoch association before the decision but keeps it only for `Applied`.
+`AlreadyReleased` cancels that provisional association and is stale; no
+Surface buffer or frame-available callback can follow. Presentation/surface
+invalidation may therefore re-offer a retained latest frame without creating
+an impossible pending image wait.
+
 Direct-Surface mode is a third, separate path. It has no application-readable
 texture and therefore cannot be described as Vulkan, OpenGL ES, libplacebo, or
 application-side tone mapping.

modern/DECISIONS.md

@@ -44,6 +44,10 @@ libplacebo.
    The decoded source must remain free of CPU mapping, readback, transfer,
    staging, or upload. An on-GPU external-format normalization pass is allowed
    when it preserves raw Y, Cb, and Cr for libplacebo.
+   The Vulkan identity sampler retains the driver component mapping, producing
+   Vulkan's sampled `(Cr,Y,Cb)` convention; the shared `.gbr` normalizer is the
+   only step that reorders it to `(Y,Cb,Cr)`. Platform interop must not perform
+   the same rotation in advance.
 4. Hardware decode, ZeroCopy, Vulkan, and HDR are the Android player defaults.
    Disabling ZeroCopy while hardware decode remains enabled selects MediaCodec
    direct-Surface presentation and therefore does not use libplacebo.
@@ -240,7 +244,8 @@ ZeroCopy selection, and HDR output policy.
 
 - Date: 2026-08-03
 - Amended: 2026-08-12 to make seek-generation invalidation and late AImage
-  rejection an interop responsibility shared with the Vulkan path
+  rejection an interop responsibility shared with the Vulkan path; 2026-08-13
+  to make an already-consumed MediaCodec output an explicit stale result
 - Status: Accepted
 - Scope: Android player OpenGL ES MediaCodec/AImageReader scheduling and EGL
   release synchronization, plus Android AImageReader producer-generation
@@ -281,6 +286,17 @@ does not wake its bounded image wait; OpenGL has an interop `flush()` but no
 automatic renderer-to-interop invalidation path. Correctness must not depend on
 the Android application calling those interop methods before Player controls.
 
+A surface-size/fullscreen refresh exposed one more ownership ambiguity shared
+with the OHOS failure. Android's renderer invalidates pending presentation
+state even when `surfaceChanged` republishes the same native window. A retained
+latest `VideoFrame` can then be offered again in the new interop epoch. FFmpeg
+8.1.2 atomically suppresses the second native MediaCodec release, but returned
+success for both that duplicate and an output retired by decoder flush.
+QtAVCore therefore installed a pending timestamp association and waited for a
+frame-available callback that could never be emitted. Producer epochs can
+reject late callbacks; they cannot infer that an upstream release reported a
+false success.
+
 ### Decision
 
 1. `MediaCodecOpenGLInterop::queueFrame()` only registers the exact frame key
@@ -316,6 +332,16 @@ the Android application calling those interop methods before Player controls.
    overlapping old render crosses the Player generation boundary. It never
    waits for already-submitted GPU work; imported images keep their existing
    native-fence or timeline-based lifetime.
+9. The repository FFmpeg Android overlay returns `AVERROR(EALREADY)` when a
+   MediaCodec output was already released by another retained frame view or
+   retired by decoder flush. Zero means that the current call actually invoked
+   the native release. Installed-package verification rejects the old
+   ambiguous contract.
+10. `MediaCodecFrame` maps the boundary to `Applied`, `AlreadyReleased`, or
+    `Failed`. Both AImageReader interops cancel their provisional epoch
+    association and return stale for `AlreadyReleased`; direct-Surface output
+    also ignores it as stale. No path creates a pending producer association,
+    retries the frame, probes dimensions, or waits for a callback in that case.
 
 ### Consequences
 
@@ -339,6 +365,10 @@ the Android application calling those interop methods before Player controls.
 - This amendment specializes AD-008's generic exact-deferred-frame and
   pending-producer invalidation contract for Android AImageReader. It does not
   change direct-Surface present/drop ownership.
+- The repeat-release repair is at the shared FFmpeg/native-output ownership
+  boundary, so Vulkan, OpenGL ES, and direct-Surface paths receive the same
+  truth. It adds no redraw suppression, timeout, decoder reopen, pixel copy, or
+  media-size special case.
 
 ### Seek-generation amendment status