fix(youtube,tiktok): request identity encoding on the media reads - #1853
Merged
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
giladresisi
force-pushed
the
fix/youtube-chunk-transient-errors
branch
from
August 7, 2026 14:44
4ba5359 to
8b7cb0b
Compare
The YouTube and TikTok chunked uploaders read the stored video back from media storage with their own HEAD/ranged GET helpers, and those helpers never asked for identity encoding. Every other media read in the project does: #1835 added `accept-encoding: identity` to SocialAbstract's mediaSize / mediaChunk / mediaStream and to the providers that read media themselves, because a transformed (compressed) response loses its Content-Length - and Cloudflare answers a range request on an object with no Content-Length by returning the full body with a 200 instead of the requested 206. That is very likely what has been failing ~1GB YouTube uploads: a single 200 out of the ~128 ranged reads a 1GB video needs is thrown as BadBody, which the v1.0.6 workflow treats as a permanent platform rejection, so the whole post is marked ERROR even though the upload session is still resumable. The gap looks like an artifact of merge ordering rather than a decision: #1813 introduced these two bespoke helpers on Aug 3, and #1835 swept identity encoding across the project on Aug 4 - by then the sweep had been written against a tree that did not contain them, so YouTube and TikTok were missed while X and LinkedIn (bespoke at the time too) were consolidated onto the shared helpers and fixed. Behaviour on a non-206 is deliberately left untouched: if the customer report recurs after this ships, the next step is to reconsider treating the documented 200-with-full-body answer as transient (a plain Error the workflow retries and resumes from the committed byte offset) instead of as a permanent failure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
giladresisi
force-pushed
the
fix/youtube-chunk-transient-errors
branch
from
August 7, 2026 15:54
8b7cb0b to
5a9b1cc
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What kind of change does this PR introduce?
Bug fix.
Why was this change needed?
The customer report
A customer reported (Discord) that ~1 GB YouTube uploads through Postiz fail often enough that they built a heartbeat agent just to check whether their posts actually went out. Their own investigation was accurate: the same hosted files alternated between passing and failing without being changed, an immediately-failed post succeeded a minute later after restarting the same row, and larger files fail more often without there being a fixed size limit.
Root cause
YoutubeProvider.youtubeMediaSize/youtubeChunkStreamandTiktokProvider.tiktokChunkStreamread the stored video back from media storage with their own HEAD and ranged-GET helpers, and none of them asked for identity encoding. Every other media read in the project does — #1835 addedaccept-encoding: identitytoSocialAbstract.mediaSize/mediaChunk/mediaStreamand to the providers that read media themselves.That header matters here for two reasons, both documented by Cloudflare:
Content-LengthHTTP header of responses delivered to website visitors". Which compression is applied is driven by "the values visitors provide in theaccept-encodingrequest header" — so asking foridentityis the client-side way to keep the response untransformed and its length intact. (The origin-side equivalent Cloudflare documents iscache-control: no-transform, which "must be set by the origin — it cannot be added in client requests".)Content-Lengththe edge cannot serve a byte range: "If the origin response does not include theContent-Lengthheader, the cache will return the full content with an HTTP 200 response", instead of the 206 the uploader asked for.A 200-with-full-body is currently thrown as
BadBody, whichpostWorkflowV106treats as a permanent platform rejection: the post is marked ERROR immediately with no retry. A 1 GB video needs ~128 ranged reads, so one such answer kills the whole post even though the YouTube upload session itself is still perfectly resumable. That matches the reported symptom, including why it scales with file size and why an identical file behaves differently between attempts.Why the header was missing here
This looks like an artifact of PR merge ordering rather than a decision. #1813 introduced these bespoke YouTube/TikTok helpers on 2026-08-03. #1835 swept identity encoding across the project on 2026-08-04, but it had been written against a tree that did not yet contain them — so X and LinkedIn (which had their own bespoke ranged reads at the time, from #1796 and #1786) were consolidated onto the shared helpers and fixed, while YouTube and TikTok were missed.
SocialAbstractand every provider that reads media through it has had the header since; these three call sites were the only ones left without it.Other information:
Errorinstead ofBadBody, which the v1.0.6 workflow already knows how to handle — it re-probes the upload session and resumes from the exact committed byte offset, with a bounded consecutive-failure budget. That behaviour was prototyped and verified end-to-end against a storage server injecting Cloudflare-style 200s: pre-change the post died at the third chunk; with the reclassification the same upload survived five injected failures, resuming byte-exactly each time, and published in 98 s. It is being held back deliberately in favour of trying the smaller, in-pattern fix first.youtubeMediaSizeand callSocialAbstract.mediaSizeinstead, as TikTok already does. The generic helper is a strict superset: it also rejects a non-ok HEAD and a zero/NaN length (a failed HEAD can still carry theContent-Lengthof its error body, which would poison the chunk-count math downstream).youtubeChunkStream/tiktokChunkStreaminto a singlemediaChunkStreamonSocialAbstract, next tomediaChunk(ranged, buffered) andmediaStream(whole file, streamed). The ranged-stream shape is the one combination the abstract does not offer today, which is why both providers hand-rolled it; a shared helper would have been swept automatically by fix: stream provider media uploads instead of buffering files in memory #1835 and would keep the non-206 classification in one place. Note the two callers do not want identical error semantics — YouTube can probe and resume its session, TikTok cannot — so the shared helper should throw one error type and let each caller classify it.Checklist:
Put a "X" in the boxes below to indicate you have followed the checklist;
🤖 Generated with Claude Code