TIKA-4869: Emit the video of a motion photo as an embedded document

[dschmidt]

A motion photo carries a short video after the image, and Tika already reads the XMP that says where it is, but not the video itself.

Two formats, one arithmetic: a Motion Photo lists its parts in Container:Directory, the primary image first and the video last, with nothing behind it; the older MicroVideo gives Camera:MicroVideoOffset, the number of bytes from the end of the file to the start of the video. Either way the video ends at the end of the file, so its start follows from its own declared length. The same computation covers HEIC motion photos, whose video sits in a trailing mpvd box whose 8 byte header is the primary item's padding; that path is reachable through the same extractor, but I have no sample file to prove it with.

The image parsers emit it as an ATTACHMENT embedded document named motion-photo.<ext>, parsed like any other embedded document. ATTACHMENT rather than INLINE because the video is not another rendering of the picture: it is literally appended behind the image, the JPEG ends at its EOI marker, and a reader that ignores the trailer still sees a complete image. It also carries what the still cannot: sound, motion, its own timeline and metadata.

What is at the computed offset is typed by content, without the declared Item:Mime as a hint: a hint would let a wrong length pass as a video, since the mime magic returns the hint when it recognizes nothing. When the declared length does not fit the file, or the bytes there are not a video, nothing is emitted and nothing is recorded either. That is not a corrupt file worth an exception: sharing a motion photo out of the Android gallery keeps the XMP and takes the video, and the XMP that promised it is in the metadata for a client to see. The tests cover it with a file shaped the way sharing leaves one.

The name follows the file's own declaration: motion-photo.mp4 where Item:Mime says so, and plain motion-photo for a MicroVideo, which declares nothing. The parse that follows knows the format for certain, but by then the name is fixed: the recursive wrapper takes the embedded metadata when the embedded parse ends, so naming it afterwards has no effect (measured). Guessing an extension from detection seemed worse than carrying none.

The XMP keys are keyed by namespace URI rather than by the prefix the document happens to use, the way Camera: already was. An XMP prefix is the writer's choice, and exiftool rewrites these two namespaces to GContainer:, which would have silently cost the video on any file it has touched.

The two test files declared a video in their XMP without carrying one; they now hold a small real MP4, and a third one covers the shared-without-video case.

Verified on a server built from this branch: both files answer with the video as an embedded document, /unpack returns its bytes unchanged (identical SHA-256 to the appended MP4), a plain JPEG is untouched, and a file whose video has been stripped yields no embedded document and no warning.

https://issues.apache.org/jira/browse/TIKA-4869

[tballison]

This is really nice.

My bot has maybe one or two things worth changing. The hygiene thing with MediaType.parse should be fixed now in main(?).

Let me know what you think, and thank you!

  1. edge-case — the offset gate accepts any recognized type, not a video; that makes a JPEG recursive

  MotionPhoto.java:222 — if (type == null || MediaType.OCTET_STREAM.equals(type)) { return; }

  Two measured consequences:

  - Replace the trailing 1583 bytes of testJPEG_MotionPhoto.jpg with ASCII text → a second document is emitted, text/plain, named motion-photo.mp4. CHANGES.txt and the PR description both say nothing is emitted "when the bytes there are not a video". That is
    not what the code does.
  - outer || inner, where outer is the fixture with Item:Length rewritten to |inner| and inner is a motion photo → 3 documents: depth 1 image/jpeg named motion-photo.mp4, depth 2 its own video/mp4. Chained 1500 deep (~1.4 KB per link, 2.19 MB input), a plain
    AutoDetectParser.parse → StackOverflowError.

  The depth cap that saves the /rmeta path is AbstractRecursiveParserWrapperHandler.MAX_DEPTH = 100, which only exists on the RecursiveParserWrapper path. ParseRecord.maxEmbeddedDepth defaults to -1, so /tika, tika-app text output and the Tika facade have no
  guard. Even at the capped 100 levels each level re-spools the whole remaining suffix via tis.getPath(), so a 50 MB input writes gigabytes of temp files.

  Fix is one line and is what the PR already claims: require "video".equals(type.getType()). It matches both specs (the item's semantic is the video), and it closes the recursion for good, since a video parser never re-enters this code. Add the missing test —
  a trailer that is recognized but not a video; testDeclaredVideoIsNotOne only covers unrecognized zeros.

  2. edge-case — extension from the declaration, content type from detection

  MotionPhoto.java:221-233. The two can disagree (motion-photo.mp4 on text/plain, above). The PR's rationale — "the parse that follows knows the format for certain, but by then the name is fixed" — is about the parse; the detected type is already in hand at
  line 221, before the name is set at 229. Also RESOURCE_NAME_EXTENSION_INFERRED=true is documented as "inferred by Tika (e.g. from content type detection) rather than provided by the original document", and here it is the document's declaration. If the
  declaration is deliberately preferred, worth saying why detection-at-221 was rejected.

  3. edge-case (low) — IOException on the video path fails the whole image

  file(tis) maps getPath()'s IOException to null, but Files.size (:217), detect (:221) and region (:239) propagate out of the image parser. That contradicts the stated contract ("a video out of reach is no reason to fail an image that parsed"). Cheap fix:
  extend the guard over the rest of the body.

  Hygiene
  
  - MediaType.parse(metadata.get(... "Item:Mime")) (:289) interns an attacker-controlled string into the static, never-evicted MediaType.SIMPLE_TYPES — the same class of thing TIKA-4826/4862 are removing. The value is only used for an extension, and
    EmbeddedDocumentUtil.getExtensionForMediaType(String) already takes a String and normalizes itself; dropping the parse removes the interning and a field.
  - CHANGES.txt "the bytes there are not a video" — fixed by finding 1.
  - parseEmbedded(..., outputHtml=false) with no <div class="embedded"> of its own; EMFParser, the closest precedent, passes true.
  - 1583 is hardcoded in four places across two tests; derive it from the fixture.

[tballison]

On 1., I'm frankly not certain that we need to require video. If someone sticks a PDF after an image, we should extract it. I do want to avoid the Stackoverflow, though.

[dschmidt]

Thank you, and all of it is in, pushed just now.

On 1., I agree, and I did not add the video requirement. The recursion is closed at its source instead: the branch below an emitted trailer is not searched for a trailer of its own. A PDF behind an image is still emitted, an image behind an image too, but it cannot chain. The new test builds the nesting your bot measured and asserts two documents where it got three.

Worth its own ticket, maybe: what made this reachable at all is that ParseRecord.maxEmbeddedDepth defaults to -1, so /tika, tika-app and the facade have no depth guard, only the RecursiveParserWrapper path does. Our chain is closed now, but any parser that emits something derived from its own input has the same shape.

On 2., the reason for preferring the declaration is that detection cannot tell these apart: an MP4 with the isom brand types as video/quicktime (that is TIKA-3646), so the extension from detection would be .mov on a file whose container says video/mp4, and MicroVideo declares no type at all, where detection would guess one. So the declaration still wins, but only within the same top level type. Where the two disagree about the kind of file it is, as with your text trailer, the bytes win and RESOURCE_NAME_EXTENSION_INFERRED is set, since that is the case the property describes.

On 3., Files.size, the detection and the region stream are inside the guard now: locating the trailer is one step that yields nothing rather than throwing, and so is opening it.

On the hygiene list: the declared Item:Mime stays a string and is only looked up for an extension through EmbeddedDocumentUtil, so neither MediaType.parse nor MimeTypes.forName sees it. MediaType.parse on main still caches into SIMPLE_TYPES, capped at 10000, and MimeTypes.forName still registers what it does not know, which is TIKA-4826. CHANGES.txt now says "not recognized", the embedded parse writes its own div like EMFParser and RawTiffParser, and the tests read the declared length from the fixture.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

This PR adds support for extracting the trailing video from Google/Android Motion Photos (and older MicroVideo) and emitting it as an embedded ATTACHMENT, while also stabilizing XMP raw keys by canonicalizing certain namespace prefixes.

Changes:

• Add MotionPhoto trailer extraction and wire it into image parsing to emit the appended video as an embedded document.
• Canonicalize Google container XMP namespace prefixes in XmpSaxFlattener so downstream consumers can rely on stable Container:/Item: paths.
• Add unit + integration tests covering extraction, naming, detection behavior, and “shared without video” cases.

Reviewed changes

Copilot reviewed 8 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tika-parser-xmp-commons/.../XmpSaxFlattenerTest.java	Adds a test ensuring canonical Container:/Item: paths even when the source uses rewritten prefixes (e.g., ExifTool).
tika-parser-xmp-commons/.../XmpSaxFlattener.java	Introduces canonical prefix mapping for specific namespaces and applies it to element/attribute path construction.
tika-parser-image-module/.../MotionPhotoXmpTest.java	Updates expected XMP values in tests to match new fixture/trailer sizes.
tika-parser-image-module/.../MotionPhotoVideoTest.java	Adds focused unit tests for motion photo trailer extraction and edge cases.
tika-parser-image-module/.../MotionPhoto.java	Implements trailer location, detection, naming, and embedded-document emission logic.
tika-parser-image-module/.../AbstractImageParser.java	Calls MotionPhoto.extract(...) so trailing motion-photo video is emitted during image parsing.
tika-parsers-standard-integration-tests/.../MotionPhotoVideoIntegrationTest.java	Adds an integration test asserting MP4 typing + some parsed metadata in a full parser set.
CHANGES.txt	Documents the new Motion Photo video embedded-document behavior.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.