feat: original-sized previews for non-web-friendly images

[eligao]

Implementation details:

• Server: extract and keep the original-sized preview image for non-web-friendly images

  • Definition of web-friendly images:
    • From MDN: https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Image_types
    • In the code: https://github.com/eligao/immich/blob/feat/raw-fullsize-preview/server/src/utils/mime-types.ts#L62
  • A new config for this feature: image.fullsize
    • image.fullsize.enabled: toggles original-sized previews
      • When image.extractEmbedded is on, it will attempt to extract and use the JPG preview from RAW images instead of converting from RAW.
  • The result will be saved as a separate AssetFileType.FULLSIZE file.

• OpenAPI: a new AssetMediaSize.FULLSIZE size is available under the viewThumbnail API

  • It returns the AssetFileType.FULLSIZE file when available
  • Or redirects to the original file when it's web-friendly
  • Or otherwise redirects back to AssetFileSize.PREVIEW
    • Happens when an image is not web-friendly and a FULLSIZE preview is not yet generated.
  • HTTP 302 is used here to make full use of caching

• Web: photo-viewer

  • When zoomed in, the browser loads the AssetMediaSize.FULLSIZE preview via the viewThumbnail API
  • This applies to ALL images, instead of previously only loading the original file for web-friendly images.
  • The server will determine what to return or redirect, simplifying the logic on the client, and the client never knows if a fullsize preview is available anyway.
  • This file can be right-clicked and downloaded/copied for sharing.

Some future ideas:

• Load the FULLSIZE image inthe mobile client. Unfortunately I'm not familiar with flutter yet.
• Download the RAW as JPEG with full EXIF preserved, for easier sharing
• Trim the RAW originals and keep the JPEG only, freeing up storage space

[mertalev]

Thank you for the PR! I mostly looked at the server code, where I think it needs some more polishing.

Re: caching, this also affects the case where the user zooms in and no extracted image exists, right? It will load the preview again.

[eligao]

any more comments from maintainers to get this merged, so that I can continue with previews of other images?
@danieldietzler @alextran1502

[alextran1502]

@eligao no rush here

[mertalev]

I think we should wait until we're done with hot fixes for the current release.

[michelheusschen]

For a DNG image with a resolution of 5376x3956 and 41.3MiB in size, the following variants are generated using the default settings:

Variant	Format	Resolution	Size
Thumbnail	webp	5280x3956	887KiB
Preview	jpeg	5280x3956	1794KiB
Original	jpeg	5280x3956	1794KiB

These generated variants seem larger than expected and don't match the configured settings, with the preview and original being identical. It's also strange that the resolution has slightly changed.

[eligao]

For a DNG image with a resolution of 5376x3956 and 41.3MiB in size, the following variants are generated using the default settings:

Variant Format Resolution Size
Thumbnail webp 5280x3956 887KiB
Preview jpeg 5280x3956 1794KiB
Original jpeg 5280x3956 1794KiB
These generated variants seem larger than expected and don't match the configured settings, with the preview and original being identical. It's also strange that the resolution has slightly increased.

nice catch, the file sizes are indeed wrong, let me fix that and add checks to tests.
as for the sizes being slightly different, it's likely due to the crop tags in exif:

[mertalev]

Hi again @eligao! We've been talking about the PR and think there are a few changes it'll need for it to be merge-able.

1. It should apply to all images that aren't web compatible (using the same list as web, ignoring that Safari can view HEIF images)
2. It should be configurable in the image settings, similar to the thumbnail and preview sections
   • There should be an "enabled" toggle that is disabled by default
   • The target format should be configurable in the case where an embedded image is not used and a new image has to be generated
3. Rename the "converted" enum to "fullsize"

[mertalev]

I did a bit of testing with this. The image generation itself works well, but it doesn't get displayed correctly in the client. The initial image shown is the preview and zooming in shows the thumbnail. Also, the "Display original photos" user setting should probably make it load the full-size image directly and skip the preview.

[mertalev]

Just a heads up that I will be away for the next two weeks and won't be able to test this until I'm back. But feel free to continue working on it! I'm happy for it to be merged once the bugs are fixed and someone can confirm that it works as expected.