feat: original-sized previews for non-web-friendly images #14446

eligao · 2024-12-02T12:38:32Z

Implementation details:

Server: extract and keep the original-sized preview image for non-web-friendly images
- Definition of web-friendly images:
  - From MDN: https://developer.mozilla.org/en-US/docs/Web/Media/Formats/Image_types
  - In the code: https://github.com/eligao/immich/blob/feat/raw-fullsize-preview/server/src/utils/mime-types.ts#L62
- A new config for this feature: image.fullsize
  - image.fullsize.enabled: toggles original-sized previews
    - When image.extractEmbedded is on, it will attempt to extract and use the JPG preview from RAW images instead of converting from RAW.
- The result will be saved as a separate AssetFileType.FULLSIZE file.
OpenAPI: a new AssetMediaSize.FULLSIZE size is available under the viewThumbnail API
- It returns the AssetFileType.FULLSIZE file when available
- Or redirects to the original file when it's web-friendly
- Or otherwise redirects back to AssetFileSize.PREVIEW
  - Happens when an image is not web-friendly and a FULLSIZE preview is not yet generated.
- HTTP 302 is used here to make full use of caching
Web: photo-viewer
- When zoomed in, the browser loads the AssetMediaSize.FULLSIZE preview via the viewThumbnail API
- This applies to ALL images, instead of previously only loading the original file for web-friendly images.
- The server will determine what to return or redirect, simplifying the logic on the client, and the client never knows if a fullsize preview is available anyway.
- This file can be right-clicked and downloaded/copied for sharing.

Some future ideas:

Load the FULLSIZE image inthe mobile client. Unfortunately I'm not familiar with flutter yet.
Download the RAW as JPEG with full EXIF preserved, for easier sharing
Trim the RAW originals and keep the JPEG only, freeing up storage space

mertalev

Thank you for the PR! I mostly looked at the server code, where I think it needs some more polishing.

Re: caching, this also affects the case where the user zooms in and no extracted image exists, right? It will load the preview again.

server/src/repositories/media.repository.ts

server/src/services/media.service.ts

eligao · 2024-12-06T06:15:56Z

any more comments from maintainers to get this merged, so that I can continue with previews of other images?
@danieldietzler @alextran1502

alextran1502 · 2024-12-06T06:17:12Z

@eligao no rush here

mertalev · 2024-12-06T06:20:14Z

I think we should wait until we're done with hot fixes for the current release.

michelheusschen · 2024-12-07T07:50:47Z

For a DNG image with a resolution of 5376x3956 and 41.3MiB in size, the following variants are generated using the default settings:

Variant	Format	Resolution	Size
Thumbnail	webp	5280x3956	887KiB
Preview	jpeg	5280x3956	1794KiB
Original	jpeg	5280x3956	1794KiB

These generated variants seem larger than expected and don't match the configured settings, with the preview and original being identical. It's also strange that the resolution has slightly changed.

eligao · 2024-12-07T08:22:27Z

For a DNG image with a resolution of 5376x3956 and 41.3MiB in size, the following variants are generated using the default settings:

Variant Format Resolution Size
Thumbnail webp 5280x3956 887KiB
Preview jpeg 5280x3956 1794KiB
Original jpeg 5280x3956 1794KiB
These generated variants seem larger than expected and don't match the configured settings, with the preview and original being identical. It's also strange that the resolution has slightly increased.

nice catch, the file sizes are indeed wrong, let me fix that and add checks to tests.
as for the sizes being slightly different, it's likely due to the crop tags in exif:

mertalev · 2024-12-10T23:19:19Z

Hi again @eligao! We've been talking about the PR and think there are a few changes it'll need for it to be merge-able.

It should apply to all images that aren't web compatible (using the same list as web, ignoring that Safari can view HEIF images)
It should be configurable in the image settings, similar to the thumbnail and preview sections
- There should be an "enabled" toggle that is disabled by default
- The target format should be configurable in the case where an embedded image is not used and a new image has to be generated
Rename the "converted" enum to "fullsize"

server/src/repositories/media.repository.ts

server/src/services/media.service.ts

e2e/src/api/specs/asset.e2e-spec.ts

open-api/immich-openapi-specs.json

server/src/repositories/media.repository.ts

server/src/services/media.service.ts

server/src/repositories/media.repository.ts

open-api/immich-openapi-specs.json

server/src/services/asset-media.service.ts

mertalev · 2025-02-04T08:56:29Z

Just a heads up that I will be away for the next two weeks and won't be able to test this until I'm back. But feel free to continue working on it! I'm happy for it to be merged once the bugs are fixed and someone can confirm that it works as expected.

mertalev · 2025-03-05T15:16:50Z

Hey @eligao, just checking in! Were you able to take a look at the issues raised above? Is there anything I can do to help bring this PR to the finish line?

eligao · 2025-03-05T16:18:50Z

@mertalev thanks for checking in, sorry I've been busy recently but I still remember this PR. Let me update the branch and see if I can fix the small issues you mentioned above :)

…review

eligao · 2025-03-05T20:15:55Z

@mertalev I've updated the branch and did some small fixes and tests:

The initial image shown is the preview and zooming in shows the fullsize.
The "Display original photos" user setting should not make it load the full-size image directly, otherwise it slows the browser down.

mertalev · 2025-03-06T00:46:06Z

Nice! I'll do some testing with it tomorrow and see if we can get this merged :)

…review

eligao · 2025-03-19T16:23:01Z

@mertalev just checking in, any further suggestions for this PR? 👀

mertalev

The PR works as expected, tested on web with embedded preview enabled and disabled and format set to WebP and JPEG. Disabling the full-size feature and regenerating deletes both the file and the DB entry. Requesting a full-size image where there is none redirects to the preview and uses the cached preview (tested on Chrome).

…is enabled

mertalev · 2025-03-24T18:40:44Z

I made two tweaks to the PR:

Continue displaying the full-size/original image after zooming back out. It makes things smoother and avoids jarring effects like going from an HDR image back to SDR after zooming out.
Display full-size/original image when opening the viewer if "display original photos" is enabled. You mentioned that it's slower, but this is the current behavior for e.g. high-res web compatible images. For users who want to see the originals whenever possible (hence opting in to this setting), it's better to avoid requiring a zoom.

eligao · 2025-03-26T18:15:57Z

@mertalev thanks for the tweaks :)

Nice catch, I totally missed the case when zooming back out.
Sounds good as long as it is the current behavior.

eligao · 2025-03-27T04:25:23Z

@danieldietzler any more suggestions before we can get this merged? 👀

mertalev · 2025-03-27T05:27:32Z

We're planning to include it in 1.131.0, just needs testing from @alextran1502

…#14446) * feat(server): extract full-size previews from RAW images * feat(web): load fullsize preview for RAW images when zoomed in * refactor: tweaks for code review * refactor: rename "converted" preview/assets to "fullsize" * feat(web/server): fullsize preview for non-web-friendly images * feat: tweaks for code review * feat(server): require ASSET_DOWNLOAD premission for fullsize previews * test: fix types and interfaces * chore: gen open-api * feat(server): keep only essential exif in fullsize preview * chore: regen openapi * test: revert unnecessary timeout * feat: move full-size preview config to standalone entry * feat(i18n): update en texts * fix: don't return fullsizePath when disabled * test: full-size previews * test(web): full-size previews * chore: make open-api * feat(server): redirect to preview/original URL when fullsize thumbnail not available * fix(server): delete fullsize preview image on thumbnail regen after fullsize preview turned off * refactor(server): AssetRepository.deleteFiles with Kysely * fix(server): type of MediaRepository.writeExif * minor simplification * minor styling changes and condensed wording * simplify * chore: reuild open-api * test(server): fix media.service tests * test(web): fix photo-viewer test * fix(server): use fullsize image when requested * fix file path extension * formatting * use fullsize when zooming back out or when "display original photos" is enabled * simplify condition --------- Co-authored-by: mertalev <[email protected]>

eligao requested a review from danieldietzler as a code owner December 2, 2024 12:38

github-actions bot added 🖥️web 🗄️server labels Dec 2, 2024

danieldietzler added the changelog:enhancement label Dec 2, 2024

eligao changed the title ~~feat/raw fullsize preview~~ feat: original-sized previews for RAW images Dec 2, 2024

eligao force-pushed the feat/raw-fullsize-preview branch from f2b631a to 54b91b4 Compare December 2, 2024 16:35

mertalev reviewed Dec 2, 2024

View reviewed changes

server/src/services/media.service.ts Outdated Show resolved Hide resolved

mertalev reviewed Dec 2, 2024

View reviewed changes

server/src/services/media.service.ts Outdated Show resolved Hide resolved

server/src/services/media.service.ts Outdated Show resolved Hide resolved

eligao force-pushed the feat/raw-fullsize-preview branch 5 times, most recently from 076d59d to a7a9b4d Compare December 3, 2024 21:24

eligao commented Dec 3, 2024

View reviewed changes

server/src/services/media.service.ts Show resolved Hide resolved

eligao force-pushed the feat/raw-fullsize-preview branch from a7a9b4d to 34592b3 Compare December 3, 2024 21:29

eligao force-pushed the feat/raw-fullsize-preview branch from ebcfe9c to e22a123 Compare December 16, 2024 20:37

eligao changed the title ~~feat: original-sized previews for RAW images~~ [WIP] feat: original-sized previews for non-web-friendly images Dec 16, 2024

mertalev reviewed Dec 16, 2024

View reviewed changes

server/src/repositories/media.repository.ts Outdated Show resolved Hide resolved

server/src/repositories/media.repository.ts Outdated Show resolved Hide resolved

server/src/services/media.service.ts Outdated Show resolved Hide resolved

mertalev reviewed Dec 16, 2024

View reviewed changes

e2e/src/api/specs/asset.e2e-spec.ts Outdated Show resolved Hide resolved

open-api/immich-openapi-specs.json Outdated Show resolved Hide resolved

eligao commented Dec 17, 2024

View reviewed changes

eligao changed the title ~~[WIP] feat: original-sized previews for non-web-friendly images~~ feat: original-sized previews for non-web-friendly images Dec 17, 2024

eligao force-pushed the feat/raw-fullsize-preview branch from 8d43411 to 4cee0c7 Compare December 17, 2024 16:45

eligao commented Dec 23, 2024

View reviewed changes

server/src/services/asset-media.service.ts Outdated Show resolved Hide resolved

eligao added 2 commits January 27, 2025 15:25

Merge branch 'main' into feat/raw-fullsize-preview

c26eaa9

chore: reuild open-api

bd62c2f

eligao added 4 commits March 6, 2025 01:37

Merge remote-tracking branch 'upstream/main' into feat/raw-fullsize-p…

277f6de

…review

test(server): fix media.service tests

ccb3318

test(web): fix photo-viewer test

1836d10

fix(server): use fullsize image when requested

ac75f56

Merge remote-tracking branch 'upstream/main' into feat/raw-fullsize-p…

d2c8f24

…review

mertalev added 3 commits March 24, 2025 12:15

Merge branch 'main' into feat/raw-fullsize-preview

8630ce0

fix file path extension

9f8dcdc

formatting

dc37e0b

mertalev approved these changes Mar 24, 2025

View reviewed changes

use fullsize when zooming back out or when "display original photos" …

d9a0d5a

…is enabled

simplify condition

48c3806

Merge branch 'main' into feat/raw-fullsize-preview

d11f92c

mertalev merged commit 5c80e87 into immich-app:main Mar 31, 2025
40 checks passed

eligao deleted the feat/raw-fullsize-preview branch March 31, 2025 18:30

hackrush01 mentioned this pull request Apr 2, 2025

HEIC/HEIF original images not loading in Safari 18.3.1 #17334

Closed

4 tasks

mertalev mentioned this pull request Apr 2, 2025

fix(web): use original image if web compatible #17347

Merged

Uh oh!

feat: original-sized previews for non-web-friendly images #14446

feat: original-sized previews for non-web-friendly images #14446

Uh oh!

Conversation

eligao commented Dec 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Implementation details:

Some future ideas:

Uh oh!

mertalev left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eligao commented Dec 6, 2024

Uh oh!

alextran1502 commented Dec 6, 2024

Uh oh!

mertalev commented Dec 6, 2024

Uh oh!

michelheusschen commented Dec 7, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eligao commented Dec 7, 2024

Uh oh!

mertalev commented Dec 10, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mertalev commented Feb 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mertalev commented Mar 5, 2025

Uh oh!

eligao commented Mar 5, 2025

Uh oh!

eligao commented Mar 5, 2025

Uh oh!

mertalev commented Mar 6, 2025

Uh oh!

eligao commented Mar 19, 2025

Uh oh!

mertalev left a comment

Choose a reason for hiding this comment

Uh oh!

mertalev commented Mar 24, 2025

Uh oh!

eligao commented Mar 26, 2025

Uh oh!

eligao commented Mar 27, 2025

Uh oh!

mertalev commented Mar 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eligao commented Dec 2, 2024 •

edited

Loading

michelheusschen commented Dec 7, 2024 •

edited

Loading

mertalev commented Feb 4, 2025 •

edited

Loading

mertalev commented Mar 27, 2025 •

edited

Loading