Thumbnails #1262
Thumbnails #1262
Conversation
leejuyuu
force-pushed
the
thumbs
branch
2 times, most recently
from
dac42a0 to
bee51e0
Compare
|
Please rebase the branch so we have fresh Windows build on appveyor. |
There was a problem hiding this comment.
This is a lot slower for me (reading from NAS which tops out around 300MB/s). Which is not very fast by today's standards (nvme etc well over 1GB/s).
I'm not sure what to do about this but I think there needs to be a multi-threaded option. Maybe we can scale the number of threads based on how much time is spent in I/O vs decoding/scaling.
If the computed thumbs were cached somewhere then maybe we don't need multi-threading.
|
@novomesk This branch has been rebased. @scrubbbbs Did you observe slow down when loading files without EXIF thumbnail, or was it slow even for those with EXIF thumbnail? |
It is quite a bit slower in both cases, of course not as much for EXIF since it can load much faster. However much more noticeable when loading thousands of files in the grid view. I've just noticed the embedded thumb is also not transformed correctly anymore, orientation "2" is not flipped anymore. Easy to see in formats_testset/orientations/with-thumb |
Do you mean scrolling down the grid view? I think it should only load what is currently visible, which is ~50 files for my screen. It's strange that EXIF thumbnails loads almost instantly for me (on NVME SSD), so they should not be compute bound...
Thanks! I've fixed that. |
|
Scrolling is where this is most apparent, resorting is not too bad. I prepared a couple of clips to show the difference. This folder has about 5000 files, they are small jpgs, most have exif thumbs but many do not. I tried to scroll both at the same rate. Clip 1 new version] thumbs.mp4Clip 2 git master thumbs-threads.mp4If the grid view could reschedule items when scrolling, the difference would be a lot less. Maybe after the view scrolls we set a timer to reschedule provided there is not another scroll in the meantime? I think I would prefer not reordering EXIF vs computed thumbs, just seems weird having gaps in the thumbs that fill in later. I guess I don't understand the benefit of rescheduling based on missing EXIF thumbs or not. |
|
I sorry but I haven't seen the videos. |
Thanks, reencoded to fit the github limit now. |
|
I seems to me from the videos, that no-longer visible thumbnails are still being loaded before those active on the screen. |
This is legacy behavior AFAIK and yes I agree it is probably most of the apparent slowdown. I would suggest fixing this first as it will have a bigger impact. However, this clearly shows that loading thumbnails is not always going to be I/O bound as @leejuyuu has asserted here with a single-threaded implementation. Given it the difference is around 2x in this testcase, multi-threaded thumbnail support should probably be retained. |
The byte array parameter of this function is always empty. Remove it for simplicity.
- Return early if scaling is not needed in DkImage::createThumb - Change duplicated code in DkThumbNail::computeIntern into DkImage::createThumb call. - Remove unused maxThumbSize parameter of DkThumbNail::computeIntern - Remove unused mMaxThumbSize member of DkThumbNail
- Remove the unused forceLoad flag of DkThumbNail::compute() - Absorb common DkImage::createThumb() call into DkThumbNail::computeIntern(). - Use the QtConcurrent::run() feature to pass arguments instead of using lambda. The copying is handled by Qt.
This is part of the effort to move DkThumbNailT out of DkImageContainer. The widgets that needs thumbnails should make their own. - Add function to create DkThumbNailT from DkImageContainer because the creation accesses internal state for zip files. - Create the DkThumbNailT inside DkThumbSaver instead of using DkImageContainer::getThumb(). - Remove the excess DkThumbSaver::loadNext call
This is part of the effort to move DkThumbNailT out of DkImageContainer. The widgets that needs thumbnails should make their own.
|
One more regression, when toggling the thumbs panel thumbs are re-computed - the were retained before. Memory usage indicates they are retained, but are forced to recompute. Update: also affects history/recent files panel. I'll try to get through the code review tomorrow, so far it is looking good. Overall, this PR is a big improvement, just have to solve the zip and performance regressions. |
| : DkBatchContent(parent, f) | ||
| { | ||
| setObjectName("DkBatchInput"); | ||
| createLayout(); | ||
| createLayout(thumbLoader); | ||
| setMinimumHeight(300); |
There was a problem hiding this comment.
I think it would be better for DkThumbLoader to use singleton pattern and not require storing a reference to it in everything that uses thumbs. Or else each thing can construct its own loader and the shared data can be a singleton member of DkThumbLoader
There was a problem hiding this comment.
I think using a singleton (i.e., a global) to avoid passing values should be generally avoided. Dependency injection is more verbose but much more explicit. For example, any class or function can use the singleton, while I can define what classes can use DkThumbLoader.
There was a problem hiding this comment.
I think it is adding clutter. There are many global objects in nomacs and we should not have to pass them through constructors down through many layers to get them where they are needed.
If it was only used in one or two places that would be fine but this type of object will be used all over like settings.
There was a problem hiding this comment.
I do not want the object to be used all over the places, and that is why I pass them explicitly.
|
This seems to fix #857, the heap does not grow provided the thumbnail ribbon is disabled. Do you have any plans for reducing the memory footprint of thumbnails? |
- Load thumnail directly from memory, no need to use the DkThumbNailT and the extra threading. - There is no need to store DkImageContainerT, so remove them. - Remove QImage.setText calls that are no longer needed.
Use QtConcurrent::run to call function directly without using the DkThumbNailT to manage state. Also, this avoids storing the thumbnails in memory in the computing process.
Pass whether the thumbnail is loaded from EXIF data.
Dispatch thumbnailLoaded signal on DkThumbLoader when a full image is loaded to the viewport. This brings back the thumbnail with higher res and allows the thumbnails to be aware of saved image edits.
After some testing, it was found that the image decoding part when loading thumbnails is significant enough that single-threaded implementation results in performance regression. This adds (bounded) concurrency back, while preserve the current interface. Some decision notes: - Down-scaling large images in the main thread freezes the UI visible when increase the concurrency, so move the scaling out of main thread. - Because of the concurrency, there is no need to prioritize EXIF thumbnails anymore. - Use a combination of queue and map to keep track of the thumbnail requests, instead of using the queued signal connection. This allows deduplication of the requests and makes cancelation possible.
The original design makes DkThumbLabel request thumbnail load when `paint` method found no available thumbnail. This make thumbnail loading slow when scrolling quickly in a directory with thousands of images because we need to wait for all the thumbnails scrolled past to load even if they are not visible anymore. Modify the DkThumbLabel::cancelLoading() method to properly handle cancelation of requests. Subscribe for scroll of DkThumbsView and cancel the thumbnail requests from DkThumbLabel that are not visible.
For similar reason as the previous commit, cancel load for the obscured thumbnails in DkFilePreview if the thumbnail is not currently visible.
There was a crash at setAlphaChannel call that was probably due to a race condition when converting the mask image to grayscale8. See nomacs#1262 (comment) Specify the format when constructing the mask to bypass this conversion.
- Pass references - Return modified image instead of modifying pointer
Fix DkFilePreview current index not found when viewing zip files. Images inside zip files originally have encoded file path. However, `DkImageContainerT::checkForFileUpdates` later replace the file path with the path inside the archive, causing the path comparison to always fail. DkThumbLabel is also affected. Record a copy of the original path that is not mutated inside DkImageContainer to fix this.
Check if the thumbnail load was failed before requesting to avoid infinite loop.
These booleans control whether the thumbnail is to be requested. This fixes a weird bug that some thumbnails are never loaded and the gray boxes in the DkThumbScrollWidget.
Release the loaded thumbnail when the widget is closed.
Should be fixed now. |
This is intentional. The thumbnails should not hold memory when they are not used anymore. There might still be some leaks, I am trying to track them down. |
|
I did not find leaks related to the thumbnails.
I'd like to test QPixmapCache. Currently, we store the thumbnail (with max size 400 px). The thumbnail labels are usually smaller. If we can store the paint result instead, we can further save some memory. |
This is a pretty big change, I think a lot of users wouldn't like it. To reduce the memory footprint we could use smaller thumbnails (400px is a lot) and keep fewer of them..we don't need more than a screenful or two of the grid view. |
|
I have a problem with the following sequence:
Repeat this several times and on step 2 , one of these bugs occurs
Update: this seems to occur more frequently if there are only a small number of thumbs showing (maybe 20 or so). Stack Trace
Thread 1 "nomacs" received signal SIGSEGV, Segmentation fault.
0x00007ffff5fe385d in QDir::fromNativeSeparators(QString const&) () from /lib/x86_64-linux-gnu/libQt5Core.so.5
(gdb) bt
#0 0x00007ffff5fe385d in QDir::fromNativeSeparators(QString const&) ()
at /lib/x86_64-linux-gnu/libQt5Core.so.5
#1 0x00007ffff5ff834c in QFileInfo::QFileInfo(QString const&) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#2 0x00007ffff7d4ec7b in nmc::DkImageLoader::load (this=0x555555c6a8e0, filePath=...)
at /home/test/mnt/hd4/sw/nomacs/ImageLounge/src/DkCore/DkImageLoader.cpp:862
#3 0x00007ffff7cc18d0 in nmc::DkViewPort::loadFile (this=0x5555558396f0, filePath=...)
at /home/test/mnt/hd4/sw/nomacs/ImageLounge/src/DkGui/DkViewPort.cpp:1697
#4 0x00007ffff7bb2bb3 in nmc::DkCentralWidget::loadFile (this=0x5555558f3b50, filePath=..., newTab=false)
at /home/test/mnt/hd4/sw/nomacs/ImageLounge/src/DkGui/DkCentralWidget.cpp:1129
#5 0x00007ffff7bbdbac in QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<QString const&, bool>, void, void (nmc::DkCentralWidget::*)(QString const&, bool)>::call
(f=(void (nmc::DkCentralWidget::*)(nmc::DkCentralWidget * const, const QString &, bool)) 0x7ffff7bb2b52 <nmc::DkCentralWidget::loadFile(QString const&, bool)>, o=0x5555558f3b50, arg=0x7fffffffc760)
at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:152
#6 0x00007ffff7bbce7f in QtPrivate::FunctionPointer<void (nmc::DkCentralWidget::*)(QString const&, bool)>::call<QtPrivate::List<QString const&, bool>, void>
(f=(void (nmc::DkCentralWidget::*)(nmc::DkCentralWidget * const, const QString &, bool)) 0x7ffff7bb2b52 <nmc::DkCentralWidget::loadFile(QString const&, bool)>, o=0x5555558f3b50, arg=0x7fffffffc760)
at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:185
#7 0x00007ffff7bbc475 in QtPrivate::QSlotObject<void (nmc::DkCentralWidget::*)(QString const&, bool), QtPrivate::List<QString const&, bool>, void>::impl
(which=1, this_=0x555555d5ebd0, r=0x5555558f3b50, a=0x7fffffffc760, ret=0x0)
at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:418
#8 0x00007ffff6112e16 in ??? () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#9 0x00007ffff7b4f6e0 in nmc::DkThumbScene::loadFileSignal (this=0x55555610f8d0, _t1=..., _t2=false)
at /home/test/mnt/hd4/sw/nomacs-testing/build/nomacsCore_autogen/XJ2BUHI2UY/moc_DkThumbsWidgets.cpp:619
#10 0x00007ffff7ca08a0 in QtPrivate::FunctorCall<QtPrivate::IndexesList<0, 1>, QtPrivate::List<QString const&, b-----T--Ty------Type <RET> for more, q to quit, c to continue without paging--
ool>, void, void (nmc::DkThumbScene::*)(QString const&, bool) const>::call
(f=(void (nmc::DkThumbScene::*)(const nmc::DkThumbScene * const, const QString &, bool)) 0x7ffff7b4f66c <nmc::DkThumbScene::loadFileSignal(QString const&, bool) const>, o=0x55555610f8d0, arg=0x7fffffffc940)
at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:158
#11 0x00007ffff7c9f995 in QtPrivate::FunctionPointer<void (nmc::DkThumbScene::*)(QString const&, bool) const>::call<QtPrivate::List<QString const&, bool>, void>
(f=(void (nmc::DkThumbScene::*)(const nmc::DkThumbScene * const, const QString &, bool)) 0x7ffff7b4f66c <nmc::DkThumbScene::loadFileSignal(QString const&, bool) const>, o=0x55555610f8d0, arg=0x7fffffffc940)
at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:197
#12 0x00007ffff7c9cd47 in QtPrivate::QSlotObject<void (nmc::DkThumbScene::*)(QString const&, bool) const, QtPrivate::List<QString const&, bool>, void>::impl (which=1, this_=0x555556a26550, r=0x55555610f8d0, a=0x7fffffffc940, ret=0x0)
at /usr/include/x86_64-linux-gnu/qt5/QtCore/qobjectdefs_impl.h:418
#13 0x00007ffff6112e16 in ??? () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#14 0x00007ffff7b4ef82 in nmc::DkThumbLabel::loadFileSignal (this=0x555555fe63b0, _t1=..., _t2=false)
at /home/test/mnt/hd4/sw/nomacs-testing/build/nomacsCore_autogen/XJ2BUHI2UY/moc_DkThumbsWidgets.cpp:374
#15 0x00007ffff7c860bf in nmc::DkThumbLabel::mouseDoubleClickEvent (this=0x555555fe63b0, event=0x7fffffffcce0)
at /home/test/mnt/hd4/sw/nomacs/ImageLounge/src/DkGui/DkThumbsWidgets.cpp:1014
#16 0x00007ffff709ea86 in QGraphicsItem::sceneEvent(QEvent*) () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#17 0x00007ffff70c2a99 in ??? () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#18 0x00007ffff70cb67f in ??? () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#19 0x00007ffff70d5903 in QGraphicsScene::event(QEvent*) () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#20 0x00007ffff6d6bd45 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#21 0x00007ffff60d8118 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#22 0x00007ffff70f395f in QGraphicsView::mouseDoubleClickEvent(QMouseEvent*) () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#23 0x00007ffff6db1b21 in QWidget::event(QEvent*) () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#24 0x00007ffff6e62647 in QFrame::event(QEvent*) () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#25 0x00007ffff60d7e82 in QCoreApplicationPrivate::sendThroughObjectEventFilters(QObject*, QEvent*) ()
at /lib/x86_64-linux-gnu/libQt5Core.so.5
#26 0x00007ffff6d6bd35 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#27 0x00007ffff6d746b0 in QApplication::notify(QObject*, QEvent*) () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#28 0x00007ffff60d8118 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#29 0x00007ffff6d72874 in QApplicationPrivate::sendMouseEvent(QWidget*, QMouseEvent*, QWidget*, QWidget*, QWidget**, QPointer<QWidget>&, bool, bool) () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
--Type <RET> for more, q to quit, c to continue without paging--c
#30 0x00007ffff6dcaa39 in ??? () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#31 0x00007ffff6dcdfbf in ??? () at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#32 0x00007ffff6d6bd45 in QApplicationPrivate::notify_helper(QObject*, QEvent*) ()
at /lib/x86_64-linux-gnu/libQt5Widgets.so.5
#33 0x00007ffff60d8118 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#34 0x00007ffff65459c8 in QGuiApplicationPrivate::processMouseEvent(QWindowSystemInterfacePrivate::MouseEvent*) ()
at /lib/x86_64-linux-gnu/libQt5Gui.so.5
#35 0x00007ffff6517bfc in QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
at /lib/x86_64-linux-gnu/libQt5Gui.so.5
#36 0x00007fffeeefcd06 in ??? () at /lib/x86_64-linux-gnu/libQt5XcbQpa.so.5
#37 0x00007ffff45145b5 in ??? () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#38 0x00007ffff4573717 in ??? () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#39 0x00007ffff4513a53 in g_main_context_iteration () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#40 0x00007ffff6135279 in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) ()
at /lib/x86_64-linux-gnu/libQt5Core.so.5
#41 0x00007ffff60d6a7b in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#42 0x00007ffff60df3e8 in QCoreApplication::exec() () at /lib/x86_64-linux-gnu/libQt5Core.so.5
#43 0x000055555557290f in main (argc=1, argv=0x7fffffffdc18) at /home/test/mnt/hd4/sw/nomacs/ImageLounge/src/main.cpp:370 |
| mThumbsScene->cancelLoading(); | ||
| mThumbsScene->updateThumbs({}); | ||
| } |
There was a problem hiding this comment.
This has not fixed the memory issue for me. I'm testing with a folder that has ~4000 images and I make the icons really small so they all have to loaded in. nomacs uses 2.4G resident/4.2G virtual memory to load the thumbnails, then does not release it when hiding.
|
|
||
| if (!visible) { | ||
| mThumbs.clear(); | ||
| } |
There was a problem hiding this comment.
Note that with fade widgets, setVisible() does not immediately hide the widget, it could be redrawn many times to fade it out. I did not notice a problem with this however, perhaps it was not redrawn or mThumbs.clear() didn't have the desired effect.
No description provided.