Home Tab Library Document Count Does Not Match With Actual Number of Documents in Library

[cjpell1]

See similar issue #309
Build 83.0.7649-30836

The number of documents in a libary reflected on the home tab does not match the actual number of documents in the library.
I ran the "recover unregistered PDFs" tool and some broken links (semi-opaque pdf icons) but the numbers still dont add up.

I think my issue is most likely related to 4. on the #309.

My library is actually about ~2500 documents but the home page says ~5000. I compiled a huge collection by copying over some other smaller qiqqa libraries (legacy ones from before open source), as well as importing a couple thousand references from EndNote with the goal of making one large library instead of several smaller ones. Copying from my previous qiqqa libraries didn't cause an issue, and importting from Endnote worked well. The problem with endnote is that it gave me several vanilla references because not every refrence had a PDF.

I think the core of the problem was when I then went to remove all of the vanilla references by moving them into their own library. The goal was to have just pdfs in one and just vanilla in another.

When my library loads it still says its loading the 5000+ documents but then when you go to the actual library its still actually only half that.

[GerHobbelt]

Thanks for the report! Good hint about your historic workflow, using EndNote and the Vanilla references: hadn't considered the Vanilla References as a potential trigger for this issue. Only noticed I had a hard time reproducing the issue and knew I had to come up with a way to approach it through code review; having a description of your process how you arrived at this point will help me discovering what could've gone wrong easier than digging through a ream of code by hand & eye alone. :-) Incidentally, just curious: how do you know your lib is about 2500 large? (Just wondering where you observe the different numbers precisely, e.g. by counting the number of PDFs stored in the library's /documents/ directory tree and/or other means to verify document count. Just checking so that I'll know where to look and how to observe the same as you do, when I dig in and debug this thing. Cheers, Ger Met vriendelijke groeten / Best regards, Ger Hobbelt

…

-------------------------------------------------- web: http://www.hobbelt.com/ http://www.hebbut.net/ mail: ***@***.*** mobile: +31-6-11 120 978 --------------------------------------------------

On Sun, May 30, 2021 at 9:04 PM cjpell1 ***@***.***> wrote: See similar issue #309 <#309> Build 83.0.7649-30836 The number of documents in a libary reflected on the home tab does not match the actual number of documents in the library. I ran the "recover unregistered PDFs" tool and some broken links (semi-opaque pdf icons) but the numbers still dont add up. I think my issue is most likely related to 4. on the #309 <#309>. My library is actually about ~2500 documents but the home page says ~5000. I compiled a huge collection by copying over some other smaller qiqqa libraries (legacy ones from before open source), as well as importing a couple thousand references from EndNote with the goal of making one large library instead of several smaller ones. Copying from my previous qiqqa libraries didn't cause an issue, and importting from Endnote worked well. The problem with endnote is that it gave me several vanilla references because not every refrence had a PDF. I think the core of the problem was when I then went to remove all of the vanilla references by moving them into their own library. The goal was to have just pdfs in one and just vanilla in another. When my library loads it still says its loading the 5000+ documents but then when you go to the actual library its still actually only half that. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#331>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADCIHQ5YVMP2OJTOCKCQMTTQKD4PANCNFSM45ZUIYVQ> .

[cjpell1]

I’ll try to recreate my workflow as best as I can remember.

-Started with a clean install of release 83.0.7649-30836 version prior to that was v79commercial (I think?)
-Created a new library
-Imported ~5300 references from end note following the import instructions
-Opened up a few legacy libraries and selected pdfs (~600) and used copy to library to put them in my new library
-Created a second new library
-In the first library I sorted by "has pdf", selected all those that didn’t (all vanilla references from end note, I know that the legacy library all had pdfs) and selected “move to another library”. There were about 2500 vanilla references that got moved. I think some broken link ones might have gotten moved as well, but my main objective was complete and I was happy.
-The second library was populated and the first library was left with just references with pdfs.

Problem was I still had a lot of “ghost” references. By that I mean in my new library it was saying I still had 5000+ references. But there was no entry, no pdf, nothing for me to select. In addition when it loads up each time I open qiqqa the progress bar says its loading 5000 documents.

I just tried moving a single reference out of the "mega library" into a new empty library and both document counts changed appropriately.

As for how I got these numbers exactly. I'm getting the Home tab number just from the library description “x document(s) in this library”. And for the “actual” numbers I just use Ctrl+A to select all in the open library.

Also I just noticed that when I open the library in question it says it is being filtered by: (2561 matching documents) which is the same amount I get when I use ctrl+a, but not the 5071 it says it contains on the home tab. I can’t see any options for filters that are selected plus that’s coming from a freshly opened qiqqa.

I should add, so far I haven’t noticed any major slow down with a “5000+” library. So while it says its loading all those references Im not sure if they are even really being worked on when I run things like expedition or brainstorm. So really its just an initial loading slow down when I first open qiqqa.

[GerHobbelt]

Commit SHA-1: 90c573a

• fix Home Tab Library Document Count Does Not Match With Actual Number of Documents in Library #331 : cause was the use of the internal FAST COUNT member, which reported the total number of Library DB records, INCLUDING DELETED RECORDS (which can be due to manual delete or document move actions where documents have been moved between libraries: they are then deleted in the source library, while being added in the destination library.

---

To be made available in next Qiqqa release.

[GerHobbelt]

(Keep this one pending until Qiqqa release)