Migrating old auto-exports to newer version

[John-foobar-Doe]

As a follow-up of #2135, I'm also migrating auto-exports from old version (Better BibTeX 1.6.100 installed alongside Zotero 4 for Firefox) to new version (Zotero 6 with Better BibTeX 6.5.2 installed).

Under the data directory of Zotero 4 for Firefox: U:\PortableApps\FirefoxPortableESR\Data\profile\zotero, I have the following better-bibtex related folders and files:

\better-bibtex\cache.json
\better-bibtex\cache.json.1
\better-bibtex\cache.json.2
\better-bibtex\cache.json.3
\better-bibtex\cache.json.4
\better-bibtex\cache.json.5
\better-bibtex\db.json
\better-bibtex\db.json.1
\better-bibtex\db.json.2
\better-bibtex\db.json.3
\better-bibtex\db.json.4
\better-bibtex\db.json.5
better-bibtex-serialized-items.json
betterbibtex.sqlite.1.bak
betterbibtex.sqlite.bak
betterbibtex.sqlite-journal
betterbibtex-lokijs.sqlite.1.bak
betterbibtex-lokijs.sqlite.bak
betterbibtex-lokijs.sqlite.failedmigration
betterbibtex-lokijs.sqlite-journal

And it seems that the auto-exports information is stored at the end of U:\PortableApps\FirefoxPortableESR\Data\profile\zotero\better-bibtex\db.json.

When doing the migration to newer version, I copied all folders and files under U:\PortableApps\FirefoxPortableESR\Data\profile\zotero to U:\PortableApps\ZoteroPortable\Data\profile, and after running Zotero 6, Zotero's own database has been upgraded and I can see all Zotero collections and items are migrated and fine, except that the auto-exports in older version of better-bibtex are not migrated to the newer version, and I noticed that U:\PortableApps\ZoteroPortable\Data\profile\better-bibtex\db.json is unchanged after migration.

So how can I migrate these auto-exports without re-doing the manual exporting of the collections one-by-one?

[retorquere]

If you get me a copy, I could transform them into a script that you can run in Zotero's javascript runner. On windows, that's most likely the simplest.

[retorquere]

https://gist.github.com/efb79f8b2c263f99c27310b3b8cd82b5 has two files, ae.json which is the upgraded collection, and ae.py ae.json path\to\better-bibtex.sqlite will patch it into the database. Zotero must not be running while running ae.py. I've tested this locally and it works for me.

[John-foobar-Doe]

Thanks for that! I can confirm it works, and after running the python script, Zotero/BBT won't auto-update the timestamps any more, unless there is an actual change to the involved collections/items. So thanks very much for your hard work! And I've now closed the issue #2151.

[John-foobar-Doe]

On the other hand, I noticed another issue: BBT no longer supports auto-export of a saved search in Zotero. When I tried to export a saved search in Zotero to Better BibTeX and select "Keep updated", I got the following error:

Moreover, in the db.json that I uploaded (db.json.zip) before, there is one auto-export of a saved search:

				{
					"collection": "search:2",
					"path": "U:\\Documents\\Work\\Research\\LaTeXCV\\CVDraft\\MyPub4.bib",
					"translatorID": "ca65189f-8815-4afe-8c8b-8c7c15f0edca",
					"exportCharset": "UTF-8",
					"exportNotes": false,
					"useJournalAbbreviation": true,
					"status": "done",
					"updated": "5/1/2022, 3:05:41 AM",
					"meta": {
						"revision": 12527,
						"created": 1473793853057,
						"version": 0,
						"updated": 1651370741056
					},
					"$loki": 17
				},

where the "collection": "search:2" is converted to

      "type": "collection",
      "id": 2,

in the conversion ae.json and also in the previous conversion JavaScript. So after upgrading to BBT 6.7.x, this auto-export's corresponding collection - as shown in Zotero & BBT preferences UI - is a very old collection (whose "id" is 2, the 2nd collection I created more than 10 years ago...), but not the saved search in Zotero (which is supposed to be "search:2", i.e., the 2nd saved search I created in Zotero's database).

So, did you decide not to support auto-export of saved search any more (in BBT version > 1.6.100)? (Please let me know should I open another discussion for this, as this discussion is a bit too long.)

[retorquere]

#904

[John-foobar-Doe]

Oh, thanks. I didn't navigate to the 2nd page of the issue board. And now I've joined the discussion there.