regenerate parts database not clearing part search cache

[mMerlin]

Current Behaviour

After changing parts library path, and regenerating parts database, part search errors with unable to find parts, referencing parts from the previous parts library.

Fritzing remembered the previous search criteria and it is still in the input box. This results in the Unable to find popup with the part names and paths pointing to the test parts library. Clicking OK on the popup, and executing the search clears the error. It looks like the previous search results are being remembered, then the initial parts view update fails because the parts are not in the freshly regenerated database.

This is using flatpak, but looks like an application code issue, not anything to do with the running environment.

Build:
Version 0.9.4 (CD-498 2019-12-01) 64 [Qt 5.13.2]

Operating System:
Fedora 31 Workstation
Flatpak 1.4.3
org.fritzing.Fritzing
Commit: c595162a1d06f01eab9b66092c7ca68518974b9887295cd59950c439bfba3a72

Steps to reproduce:

• start Fritzing application with --parts pointing to my test repository
• do Help ¦ Check for updates… to verify I have the correct parts library
• select Breadboard view
• do Part ¦ Regenerate part database …
• start Fritzing application with same --parts path
• do Help ¦ Check for updates… to verify I have the correct parts library
• click find and search for my parts
• place and test my parts
• exit Fritzing
• start Fritzing application with --parts pointing to my regular repository
• do Help ¦ Check for updates… to verify I have the correct parts library
• select Breadboard view
• do Part ¦ Regenerate part database …
• start Fritzing application with same --parts path
• do Help ¦ Check for updates… to verify I have the correct parts library
• click the find icon in the parts view

Expected Behaviour

The search results from the previous library should be cleared when regenerating parts database. Either the initial search should be blank, or it should just show the parts that match in the current database and library.

[vanepp]

If you are using a local parts repository, you don't want to do the check for updates. That will attempt to sync the parts data base to the parts repository on github. It should fail with an error message that you have made local changes to the parts database though. You normally should not do a regenerate parts database as it is meant to correct corruption not do updates. If you want to change to a test repository you need to delete the user directories before doing so (Fritzing will recreate them when it starts, note that this will delete your mine parts bin so you may want to keep a copy of the original directories by renaming them rather than deleting them.) On linus the user directories are in

~/Documents/Fritzing/parts
~/.config/Fritzing

[mMerlin]

The error message is actually why I have been using the check for updates. It quickly tells me if I have pointed to the intended repository. I was have some spelling mistake issues for the parts path that Fritzing silently ignored. It also then ignored the specified parts path, falling back to the system wide default. Causing confusion about where MY parts were.

I am doing the regenerate to get the database synchronized with the changed parts library (path). I do not know of another way to do that. The parts I am testing are intended to end up in core parts, so I do not have any issues with user directories. Those are staying empty. Or they were supposed to be. I just noticed a downloaded example imported and sitting in /user.

If I were creating fzpz part files, I would not need to switch parts libraries. Just delete and reimport the part. But I want a library / repository that I can safely play in, and have a stable environment for other things.

[KjellMorgenstern]

Can you check your .config/Fritzing/Fritzing.conf ? On the first start, Fritzing stores paths in this config file. Maybe this file overrides the --parts option in some use cases. For example, the --parts option seems to affect the config file only on first start, when the config file is created, not on subsequent calls.

Just to be clear, the --parts option is meant to configure the core library, usually by developers or distributors.nThis should not affect your Documents folder in any way (user data is stored there).
If you just want to use a custom bin or part, simply open it and it will be added to you Documents/Fritzing folder.

[mMerlin]

I do not see anything in the config file that directly references the parts library. I see [bins] section with app and more location entries pointing to the system default folder /usr/share/fritzing/bins/. I suspect this configuration file got carried over from an earlier, non flatpak, install. I see version=0.9.2b.11.19.8d2d5970658f0bed09c661c9ea9a515b5f40f44c, which is the version that dnf install fritzing provides, that I installed in a toolbox container while exploring. Trying to get something to work. Separate issue reported, and solved, for the flatpak version had it refusing to place parts from the library. Turned out to be a wayland versus x11 issue, but in the mean time, I tried other options.

However, the --parts flag is working on a per run basis. I can see that I have different libraries after changing the path, and regenerating the database. Though this probably accounts for why I have to specify the path on every run. It is not actually getting stored in the config file.

You are probably going to tell me to delete/uninstall all versions, cleanup left overs, then install only one, specify --parts on the first run, and check the config file. That could cleanup some of this, but I still want to be able to override the parts library for specific cases. What I am working on is not just a custom part. It is a family of parts, with shared svg files. That I want in the core. Once stabilized, I'll push them to my fritzing-part repository fork, and do a pull request. But in the mean time, I need a working area parts library I can switch to, plus a standard parts library that is not 'messed up' by my partial changes. If necessary, I can do a total split, by move the development to a virtual machine, but it looked to me like one off overrides with the --parts flag would do what I needed. Along with parts database rebuilds every time I switched between parts development and circuit diagrams.

Count me as a developer. Currently just for the parts, but could be for code later too.

[KjellMorgenstern]

Yes, indeed that would be the workaround: clean up everything, especially the config file. I think it should work even by just renaming the config file. The config file is created when you first run fritzing. I don’t know how flatpak handles this, and maybe it comes with a preset config, but our package does not.

In the long term, and the sake of sanity, the parts option should also cover the use case you describe.

[vanepp]

I tend to make parts via the fzpz route, then when finished the testing, create a local parts repo from the fzpz files, move aside the user directories install the new repo and regenerate the database then test the parts. One of the changes in the new check parts script (which isn't finished yet) will be to take an fzpz file, unzip it check it and optionally write it to a selected repository (converting the file names in the process) which should make that easier. In the long term I agree that an easier way to select test repositories is a good bet, but I think it will need code changes (or at the very least understanding and documenting what the current code does which AFAIK doesn't exist.) either of which will take time and resources and which so far no one has done.

[mMerlin]

@KjellMorgenstern I wiped out the no longer needed toolbox (0.9.2b) version of Fritzing, cleaned out the Documents/Fritzing/parts and .config/Fritzing folder, then started the flatpak (0.9.4b) version with --parts pointing to my local development repo. Since this was now 'first time in', I did a regenerate part database. The Documents/Fritzing/parts folder structure got recreated (empty), but not .config/Fritzing. It seems that the flatpak environment uses its own internal configuration storage area, that is invisible. At least with my current knowledge of flatpak.

I then started the flatpak fritzing again, this time without specifying a --parts path. The Check for Updates gave a permissions message pointing to /app/share/fritzing/parts. Which actually maps to .local/share/flatpak/app/org.fritzing.Fritzing/current/active/files/share/fritzing/parts. So it did not remember the previous path. Closing and starting with the --parts override got to where things were supposed to be.

That 'not remembering' would seem to be because of the hidden config file. I did not find it in the flatpak application folder structure When I first went, I got a message about a missing user part that was previously in Documents/Fritzing/parts. So it has some other history information that I did not clear out. Which, if --parts is only supposed to write to the config on first run (when Fritzing.config does not yet exist), was still hanging around some place, so my first time in wasn't really. I'll need to remove and reinstall the flatpak version to test that further.

But ignoring a lot of those details, regenerate parts database should still clear parts cache information being stored by the applications. That sort of information immediately 'goes stale' when the database starts to be rebuilt.

[mMerlin]

@vanepp This would be a bit big for that structure. When done what I know about, there will be:

• 1 icon svg file
• 4 breadboard svg files
• 2 schematic svg files
• 8 pcb svg files
• 16 part fzp files

4 sizes, 2 polarities, 2 footprint options.

Getting all of those to match the part#_description_package_pins_spacing_color_view.svg and package_pins_spacing_pcb.svg naming convention is easier to do to start with. Instead of renaming at least all of the svg files, and editing all of the fzp to match.

[vanepp]

@mMerlin Yep, that would be a pain done as fzpzs OK, so I guess you do need the rebuild the database. For now I expect you will need to clear the user directories to make that work successfully but in the longer term (assuming we can find/fund manpower to do it) it would be a useful enhancement.

"Getting all of those to match the part#_description_package_pins_spacing_color_view.svg and package_pins_spacing_pcb.svg naming convention is easier to do to start with. Instead of renaming at least all of the svg files, and editing all of the fzp to match."

Where do I find these two? I've never heard of them before. In case you are not aware of it, the variant field in the fzp needs to be of the form "variant variant-name" to be recognized by Inspector to switch between the variants. I.e:

<property name="variant">variant 2</property>

will switch via the pull down menu where

<property name="variant">2</property>

will display the variant in Inspector but not let you change it to swap in one of the other variants assuming you want to allow that. Assuming there is a variant 1 to swap to of course and the family properties are identical. Got bit by that lately. Yet another undocumented feature.

[mMerlin]

@vanepp 4. File Naming Conventions on fritzing.org/fritzings-graphic-standards/

re variant. Yep. That is what I use to allow toggling between polarities in the family. Actually, it looks like do not need the variant prefix (for the value). Just needs to be a text string instead of (only) a number. Not quite what I am doing, but

<property name="variant">positive</property>
<property name="variant">negative</property>

I 2 different fzp files with the same family should work. That is, allow the user to switch between the parts from the inspector.

Going to be posting over in the forum about what more can be done with properties. Something I am trying in addition to variant is not working the way I expected.

[KjellMorgenstern]

Related to #3454

The title of the issue is "regenerate parts database not clearing part search cache".
I see the discussion here is about file naming conventions, is this still related to the cache issue?

I have observed issues when editing parts, and changes not showing up in Fritzing by myself.
My workaround usually is to have a hash in the parts, and update this each time I want to test the part in Fritzing.

[mMerlin]

@KjellMorgenstern The issue is the cache. The thread went to discussion about a different work flow that avoided the need for regenerating the database, thus avoiding the error, which then went to why I was using the existing work flow, which slipped over to the file naming conventions for parts.

So it all connected, but ended up rather far from the actual code related issue.

I looked a the referenced related issue. I don't see why caching has anything in particular to do with where the parts library actually exists. Whatever the location, there will probably be some caching. And that caching should be cleared when either the path to the library changes, or library is updated in place. Not a major bug, since it cleans itself up after the next search.