Fix hasProcesses lookup and improve ProjectForm performance

[BartChris]

Fixes #6777
Fixes #6776

[solth]

@BartChris just one question concerning the following remark

It seems you have to reindex the processes by hand now, to have the project name change reflected in the index and be able to search for it without prefix (search over all fields))

which you made here #6776 (comment)

Doesn't that mean we introduce a new issue when merging this pull request? Currently, processes can be found by updated project titles immediately after saving the project, don't they? Do you think performance gains should have higher priority over bug-free functions? Maybe I just missed something, but that seems like a suboptimal tradeoff to me. In my opinion we should have a solution for this new issue ready (for example making use of the mentioned @IndexEmbedded) when changing the code to improve performance.

[BartChris]

@solth Speaking about effects you are right. But i think the reindexing of processes happened just by accident. Because we triggered some Hibernate behaviour, not by clear intent. If we want a reindex, it should by done in a better way, not by breaking the app with catastrophic runtimes because we reindex thousands of processes during a UI cycle, as it seems to me.

But you are right in a way, so i am conflicted and have no clear answer. My general stance on this issue is, that we allow project search by the database and the indexing of project title in a catch all field brings little value, so i would not index project title in the free search which is mostly intended for metadata. But this is debatable as well.

Edit: Using the normal: "project:ABC" will still work after this change, as projetc search uses the database not the index. Only in the case of someone just putting "ABC" and expects to find projects which are still indexed as "DEF" under their new name "ABC" in the index wil break.

[BartChris]

Looking at the entity and indexing (Process, ProcessKeywords class) code i think it was not intentionally considered that on a change of a project title, we need to reindex all processes, which belong to the project. This reindexing just "happened" based on some internal Hibernate behavior.
Unfortunately indexing a parent attribute in a child is very hard - independent of the search engine used - when the collection of children is large. I am afraid that even when using Hibernate Search annotations (@IndexEmbedded) performance for reindexing huge amounts of processes will be bad. The right way to do something like that probably includes async processing in a seperate thread.

This leaves us in a constellation where we have only non-optimal options

• We can leave the system as it is right now. But i consider the current state of the system not just a performance problem. The numbers given by @henning-gerhardt indicate a broken system, which is dysfunctional. 300 seconds just to open a form risks running into timeouts and is broken in my opinion as the project page is basically unusable.
  In addition to that we rely on Hibernate doing things in the background based on some annotations which maybe change their behaviour in the future. See in general: Resetting process state results in Java out of memory error #5813 (comment).

• We can merge the PR and have a functional project page, but we lose the reindexing.

• We can try to find a good reindexing solution on project change, but i suppose implementing it will require a lot of effort. And i consider the net benefit very low as this would only be about indexing project titles in the free text search, which is in my opinion not the right mechanism to search by project name.

[solth]

Edit: Using the normal: "project:ABC" will still work after this change, as projetc search uses the database not the index. Only in the case of someone just putting "ABC" and expects to find projects which are still indexed as "DEF" under their new name "ABC" in the index wil break.

I understand and agree that still being able to find processes using the updated project title via a search/filter with "project:" prefix should be sufficient.

Considering that it is possible to immediately find processes via general index search using a recently updated project title right now, though, this should be documented as a regression when we merge this pull request. After all, we cannot be sure if there are some institutions who specifically use this functionality and consider it a deal breaker if it does not work anymore in the next release.

I would prospose to open an issue or discussion for this afterwards to gauge how important this feature is after all and if we have to somehow re-implement it or not.

@BartChris please rebase this pull request against the current main branch, though, as there have been some changes made to the code recently.