Skip to content

Conversation

@pmauduit
Copy link
Member

@pmauduit pmauduit commented Apr 28, 2025

geOrchestra/geonetwork checklist

  • PR only involves cherry-picked commits from upstream.
  • PR contains custom code which will soon be available in an upstream release and can be overriden => mention core-geonetwork version if possible.
  • PR contains custom geOrchestra code, which need to be verified during future migrations.

Note: this PR suggests to change the way we are building GN in geOrchestra. About the docker profile, it attaches the build docker-maven-plugin goal to the package maven phase. It is then unecessary to mvn docker:build anymore, as mvn package will do, as long as the docker profile is activated.

Also, the datafeeder-integration has been made optional upstream, we probably want it as a standard in the geOrchestra fork, but if we don't want to diverge much from upstream, I suggest to keep it "optional", but enforce its build in our CIs (GHA, buildbot). It is just a matter of activating a maven plugin anyway.

tylerjmchugh and others added 30 commits June 19, 2024 13:14
* Add settings fields for localized emails

* Add spring bean to initialize feedback email locales from settings field

* Modify settings manager to update feedback locales when the settings fields are saved

* Create classes for localized emails, their components, and the components' parameters

* Implement lang in getIndexField

* Localize workflow status emails

* Localize metadata publication emails

* Localize user feedback emails

* Localize RegisterApi emails

* Localize PasswordApi emails

* Localize WatchListNotifier emails

* Localize MailApi emails

* Update migration script to only insert settings fields if not present

* Add static enum imports for readability

* Log a warning when a locale is invalid or missing

* Update log modules and messages

* Trim language codes defined in settings to handle spaces after commas

* Rename translation follows label to translation follows text for consistency

* Add back resource bundle 'messages' that was unused before merging main

* Add logic to break from loop when email subject and text messages fail
* Associated resource / Consistent labels

Create one directive to display associated resources labels with icon.

The directive is used in all places:
* Record view (list and card layout)
* Editor side panel
* Editor popup

Add the possibility to customize an association label by adding custom translation key in `${lang}-custom.json`

eg.
```json
{
  "crossReference-study": "Publication",
  "crossReference-campaign": "Campaign",
  "crossReference-reuse": "Reuse"
}
```

* Editor / Associated resource popup / Fix element using same DOM id.
* Record view / Improve alignement of source datasets

* Update UtilityDirective.js

* Update related.html

* Associated resource / Consistent labels / Missing template.

* Record view / Associated / Fallback on URL if no title. Do not display twice the list
…#8152)

* Fix canViewRecord function so that it returned the workflow record.
Prior to this fix, it would always return working copy record if it exists.
Also updated getInternalId to have better error handling

* Update services/src/main/java/org/fao/geonet/api/ApiUtils.java

Co-authored-by: Jose García <[email protected]>

* Fix number check based on review - was checking wrong value.

---------

Co-authored-by: Jose García <[email protected]>
* removed double translation

* additional fix for double translation
When using tools to convert OpenAPI to client code eg.

```bash
npx openapi-ts -i ./src/gapi/gapi.json -o src/gapi
```

Some reserved word may create invalid generated code. eg. when building typescript:

```
 TS1102: 'delete' cannot be called on an identifier in strict mode. [
```
…e (#8178)

* Add info logs to make transaction of working copy merge more traceable

* build fix

* Update listeners/src/main/java/org/fao/geonet/listener/metadata/draft/DraftUtilities.java

Co-authored-by: Jose García <[email protected]>

* Update listeners/src/main/java/org/fao/geonet/listener/metadata/draft/DraftUtilities.java

Co-authored-by: Jose García <[email protected]>

* Update datastorages/cmis/src/main/java/org/fao/geonet/api/records/attachments/CMISStore.java

Co-authored-by: Jose García <[email protected]>

* Update datastorages/cmis/src/main/java/org/fao/geonet/api/records/attachments/CMISStore.java

Co-authored-by: Jose García <[email protected]>

* add more logs for delete folder

* build fix

* build fix

* Update core/src/main/java/org/fao/geonet/api/records/attachments/FilesystemStore.java

Co-authored-by: Jose García <[email protected]>

---------

Co-authored-by: Jose García <[email protected]>
Co-authored-by: Ian <[email protected]>
Related to JQuery update.
…able (#8229)

* Standard / ISO19139 / Formatter / Do not display extent if none available

* Update schemas/iso19139/src/main/plugin/iso19139/formatter/xsl-view/view.xsl

Co-authored-by: Jose García <[email protected]>

---------

Co-authored-by: Jose García <[email protected]>
Show a warning to the user when an error happens while deleting one 
or more metadata records.
If an error happens while using the delete, validate or other
option of the editor dashboard metadata actions menu then remove
the text "Please wait" and the spinner from the button to allow
to perform a new action.
Improve documentation and properly target node to delete in case the
forEach loop element is not the one to remove.

eg.
```xml
<section name="Axe - Time"
         forEach="/mdb:MD_Metadata/mdb:spatialRepresentationInfo/*/msr:axisDimensionProperties/*[msr:dimensionName/*/@codeListValue = 'time']"
         del="ancestor::msr:axisDimensionProperties">
   <field xpath="msr:dimensionSize"/>
   <field xpath="msr:resolution"/>
</section>
```

Before the change, the remove button was not displayed.
Most of the time the loop element is the one to delete and this case was
fine.
Fixes "md:scope will be gmd:MD_Scope when it should be gmd:DQ_Scope" 

See geonetwork/core-geonetwork#8220
…etadataUuid}/formatters/{formatterId:.+} (#8269)

This affect PDF exports on working copies.
…69 (#8262)

Use the UI language for the metadata information, when exporting a metadata selection to CSV / PDF.
The API for these services accept a new optional parameter language that defaults to English as previously.

Fixes #7969.
* Zoom to map popup remains active on non-map pages. Fixes #8260

Fixes also wrong values for add alert delay, provided in some pages in milliseconds, but the method expects seconds.

* Zoom to map popup remains active on non-map pages - remove alerts only when switching from the map

* Increase timeout to hide layer added popup to 15 seconds
In the register user, metadata feedback and user feedback forms, sometimes the settings with the 
recaptcha keys takes a bit longer to load making the recaptcha widget not to load. This commit fixes 
that updating the values of the recaptcha settings when the settings have finished loading.

It also resets the recaptcha widget if there is any problem returned by the server
after sending the form.
…sfer options block are used. (#8281)

* Standard / ISO19139 / Fix removal of online source when multiple
transfer options block are used.

Follow up of geonetwork/core-geonetwork#7431

* Fix online resource update/delete so that it supports multiple gmd:MD_DigitalTransferOptions blocks.

---------

Co-authored-by: Ian Allen <[email protected]>
* Update batch PDF export to skip working copies

* Capitalize boolean operators in lucene query
Before only maintenance frequency was displayed but more information may be provided about the maintenance (eg. custom frequency, next update date, note).
On application startup, the translation pack may report an error like the following

```
org.fao.geonet.api.exception.ResourceNotFoundException
        at org.fao.geonet.api.standards.StandardsUtils.getCodelistOrLabel(StandardsUtils.java:72)
        at org.fao.geonet.api.standards.StandardsUtils.getLabel(StandardsUtils.java:55)
        at org.fao.geonet.api.tools.i18n.TranslationPackBuilder.getStandardLabel(TranslationPackBuilder.java:217)
```
due to a missing french translation for element `DQ_EvaluationMethodTypeCode`

Follow up of geonetwork/core-geonetwork#7180
- other harvester have same or higher loglevel as well
- helps to identify invalid metadata
…317)

* Repository Citation.cff metadata for DUI regestration with zenodo

* Update CITATION.cff

Minor updates, additions and clean up

* Update CITATION.cff

Simplified the list to only include active PSC members

---------

Co-authored-by: Jeroen Ticheler <[email protected]>
@pmauduit pmauduit requested a review from f-necas April 28, 2025 08:29
@landryb
Copy link
Member

landryb commented Apr 28, 2025

Fwiw this PR builds fine here, and produces a georchestra-geonetwork_4.4.7-georchestra.20250428084638~0e60062-1_all.deb debian package.

minor nit: the scmVersion in web/pom.xml should point at master..

@pmauduit
Copy link
Member Author

minor nit: the scmVersion in web/pom.xml should point at master..

right

@landryb
Copy link
Member

landryb commented Apr 28, 2025

can we also merge #320 & #319 in it while here ? as for the datahub integration build profile, my understanding is that it's the per-portal-datahub-instance-shipped-within-geonetwork feature ?

@pmauduit
Copy link
Member Author

my understanding is that it's the per-portal-datahub-instance-shipped-within-geonetwork feature ?

Indeed:
Screenshot from 2025-04-24 17-23-22

@landryb
Copy link
Member

landryb commented Apr 28, 2025

Also, the datafeeder-integration has been made optional upstream, we probably want it as a standard in the geOrchestra fork, but if we don't want to diverge much from upstream, I suggest to keep it "optional", but enforce its build in our CIs (GHA, buildbot). It is just a matter of activating a maven plugin anyway.

i had a quick look at plugins/README.md, trying to add it to the parent/toplevel Makefile with this:

--- i/Makefile
+++ w/Makefile
@@ -80,7 +80,7 @@ war-build-geowebcache: build-deps

 war-build-gn: build-deps
        mvn clean install -pl testcontainers,ldap-account-management,security-proxy-spring-integration -DskipTests
-       mvn clean install -f geonetwork/pom.xml -DskipTests
+       mvn clean install -f geonetwork/pom.xml -DskipTests -Pdatahub-integration

does actually try to build the plugin, one just needs to remember to git submodule init and git submodule update to fill the plugins/datahub-integration/geonetwork-ui submodule.

[INFO] --- frontend-maven-plugin:1.15.0:npm (npm-build) @ gn-datahub-integration ---
[INFO] Running 'npm run nx -- build datahub --base-href=./' in /data/src/georchestra/georchestra/geonetwork/plugins/datahub-integration/geonetwork-ui
[INFO] 
[INFO] > [email protected] nx
[INFO] > nx build datahub --base-href=./
[INFO] 
[INFO] 
[INFO]  NX   Running target build for project datahub and 1 task it depends on:
[INFO] 
[INFO] 
[INFO] Nx is configured to use the legacy cache. This cache will be removed in Nx 21. Read more at https://nx.dev/deprecated/legacy-cache.
[INFO] 
[INFO] > nx run util-data-fetcher:build
[INFO] 
[INFO] Your library compilation option specifies that the compiler external helper (tslib) is needed but it is not installed.
[INFO] Compiling TypeScript files for project "util-data-fetcher"...
[INFO] Done compiling TypeScript files for project "util-data-fetcher".
[INFO] 
[INFO] > nx run datahub:build:production --base-href=./
[INFO] 
[INFO] - Generating browser application bundles (phase: setup)...
....
[INFO] ✔ Browser application bundle generation complete.
[INFO] ✔ Browser application bundle generation complete.
[INFO] - Copying assets...
[INFO] ✔ Copying assets complete.
[INFO] - Generating index html...
[INFO] ✔ Index html generation complete.
...
[INFO]  NX   Successfully ran target build for project datahub and 1 task it depends on

i havent tested it yet at runtime, but that generates a jar in the webapp:

georchestra/geonetwork $find  web/target/ -name *datah*
web/target/geonetwork/WEB-INF/lib/gn-datahub-integration-4.4.7-georchestra.jar

@pmauduit
Copy link
Member Author

havent tested it yet at runtime, but that generates a jar in the webapp

if so, then we should have everything in place, huge thanks for testing it

@f-necas
Copy link
Collaborator

f-necas commented Apr 29, 2025

As said in private, it's just missing for gn-datahub-integration.jar by default :)

Also to check:


Changes in georchestra/docker which works for me:

docker-compose.yml

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.14.3
    ...
    environment:
      ...
      xpack.security.enabled: false
      xpack.security.enrollment.enabled: false
 
  kibana:
    image: docker.elastic.co/kibana/kibana:8.14.3

resources/kibana/kibana.yml

#Comment kibana.index line
#kibana.index: ".dashboards"

@pmauduit
Copy link
Member Author

As said in private, it's just missing for gn-datahub-integration.jar by default :)

it should not as long as the datahub-integration maven profile is being activated (which is one thing to do afterwards, e.g. adapting the CICD processes in geOrchestra to do so). This is what I suggested in the PR comment:

Also, the datafeeder-integration has been made optional upstream, we probably want it as a standard in the geOrchestra fork, but if we don't want to diverge much from upstream, I suggest to keep it "optional", but enforce its build in our CIs (GHA, buildbot). It is just a matter of activating a maven plugin anyway.

@pmauduit
Copy link
Member Author

About the PR on geor/docker, can it wait to have geonetwork migrated to 4.4.7 and integrated to georchestra/georchestra:master ? For sure I can prepare a PR to do so, but this one should be merged (and the wiring into geor/geor / CICD modifications) prior to this.

@f-necas
Copy link
Collaborator

f-necas commented Apr 29, 2025

About the PR on geor/docker, can it wait to have geonetwork migrated to 4.4.7 and integrated to georchestra/georchestra:master ? For sure I can prepare a PR to do so, but this one should be merged (and the wiring into geor/geor / CICD modifications) prior to this.

Ofc ! It's just to have a reminder somewhere and links between PRs

@pmauduit
Copy link
Member Author

pmauduit commented Apr 29, 2025

The docker composition is making use of official images, so I don't think there is a need for a fork on our side anymore ? I'll contribute a sample-docker-composition to runtime test easily though.

@f-necas
Copy link
Collaborator

f-necas commented May 5, 2025

ready to merge ?

@pmauduit pmauduit marked this pull request as ready for review May 9, 2025 14:58
@f-necas f-necas merged commit d9311ca into georchestra-gn4.4.x Jun 4, 2025
1 check passed
@f-necas f-necas deleted the georchestra-gn4.4.7 branch June 4, 2025 09:05
landryb added a commit to georchestra/ansible that referenced this pull request Jun 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.