diff --git a/doc/release-notes/ 11065-extend-search-api-to-include-type-counts.md b/doc/release-notes/ 11065-extend-search-api-to-include-type-counts.md deleted file mode 100644 index 0ba188c8637..00000000000 --- a/doc/release-notes/ 11065-extend-search-api-to-include-type-counts.md +++ /dev/null @@ -1 +0,0 @@ -The JSON payload of the search endpoint has been extended to include total_count_per_object_type for types: dataverse, dataset, and files when the search parameter "&show_type_counts=true" is passed in. diff --git a/doc/release-notes/10108-stata-version-detected-in-direct-upload.md b/doc/release-notes/10108-stata-version-detected-in-direct-upload.md deleted file mode 100644 index a69680e1c48..00000000000 --- a/doc/release-notes/10108-stata-version-detected-in-direct-upload.md +++ /dev/null @@ -1 +0,0 @@ -The version of Stata files is now detected during S3 direct upload (as it was for normal uploads), allowing ingest of Stata 14 and 15 files that have been uploaded directly. See [the guides](https://dataverse-guide--11054.org.readthedocs.build/en/11054/developers/big-data-support.html#features-that-are-disabled-if-s3-direct-upload-is-enabled), #10108, and #11054. diff --git a/doc/release-notes/10171-exlude-metadatablocks.md b/doc/release-notes/10171-exlude-metadatablocks.md deleted file mode 100644 index 7c0a9b030eb..00000000000 --- a/doc/release-notes/10171-exlude-metadatablocks.md +++ /dev/null @@ -1,4 +0,0 @@ -Extension of API `{id}/versions` and `{id}/versions/{versionId}` with an optional ``excludeMetadataBlocks`` parameter, -that specifies whether the metadataBlocks should be listed in the output. It defaults to ``false``, preserving backward -compatibility. (Note that for a dataset with a large number of versions and/or metadataBlocks having the metadata blocks -included can dramatically increase the volume of the output). See also [the guides](https://dataverse-guide--10778.org.readthedocs.build/en/10778/api/native-api.html#list-versions-of-a-dataset), #10778, and #10171. diff --git a/doc/release-notes/10241-new-solr-client.md b/doc/release-notes/10241-new-solr-client.md deleted file mode 100644 index 67ccdd4f184..00000000000 --- a/doc/release-notes/10241-new-solr-client.md +++ /dev/null @@ -1,9 +0,0 @@ -[HttpSolrClient](https://solr.apache.org/docs/9_4_1/solrj/org/apache/solr/client/solrj/impl/HttpSolrClient.html) is deprecated as of Solr 9, and which will be removed in a future major release of Solr. It's recommended to use [Http2SolrClient](https://solr.apache.org/docs/9_4_1/solrj/org/apache/solr/client/solrj/impl/Http2SolrClient.html) instead. - -[Solr documentation](https://solr.apache.org/guide/solr/latest/deployment-guide/solrj.html#types-of-solrclients) describe it as a _async, non-blocking and general-purpose client that leverage HTTP/2 using the Jetty Http library_. - -With Solr 9.4.1, the Http2SolrClient is indicate as experimental. But since the 9.6 version of Solr, this mention is no longer maintained. - -The ConcurrentUpdateHttp2SolrClient is now also used in some cases, which is supposed to be more efficient for indexing. - -For more information, see issue [#10161](https://github.com/IQSS/dataverse/issues/10161) and pull request [#10241](https://github.com/IQSS/dataverse/pull/10241) diff --git a/doc/release-notes/10304-add-move-dataverse-collection.md b/doc/release-notes/10304-add-move-dataverse-collection.md deleted file mode 100644 index 1ddeb526c61..00000000000 --- a/doc/release-notes/10304-add-move-dataverse-collection.md +++ /dev/null @@ -1,5 +0,0 @@ -### Move a collection from the dashboard - -In addition to the api [Move a Dataverse Collection](https://guides.dataverse.org/en/latest/admin/dataverses-datasets.html#move-a-dataverse-collection), it is now possible for a Dataverse administrator to move a collection from the Dataverse dashboard. - -For more information, see #10304. \ No newline at end of file diff --git a/doc/release-notes/10320-cookie-consent.md b/doc/release-notes/10320-cookie-consent.md deleted file mode 100644 index a3fd7f5466b..00000000000 --- a/doc/release-notes/10320-cookie-consent.md +++ /dev/null @@ -1,3 +0,0 @@ -## Cookie Consent Popup (GDPR) - -For compliance with GDPR and other privacy regulations, advice on adding a cookie consent popup has been added to the guides. See the new [cookie consent](https://dataverse-guide--10320.org.readthedocs.build/en/10320/installation/config.html#adding-cookie-consent-for-gdpr-etc) section and #10320. diff --git a/doc/release-notes/10340-forbidden.md b/doc/release-notes/10340-forbidden.md deleted file mode 100644 index 5997f717d64..00000000000 --- a/doc/release-notes/10340-forbidden.md +++ /dev/null @@ -1,3 +0,0 @@ -### Backward Incompatible Changes - -The [Show Role](https://dataverse-guide--11116.org.readthedocs.build/en/11116/api/native-api.html#show-role) API endpoint was returning 401 Unauthorized when a permission check failed. This has been corrected to return 403 Forbidden instead. That is, the API token is known to be good (401 otherwise) but the user lacks permission (403 is now sent). See also the [API Changelog](https://dataverse-guide--11116.org.readthedocs.build/en/11116/api/changelog.html), #10340, and #11116. diff --git a/doc/release-notes/10384-link.md b/doc/release-notes/10384-link.md deleted file mode 100644 index 7092241adf4..00000000000 --- a/doc/release-notes/10384-link.md +++ /dev/null @@ -1,3 +0,0 @@ -### Broken Link in Email When Users Request Access to Files - -When users request access to a files, the people who have permission to grant access receive an email with a link in it that didn't work due to a trailing period (full stop) right next to the link (e.g. `https://demo.dataverse.org/permissions-manage-files.xhtml?id=9.`) A space has been added to fix this. See #10384 and #11115. diff --git a/doc/release-notes/10472-review-modify-jsonprinter-for-datasetfieldtype.md b/doc/release-notes/10472-review-modify-jsonprinter-for-datasetfieldtype.md deleted file mode 100644 index f0b9c30c9cd..00000000000 --- a/doc/release-notes/10472-review-modify-jsonprinter-for-datasetfieldtype.md +++ /dev/null @@ -1,8 +0,0 @@ -### Json Printer Bug fix - -DatasetFieldTypes in MetadataBlock response that are also a child of another DatasetFieldType were being returned twice. The child DatasetFieldType was included in the "fields" object as well as in the "childFields" of it's parent DatasetFieldType. This fix suppresses the standalone object so only one instance of the DatasetFieldType is returned (in the "childFields" of its parent). -This fix changes the Json output of the API `/api/dataverses/{dataverseAlias}/metadatablocks` - -## Backward Incompatible Changes - -The Json response of API call `/api/dataverses/{dataverseAlias}/metadatablocks` will no longer include the DatasetFieldTypes in "fields" if they are children of another DatasetFieldType. The child DatasetFieldType will only be included in the "childFields" of it's parent DatasetFieldType. diff --git a/doc/release-notes/10476-display-on-create-field-option.md b/doc/release-notes/10476-display-on-create-field-option.md deleted file mode 100644 index e4a38e181a2..00000000000 --- a/doc/release-notes/10476-display-on-create-field-option.md +++ /dev/null @@ -1,6 +0,0 @@ -New feature: Collection administrators can now configure which metadata fields appear during dataset creation through the `displayOnCreate` property, even when fields are not required. This provides greater control over metadata visibility and can help improve metadata completeness. - -- The feature is currently available through the API endpoint `/api/dataverses/{alias}/inputLevels` -- UI implementation will be available in a future release [#11221](https://github.com/IQSS/dataverse/issues/11221) - -For more information, see the [API Guide](https://guides.dataverse.org/en/latest/api/native-api.html#update-collection-input-levels) and issues [#10476](https://github.com/IQSS/dataverse/issues/10476) and [#11224](https://github.com/IQSS/dataverse/pull/11224). \ No newline at end of file diff --git a/doc/release-notes/10516_legacy_permalink_config_fix.md b/doc/release-notes/10516_legacy_permalink_config_fix.md deleted file mode 100644 index d78395252d4..00000000000 --- a/doc/release-notes/10516_legacy_permalink_config_fix.md +++ /dev/null @@ -1 +0,0 @@ -Support for legacy configuration of a PermaLink PID provider, e.g. using the :Protocol,:Authority, and :Shoulder settings, is fixed. \ No newline at end of file diff --git a/doc/release-notes/10519-dataset-types.md b/doc/release-notes/10519-dataset-types.md deleted file mode 100644 index 99cf79a796f..00000000000 --- a/doc/release-notes/10519-dataset-types.md +++ /dev/null @@ -1,12 +0,0 @@ -## Dataset Types can be linked to Metadata Blocks - -Metadata blocks (e.g. "CodeMeta") can now be linked to dataset types (e.g. "software") using new superuser APIs. - -This will have the following effects for the APIs used by the new Dataverse UI ( https://github.com/IQSS/dataverse-frontend ): - -- The list of fields shown when creating a dataset will include fields marked as "displayoncreate" (in the tsv/database) for metadata blocks (e.g. "CodeMeta") that are linked to the dataset type (e.g. "software") that is passed to the API. -- The metadata blocks shown when editing a dataset will include metadata blocks (e.g. "CodeMeta") that are linked to the dataset type (e.g. "software") that is passed to the API. - -Mostly in order to write automated tests for the above, a [displayOnCreate](https://dataverse-guide--11001.org.readthedocs.build/en/11001/api/native-api.html#set-displayoncreate-for-a-dataset-field) API endpoint has been added. - -For more information, see the guides ([overview](https://dataverse-guide--11001.org.readthedocs.build/en/11001/user/dataset-management.html#dataset-types), [new APIs](https://dataverse-guide--11001.org.readthedocs.build/en/11001/api/native-api.html#link-dataset-type-with-metadata-blocks)), #10519 and #11001. diff --git a/doc/release-notes/10541-root-alias-name2.md b/doc/release-notes/10541-root-alias-name2.md deleted file mode 100644 index e699d40994b..00000000000 --- a/doc/release-notes/10541-root-alias-name2.md +++ /dev/null @@ -1 +0,0 @@ -The [tutorial](https://dataverse-guide--11201.org.readthedocs.build/en/11201/container/running/demo.html#root-collection-customization-alias-name-etc) on running Dataverse in Docker has been updated to explain how to configure the root collection using a JSON file. See also #10541 and #11201. diff --git a/doc/release-notes/10542-signposting.md b/doc/release-notes/10542-signposting.md deleted file mode 100644 index f847e6ba94a..00000000000 --- a/doc/release-notes/10542-signposting.md +++ /dev/null @@ -1,11 +0,0 @@ -# Signposting Output Now Contains Links to All Dataset Metadata Export Formats - -When Signposting was added in Dataverse 5.14 (#8981), it only provided links for the `schema.org` metadata export format. - -The output of HEAD, GET, and the Signposting "linkset" API have all been updated to include links to all available dataset metadata export formats (including any external exporters, such as Croissant, that have been enabled). - -This provides a lightweight machine-readable way to first retrieve a list of links (via a HTTP HEAD request, for example) to each available metadata export format and then follow up with a request for the export format of interest. - -In addition, the content type for the `schema.org` dataset metadata export format has been corrected. It was `application/json` and now it is `application/ld+json`. - -See also [the docs](https://preview.guides.gdcc.io/en/develop/api/native-api.html#retrieve-signposting-information) and #10542. diff --git a/doc/release-notes/10626-archival-tsv.md b/doc/release-notes/10626-archival-tsv.md deleted file mode 100644 index 2cd7c4e60b1..00000000000 --- a/doc/release-notes/10626-archival-tsv.md +++ /dev/null @@ -1,2 +0,0 @@ -Release Highlights: -An experimental "Archival" metadata block has been added, [downloadable](https://dataverse-guide--10626.org.readthedocs.build/en/10626/user/appendix.html) from the User Guide. The purpose of the metadata block is to enable repositories to register metadata relating to the potential archiving of the dataset at a depositor archive, whether that being your own institutional archive or an external archive, i.e. a historical archive. See also #10626. diff --git a/doc/release-notes/10713-Solr9.8.0 and lib updates.md b/doc/release-notes/10713-Solr9.8.0 and lib updates.md deleted file mode 100644 index a42960301ef..00000000000 --- a/doc/release-notes/10713-Solr9.8.0 and lib updates.md +++ /dev/null @@ -1,9 +0,0 @@ -Solr 9.8.0 is now the version recommended in our installation guides and used with automated testing. Other libraries Dataverse uses have been updated as well. - -For the upgrade instructions section: - -[note that 6.6 may contain other solr-related changes, so the instructions may need to contain information merged from multiple release notes!] - -If you are upgrading Solr: - - Install solr-9.8.0 following the instructions from the Installation guide. - - Run a full reindex to populate the search catalog. diff --git a/doc/release-notes/10714-access-requests-missing-since-upgrade-v6-0.md b/doc/release-notes/10714-access-requests-missing-since-upgrade-v6-0.md deleted file mode 100644 index a220c150791..00000000000 --- a/doc/release-notes/10714-access-requests-missing-since-upgrade-v6-0.md +++ /dev/null @@ -1,6 +0,0 @@ -### Flyway Script added to Fix File Access Requests when upgrading from Dataverse 6.0 - -Database update script added to prevent duplicate keys when upgrading from V6.0 -This script will delete access requests made after the initial request and will set the initial request to "Created" - -See: https://github.com/IQSS/dataverse/issues/10714 diff --git a/doc/release-notes/10735-search-dataset-name-and-dataset-citation-different.md b/doc/release-notes/10735-search-dataset-name-and-dataset-citation-different.md deleted file mode 100644 index 6a6b2008772..00000000000 --- a/doc/release-notes/10735-search-dataset-name-and-dataset-citation-different.md +++ /dev/null @@ -1,4 +0,0 @@ - -### Search files Bug fix - -dataset-citation was displaying DRAFT version instead of latest released version diff --git a/doc/release-notes/10790-fix pid handling in exporters and citations.md b/doc/release-notes/10790-fix pid handling in exporters and citations.md deleted file mode 100644 index da430ed9671..00000000000 --- a/doc/release-notes/10790-fix pid handling in exporters and citations.md +++ /dev/null @@ -1,16 +0,0 @@ -### Improvements to PID formatting in exports and citations - -Multiple small issues with the formatting of PIDs in the -DDI exporters, and EndNote and BibTeX citation formats have -been addressed. These should improve the ability to import -Dataverse citations into reference managers and fix potential -issues harvesting datasets using PermaLinks. - -Backward Incompatibility - -Changes to PID formatting occur in the DDI/DDI Html export formats -and the EndNote and BibTex citation formats. These changes correct -errors and improve conformance with best practices but could break -parsing of these formats. - -For more information, see #10790. diff --git a/doc/release-notes/10809-oai-ore-nested-compound.md b/doc/release-notes/10809-oai-ore-nested-compound.md deleted file mode 100644 index d67f2b65592..00000000000 --- a/doc/release-notes/10809-oai-ore-nested-compound.md +++ /dev/null @@ -1 +0,0 @@ -The OAI-ORE exporter can now export metadata containing nested compound fields (i.e. compound fields within compound fields). See #10809 and #11190. diff --git a/doc/release-notes/10883-license improvements.md b/doc/release-notes/10883-license improvements.md deleted file mode 100644 index 30b161f9bd5..00000000000 --- a/doc/release-notes/10883-license improvements.md +++ /dev/null @@ -1,12 +0,0 @@ -- License metadata enhancements (#10883): - - Added new fields to licenses: rightsIdentifier, rightsIdentifierScheme, schemeUri, languageCode - - Updated DataCite metadata export to include rightsIdentifier, rightsIdentifierScheme, and schemeUri consistent with the DataCite 4.5 schema and examples - - Enhanced metadata exports to include all new license fields - - Existing licenses from the example set included with Dataverse will be automatically updated with new fields - - Existing API calls support the new optional fields - - Setup: For existing published datasets, the additional license metadata will not be available from DataCite or in metadata exports until the dataset is republished or - - the /api/admin/metadata/{id}/reExportDataset is run for the dataset - - the api/datasets/{id}/modifyRegistrationMetadata API is run for the dataset, - or the global version of these api calls (/api/admin/metadata/reExportAll, /api/datasets/modifyRegistrationPIDMetadataAll) are used. - \ No newline at end of file diff --git a/doc/release-notes/10887-solr-field-types.md b/doc/release-notes/10887-solr-field-types.md deleted file mode 100644 index 1afbae34e90..00000000000 --- a/doc/release-notes/10887-solr-field-types.md +++ /dev/null @@ -1,102 +0,0 @@ -This release enhances how numerical and date fields are indexed in Solr. Previously, all fields were indexed as English text (text_en), but with this update: - -* Integer fields are indexed as `plong` -* Float fields are indexed as `pdouble` -* Date fields are indexed as `date_range` (`solr.DateRangeField`) - -Specifically, the following fields were updated: - -- coverage.Depth -- coverage.ObjectCount -- coverage.ObjectDensity -- coverage.Redshift.MaximumValue -- coverage.Redshift.MinimumValue -- coverage.RedshiftValue -- coverage.SkyFraction -- coverage.Spectral.CentralWavelength -- coverage.Spectral.MaximumWavelength -- coverage.Spectral.MinimumWavelength -- coverage.Temporal.StartTime -- coverage.Temporal.StopTime -- dateOfCollectionEnd -- dateOfCollectionStart -- dateOfDeposit -- distributionDate -- dsDescriptionDate -- journalPubDate -- productionDate -- resolution.Redshift -- targetSampleActualSize -- timePeriodCoveredEnd -- timePeriodCoveredStart - -This change enables range queries when searching from both the UI and the API, such as `dateOfDeposit:[2000-01-01 TO 2014-12-31]` or `targetSampleActualSize:[25 TO 50]`. - -Dataverse administrators must update their Solr schema.xml (manually or by rerunning `update-fields.sh`) and reindex all datasets. - -Additionally, search result highlighting is now more accurate, ensuring that only fields relevant to the query are highlighted in search results. If the query is specifically limited to certain fields, the highlighting is now limited to those fields as well. - -## Upgrade Instructions - -7\. Update Solr schema.xml file. Start with the standard v6.5 schema.xml, then, if your installation uses any custom or experimental metadata blocks, update it to include the extra fields (step 7a). - -Stop Solr (usually `service solr stop`, depending on Solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/6.5/installation/prerequisites.html#solr-init-script)). - -```shell -service solr stop -``` - -Replace schema.xml - -```shell -wget https://raw.githubusercontent.com/IQSS/dataverse/v6.5/conf/solr/schema.xml -cp schema.xml /usr/local/solr/solr-9.4.1/server/solr/collection1/conf -``` - -Start Solr (but if you use any custom metadata blocks, perform the next step, 7a first). - -```shell -service solr start -``` - -7a\. For installations with custom or experimental metadata blocks: - -Before starting Solr, update the schema to include all the extra metadata fields that your installation uses. We do this by collecting the output of the Dataverse schema API and feeding it to the `update-fields.sh` script that we supply, as in the example below (modify the command lines as needed to reflect the names of the directories, if different): - -```shell - wget https://raw.githubusercontent.com/IQSS/dataverse/v6.5/conf/solr/update-fields.sh - chmod +x update-fields.sh - curl "http://localhost:8080/api/admin/index/solr/schema" | ./update-fields.sh /usr/local/solr/solr-9.4.1/server/solr/collection1/conf/schema.xml -``` - -Now start Solr. - -8\. Reindex Solr - -Note: these instructions are a little different than usual because we observed a strange error about `DOCS_AND_FREQS_AND_POSITIONS` when testing upgrades (see #11139 for details). Extra steps about explicitly clearing the index and reloading the core are included. If you run into trouble, as a last resort, you could reinstall Solr completely and then reindex. - -Clear the Solr index: - -```shell -curl http://localhost:8080/api/admin/index/clear -``` - -Make sure the Solr index is empty: - -```shell -curl "http://localhost:8983/solr/collection1/select?rows=1000000&wt=json&indent=true&q=*%3A*" -``` - -Reload the Solr core: - -```shell -curl "http://localhost:8983/solr/admin/cores?action=RELOAD&core=collection1" -``` - -Below is the simplest way to reindex Solr: - -```shell -curl http://localhost:8080/api/admin/index -``` - -The API above rebuilds the existing index "in place". If you want to be absolutely sure that your index is up-to-date and consistent, you may consider wiping it clean and reindexing everything from scratch (see [the guides](https://guides.dataverse.org/en/latest/admin/solr-search-index.html)). Just note that, depending on the size of your database, a full reindex may take a while and the users will be seeing incomplete search results during that window. diff --git a/doc/release-notes/10909-datacite-oai-harvesting.md b/doc/release-notes/10909-datacite-oai-harvesting.md deleted file mode 100644 index be8265263fd..00000000000 --- a/doc/release-notes/10909-datacite-oai-harvesting.md +++ /dev/null @@ -1,10 +0,0 @@ -### OAI Harvesting from DataCite - -DataCite maintains an OAI server (https://oai.datacite.org/oai) that serves records for every DOI they have registered. There's been a lot of interest in the community in being able to harvest from them. This way, it will be possible to harvest metadata from institution X even if the institution X does not maintain an OAI server of their own, if they happen to register their DOIs with DataCite. One extra element of this harvesting model that makes it especially powerful and flexible is the DataCite's concept of a "dynamic OAI set": a harvester is not limited to harvesting the pre-defined set of ALL the records registered by the Institution X, but can instead harvest virtually any arbitrary subset thereof; any query that the DataCite search API understands can be used as an OAI set (!). The feature is already in use at IQSS, as a beta version patch. - -For various reasons, in order to take advantage of this feature harvesting clients must be created using the `/api/harvest/clients` API. Once configured however, harvests can be run from the Harvesting Clients control panel in the UI. - -DataCite-harvesting clients must be configured with 2 new feature flags, `useListRecords` and `useOaiIdentifiersAsPids` (added in v6.5). Note that these features may be of use when harvesting from other sources, not just from DataCite. - -See "Harvesting from DataCite" under https://guides.dataverse.org/en/latest/api/native-api.html#managing-harvesting-clients for more information. - diff --git a/doc/release-notes/10930-marketplace-external-tools-apis.md b/doc/release-notes/10930-marketplace-external-tools-apis.md deleted file mode 100644 index e3350a8b2d2..00000000000 --- a/doc/release-notes/10930-marketplace-external-tools-apis.md +++ /dev/null @@ -1,14 +0,0 @@ -## New APIs for External Tools Registration for Marketplace - -New API base path /api/externalTools created that mimics the admin APIs /api/admin/externalTools. These new add and delete apis require an authenticated superuser token. - -Example: -``` - API_TOKEN='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' - export TOOL_ID=1 - - curl http://localhost:8080/api/externalTools - curl http://localhost:8080/api/externalTools/$TOOL_ID - curl -s -H "X-Dataverse-key:$API_TOKEN" -X POST -H 'Content-type: application/json' http://localhost:8080/api/externalTools --upload-file fabulousFileTool.json - curl -s -H "X-Dataverse-key:$API_TOKEN" -X DELETE http://localhost:8080/api/externalTools/$TOOL_ID -``` diff --git a/doc/release-notes/10943-featured-items.md b/doc/release-notes/10943-featured-items.md deleted file mode 100644 index fa61d4e4875..00000000000 --- a/doc/release-notes/10943-featured-items.md +++ /dev/null @@ -1,15 +0,0 @@ -CRUD endpoints for Collection Featured Items have been implemented. In particular, the following endpoints have been implemented: - -- Create a feature item (POST /api/dataverses//featuredItems) -- Update a feature item (PUT /api/dataverseFeaturedItems/) -- Delete a feature item (DELETE /api/dataverseFeaturedItems/) -- List all featured items in a collection (GET /api/dataverses//featuredItems) -- Delete all featured items in a collection (DELETE /api/dataverses//featuredItems) -- Update all featured items in a collection (PUT /api/dataverses//featuredItems) - -New settings: - -- dataverse.files.featured-items.image-maxsize - It sets the maximum allowed size of the image that can be added to a featured item. -- dataverse.files.featured-items.image-uploads - It specifies the name of the subdirectory for saving featured item images within the docroot directory. - -See also #10943 and #11124. diff --git a/doc/release-notes/10959-oidc-api-auth-ext.md b/doc/release-notes/10959-oidc-api-auth-ext.md deleted file mode 100644 index 04ee2099f68..00000000000 --- a/doc/release-notes/10959-oidc-api-auth-ext.md +++ /dev/null @@ -1,14 +0,0 @@ -Extends the OIDC API auth mechanism (available through feature flag ``api-bearer-auth``) to properly handle cases -where ``BearerTokenAuthMechanism`` successfully validates the token but cannot identify any Dataverse user because there -is no account associated with the token. - -To register a new user who has authenticated via an OIDC provider, a new endpoint has been -implemented (``/users/register``). A feature flag named ``api-bearer-auth-provide-missing-claims`` has been implemented -to allow -sending missing user claims in the request JSON. This is useful when the identity provider does not supply the necessary -claims. However, this flag will only be considered if the ``api-bearer-auth`` feature flag is enabled. If the latter is -not enabled, the ``api-bearer-auth-provide-missing-claims`` flag will be ignored. - -A feature flag named ``api-bearer-auth-handle-tos-acceptance-in-idp`` has been implemented. When enabled, it specifies -that Terms of Service acceptance is managed by the identity provider, eliminating the need to explicitly include the -acceptance in the user registration request JSON. diff --git a/doc/release-notes/10975-fix-file-replace-via-api b/doc/release-notes/10975-fix-file-replace-via-api deleted file mode 100644 index 71646015a32..00000000000 --- a/doc/release-notes/10975-fix-file-replace-via-api +++ /dev/null @@ -1,3 +0,0 @@ -A bug that caused replacing files via API when file PIDs were enabled has been fixed. - - For testing purposes, the FAKE PID provider can now be used with file PIDs enabled. (The FAKE provider is not recommended for any production use.) \ No newline at end of file diff --git a/doc/release-notes/10981-published-datasets-should-contain-files.md b/doc/release-notes/10981-published-datasets-should-contain-files.md deleted file mode 100644 index 964e7ff1937..00000000000 --- a/doc/release-notes/10981-published-datasets-should-contain-files.md +++ /dev/null @@ -1,11 +0,0 @@ -## Feature: Prevent publishing Datasets without files -A new attribute was added to Collections in order to control the publishing of Datasets without files. -Once set to "True", the publishing of a Dataset within a Collection, without files, will be blocked for all non superusers. -In order to configure a Collection to block publishing a superuser must set the attribute "requireFilesToPublishDataset" to "True". -The collection's hierarchy will be checked if the collection's "requireFilesToPublishDataset" attribute is not set explicitly to "True" or "False". - -```shell -curl -X PUT -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/dataverses/$ID/attribute/requireFilesToPublishDataset?value=true" -``` - -See also #10981. diff --git a/doc/release-notes/11027-extend-datasets-files-from-search-api.md b/doc/release-notes/11027-extend-datasets-files-from-search-api.md deleted file mode 100644 index 7b20daeeb0f..00000000000 --- a/doc/release-notes/11027-extend-datasets-files-from-search-api.md +++ /dev/null @@ -1,22 +0,0 @@ -### Feature to extend Search API for SPA - -Added new fields to search results type=files - -For Files: -- restricted: boolean -- canDownloadFile: boolean ( from file user permission) -- categories: array of string "categories" would be similar to what it is in metadata api. -For tabular files: -- tabularTags: array of string for example,{"tabularTags" : ["Event", "Genomics", "Geospatial"]} -- variables: number/int shows how many variables we have for the tabular file -- observations: number/int shows how many observations for the tabular file - - - -New fields added to solr schema.xml (Note: upgrade instructions will need to include instructions for schema.xml): - - - - - -See https://github.com/IQSS/dataverse/issues/11027 diff --git a/doc/release-notes/11038-unconsidered-harvesting-granularity.md b/doc/release-notes/11038-unconsidered-harvesting-granularity.md deleted file mode 100644 index 72ebd522831..00000000000 --- a/doc/release-notes/11038-unconsidered-harvesting-granularity.md +++ /dev/null @@ -1,2 +0,0 @@ -Bug Fix: -OAI Client harvesting now uses the correct granularity while re-run a partial harvest (using the `from` parameter). The correct granularity comes from the `Identify` verb request. \ No newline at end of file diff --git a/doc/release-notes/11053-metadata-styling-fixes b/doc/release-notes/11053-metadata-styling-fixes deleted file mode 100644 index c1f2c69c283..00000000000 --- a/doc/release-notes/11053-metadata-styling-fixes +++ /dev/null @@ -1 +0,0 @@ -Minor styling fixes for the Related Publication Field and fields using ORCID or ROR have been made (see #11053, #10964, #11106) diff --git a/doc/release-notes/11057-globus-improvements.md b/doc/release-notes/11057-globus-improvements.md deleted file mode 100644 index d559d7afa82..00000000000 --- a/doc/release-notes/11057-globus-improvements.md +++ /dev/null @@ -1,11 +0,0 @@ -## Globus framework improvements - -The improvements and optimizations in this release build on top of the earlier work (such as PR #10781). They are based on the experience gained at IQSS as part of the production rollout of the Large Data Storage services that utilizes Globus. - -The changes in this PR (#11125) focus on improving Globus *downloads* (i.e., transfers from Dataverse-linked Globus volumes to users' Globus collections). Most importantly, th mechanism of "Asynchronous Task Monitoring", first introduced in #10781 for *uploads*, has been extended to handle downloads as well. This generally makes downloads more reliable (specifically, in how Dataverse manages temporary access rules granted to users, minimizing the risk of consequent downloads failing because of stale access rules left in place). - -See `globus-use-experimental-async-framework` under [Feature Flags](https://guides.dataverse.org/en/latest/installation/config.html#feature-flags) and [dataverse.files.globus-monitoring-server](https://guides.dataverse.org/en/latest/installation/config.html#dataverse-files-globus-monitoring-server) in the Installation Guide. - -Multiple other improvements have been made making the underlying Globus framework more reliable and robust. - - diff --git a/doc/release-notes/11064-update-metadata-customization.md b/doc/release-notes/11064-update-metadata-customization.md deleted file mode 100644 index bcead3497f8..00000000000 --- a/doc/release-notes/11064-update-metadata-customization.md +++ /dev/null @@ -1 +0,0 @@ -Metadata Customization guide has been updated to explain how to implement a kind of boolean fieldtype (see [Metadata Customization Guide](https://guides.dataverse.org/en/latest/admin/metadatacustomization.html#controlledvocabulary-enumerated-properties)) \ No newline at end of file diff --git a/doc/release-notes/11075-ror.md b/doc/release-notes/11075-ror.md deleted file mode 100644 index de32eebccdf..00000000000 --- a/doc/release-notes/11075-ror.md +++ /dev/null @@ -1,15 +0,0 @@ -### ROR (Research Organization Registry) as Author Identifier Type - -ROR (Research Organization Registry) has been added as an Author Identifier Type (alongside ORCID, etc.) for when the author is an organization rather than a person. As with all author identifiers, be sure to select the proper identifier type (e.g. "ROR") and enter just the unique identifier (e.g. "03vek6s52") rather than the full URL (e.g. "https://ror.org/03vek6s52"). Like ORCID, ROR will appear in the "Datacite" metadata export format. See also the [ROR](https://ror.org) website, a new [note](https://dataverse-guide--11118.org.readthedocs.build/en/11118/user/dataset-management.html#adding-a-new-dataset) in a guides about entering author identifiers, #11075, and #11118. - -## Upgrade Instructions - -6\. Update metadata blocks - -These changes reflect incremental improvements made to the handling of core metadata fields. Expect the loading of the citation block to take several seconds because of its size (especially due to the number of languages). - -```shell -wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/scripts/api/data/metadatablocks/citation.tsv - -curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file citation.tsv -``` diff --git a/doc/release-notes/11083-mydata-npe-with-harvested-dataverses.md b/doc/release-notes/11083-mydata-npe-with-harvested-dataverses.md deleted file mode 100644 index 230d69c9b9f..00000000000 --- a/doc/release-notes/11083-mydata-npe-with-harvested-dataverses.md +++ /dev/null @@ -1 +0,0 @@ -Fix a bug with My Data where listing dataverses for a user with only rights on harvested dataverses would result in a server error response. \ No newline at end of file diff --git a/doc/release-notes/11095-fix-extcvoc-indexing.md b/doc/release-notes/11095-fix-extcvoc-indexing.md deleted file mode 100644 index a072491d04b..00000000000 --- a/doc/release-notes/11095-fix-extcvoc-indexing.md +++ /dev/null @@ -1,7 +0,0 @@ -Some External Controlled Vocabulary scripts/configurations, when used on a metadata field that is single-valued could result -in indexing failure for the dataset (e.g. when the script tried to index both the identifier and name of the identified entity for indexing). -Dataverse has been updated to correctly indicate the need for a multi-valued Solr field in these cases in the call to /api/admin/index/solr/schema. -Configuring the Solr schema and the update-fields.sh script as usually recommended when using custom metadata blocks will resolve the issue. - -The overall release notes should include a Solr update (which hopefully is required by an update to 9.8.0 anyway) and our standard instructions -should change to recommending use of the update-fields.sh script when using custom metadatablocks *and/or external vocabulary scripts*. diff --git a/doc/release-notes/11107-fake-to-perma-demo.md b/doc/release-notes/11107-fake-to-perma-demo.md deleted file mode 100644 index afb6b8a7917..00000000000 --- a/doc/release-notes/11107-fake-to-perma-demo.md +++ /dev/null @@ -1,3 +0,0 @@ -### Demo/Eval Container Tutorial - -The demo/eval container tutorial has been updated to use the Permalink PID provider instead of the FAKE DOI Provider. See also #11107. diff --git a/doc/release-notes/11113-avoid-orphan-perm-docs.md b/doc/release-notes/11113-avoid-orphan-perm-docs.md deleted file mode 100644 index 93c030d0afe..00000000000 --- a/doc/release-notes/11113-avoid-orphan-perm-docs.md +++ /dev/null @@ -1,5 +0,0 @@ -This release fixes a bug that caused Dataverse to generate unnecessary solr documents for files when a file is added/deleted from a draft dataset. These documents could accumulate and potentially impact performance. - -Assuming the upgrade to solr 9.8.0 also occurs in this release, there's nothing else needed for this PR. (Starting with a new solr insures the solr db is empty and that a reindex is already required.) - - diff --git a/doc/release-notes/11120-add-3d-object-metadata-block.md b/doc/release-notes/11120-add-3d-object-metadata-block.md deleted file mode 100644 index a43be86bc7d..00000000000 --- a/doc/release-notes/11120-add-3d-object-metadata-block.md +++ /dev/null @@ -1,60 +0,0 @@ -### New 3D Object Data Metadata Block - -A new metadata block has been added for describing 3D object data. You can download it from the [guides](https://dataverse-guide--11167.org.readthedocs.build/en/11167/user/appendix.html). See also #11120 and #11167. - -All new Dataverse installations will receive this metadata block by default. We recommend adding it by following the upgrade instructions below. - -## Upgrade Instructions - -### For 6.6-Release-notes.md - -6\. Restart Payara - -7\. Update metadata blocks - -These changes reflect incremental improvements made to the handling of core metadata fields. - -```shell -wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/scripts/api/data/metadatablocks/citation.tsv - -curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file citation.tsv -``` -```shell -wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/scripts/api/data/metadatablocks/3d_objects.tsv - -curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file 3d_objects.tsv -``` - -8\. Update Solr schema.xml file. Start with the standard v6.6 schema.xml, then, if your installation uses any custom or experimental metadata blocks, update it to include the extra fields (step 8a). - -Stop Solr (usually `service solr stop`, depending on Solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/6.6/installation/prerequisites.html#solr-init-script)). - -```shell -service solr stop -``` - -Replace schema.xml - -```shell -wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/conf/solr/schema.xml -cp schema.xml /usr/local/solr/solr-9.4.1/server/solr/collection1/conf -``` - -Start Solr (but if you use any custom metadata blocks or adding 3D Objects, perform the next step, 8a first). - -```shell -service solr start -``` -8a\. For installations with custom or experimental metadata blocks: - -Before starting Solr, update the schema to include all the extra metadata fields that your installation uses. We do this by collecting the output of the Dataverse schema API and feeding it to the `update-fields.sh` script that we supply, as in the example below (modify the command lines as needed to reflect the names of the directories, if different): - -```shell - wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/conf/solr/update-fields.sh - chmod +x update-fields.sh - curl "http://localhost:8080/api/admin/index/solr/schema" | ./update-fields.sh /usr/local/solr/solr-9.4.1/server/solr/collection1/conf/schema.xml -``` - -Now start Solr. - -9\. Reindex Solr diff --git a/doc/release-notes/11127-update-search-api-to-show-dvobject-type-counts.md b/doc/release-notes/11127-update-search-api-to-show-dvobject-type-counts.md deleted file mode 100644 index c9b63340a1f..00000000000 --- a/doc/release-notes/11127-update-search-api-to-show-dvobject-type-counts.md +++ /dev/null @@ -1,5 +0,0 @@ -### show-type-counts Behavior changed in Search API - -In the Search API if you set show_type_counts=true the response will include all object types (Dataverses, Datasets, and Files) even if the search result for any given type is 0. - -See also the [guides](https://preview.guides.gdcc.io/en/develop/api/search.html#parameters), #11127 and #11138. diff --git a/doc/release-notes/11129-send-feedback-to-contacts.md b/doc/release-notes/11129-send-feedback-to-contacts.md deleted file mode 100644 index 56eff133e0d..00000000000 --- a/doc/release-notes/11129-send-feedback-to-contacts.md +++ /dev/null @@ -1,14 +0,0 @@ -This feature adds a new API to send feedback to the Collection, Dataset, or DataFile's contacts. -Similar to the "admin/feedback" API the "sendfeedback" API sends an email to all the contacts listed for the Dataset. The main differences for this feature are: -1. This API is not limited to Admins -2. This API does not return the email addresses in the "toEmail" and "ccEmail" elements for privacy reasons -3. This API can be rate limited to avoid spamming -4. The body size limit can be configured -5. The body will be stripped of any html code to prevent malicious scripts or links -6. The fromEmail will be validated for correct format - -To set the Rate Limiting for guest users (See Rate Limiting Configuration for more details. This example allows 1 send per hour for any guest) -``curl http://localhost:8080/api/admin/settings/:RateLimitingCapacityByTierAndAction -X PUT -d '[{\"tier\": 0, \"limitPerHour\": 1, \"actions\": [\"CheckRateLimitForDatasetFeedbackCommand\"]}]'`` - -To set the message size limit (example limit of 1080 chars): -``curl -X PUT -d 1080 http://localhost:8080/api/admin/settings/:ContactFeedbackMessageSizeLimit`` diff --git a/doc/release-notes/11130-update-dataverse-api-remove-metadatablocks.md b/doc/release-notes/11130-update-dataverse-api-remove-metadatablocks.md deleted file mode 100644 index e71a67142b3..00000000000 --- a/doc/release-notes/11130-update-dataverse-api-remove-metadatablocks.md +++ /dev/null @@ -1,8 +0,0 @@ -### Fixes consequences for not adding some optional fields in update dataverse API - -Omitting optional fields inputLevels, facetIds, or metadataBlockNames caused data to be deleted. -This fix no longer deletes data for these fields. Two new flags have been added to the ``metadataBlocks`` Json object to signal the deletion of the data. -- ``inheritMetadataBlocksFromParent: true`` will remove ``metadataBlockNames`` and ``inputLevels`` if the Json objects are omitted. -- ``inheritFacetsFromParent: true`` will remove ``facetIds`` if the Json object is omitted. - -For more information, see issue [#11130](https://github.com/IQSS/dataverse/issues/11130) diff --git a/doc/release-notes/11133-search-fix.md b/doc/release-notes/11133-search-fix.md deleted file mode 100644 index 88962b70ea0..00000000000 --- a/doc/release-notes/11133-search-fix.md +++ /dev/null @@ -1,3 +0,0 @@ -### Search fix when using AVOID_EXPENSIVE_SOLR_JOIN=true - -Dataverse v6.5 introduced a bug which causes search to fail for non-superusers in multiple groups when the AVOID_EXPENSIVE_SOLR_JOIN feature flag is set to true. This releases fixes the bug. diff --git a/doc/release-notes/11136-solr-nested.md b/doc/release-notes/11136-solr-nested.md deleted file mode 100644 index f85c02cde20..00000000000 --- a/doc/release-notes/11136-solr-nested.md +++ /dev/null @@ -1 +0,0 @@ -Deeply nested metadata fields are not supported but the code used to generate the Solr schema has been adjusted to support them. See #11136. diff --git a/doc/release-notes/11142-more-detailed-file-differences.md b/doc/release-notes/11142-more-detailed-file-differences.md deleted file mode 100644 index d7425b8d2fa..00000000000 --- a/doc/release-notes/11142-more-detailed-file-differences.md +++ /dev/null @@ -1,3 +0,0 @@ -The file page version table now shows more detail, e.g. when there are metadata changes or whether a file has been replaced. -A bug that causes adding free-form provenance to a file to fail has been fixed. -See also #11142 and #11145. diff --git a/doc/release-notes/11159-preview-url-update.md b/doc/release-notes/11159-preview-url-update.md deleted file mode 100644 index 2a84540ae75..00000000000 --- a/doc/release-notes/11159-preview-url-update.md +++ /dev/null @@ -1,7 +0,0 @@ -### Preview URL popup updated - -The Preview URL popup and related documentation has been updated to give the dataset more information about anonymous access including the names of the dataset fields that will be withheld from the url user and suggesting how to review the url before releasing it. See also #11159 and #11164. - -###Bug Fix - -Bug which causes users of the Anonymous Review URL to have some metadata of published datasets withheld has been fixed. See #11202 \ No newline at end of file diff --git a/doc/release-notes/11163-CSL citation formats.md b/doc/release-notes/11163-CSL citation formats.md deleted file mode 100644 index d02a0ba4c3b..00000000000 --- a/doc/release-notes/11163-CSL citation formats.md +++ /dev/null @@ -1,5 +0,0 @@ -This release adds support for generating citations in any of the standard independent formats specified using the [Citation Style Language](https://citationstyles.org/). -The CSL formats are available to copy/paste from a new "Cite Dataset" menu "View Styles Citations" pop-up the dataset page. -An API call to retrieve a dataset citation in EndNote, RIS, BibTeX, and CSLJson format has also been added. (The first three have been available as downloads from the UI but have not been directly accessible via API. The CSLJson format is new to Dataverse and can be used with open-source libraries to generate all of the other CSL stypes citations.) - -Admins can use a new dataverse.csl.common-styles setting to highlight commonly used styles - common styles are listed in the pop-up, others can be found by type-ahead search in a list of 1000+ options. diff --git a/doc/release-notes/11178-bug-fix-sort-by-newest-first.md b/doc/release-notes/11178-bug-fix-sort-by-newest-first.md deleted file mode 100644 index 8fb3f65b2f1..00000000000 --- a/doc/release-notes/11178-bug-fix-sort-by-newest-first.md +++ /dev/null @@ -1,8 +0,0 @@ -### Bug fix: Sorting by "newest first" - -Fixed an issue where draft versions of datasets were sorted using the release timestamp of their most recent major version. -This caused newer drafts to appear incorrectly alongside their corresponding major version, instead of at the top, when sorted by "newest first". -Sorting now uses the last update timestamp when sorting draft datasets. -The sorting behavior of published dataset versions (major and minor) is unchanged. - -**Upgrade instructions**: draft datasets must be reindexed for this fix to take effect. \ No newline at end of file diff --git a/doc/release-notes/11188-dataset-version-api.md b/doc/release-notes/11188-dataset-version-api.md deleted file mode 100644 index 2367286d05b..00000000000 --- a/doc/release-notes/11188-dataset-version-api.md +++ /dev/null @@ -1,4 +0,0 @@ -### Retreiving a list of Dataset Versions - -An API has been added to get dataset versions including a summary of diiferences between consecutive versions where available. See [the docs](https://guides.dataverse.org/en/6.6/api/native-api.html#get-versions-of-a-dataset-with-summary-of-changes ), and #10888. - diff --git a/doc/release-notes/11198-list-file-versions-tab b/doc/release-notes/11198-list-file-versions-tab deleted file mode 100644 index 83a4aa00e2c..00000000000 --- a/doc/release-notes/11198-list-file-versions-tab +++ /dev/null @@ -1,10 +0,0 @@ -Feature: New API for SPA to get the file versions metadata to populate the File Page Version Tab - - export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx - export SERVER_URL=https://demo.dataverse.org - export ID=19 - - ID can be either a DataFileID or ":persistentId" with a query parameter ?persistentId=doi:10.5072/FK2/ADMYJF - -``curl -H "X-Dataverse-key: $API_TOKEN" -X GET "$SERVER_URL/api/files/${ID}/versionDifferences"`` -``curl -H "X-Dataverse-key: $API_TOKEN" -X GET "$SERVER_URL/api/files/:persistentId/versionDifferences?persistentId=${ID}"`` diff --git a/doc/release-notes/11203-request-access-fixes.md b/doc/release-notes/11203-request-access-fixes.md deleted file mode 100644 index 7087588c916..00000000000 --- a/doc/release-notes/11203-request-access-fixes.md +++ /dev/null @@ -1,2 +0,0 @@ -Bugs that caused 1) guestbook questions to appear along with terms of use/terms of access in the request access dialog when no guestbook was configured, and 2) terms of access to not be shown when using the per-file request access/download menu items have been fixed. -Text related to configuring the choice to have guestbooks appear when file access is requested or when files are downloaded has been updated to make it clearer that this only affects datasets where guestbooks have been configured. \ No newline at end of file diff --git a/doc/release-notes/11212-postgresql-16.md b/doc/release-notes/11212-postgresql-16.md deleted file mode 100644 index b1a9c8691bf..00000000000 --- a/doc/release-notes/11212-postgresql-16.md +++ /dev/null @@ -1,5 +0,0 @@ -Per PostgreSQL 13 reaches EOL on 13 November 2025. Our first step toward moving off version 13 was to [switch](https://github.com/gdcc/dataverse-ansible/commit/8ebbd84ad2cf3903b8f995f0d34578250f4223ff) our testing to version 16, as we've [noted](https://dataverse-guide--11215.org.readthedocs.build/en/11215/installation/prerequisites.html#postgresql) in the guides. You are encouraged to start planning your upgrade and may want to review the [Dataverse 5.4 release notes](https://github.com/IQSS/dataverse/releases/tag/v5.4) as the upgrade process (e.g. `pg_dumpall`, etc.) will likely be similar. If you notice any bumps along the way, please let us know! - -Dataverse developers [using Docker](https://guides.dataverse.org/en/6.5/container/dev-usage.html) have been using PostgreSQL 17 since Dataverse 6.5 (#10912). (Developers not using Docker who are still on PostgreSQL 13 are encouraged to upgrade.) Older or newer versions should work, within reason. - -See also #11212 and #11215. diff --git a/doc/release-notes/11217-source-name-harvesting-client.md b/doc/release-notes/11217-source-name-harvesting-client.md deleted file mode 100644 index 53347de694c..00000000000 --- a/doc/release-notes/11217-source-name-harvesting-client.md +++ /dev/null @@ -1,13 +0,0 @@ -### Metadata Source Facet Can Now Differentiate Between Harvested Sources - -The behavior of the feature flag `index-harvested-metadata-source` and the "Metadata Source" facet, which were added and updated, respectively, in [Dataverse 6.3](https://github.com/IQSS/dataverse/releases/tag/v6.3) (through pull requests #10464 and #10651), have been updated. A new field called "Source Name" has been added to harvesting clients. - -Before Dataverse 6.3, all harvested content (datasets and files) appeared together under "Harvested" under the "Metadata Source" facet. This is still the behavior of Dataverse out of the box. Since Dataverse 6.3, enabling the `index-harvested-metadata-source` feature flag (and reindexing) resulted in harvested content appearing under the nickname for whatever harvesting client was used to bring in the content. This meant that instead of having all harvested content lumped together under "Harvested", content would appear under "client1", "client2", etc. - -Now, as this release, enabling the `index-harvested-metadata-source` feature flag, populating a new field for harvesting clients called "Source Name" ("sourceName" in the [API](https://dataverse-guide--11217.org.readthedocs.build/en/11217/api/native-api.html#create-a-harvesting-client)), and reindexing (see upgrade instructions below), results in the source name appearing under the "Metadata Source" facet rather than the harvesting client nickname. This gives you more control over the name that appears under the "Metadata Source" facet and allows you to group harvested content from various harvesting clients under the same name if you wish (by reusing the same source name). - -Previously, `index-harvested-metadata-source` was not documented in the guides, but now you can find information about it under [Feature Flags](https://dataverse-guide--11217.org.readthedocs.build/en/11217/installation/config.html#feature-flags). See also #10217 and #11217. - -## Upgrade instructions - -If you have enabled the `dataverse.feature.index-harvested-metadata-source` feature flag and given some of your harvesting clients a source name, you should reindex to have those source names appear under the "Metadata Source" facet. diff --git a/doc/release-notes/11222-ORCID in profile.md b/doc/release-notes/11222-ORCID in profile.md deleted file mode 100644 index a4e0a5cbcc0..00000000000 --- a/doc/release-notes/11222-ORCID in profile.md +++ /dev/null @@ -1,9 +0,0 @@ -Dataverse now includes improved integration with ORCID, supported through a grant to GDCC from the ([ORCID Global Participation Fund](https://info.orcid.org/global-participation-fund-announces-fourth-round-of-awardees/).)[https://info.orcid.org/global-participation-fund-announces-fourth-round-of-awardees/]. - -Specifically, Dataverse users can now link their Dataverse account with their ORCID profile which then allows Dataverse to use automatically add their ORCID to their author metadata when they create a dataset. - -This functionality leverages Dataverse's existing support for login via ORCID, but can be turned on independently of it. If ORCID login is enabled, the user's ORCID will automatically be added to their profile. If the user has logged in via some other mechanism, they are able to click a button to initiate a similar authentication process in which the user must login to their ORCID account and approve the connection. - -Feedback from installations that enable this functionality is requested and we expect that updates can be made in the next Dataverse release. - -See also [the](https://dataverse-guide--11222.org.readthedocs.build/en/11222/user/account.html#linking-orcid-with-your-account-profile) [guides](https://dataverse-guide--11222.org.readthedocs.build/en/11222/installation/orcid.html), #7284, and #11222. diff --git a/doc/release-notes/11228-summaryfields.md b/doc/release-notes/11228-summaryfields.md deleted file mode 100644 index 63a55637934..00000000000 --- a/doc/release-notes/11228-summaryfields.md +++ /dev/null @@ -1 +0,0 @@ -The :CustomDatasetSummaryFields setting now allows spaces along with a comma separating field names. In addition, a bug that caused license information to be hidden if there are no values for any of the custom fields specified has been fixed. \ No newline at end of file diff --git a/doc/release-notes/11230-deleteFiles api call.md b/doc/release-notes/11230-deleteFiles api call.md deleted file mode 100644 index 98754f24e60..00000000000 --- a/doc/release-notes/11230-deleteFiles api call.md +++ /dev/null @@ -1 +0,0 @@ -A new /api/datasets/{id}/deleteFiles call has beed added to the API, allowing delete of multiple file from the latest version of a dataset in one operation. diff --git a/doc/release-notes/11234-DOI Publishing Bug.md b/doc/release-notes/11234-DOI Publishing Bug.md deleted file mode 100644 index 2f8743cbe5c..00000000000 --- a/doc/release-notes/11234-DOI Publishing Bug.md +++ /dev/null @@ -1 +0,0 @@ -This release fixes a bug which could cause publication of datasets to fail in cases where they were not assigned a DOI at creation. \ No newline at end of file diff --git a/doc/release-notes/11242-fix-oricid-recognition.md b/doc/release-notes/11242-fix-oricid-recognition.md deleted file mode 100644 index 268a63bb05e..00000000000 --- a/doc/release-notes/11242-fix-oricid-recognition.md +++ /dev/null @@ -1,3 +0,0 @@ -A bug that caused ORCIDs start with https://orcid.org/ entered as author identifier to be ignored when creating the DataCite metadata has been fixed. This primarily affected users of the ORCID external vocabulary script as the manual entry form recommends not using the URL form. - -The display of authorIdentifier, when not using any external vocabulary scripts, has been improved so that either the plain identifier (e.g. "0000-0002-1825-0097") or its URL form (e.g. "https://orcid.org/0000-0002-1825-0097") will result in valid links in the display (for identifier types that have a URL form). The URL form is now recommended when doing manual entry. diff --git a/doc/release-notes/11244-get-dataset-download-count-api.md b/doc/release-notes/11244-get-dataset-download-count-api.md deleted file mode 100644 index a03510cf1f2..00000000000 --- a/doc/release-notes/11244-get-dataset-download-count-api.md +++ /dev/null @@ -1,12 +0,0 @@ -### New API for the SPA, in order to replicate the "classic" download count. -- ``includeMDC`` parameter is optional. False or not included will return the total count of downloads if MDC is not running. Or, if MDCStartDate is set the count returned will be limited to the time prior to the MDCStartDate. Setting ``includeMDC`` to true will ignore any MDCStartDate and return the total download count for the dataset. - -Example: -``` - API_TOKEN='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' - export DATASET_ID=1 - export includeMDC=False - - curl -s -H "X-Dataverse-key:$API_TOKEN" -X GET http://localhost:8080/api/datasets/$DATASET_ID/download/count - curl -s -H "X-Dataverse-key:$API_TOKEN" -X GET http://localhost:8080/api/datasets/$DATASET_ID/download/count?includeMDC=true -``` diff --git a/doc/release-notes/11255-get-dataverse-api-return-child-count.md b/doc/release-notes/11255-get-dataverse-api-return-child-count.md deleted file mode 100644 index f91cc9ea9f9..00000000000 --- a/doc/release-notes/11255-get-dataverse-api-return-child-count.md +++ /dev/null @@ -1 +0,0 @@ -Added a new query param, ``returnChildCount``, to getDataverse endpoint(``/api/dataverses/``) for optionally retrieving the child count, which represents the number of dataverses, datasets, or files within the dataverse (direct children only). See also #11255 and #11259. diff --git a/doc/release-notes/11256-news-on-support-for-external-vocabulary.md b/doc/release-notes/11256-news-on-support-for-external-vocabulary.md deleted file mode 100644 index 180f888e4df..00000000000 --- a/doc/release-notes/11256-news-on-support-for-external-vocabulary.md +++ /dev/null @@ -1,4 +0,0 @@ -### News on Support for External Vocabulary Services - -It is now possible to fill Dataverse keywords metadata using an OntoPortal service. -The code has been shared on [GDCC GitHub Repository](https://github.com/gdcc/dataverse-external-vocab-support#scripts-in-production). \ No newline at end of file diff --git a/doc/release-notes/11264-slower refresh for long running locks.md b/doc/release-notes/11264-slower refresh for long running locks.md deleted file mode 100644 index 96e0cec0ccb..00000000000 --- a/doc/release-notes/11264-slower refresh for long running locks.md +++ /dev/null @@ -1 +0,0 @@ -When a dataset has a long running lock, including when it is 'in review', Dataverse will now slow the page refresh rate over time. diff --git a/doc/release-notes/11272-fix-order-on-advancedsearchpage.md b/doc/release-notes/11272-fix-order-on-advancedsearchpage.md deleted file mode 100644 index 9db098be3c4..00000000000 --- a/doc/release-notes/11272-fix-order-on-advancedsearchpage.md +++ /dev/null @@ -1,5 +0,0 @@ -### Fix order of metadata fields on the advanced search page - -The metadata fields are now displayed in the correct order as defined in the TSV file via the displayOrder value. -The order of the fields on the AdvancedSearch page will then be more similar to that on the metadata forms. -Note that fields that are not defined in the TSV file, like the 'Persistent ID' and 'Publication Date', will be displayed at the end. diff --git a/doc/release-notes/6.2025.2_update.md b/doc/release-notes/6.2025.2_update.md deleted file mode 100644 index 5724612c6e9..00000000000 --- a/doc/release-notes/6.2025.2_update.md +++ /dev/null @@ -1,35 +0,0 @@ -- Payara application server has been upgraded to version 6.2025.2. See #11126 and #11128. - -## Installation - -The following assumes the standard instructions to `export PAYARA=/usr/local/payara6`, undeploy v6.5, stop payara have been done already. - -### Upgrade to Payara 6.2025.2 - -The steps below reuse your existing domain directory with the new distribution. You may also want to review the Payara upgrade instructions as it could be helpful during any troubleshooting: -[Payara Release Notes](https://docs.payara.fish/community/docs/6.2025.2/Release%20Notes/Release%20Notes%206.2025.2.html). -We also recommend you ensure you followed all update instructions from the past releases regarding Payara. -(The most recent Payara update was for [v6.3](https://github.com/IQSS/dataverse/releases/tag/v6.3).) - -Move the current Payara directory out of the way: - -```shell -mv $PAYARA $PAYARA.6.2024.6 -``` - -Download the new Payara version 6.2025.2 (from https://www.payara.fish/downloads/payara-platform-community-edition/ or https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2025.2/payara-6.2025.2.zip), and unzip it in its place: - -```shell -cd /usr/local -unzip payara-6.2025.2.zip -``` - -Replace the brand new `payara/glassfish/domains/domain1` with your old, preserved domain1: - -```shell -mv payara6/glassfish/domains/domain1 payara6/glassfish/domains/domain1_DIST -mv payara6.6.2024.6/glassfish/domains/domain1 payara6/glassfish/domains/ -``` - -Deploy/install as normal - diff --git a/doc/release-notes/6.6-release-notes.md b/doc/release-notes/6.6-release-notes.md new file mode 100644 index 00000000000..751c471a0a3 --- /dev/null +++ b/doc/release-notes/6.6-release-notes.md @@ -0,0 +1,549 @@ +# Dataverse 6.6 + +Please note: To read these instructions in full, please go to https://github.com/IQSS/dataverse/releases/tag/v6.6 rather than the [list of releases](https://github.com/IQSS/dataverse/releases), which will cut them off. + +This release brings new features, enhancements, and bug fixes to Dataverse. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project! + +## Release Highlights + +Highlights for Dataverse 6.6 include: + +- metadata fields can be "display on create" per collection +- ORCIDs linked to accounts +- version notes +- harvesting from DataCite +- citations using Citation Style Language (CSL) +- license metadata enhancements +- metadata fields now support range searches (dates, integers, etc.) +- more accurate search highlighting +- collections can be moved by using the superuser dashboard +- new 3D Objects metadata block +- new Archival metadata block (experimental) +- optionally prevent publishing of datasets without files +- Signposting output now contains links to all dataset metadata export formats +- infrastructure updates (Payara and Solr) + +In a recent community call, we talked about many of these highlights if you'd like to watch the [video](https://harvard.zoom.us/rec/share/Ir5CkFHkzoya9b5Nk69rLFUpTyGics3-KGLl9WITSLMy4ezHRsB8CnY22cUNg2g.JPpxrjzHMeCii_zO) (around 22:30). + +## Features Added + +### Metadata Fields Can Be "Display on Create" Per Collection + +Collection administrators can now configure which metadata fields appear during dataset creation through the `displayOnCreate` property, even when fields are not required. This provides greater control over metadata visibility and can help improve metadata completeness. + +Currently this feature can only be configured [via API](https://guides.dataverse.org/en/6.6/api/native-api.html#update-collection-input-levels), but a UI implementation is planned in #11221. See #10476, #11224, and #11312. + +### ORCIDs Linked to Accounts + +Dataverse now includes improved integration with ORCID, supported through a grant to GDCC from the ([ORCID Global Participation Fund](https://info.orcid.org/global-participation-fund-announces-fourth-round-of-awardees/)). + +Specifically, Dataverse users can now link their Dataverse account with their ORCID profile. Previously, this was only available to users who logged in with ORCID. Once linked, Dataverse will automatically prepopulate their ORCID to their author metadata when they create a dataset. + +This functionality leverages Dataverse's existing support for login via ORCID, but can be turned on independently of it. If ORCID login is enabled, the user's ORCID will automatically be added to their profile. If the user has logged in via some other mechanism, they are able to click a button to initiate a similar authentication process in which the user must login to their ORCID account and approve the connection. + +Feedback from installations that enable this functionality is requested and we expect that updates can be made in the next Dataverse release. + +See the [User Guide](http://guides.dataverse.org/en/6.6/user/account.html#linking-orcid-with-your-account-profile), [Installation Guide](http://guides.dataverse.org/en/6.6/installation/orcid.html), #7284, and #11222. + +### Version Notes + +Dataverse now supports the option of adding a version note before or during the publication of a dataset. These notes can be used, for example, to indicate why a version was created or how it differs from the prior version. Whether this feature is enabled is controlled by the flag `dataverse.feature.enable-version-note`. Version notes are shown in the user interface (in the dataset page version table), indexed (as `versionNote`), available via the API, and have been added to the JSON, DDI, DataCite, and OAI-ORE exports. + +With the addition of this feature, work has been done to clean-up and rename fields that have been used for specifying the reason for deaccessioning a dataset and providing an optional link to a non-Dataverse location where the dataset still can be found. The former was listed in some JSON-based API calls and exports as "versionNote" and is now "deaccessionNote", while the latter was referred to as "archiveNote" and is now "deaccessionLink". + +Further, some database consolidation has been done to combine the deaccessionlink and archivenote fields, which appear to have both been used for the same purpose. The deaccessionlink database field is older and also was not displayed in the current UI. Going forward, only the deaccessionlink column exists. + +See the [User Guide](https://guides.dataverse.org/en/6.6/user/dataset-management.html#data-provenance), [API Guide](https://guides.dataverse.org/en/6.6/api/native-api.html#dataset-version-notes) #8431, and #11068. + +### OAI-PMH Harvesting from DataCite + +DataCite maintains an OAI server () that serves records for every DOI they have registered. There's been a lot of interest in the community in being able to harvest from them. This way, it will be possible to harvest metadata from institution X even if the institution X does not maintain an OAI server of their own, if they happen to register their DOIs with DataCite. One extra element of this harvesting model that makes it especially powerful and flexible is the DataCite's concept of a "dynamic OAI set": a harvester is not limited to harvesting the pre-defined set of ALL the records registered by the institution X, but can instead harvest virtually any arbitrary subset thereof; any query that the DataCite search API understands can be used as an OAI set. The feature is already in use at Harvard Dataverse, as a beta version patch. + +For various reasons, in order to take advantage of this feature harvesting clients must be created using the `/api/harvest/clients` API. Once configured however, harvests can be run from the Harvesting Clients control panel in the UI. + +DataCite-harvesting clients must be configured with 2 new feature flags, `useListRecords` and `useOaiIdentifiersAsPids` (added in Dataverse 6.5). Note that these features may be of use when harvesting from other sources, not just from DataCite. + +See the [Admin Guide](http://guides.dataverse.org/en/6.6/admin/harvestclients.html#harvesting-from-datacite), [API Guide](http://guides.dataverse.org/en/6.6/api/native-api.html#harvesting-from-datacite), #10909, and #11011. + +### Citations Using Citation Style Language (CSL) + +This release adds support for generating citations in any of the standard independent formats specified using the [Citation Style Language](https://citationstyles.org). + +The CSL formats are available to copy/paste if you click "Cite Dataset" and then "View Styled Citations" on the dataset page. An API call to retrieve a dataset citation in EndNote, RIS, BibTeX, and CSLJson format has also been added. The first three have been available as downloads from the UI (CSLJson is not) but have not been directly accessible via API until now. The CSLJson format is new to Dataverse and can be used with open source libraries to generate all of the other CSL-style citations. + +Admins can use a new `dataverse.csl.common-styles` setting to highlight commonly used styles. Common styles are listed in the pop-up, while others can be found by type-ahead search in a list of 1000+ options. + +See the [User Guide](http://guides.dataverse.org/en/6.6/user/find-use-data.html#cite-data), [Settings](http://guides.dataverse.org/en/6.6/installation/config.html#dataverse-csl-common-styles), [API Guide](http://guides.dataverse.org/en/6.6/api/native-api.html#get-citation-in-other-formats), and #11163. + +### License Metadata Enhancements + +- Added new fields to licenses: rightsIdentifier, rightsIdentifierScheme, schemeUri, languageCode. See JSON files under [Adding Licenses](https://guides.dataverse.org/en/6.6/installation/config.html#adding-licenses) in the guides +- Updated DataCite metadata export to include rightsIdentifier, rightsIdentifierScheme, and schemeUri consistent with the DataCite 4.5 schema and examples +- Enhanced metadata exports to include all new license fields +- Existing licenses from the example set included with Dataverse will be automatically updated with new fields +- Existing API calls support the new optional fields + +See below for upgrade instructions. See also #10883 and #11232. + +### Range Search + +This release enhances how numerical and date fields are indexed in Solr. Previously, all fields were indexed as English text (`text_en`), but with this update: + +* Integer fields are indexed as `plong` +* Float fields are indexed as `pdouble` +* Date fields are indexed as `date_range` (`solr.DateRangeField`) + +This change enables range queries when searching from both the UI and the API, such as `dateOfDeposit:[2000-01-01 TO 2014-12-31]` or `targetSampleActualSize:[25 TO 50]`. See below for a full list of fields that now support range search. + +Additionally, search result highlighting is now more accurate, ensuring that only fields relevant to the query are highlighted in search results. If the query is specifically limited to certain fields, the highlighting is now limited to those fields as well. See #10887. + +Specifically, the following fields were updated: + +- coverage.Depth +- coverage.ObjectCount +- coverage.ObjectDensity +- coverage.Redshift.MaximumValue +- coverage.Redshift.MinimumValue +- coverage.RedshiftValue +- coverage.SkyFraction +- coverage.Spectral.CentralWavelength +- coverage.Spectral.MaximumWavelength +- coverage.Spectral.MinimumWavelength +- coverage.Temporal.StartTime +- coverage.Temporal.StopTime +- dateOfCollectionEnd +- dateOfCollectionStart +- dateOfDeposit +- distributionDate +- dsDescriptionDate +- journalPubDate +- productionDate +- resolution.Redshift +- targetSampleActualSize +- timePeriodCoveredEnd +- timePeriodCoveredStart + +### New 3D Objects Metadata Block + +A new metadata block has been added for describing 3D object data. You can download it from the [guides](https://guides.dataverse.org/en/6.6/user/appendix.html). See also #11120 and #11167. + +All new Dataverse installations will receive this metadata block by default. We recommend adding it by following the upgrade instructions below. + +### New Archival Metadata Block (Experimental) + +An experimental "Archival" metadata block has been added, [downloadable](https://guides.dataverse.org/en/6.6/user/appendix.html) from the User Guide. The purpose of the metadata block is to enable repositories to register metadata relating to the potential archiving of the dataset at a depositor archive, whether that being your own institutional archive or an external archive, i.e. a historical archive. Feedback is welcome! See also #10626. + +### Prevent Publishing of Datasets Without Files + +Datasets without files can be optionally prevented from being published through a new "requireFilesToPublishDataset" boolean defined at the collection level. This boolean can be set only via API and only by a superuser. See [Change Collection Attributes](https://guides.dataverse.org/en/6.6/api/native-api.html#change-collection-attributes). If the boolean is not set, the parent collection is consulted. If you do not set the boolean, the existing behavior of datasets being able to be published without files will continue. Superusers can still publish datasets whether or not the boolean is set. See #10981 and #10994. + +### Metadata Source Facet Can Now Differentiate Between Harvested Sources + +The behavior of the feature flag `index-harvested-metadata-source` and the "Metadata Source" facet, which were added and updated, respectively, in [Dataverse 6.3](https://github.com/IQSS/dataverse/releases/tag/v6.3) (through pull requests #10464 and #10651), have been updated. A new field called "Source Name" has been added to harvesting clients. + +Before Dataverse 6.3, all harvested content (datasets and files) appeared together under "Harvested" under the "Metadata Source" facet. This is still the behavior of Dataverse out of the box. Since Dataverse 6.3, enabling the `index-harvested-metadata-source` feature flag (and reindexing) resulted in harvested content appearing under the nickname for whatever harvesting client was used to bring in the content. This meant that instead of having all harvested content lumped together under "Harvested", content would appear under "client1", "client2", etc. + +With this release, enabling the `index-harvested-metadata-source` feature flag, populating a new field for harvesting clients called "Source Name" ("sourceName" in the [API](https://dataverse-guide--11217.org.readthedocs.build/en/11217/api/native-api.html#create-a-harvesting-client)), and reindexing (see upgrade instructions below) results in the source name appearing under the "Metadata Source" facet rather than the harvesting client nickname. This gives you more control over the name that appears under the "Metadata Source" facet and allows you to reuse the same source name to group harvested content from various harvesting clients under the same name if you wish. + +Previously, `index-harvested-metadata-source` was not documented in the guides, but now you can find information about it under [Feature Flags](https://guides.dataverse.org/en/6.6/installation/config.html#feature-flags). See also #10217 and #11217. + +### Globus Framework Improvements + +The improvements and optimizations in this release build on top of the earlier work (such as #10781). They are based on the experience gained at IQSS as part of the production rollout of the Large Data Storage services that utilizes Globus. + +The changes in this release focus on improving Globus *downloads*, i.e., transfers from Dataverse-linked Globus volumes to users' Globus collections. Most importantly, the mechanism of "Asynchronous Task Monitoring", first introduced in #10781 for *uploads*, has been extended to handle downloads as well. This generally makes downloads more reliable, specifically in how Dataverse manages temporary access rules granted to users, minimizing the risk of consequent downloads failing because of stale access rules left in place. + +Multiple other improvements have been made making the underlying Globus framework more reliable and robust. + +See `globus-use-experimental-async-framework` under [Feature Flags](https://guides.dataverse.org/en/6.6/installation/config.html#feature-flags) and [dataverse.files.globus-monitoring-server](https://guides.dataverse.org/en/6.6/installation/config.html#dataverse-files-globus-monitoring-server) in the Installation Guide, #11057, and #11125. + +### OIDC Bearer Tokens + +The release extends the OIDC API auth mechanism, available through feature flag `api-bearer-auth`, to properly handle cases where ``BearerTokenAuthMechanism`` successfully validates the token but cannot identify any Dataverse user because there is no account associated with the token. + +To register a new user who has authenticated via an OIDC provider, a new endpoint has been implemented (`/users/register`). A feature flag named `api-bearer-auth-provide-missing-claims` has been implemented to allow sending missing user claims in the request JSON. This is useful when the identity provider does not supply the necessary claims. However, this flag will only be considered if the `api-bearer-auth` feature flag is enabled. If the latter is not enabled, the `api-bearer-auth-provide-missing-claims` flag will be ignored. + +A feature flag named `api-bearer-auth-handle-tos-acceptance-in-idp` has been implemented. When enabled, it specifies that Terms of Service acceptance is managed by the identity provider, eliminating the need to explicitly include the acceptance in the user registration request JSON. + +See [the guides](https://guides.dataverse.org/en/6.6/api/auth.html#bearer-tokens), #10959, and #10972. + +### Signposting Output Now Contains Links to All Dataset Metadata Export Formats + +When Signposting was added in Dataverse 5.14 (#8981), it provided links only for the `schema.org` metadata export format. + +The output of HEAD, GET, and the Signposting "linkset" API have all been updated to include links to all available dataset metadata export formats, including any external exporters, such as Croissant, that have been enabled. + +This provides a lightweight machine-readable way to first retrieve a list of links, such as via a HTTP HEAD request, to each available metadata export format and then follow up with a request for the export format of interest. + +In addition, the content type for the `schema.org` dataset metadata export format has been corrected. It was `application/json` and now it is `application/ld+json`. + +See also [the guides](https://guides.dataverse.org/en/6.6/api/native-api.html#retrieve-signposting-information), #10542 and #11045. + +### Dataset Types Can Be Linked to Metadata Blocks + +Metadata blocks, such as (e.g. "CodeMeta") can now be linked to dataset types (e.g. "software") using new superuser APIs. + +This will have the following effects for the APIs used by the [new Dataverse UI](https://github.com/IQSS/dataverse-frontend): + +- The list of fields shown when creating a dataset will include fields marked as "displayoncreate" (in the tsv/database) for metadata blocks (e.g. "CodeMeta") that are linked to the dataset type (e.g. "software") that is passed to the API. +- The metadata blocks shown when editing a dataset will include metadata blocks (e.g. "CodeMeta") that are linked to the dataset type (e.g. "software") that is passed to the API. + +Mostly in order to write automated tests for the above, a [displayOnCreate](https://guides.dataverse.org/en/6.6/api/native-api.html#set-displayoncreate-for-a-dataset-field) API endpoint has been added. + +For more information, see the guides ([overview](https://guides.dataverse.org/en/6.6/user/dataset-management.html#dataset-types), [new APIs](https://guides.dataverse.org/en/6.6/api/native-api.html#link-dataset-type-with-metadata-blocks)), #10519 and #11001. + +### Other Features + +- In addition to the API [Move a Dataverse Collection](https://guides.dataverse.org/en/6.6/admin/dataverses-datasets.html#move-a-dataverse-collection), it is now possible for a Dataverse administrator to move a collection using the Dataverse dashboard. See #10304 and #11150. +- The Preview URL popup and [related documentation](https://guides.dataverse.org/en/6.6/user/dataset-management.html#preview-url-to-review-unpublished-dataset) have been updated to give more information about anonymous access, including the names of the dataset fields that will be withheld from the Anonymous Preview URL user and to suggest how to review the URL before releasing it. See also #11159 and #11164. +- [ROR](https://ror.org) (Research Organization Registry) has been added as an Author Identifier Type for when the author is an organization rather than a person. Like ORCID, ROR will appear in the "Datacite" metadata export format. See #11075 and #11118. +- The publisher value of harvested datasets is now attributed to the dataset's distributor instead of its producer. This improves the citation associated with these datasets, but the change affects only newly harvested datasets. See "Upgrade Instructions" below on how to re-harvest. For more information, see [the guides](http://guides.dataverse.org/en/6.6/admin/harvestclients.html#harvesting-client-changelog), #8739, and #9013. +- A new harvest status differentiates between a complete harvest with errors ("completed with failures") and without errors ("completed"). Also, harvest status labels are now internationalized. See #9294 and #11017. +- The OAI-ORE exporter can now export metadata containing nested compound fields or compound fields within compound fields. See #10809 and #11190. +- It is now possible to edit a custom role with the same alias. See #8808 and #10612. +- The [Metadata Customization](https://guides.dataverse.org/en/6.6/admin/metadatacustomization.html#controlledvocabulary-enumerated-properties) documentation has been updated to explain how to implement a boolean fieldtype (look for "boolean"). See #7961 and #11064. +- The version of Stata files is now detected during S3 direct upload (as it was for normal uploads), allowing ingest of Stata 14 and 15 files that have been uploaded directly. See [the guides](https://guides.dataverse.org/en/6.6/developers/big-data-support.html#features-that-are-disabled-if-s3-direct-upload-is-enabled) #10108, and #11054. +- It is now possible to populate the "Keyword" metadata field from an [OntoPortal](https://ontoportal.org) service. The code has been shared to the GDCC [dataverse-external-vocab-support](https://github.com/gdcc/dataverse-external-vocab-support#scripts-in-production) GitHub repository. See #11258. +- Support for legacy configuration of a PermaLink PID provider, such as using the :Protocol,:Authority, and :Shoulder settings, has been fixed. See #10516 and #10521. +- On the home page for each guide (User Guide, etc.) there was an overwhelming amount of information in the form of a deeply nested table of contents. The depth of the table of contents has been reduced to two levels, making the home page for each guide more readable. Compare the User Guide for [6.5](https://guides.dataverse.org/en/6.5/user/index.html) vs. [6.6](https://guides.dataverse.org/en/6.6/user/index.html) and see #11166. +- For compliance with GDPR and other privacy regulations, advice on adding a cookie consent popup has been added to the guides. See the new [cookie consent](https://guides.dataverse.org/en/6.6/installation/config.html#adding-cookie-consent-for-gdpr-etc) section and #10320. +- A new file has been added to import the French Open License to Dataverse: licenseEtalab-2.0.json. You can download it from [the guides](http://guides.dataverse.org/en/6.6/installation/config.html#adding-licenses). This license, which is compatible with the Creative Commons license, is recommended by the French government for open documents. See #9301, #9302, and #11302. +- The API that lists versions of a dataset now features an optional `excludeMetadataBlocks` parameter, which defaults to "false" for backward compatibility. For a dataset with a large number of versions and/or metadataBlocks, having the metadata blocks included can dramatically increase the volume of the output. See also [the guides](https://guides.dataverse.org/en/6.6/api/native-api.html#list-versions-of-a-dataset), #10171, and #10778. +- Deeply nested metadata fields are not supported but the code used to generate the Solr schema has been adjusted to support them. See #11136. +- The [tutorial](https://guides.dataverse.org/en/6.6/container/running/demo.html) on running Dataverse in Docker has been updated to explain how to configure the root collection using a JSON file (#10541 and #11201) and now uses the Permalink PID provider instead of the FAKE DOI Provider (#11107 and #11108). +- Payara application server has been upgraded to version 6.2025.2. See #11126 and #11128. +- Solr has been upgraded to version 9.8.0. See #10713. +- For testing purposes, the FAKE PID provider can now be used with [file PIDs enabled](https://guides.dataverse.org/en/6.6/installation/config.html#filepidsenabled). (The FAKE provider is not recommended for any production use.) See #10979. + +## Bugs Fixed + +- A bug which causes users of the Anonymous Review URL to have some metadata of published datasets withheld has been fixed. See #11202 and #11164. +- A bug that caused ORCIDs starting with "https://orcid.org/" entered as author identifier to be ignored when creating the DataCite metadata has been fixed. This primarily affected users of the ORCID external vocabulary script; for the manual entry form, we used to recommend not using the URL form. The display of authorIdentifier, when not using any external vocabulary scripts, has been improved so that either the plain identifier (e.g. "0000-0002-1825-0097") or its URL form (e.g. "https://orcid.org/0000-0002-1825-0097") will result in valid links in the display (for identifier types that have a URL form). The URL form is now [recommended](http://guides.dataverse.org/en/6.6/user/dataset-management.html#adding-a-new-dataset) when doing manual entry. See #11242 and #11242. +- Multiple small issues with the formatting of PIDs in the DDI exporters, and EndNote and BibTeX citation formats have been addressed. These should improve the ability to import Dataverse citations into reference managers and fix potential issues harvesting datasets using PermaLinks. See #10768, #10769, #11165, and #10790. +- On the Advanced Search page, the metadata fields are now displayed in the correct order as defined in the TSV file via the displayOrder value, making the order the same as when you view or edit metadata. Note that fields that are not defined in the TSV file, like the "Persistent ID" and "Publication Date", will be displayed at the end. See #11272 and #11279. +- Bugs that caused 1) guestbook questions to appear along with terms of use/terms of access in the request access dialog when no guestbook was configured, and 2) terms of access to not be shown when using the per-file request access/download menu items have been fixed. Text related to configuring the choice to have guestbooks appear when file access is requested or when files are downloaded has been updated to make it clearer that this affects only datasets where guestbooks have been configured. See #11203. +- The file page version table now shows whether a file has been replaced. See #11142 and #11145. +- We fixed an issue where draft versions of datasets were sorted using the release timestamp of their most recent major version. This caused newer drafts to appear incorrectly alongside their corresponding major version, instead of at the top, when sorted by "newest first". Sorting now uses the last update timestamp when sorting draft datasets. The sorting behavior of published major and minor dataset versions is unchanged. There is no need to reindex datasets because Solr is being upgraded (see "Upgrade Instructions"), which will result in an empty database that will be reindexed. See #11178. +- Some external controlled vocabulary scripts/configurations, when used on a metadata field that is single-valued, could result in indexing failure for the dataset, e.g. when the script tried to index both the identifier and name of the identified entity for indexing. Dataverse has been updated to correctly indicate the need for a multi-valued Solr field in these cases in the call to `/api/admin/index/solr/schema`. Configuring the Solr schema and running the update-fields.sh script as usually recommended when using custom metadata blocks (see "Upgrade Instructions") will resolve the issue. See [the guides](https://guides.dataverse.org/en/6.6/admin/metadatacustomization.html#using-external-vocabulary-services), #11095, and #11096. +- The OpenAIRE metadata export format can now correctly process one or multiple productionPlaces as geolocation. See #9546 and #11194 +- We fixed a bug that caused adding free-form provenance to a file to fail. See #11145. +- A bug has been fixed which could cause publication of datasets to fail in cases where they were not assigned a DOI at creation. See #11234 and #11236. +- When users request access to files, the people who have permission to grant access received an email with a link that didn't work due to a trailing period (full stop) right next to the link, e.g. `https://demo.dataverse.org/permissions-manage-files.xhtml?id=9.` A space has been added to fix this. See #10384 and #11115. +- Harvesting clients now use the correct granularity while re-running a partial harvest, using the `from` parameter. The correct granularity comes from the `Identify` verb request. See #11020 and #11038. +- Access requests were missing on the File Permission page after upgrading from Dataverse 6.0. This has been corrected with a database update script. See #10714 and #11061. +- When a dataset has a long running lock, including when it is "in review", Dataverse will now slow the page refresh rate over time. See #11264 and #11269. +- The `/api/info/metrics/files/monthly` API call had a bug that resulted in files being counted each time they were published in a new version if those publication events occurred in different months. This resulted in an over-count. The `/api/info/metrics/files` and `/api/info/metrics/files/toMonth` API calls had a bug that resulted in files that were published but no longer in the latest published version as of the specified date (now, or the date entered in the `/toMonth` variant). This resulted in an under-count. See #11189. +- DatasetFieldTypes in MetadataBlock response that are also a child of another DatasetFieldType were being returned twice. The child DatasetFieldType was included in the "fields" object as well as in the "childFields" of its parent DatasetFieldType. This fix suppresses the standalone object so only one instance of the DatasetFieldType is returned (in the "childFields" of its parent). This fix changes the JSON output of the API `/api/dataverses/{dataverseAlias}/metadatablocks` (see "Backward Incompatible Changes", below). See #10472 and #11066. +- A bug that caused replacing files via API when file PIDs were enabled to fail has been fixed. See #10975 and #10979. +- The [:CustomDatasetSummaryFields](https://guides.dataverse.org/en/6.6/installation/config.html#customdatasetsummaryfields) setting now allows spaces along with a comma separating field names. In addition, a bug that caused license information to be hidden if there are no values for any of the custom fields specified has been fixed. See #11228 and #11229. +- Dataverse 6.5 introduced a bug which causes search to fail for non-superusers in multiple groups when the `AVOID_EXPENSIVE_SOLR_JOIN` feature flag is set to true. This release fixes the bug. See #11133 and #11134. +- We fixed a bug with My Data where listing collections for a user with only rights on harvested collections would result in a server error response. See #11083. +- Minor styling fixes for the Related Publication field and fields using ORCID or ROR have been made. See #11053, #10964, and #11106. +- In the Search API, files were displaying DRAFT version instead of latest released version under `dataset_citation`. See #10735 and #11051. +- Unnecessary Solr documents were being created when a file was added or deleted from a draft dataset. These documents could accumulate and potentially impact performance. There is no action to take because this release includes a new Solr version, which will start with an empty database. See #11113 and #11114. +- When using the API to update a collection, omitting optional fields such as `inputLevels`, `facetIds`, or `metadataBlockNames` caused data to be deleted. The fix no longer deletes data for these fields. Two new flags have been added to the `metadataBlocks` JSON object to signal the deletion of the data: `inheritMetadataBlocksFromParent: true` and `inheritFacetsFromParent: true`. See [the guides](https://guides.dataverse.org/en/6.6/api/native-api.html#update-a-dataverse-collection), #11130, and #11144. + +## API Updates + +### Search API Returns Additional Fields for Files + +Added new fields to search results type=files + +For Files: + +- restricted: boolean +- canDownloadFile: boolean (from file user permission) +- categories: array of string "categories" would be similar to what it is in metadata api. + +For tabular files: + +- tabularTags: array of string for example, `{"tabularTags" : ["Event", "Genomics", "Geospatial"]}` +- variables: number/int shows how many variables we have for the tabular file +- observations: number/int shows how many observations for the tabular file + +See #11027 and #11097. + +### Backend Support for Collection Featured Items + +CRUD endpoints for Collection Featured Items have been implemented. In particular, the following endpoints have been implemented: + +- Create a feature item (POST `/api/dataverses//featuredItems`) +- Update a feature item (PUT `/api/dataverseFeaturedItems/`) +- Delete a feature item (DELETE `/api/dataverseFeaturedItems/`) +- List all featured items in a collection (GET `/api/dataverses//featuredItems`) +- Delete all featured items in a collection (DELETE `/api/dataverses//featuredItems`) +- Update all featured items in a collection (PUT `/api/dataverses//featuredItems`) + +See also the "Settings Added" section, #10943 and #11124. + +### Other API Updates + +- Multiple files can be deleted from a dataset at once. See the [the guides](https://guides.dataverse.org/en/6.6/api/native-api.html#delete-files-from-a-dataset) and #11230. +- An API has been added to get the "classic" download count from a dataset with an optional `includeMDC` parameter (for Make Data Count). See [the guides](http://guides.dataverse.org/en/6.6/api/native-api.html#get-the-download-count-of-a-dataset), #11244 and #11282. +- An API has been added that lists the collections that the user has access to via the permission passed. See [the guides](http://guides.dataverse.org/en/6.6/api/native-api.html#list-dataverse-collections-a-user-can-act-on-based-on-their-permissions), #6467, and #10906. +- An API has been added to get dataset versions including a summary of differences between consecutive versions where available. See [the docs](https://guides.dataverse.org/en/6.6/api/native-api.html#get-versions-of-a-dataset-with-summary-of-changes ), #10888, and #10945. +- An API has been added to list of versions of a data file showing any changes that affected the file with each version. See [the guides](https://guides.dataverse.org/en/6.6/api/native-api.html#get-json-representation-of-a-file-s-versions), #11198 and #11237. +- The Search API has a new [parameter](https://guides.dataverse.org/en/6.6/api/search.html#parameters) called `show_type_counts`. If you set it to true, it will return `total_count_per_object_type` for the types dataverse, dataset, and files (#11065 and #11082) even if the search result for any given type is 0 (#11127 and #11138). +- CRUD operations for external tools are now available for superusers from non-localhost. See [the guides](https://guides.dataverse.org/en/6.6/admin/external-tools.html#managing-external-tools), #10930 and #11079. +- A new API endpoint has been added that allows a global role to be updated. See [the guides](https://guides.dataverse.org/en/6.6/api/native-api.html#update-global-role) and #10612. +- An API has been added to send feedback to the collection, dataset, or data file's contacts. If necessary, you can [rate limit](https://guides.dataverse.org/en/6.6/installation/config.html#rate-limiting) the `CheckRateLimitForDatasetFeedbackCommand` and configure the new [:ContactFeedbackMessageSizeLimit](https://guides.dataverse.org/en/6.6/installation/config.html#contactfeedbackmessagesizelimit) database setting. See [the guides](http://guides.dataverse.org/en/6.6/api/native-api.html#send-feedback-to-contact-s), #11129, and #11162. +- /api/metadatablocks is no longer returning duplicated metadata properties and does not omit metadata properties when called. See "Backward Incompatible Changes" below and #10764. +- A new query param, `returnChildCount`, has been added to the getDataverse endpoint (`/api/dataverses/{id}`) for optionally retrieving the child count, which represents the number of collections, datasets, or files within the collection (direct children only). See also #11255 and #11259. + +## End-Of-Life (EOL) Announcements + +### PostgreSQL 13 reaches EOL on 13 November 2025 + +Per PostgreSQL 13 reaches EOL on 13 November 2025. Our first step toward moving off version 13 was to [switch](https://github.com/gdcc/dataverse-ansible/commit/8ebbd84ad2cf3903b8f995f0d34578250f4223ff) our testing to version 16, as we've [noted](https://guides.dataverse.org/en/6.6/installation/prerequisites.html#postgresql) in the guides. You are encouraged to start planning your upgrade and may want to review the [Dataverse 5.4 release notes](https://github.com/IQSS/dataverse/releases/tag/v5.4) as the upgrade process (e.g. `pg_dumpall`, etc.) will likely be similar. If you notice any bumps along the way, please let us know! + +Dataverse developers [using Docker](https://guides.dataverse.org/en/6.6/container/dev-usage.html) have been using PostgreSQL 17 since Dataverse 6.5 (#10912). (Developers not using Docker who are still on PostgreSQL 13 are encouraged to upgrade.) Older or newer versions should work, within reason. + +See also #11212 and #11215. + +## Security + +### SameSite Cookie Attribute + +The SameSite cookie attribute is defined in an upcoming revision to [RFC 6265](https://datatracker.ietf.org/doc/html/rfc6265) (HTTP State Management Mechanism) called [6265bis](https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-rfc6265bis-19>) ("bis" meaning "repeated"). The possible values are "None", "Lax", and "Strict". + +"If no SameSite attribute is set, the cookie is treated as Lax by default" by browsers according to [MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#controlling_third-party_cookies_with_samesite). This was the previous behavior of Dataverse, to not set the SameSite attribute. + +New Dataverse installations now explicitly set to the SameSite cookie attribute to "Lax" out of the box through the installer (in the case of a "classic" installation) or through an updated base image (in the case of a Docker installation). Classic installations should follow the upgrade instructions below to bring their installation up to date with the behavior for new installations. Docker installations will automatically get the updated base image. + +While you are welcome to experiment with "Strict", which is intended to help prevent Cross-Site Request Forgery (CSRF) attacks, as described in the RFC proposal and an OWASP [cheatsheet](https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html#samesite-cookie-attribute), our testing so far indicates that some functionality, such as OIDC login, seems to be incompatible with "Strict". + +You should avoid the use of "None" as it is less secure than "Lax". See also [the guides](https://guides.dataverse.org/en/6.6/installation/config.html#samesite-cookie-attribute), https://github.com/IQSS/dataverse-security/issues/27, #11210, and the upgrade instructions below. + +## Settings Added + +- dataverse.feature.enable-version-note +- dataverse.csl.common-styles +- dataverse.files.featured-items.image-maxsize - It sets the maximum allowed size of the image that can be added to a featured item. +- dataverse.files.featured-items.image-uploads - It specifies the name of the subdirectory for saving featured item images within the docroot directory. +- dataverse.feature.api-bearer-auth-provide-missing-claims +- dataverse.feature.api-bearer-auth-handle-tos-acceptance-in-idp +- :ContactFeedbackMessageSizeLimit + +## Backward Incompatible Changes + +Generally speaking, see the [API Changelog](https://guides.dataverse.org/en/latest/api/changelog.html) for a list of backward-incompatible API changes. + +- /api/metadatablocks is no longer returning duplicated metadata properties and does not omit metadata properties when called. See #10764. +- The JSON response of API call `/api/dataverses/{dataverseAlias}/metadatablocks` will no longer include the DatasetFieldTypes in "fields" if they are children of another DatasetFieldType. The child DatasetFieldType will only be included in the "childFields" of its parent DatasetFieldType. See #10472 and #11066. +- `versionNote` has been renamed to `deaccessionNote`. `archiveNote` has been renamed to `deaccessionLink`. See #11068. +- The [Show Role](https://guides.dataverse.org/en/6.6/api/native-api.html#show-role) API endpoint was returning 401 Unauthorized when a permission check failed. This has been corrected to return 403 Forbidden instead. That is, the API token is known to be good (401 otherwise) but the user lacks permission (403 is now sent). See also the [API Changelog](https://guides.dataverse.org/en/6.6/11116/api/changelog.html), #10340, and #11116. +- Changes to PID formatting occur in the DDI/DDI Html export formats and the EndNote and BibTex citation formats. These changes correct errors and improve conformance with best practices but could break parsing of these formats. See #10768, #10769, #11165, and #10790. + +## Complete List of Changes + +For the complete list of code changes in this release, see the [6.6 milestone](https://github.com/IQSS/dataverse/issues?q=milestone%3A6.6+is%3Aclosed) in GitHub. + +## Getting Help + +For help with upgrading, installing, or general questions please post to the [Dataverse Community Google Group](https://groups.google.com/g/dataverse-community) or email support@dataverse.org. + +## Installation + +If this is a new installation, please follow our [Installation Guide](https://guides.dataverse.org/en/latest/installation/). Please don't be shy about [asking for help](https://guides.dataverse.org/en/latest/installation/intro.html#getting-help) if you need it! + +Once you are in production, we would be delighted to update our [map of Dataverse installations](https://dataverse.org/installations) around the world to include yours! Please [create an issue](https://github.com/IQSS/dataverse-installations/issues) or email us at support@dataverse.org to join the club! + +You are also very welcome to join the [Global Dataverse Community Consortium](https://www.gdcc.io/) (GDCC). + +## Upgrade Instructions + +Upgrading requires a maintenance window and downtime. Please plan accordingly, create backups of your database, etc. + +These instructions assume that you've already upgraded through all the 5.x releases and are now running Dataverse 6.5. + +0\. These instructions assume that you are upgrading from the immediate previous version. If you are running an earlier version, the only supported way to upgrade is to progress through the upgrades to all the releases in between before attempting the upgrade to this version. + +If you are running Payara as a non-root user (and you should be!), **remember not to execute the commands below as root**. By default, Payara runs as the `dataverse` user. In the commands below, we use sudo to run the commands as a non-root user. + +Also, we assume that Payara 6 is installed in `/usr/local/payara6`. If not, adjust as needed. + +```shell +export PAYARA=/usr/local/payara6 +``` + +(or `setenv PAYARA /usr/local/payara6` if you are using a `csh`-like shell) + +1\. List deployed applications + +```shell +$PAYARA/bin/asadmin list-applications +``` + +2\. Undeploy the previous version (should match "list-applications" above) + +```shell +$PAYARA/bin/asadmin undeploy dataverse-6.5 +``` + +3\. Stop Payara + +```shell +sudo service payara stop +``` + +4\. Upgrade to Payara 6.2025.2 + +The steps below reuse your existing domain directory with the new distribution of Payara. You may also want to review the Payara upgrade instructions as it could be helpful during any troubleshooting: +[Payara Release Notes](https://docs.payara.fish/community/docs/6.2025.2/Release%20Notes/Release%20Notes%206.2025.2.html). +We also recommend you ensure you followed all update instructions from the past releases regarding Payara. +(The most recent Payara update was for [v6.3](https://github.com/IQSS/dataverse/releases/tag/v6.3).) + +Move the current Payara directory out of the way: + +```shell +mv $PAYARA $PAYARA.6.2024.6 +``` + +Download the new Payara version 6.2025.2 (from https://www.payara.fish/downloads/payara-platform-community-edition/ or https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2025.2/payara-6.2025.2.zip), and unzip it in its place: + +```shell +cd /usr/local +unzip payara-6.2025.2.zip +``` + +Replace the brand new `payara/glassfish/domains/domain1` with your old, preserved domain1: + +```shell +mv payara6/glassfish/domains/domain1 payara6/glassfish/domains/domain1_DIST +mv payara6.6.2024.6/glassfish/domains/domain1 payara6/glassfish/domains/ +``` + +5\. Download and deploy this version + +```shell +wget https://github.com/IQSS/dataverse/releases/download/v6.6/dataverse-6.6.war +$PAYARA/bin/asadmin deploy dataverse-6.6.war +``` + +Note: if you have any trouble deploying, stop Payara, remove the following directories, start Payara, and try to deploy again. + +```shell +sudo service payara stop +sudo rm -rf $PAYARA/glassfish/domains/domain1/generated +sudo rm -rf $PAYARA/glassfish/domains/domain1/osgi-cache +sudo rm -rf $PAYARA/glassfish/domains/domain1/lib/databases +sudo service payara start +``` + +6\. For installations with internationalization or text customizations: + +Please remember to update translations via [Dataverse language packs](https://github.com/GlobalDataverseCommunityConsortium/dataverse-language-packs). + +If you have text customizations you can get the latest English files from . + +7\. Decide to enable (or not) the `index-harvested-metadata-source` feature flag + +Decide whether or not to enable the `dataverse.feature.index-harvested-metadata-source` feature flag described above, in [the guides](https://guides.dataverse.org/en/6.6/installation/config.html#feature-flags), #10217 and #11217. The reason to decide now is that reindexing is required and the next steps involve restarting Payara and upgrading Solr, which will result in a fresh index. + +8\. Configure SameSite + +To bring your Dataverse installation in line with new installations, as described above and in [the guides](https://guides.dataverse.org/en/6.6/installation/config.html#samesite-cookie-attribute), we recommend running the following commands: + +``` +./asadmin set server-config.network-config.protocols.protocol.http-listener-1.http.cookie-same-site-value=Lax + +./asadmin set server-config.network-config.protocols.protocol.http-listener-1.http.cookie-same-site-enabled=true +``` + +Please note that "None" is less secure than "Lax" and should be avoided. You can test the setting by inspecting headers with curl, looking at the JSESSIONID cookie for "SameSite=Lax" (yes, it's expected to be repeated, probably due to a bug in Payara) like this: + +``` +% curl -s -I http://localhost:8080 | grep JSESSIONID +Set-Cookie: JSESSIONID=6574324d75aebeb86dc96ecb3bb0; Path=/;SameSite=Lax;SameSite=Lax +``` + +Before making the changes above, SameSite attribute should be absent, like this: + +``` +% curl -s -I http://localhost:8080 | grep JSESSIONID +Set-Cookie: JSESSIONID=6574324d75aebeb86dc96ecb3bb0; Path=/ +``` + +8\. Restart Payara + +```shell +sudo service payara stop +sudo service payara start +``` + +9\. Update metadata blocks + +These changes reflect incremental improvements made to the handling of core metadata fields. + +Expect the loading of the citation block to take several seconds because of its size (especially due to the number of languages). + +```shell +wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/scripts/api/data/metadatablocks/citation.tsv + +curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file citation.tsv +``` + +The 3D Objects metadata block is included in all new installations of Dataverse so we recommend adding it. + +```shell +wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/scripts/api/data/metadatablocks/3d_objects.tsv + +curl http://localhost:8080/api/admin/datasetfield/load -H "Content-type: text/tab-separated-values" -X POST --upload-file 3d_objects.tsv +``` + +10\. Upgrade Solr + +Solr 9.8.0 is now the version recommended in our Installation Guide and used with automated testing. Additionally, due to the new range search support feature and the addition of fields (e.g. versionNote, fileRestricted, canDownloadFile, variableCount, and observations), the default `schema.xml` files has changed so you must upgrade. + +Install Solr 9.8.0 following the [instructions](https://guides.dataverse.org/en/6.6/installation/prerequisites.html#solr) from the Installation Guide. + +The instructions in the guide suggest to use the config files from the installer zip bundle. When upgrading an existing instance, it may be easier to download them from the source tree: + +```shell +wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/conf/solr/solrconfig.xml +wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/conf/solr/schema.xml +cp solrconfig.xml schema.xml /usr/local/solr/solr-9.8.0/server/solr/collection1/conf +``` + +10a\. For installations with additional metadata blocks or external controlled vocabulary scripts, update fields + +- Stop Solr instance (usually `service solr stop`, depending on Solr installation/OS, see the [Installation Guide](https://guides.dataverse.org/en/6.6/installation/prerequisites.html#solr-init-script)). + +- Run the `update-fields.sh` script that we supply, as in the example below (modify the command lines as needed to reflect the correct path of your Solr installation): + +```shell +wget https://raw.githubusercontent.com/IQSS/dataverse/v6.6/conf/solr/update-fields.sh +chmod +x update-fields.sh +curl "http://localhost:8080/api/admin/index/solr/schema" | ./update-fields.sh /usr/local/solr/solr-9.8.0/server/solr/collection1/conf/schema.xml +``` + +- Start Solr instance (usually `service solr start` depending on Solr/OS). + +11\. Reindex Solr + +```shell +curl http://localhost:8080/api/admin/index +``` + +12\. Run reExportAll to update dataset metadata exports + +For existing published datasets, additional license metadata will not be available from DataCite or in metadata exports until + +- the dataset is republished or +- the /api/admin/metadata/{id}/reExportDataset is run for the dataset or +- the /api/datasets/{id}/modifyRegistrationMetadata API is run for the dataset or +- the global version of these API calls (/api/admin/metadata/reExportAll, /api/datasets/modifyRegistrationPIDMetadataAll) are used. + +For this reason, we recommend reexporting all dataset metadata. For more advanced usage, please see [the guides](http://guides.dataverse.org/en/6.6/admin/metadataexport.html#batch-exports-through-the-api). + +```shell +curl http://localhost:8080/api/admin/metadata/reExportAll +``` + +13\. (Optional) Re-harvest datasets + +The publisher value of harvested datasets is now attributed to the dataset's distributor instead of its producer. For more information, see [the guides](http://guides.dataverse.org/en/6.6/admin/harvestclients.html#harvesting-client-changelog), #8739, and #9013. + +This improves the citation associated with these datasets, but the change only affects newly harvested datasets. + +If you would like to pick up this change for existing harvested datasets, you should re-harvest them. This can be accomplished by deleting and re-adding each harvesting client, followed by a harvesting run. You may want to use [harvesting client APIs](https://guides.dataverse.org/en/6.6/api/native-api.html#managing-harvesting-clients) to save (serialize), add, and remove clients. diff --git a/doc/release-notes/6467-optimize-permission-lookups-for-a-user.md b/doc/release-notes/6467-optimize-permission-lookups-for-a-user.md deleted file mode 100644 index 395c6b05c0a..00000000000 --- a/doc/release-notes/6467-optimize-permission-lookups-for-a-user.md +++ /dev/null @@ -1,9 +0,0 @@ -The following API have been added: - -/api/users/{identifier}/allowedCollections/{permission} - -This API lists the dataverses/collections that the user has access to via the permission passed. -By passing "any" as the permission the list will return all dataverse/collections that the user can access regardless of which permission is used. -This API can be executed only by the User requesting their own list of accessible collections or by an Administrator. -Valid Permissions are: AddDataverse, AddDataset, ViewUnpublishedDataverse, ViewUnpublishedDataset, DownloadFile, EditDataverse, EditDataset, ManageDataversePermissions, -ManageDatasetPermissions, ManageFilePermissions, PublishDataverse, PublishDataset, DeleteDataverse, DeleteDatasetDraft, and "any" as a wildcard option. diff --git a/doc/release-notes/8431-versionNotes.md b/doc/release-notes/8431-versionNotes.md deleted file mode 100644 index b9804833548..00000000000 --- a/doc/release-notes/8431-versionNotes.md +++ /dev/null @@ -1,12 +0,0 @@ -Dataverse now supports the option of adding a versionNote before/during publication of a dataset that can be used to indicate why a version was created and/or how it differs from the prior version. Whether this feature is enabled is controlled by a flag. Version notes are shown in the user interface (dataset page version table), indexed, available via the API, and have been added to the JSON, DDI, DataCite, and OAI-ORE exports. - -With the addition of this feature, work has been done to clean-up and rename fields that have been used for specifying the reason for deaccessioning a dataset and providing an optional link to a non-Dataverse location where the dataset still can be found. The former was listed in some JSON-based API calls and exports as "versionNote" and is now "deaccessionNote", while the latter was referred to as "archiveNote" and is now "deacccessionLink". These result in incompatibilities in the UI related to deaccessioned datasets. - -Further, some database consolidation has been done to combine the deaccessionlink and archivenote fields which appear to have both been used for the same purpose (the deaccessionlink db field is older and was not displayed in the current UI. Going forward, only the deaccessionlink column exists. - -New Feature Flags: - -VERSION_NOTE - false by default. - -Update of the solr schema (using the standard instructions) is needed. - diff --git a/doc/release-notes/8739-publisher-during-harvesting.md b/doc/release-notes/8739-publisher-during-harvesting.md deleted file mode 100644 index 3e1555396c0..00000000000 --- a/doc/release-notes/8739-publisher-during-harvesting.md +++ /dev/null @@ -1 +0,0 @@ -The publisher value of harvested datasets is now attributed to the dataset's distributor instead of its producer. This improves the citation associated with these datasets, but the change only affects newly harvested datasets. All datasets should be re-harvested if you wish to pick up this change on already harvested datasets. For more information, see [the guides](https://dataverse-guide--9013.org.readthedocs.build/en/9013/admin/harvestclients.html#harvesting-client-changelog), #8739, and #9013. \ No newline at end of file diff --git a/doc/release-notes/8808-10575-update-global-role.md b/doc/release-notes/8808-10575-update-global-role.md deleted file mode 100644 index 38142f972e8..00000000000 --- a/doc/release-notes/8808-10575-update-global-role.md +++ /dev/null @@ -1,11 +0,0 @@ -## Release Highlights - -### Update a Global Role - -A new API endpoint has been added that allows a global role to be updated. See [Native API Guide > Update Global Role](https://guides.dataverse.org/en/6.3/api/native-api.html#update-global-role) (#10612) - -## Bug fixes - -### Edition of custom role fixed - -It is now possible to edit a custom role with the same alias (reported in #8808) \ No newline at end of file diff --git a/doc/release-notes/9294-improvement-and-internationalization-of-harvest-status.md b/doc/release-notes/9294-improvement-and-internationalization-of-harvest-status.md deleted file mode 100644 index f9fc465292c..00000000000 --- a/doc/release-notes/9294-improvement-and-internationalization-of-harvest-status.md +++ /dev/null @@ -1,6 +0,0 @@ -## Improvement and internationalization of harvest status - -Added a harvest status to differentiate a complete harvest with errors (Completed with failures) and without errors (Completed) -Harvest status labels are now internationalized - -For more information, see issue [#9294](https://github.com/IQSS/dataverse/issues/9294) \ No newline at end of file diff --git a/doc/release-notes/9302-add-Etalab-License.md b/doc/release-notes/9302-add-Etalab-License.md deleted file mode 100644 index 8009f3c1b53..00000000000 --- a/doc/release-notes/9302-add-Etalab-License.md +++ /dev/null @@ -1,5 +0,0 @@ -A new file has been added to import the French Open License to Dataverse: licenseEtalab-2.0.json. You can download it from [the guides](https://dataverse-guide--9302.org.readthedocs.build/en/9302/installation/config.html#adding-licenses). - -This license, which is compatible with the Creative Commons license, is recommended by the French government for open documents. - -For more information, see #9301, #9302, and #11302. diff --git a/doc/release-notes/Metrics-FixFileAPIs.md b/doc/release-notes/Metrics-FixFileAPIs.md deleted file mode 100644 index 871f606f2f6..00000000000 --- a/doc/release-notes/Metrics-FixFileAPIs.md +++ /dev/null @@ -1,2 +0,0 @@ -The /api/info/metrics/files/monthly API call had a bug that resulted in files being counted each time they were published in a new version if those publication events occurred in different months. This resulted in an over-count. -The /api/info/metrics/files and /api/info/metrics/files/toMonth API calls had a bug that resulted in files that were published but no longer in the latest published version as of the specified date (now, or the date entered in the /toMonth variant). This resulted in an under-count. diff --git a/doc/release-notes/ds27-samesite-attr.md b/doc/release-notes/ds27-samesite-attr.md deleted file mode 100644 index f25e6f680c3..00000000000 --- a/doc/release-notes/ds27-samesite-attr.md +++ /dev/null @@ -1,35 +0,0 @@ -### SameSite Cookie Attribute - -The SameSite cookie attribute is defined in an upcoming revision to [RFC 6265](https://datatracker.ietf.org/doc/html/rfc6265) (HTTP State Management Mechanism) called [6265bis](https://datatracker.ietf.org/doc/html/draft-ietf-httpbis-rfc6265bis-19>) ("bis" meaning "repeated"). The possible values are "None", "Lax", and "Strict". - -"If no SameSite attribute is set, the cookie is treated as Lax by default" by browsers according to [MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Cookies#controlling_third-party_cookies_with_samesite). This was the previous behavior of Dataverse, to not set the SameSite attribute. - -New Dataverse installations now explicitly set to the SameSite cookie attribute to "Lax" out of the box through the installer (in the case of a "classic" installation) or through an updated base image (in the case of a Docker installation). Classic installations should follow the upgrade instructions below to bring their installation up to date with the behavior for new installations. Docker installations will automatically get the updated base image. - -While you are welcome to experiment with "Strict", which is intended to help prevent Cross-Site Request Forgery (CSRF) attacks, as described in the RFC proposal and an OWASP [cheetsheet](https://cheatsheetseries.owasp.org/cheatsheets/Cross-Site_Request_Forgery_Prevention_Cheat_Sheet.html#samesite-cookie-attribute), our testing so far indicates that some functionality, such as OIDC login, seems to be incompatible with "Strict". - -You should avoid the use of "None" as it is less secure that "Lax". See also [the guides](https://dataverse-guide--11210.org.readthedocs.build/en/11210/installation/config.html#samesite-cookie-attribute), https://github.com/IQSS/dataverse-security/issues/27, #11210, and the upgrade instructions below. - -## Upgrade instructions - -To bring your Dataverse installation in line with new installations, as described in [the guides](https://dataverse-guide--11210.org.readthedocs.build/en/11210/installation/config.html#samesite-cookie-attribute), we recommend running the following commands: - -``` -./asadmin set server-config.network-config.protocols.protocol.http-listener-1.http.cookie-same-site-value=Lax - -./asadmin set server-config.network-config.protocols.protocol.http-listener-1.http.cookie-same-site-enabled=true -``` - -Please note that "None" is less secure than "Lax" and should be avoided. You can test the setting by inspecting headers with curl, looking at the JSESSIONID cookie for "SameSite=Lax" (yes, it's expected to be repeated, probably due to a bug in Payara) like this: - -``` -% curl -s -I http://localhost:8080 | grep JSESSIONID -Set-Cookie: JSESSIONID=6574324d75aebeb86dc96ecb3bb0; Path=/;SameSite=Lax;SameSite=Lax -``` - -Before making the changes above, SameSite attribute should be absent, like this: - -``` -% curl -s -I http://localhost:8080 | grep JSESSIONID -Set-Cookie: JSESSIONID=6574324d75aebeb86dc96ecb3bb0; Path=/ -``` diff --git a/doc/release-notes/guides-toc-depth.md b/doc/release-notes/guides-toc-depth.md deleted file mode 100644 index 0f5dd39186f..00000000000 --- a/doc/release-notes/guides-toc-depth.md +++ /dev/null @@ -1,3 +0,0 @@ -### Guides Table of Contents Depth - -On the home page for each guide (User Guide, etc.) there was an overwhelming amount of information in the form of a deeply nested tabled of contents. The depth of the table of contents has been reduced to two levels, making the home page for each guide more readable. See #11166. diff --git a/doc/release-notes/master_json_fix.md b/doc/release-notes/master_json_fix.md deleted file mode 100644 index aa30b90c2cb..00000000000 --- a/doc/release-notes/master_json_fix.md +++ /dev/null @@ -1 +0,0 @@ -This pull request fixes an issue in the JsonPrinter class so that there are no duplicated entries in the JSON metadata or ommitted metadata properties. After the fix is applied the /api/metadatablocks/ endpoint should return correct JSON. \ No newline at end of file diff --git a/doc/release-notes/openaire_fix.md b/doc/release-notes/openaire_fix.md deleted file mode 100644 index 49a809db14b..00000000000 --- a/doc/release-notes/openaire_fix.md +++ /dev/null @@ -1 +0,0 @@ -bugfix: openaire implementation can now correctly process one or multiple productionPlaces as geolocation \ No newline at end of file