Skip to content

CHANGELOG.adoc

Latest commit

 

History

History
295 lines (196 loc) · 14.9 KB

CHANGELOG.adoc

File metadata and controls

295 lines (196 loc) · 14.9 KB
Raw

Changelog

Unreleased

  • Add path option to thor rr list command

  • Add progress bar to thor batch upload command

7.2.0 (2026-07-28)

  • thor rr list now can report on all mappable rectypes, not just authorities

  • Remove thor-hollaback dependency and related code. We now rely on the at_exit block in the base module to close database tunnel and connection as needed on any form of exit.

  • Rename thor tm run to thor tm run_plan - without thor-hollaback, thor starts complaining that run is a protected keyword

  • thor batch map command now outputs both CS XML and mapping report paths separately, more clearly labeling them. == 7.1.0 (2026-05-13)

  • Skip tunnel when creating a db connection if CMT.config.client.db_tunnel_skip = true

  • Add thor duplicates check_all command

  • Fix closing of db/tunnel after each duplicate check query

  • Fix bug with passing batch_config to collectionspace-mapper

7.0.1 (2026-02-25)

  • Fix thor config show error described in #113

7.0.0 (2026-02-15)

Breaking

  • Simplifly client config by pushing some things to system config and "automagically" figuring out settings for common setup

6.0.1 (2026-02-03)

Bugfix

  • Fixed the occasional spurious duplicate file error described in #69

6.0.0 (2026-01-26)

Breaking

  • Client config files need to specify site_name instead of tenant_name.

Changed

  • Use CHIA 2.0.0, which refers to sites instead of tenants.

5.2.0 (2026-01-20)

Added

  • Add db_tunnel_connection_pause to system config, so that the sleep time between tunnelling and making a connection can be increased if connections are failing.

Changed

  • Makes database connections via new read-only database user

5.1.0 (2025-10-21)

Added

  • Initial record harvesting functionality. See thor harvest

  • Initial Term Manager functionality (thor tm)

  • thor tm reset_exact_term_lists command

Bugfix

  • Fix for ruby/openssl bug

  • Do not hit CHIA if getting non-verbose current client config (Resolves #73)

5.0.1 (2025-06-10)

Bugfix

  • Fix for error message: "WARNING: unable to load thorfile "/lib/tasks/term_manager.thor": undefined method `no_command' for Thor::Sandbox::TermManager:Class"

5.0.0 (2025-06-09)

Breaking

Now requires login credentials for instances to be fetched via AWS SSM.

To upgrade:

  • System config:

    • Set the new bastion_user value if you will be doing anything that needs to connect to CollectionSpace databases

    • Change aws_profile value from collectionspacemigrationtools to collectionspace unless you are maintaining entry for same profile under different name in credentials file

    • Change aws_media_ingest_profile from cs-media to shared unless you are maintaining entry for same profile under different name in credentials file

  • Client config:

    • For hosted clients: Populate new tenant_name setting. See README’s Per-instance setup. Delete base_uri, username, and password entries/values. Also delete the whole database config section. See sample_client_config.yml for more detail.

Added

  • Write refname report for a list of authority vocabularies to CSV via rr write list thor command. The ability to produce this is handy if your project runs into this ingest limitation/bug

  • Initial work/documentation of TermManager functionality, not yet complete

Changed

  • You can now indicate the location of your system config file via setting an environment variable, via dotfile convention. See the README section on System config for details.

4.1.0 (2024-09-12)

Added

  • Support for deleting vocabularies via thor csid delete

Changes

  • s3_bucket is now optional in client configs, since this tool may be used in some cases to interact directly with an instance via its REST API, without use of Fast Import Bucket.

  • Improve map error message when a non-duplicate file name cannot be saved because Mac’s default file system (case insensitive) does treat it as a duplicate

Bugfixes

  • Fixes issue where, if you had somehow gotten into a state where the current config was invalid, you could not switch to another config using the thor config switch command.

  • Fixes issue where batches delete_done command deleted non-done batches

4.0.0 (2024-04-15)

Breaking

  • done as an explicit batch status is removed from the application. Commands to mark batches as done are removed. batches:done will now display all batches with batch_status = ingested.

  • batches:to_ingcheck command renamed batches:to_ingstat, in order to be consistent with batches:ingstat and batch:ingstat commands.

  • Columns in batches CSV have changed. You will be prompted to run thor batches:fix_csv at some point, or you can run it proactively.

Added

  • batch_status column to batches CSV

  • batch_mode column to batches CSV (added in mapping step)

  • ingest duration calculated, shown in STDOUT, and recorded in batches CSV if ingest is complete when batch:ingstat is run

  • Batch archiving is enabled by default

Changes

  • csid delete --csv option now uses ingest_dir if configured and only a file name is given

  • Reset default client config media_with_blob_upload_delay value to 500ms in accordance with the sample client config

  • batches show now shows headers and batch status

Bugfixes

  • Fix issue with vt add command, in which ampersands in terms were not escaped in the created XML, causing errors (#39)

3.0.1 (2024-03-11)

Changes

  • Speed up caching CSIDs and refnames by pipelining values directly to Redis instead of using individual collectionspace-refcache calls

  • vt add --csv option now uses ingest_dir if configured and only a file name is given (#53)

Bugfixes

  • Fix for #55 - thor batch mtprep command failing when all missing terms are vocabulary terms (not authority terms)

  • When batch config’s batch_mode is "date details", autocache vocabularies for the batch (#58)

3.0.0 (2024-03-01)

Breaking

  • Client configuration management changes. To use new version:

    • In your system_config.yml file:

    • Remove initial redis: line and indentation of subsequent lines from your redis.yml file if necessary

    • After adding client_config_dir to your system config, you will need to run thor config switch CONFIGNAME

    • Delete client_config.yml from the CMT repository base directory if it is still there.

    • Changes were made to how the active config is stored/looked up after I discovered you could get stuck unable to switch away from an invalid config. The changes have other side benefits, too.

Added

  • Optional ingest_dir setting in client config, which allows you to enter file name only when calling thor batch add --csv option for files in that default ingest directory.

  • thor cspace:reindex command to trigger fulltext reindexing of the configured CollectionSpace instance

  • Utility script: utils/copy_media_bucket_objects_in_place.rb - If media ingest bucket is created without overriding the default AWS encryption settings, we run into problems with access to the files. Infrastructure can fix the encryption settings of the bucket, but any objects already in the bucket need to be copied in place to refresh them to use the new encryption settings. This is a quick/dirty script to do all that copying.

Bugfixes

  • Report issue/don’t fail messily when there is a missing item type value in a nonhierarchicalrelationship row

  • Fix failure when reporting ingstat for batches with non-ingested items (i.e. items that failed map or upload steps and were thus not ingested)

2.2.5 (2024-02-11)

Bugfixes

  • Uses new version of collectionspace-mapper with a bugfix to prevent errors in reporting failures in date details mapping.

2.2.4 (2024-02-07)

Added

  • Use new version of collectionspace-mapper that supports ingesting fields in same repeating field group as a structured date when you are ingesting structured date details for a date in that group.

2.2.3 (2024-02-01)

Bugfixes

  • Use new version of collectionspace-mapper that supports retaining %NULLVALUE% fields as blank fields, and sets shortid correctly when mapping date details for authority records.

2.2.2 (2023-12-19)

Added

  • thor rt:delete_all command for objects, procedures, and authorities. Do thor rt help delete_all for more details. (PR#42)

  • Attempts to handle ingest errors using AWS Lambda logs — still janky (PR#43, 44, 46)

Bugfixes

  • Fix issue where application failed when there are duplicates reported in ingstat check (PR#45)

  • Issue blocking ingest of chronology authority terms (PR#46)

  • Update collectionspace-mapper to get fix for #148 (PR#47)

  • Fixes CLI output where processes didn’t report about themselves correctly (PR#47)

2.2.1 (2023-04-26)

Bugfixes (related to media file ingest)

  • Update collectionspace-mapper to get fix for #34

  • Escape spaces in mediafileuri values, then url_encode them before construction S3 object key (#36)

2.2.0 (2023-04-24)

Added

  • Optional aws_media_ingest_profile system config setting, specifying AWS profile through which to access client media ingest bucket

  • Optional media_bucket client config setting, specifying name of client media ingest bucket

  • Two new commands to control the access policy of client media ingest bucket:

    • thor bucket:public (makes bucket public, so media can be ingested)

    • thor bucket:private (makes bucket private)

2.1.0 (2023-04-19)

Added

2.0.0 (2023-03-10)

Breaking

  • Upload of CS XML objects to S3 bucket for ingest now requires AWS profile to be set up on local system, and aws_profile setting in system_config.yml

Added

  • New media_with_blob_upload_delay and max_media_upload_threads client config settings added (PR#27)

Bugfixes

Changed

  • A number of client config settings now have defaults set by the tool, and do not need to be included in your config YML files unless you need to override the default values. See CLIENT_CONFIG_DEFAULTS in lib/collectionspace_migration_tools/configuration.rb for details. (PR#27)

1.1.0 (2022-12-14)

Added

  • thor csid:delete command, allowing deletion of records by rectype+CSID. Do thor csid help delete for more details.

Changed

  • Option for passing in custom source for thor media:deriv_report changed from blob_data to csv for better consistency with other commands, and less typing

  • When uploading media with blob to S3, post-upload "sleep post #{identificationnumber}" message written to STDOUT. This is temporary behavior to give some feedback that progress is occurring with the expected upload delay, since the upload process now takes ages.

  • Default system config max_threads increased to 10

Bugfixes

  • Fixes issue in thor media:deriv_report where report generation would fail if there was only one derivative item for a blob.

1.0.0 (2022-12-08)

This is not a breaking change, but reflects this is now being used for CS migrations by more than one person.

Added

  • When mapping a batch with rectype=media, rows with blob_uri values that cannot be converted into URI objects will get a "media_uri cannot be encoded as valid ingest URI. File ingest may not work as expected" warning. This is a warning, not an error, since CS can successfully ingest from file paths that do not convert to valid URIs.

  • thor decode:objects that decodes object keys of all objects in S3 bucket, writing the results to a CSV in your base directory. This streamlines the process of getting the human-readable record id values for objects left in S3 bucket (assumed to be ingest errors)

  • thor media:blob_data command to write report of all media procedures and, if present, their blob details. See media ingest documentation.

  • thor media:deriv_report command to generate report of derivatives present for each blobcsid given. See media ingest documentation.

  • Optional :media_with_blob_upload_delay client config setting. When uploading to S3 bucket for ingest, will wait this long after each media record that has an associated mediaFileURI value.

0.1.2 (2022-12-02)

  • Use collectionspace-mapper v4.1.2 to get error handling bugfix

0.1.1 (2022-11-21)

Bugfixes

  • Use collectionspace-mapper v4.1.1 to get scalar date creation bugfix

0.1.0 (2022-11-17)

Added

Changed

  • bin/console now uses Pry instead of IRB (PR#20)

Deleted

  • Config validation constraint requiring bastion-host value to contain bastion. The host for tunneling for a project I am working on does not contain bastion (PR#19)

0.0.2 (2022-10-18)

Bugfixes

  • Bumps version of collectionspace-mapper used, to get fixes for date processing (PR#17)

Added

  • ohc as valid profile in config_client_contract (PR#15)

0.0.1 (2022-02-11)

  • Added initial working version (skeletal!)

0.0.0 (2022-02-01)

  • Added initial implementation.