Release `26XX` distribution

[jpraynaud]

Why

In order to deploy new features, improvements and bug fixes, we need to release a new distribution.

Note

The previous distribution has been created with #3271.

What

Create and release a new distribution 26XX

Important

• Rotate ancillary signer service account keys inrelease-mainnet
• Remove deprecated features of mithril-client
• Clean infra upgrade branches if any
• Upgrade to Cardano node 11.2.X?
  • pre-release-preview
  • release-preprod
  • release-mainnet
• Prepare new configurations of infra for on-chain protocol configuration (protocol-config.addr and protocol-config.vk):
  • pre-release-preview
  • release-preprod
  • release-mainnet
• Upgrade to on-chain protocol configuration (PROTOCOL_CONFIGURATION_READER_ADDRESS_URL, PROTOCOL_CONFIGURATION_READER_VERIFICATION_KEY_URL new GitHub environment variables, remove AGGREGATOR_CARDANO_BLOCKS_TRANSACTIONS_DATABASE_CONNECTION_POOL_SIZE, AGGREGATOR_CARDANO_BLOCKS_TRANSACTIONS_PROVER_CACHE_POOL_SIZE,AGGREGATOR_CARDANO_BLOCKS_TRANSACTIONS_SIGNING_CONFIG_STEP, AGGREGATOR_CARDANO_BLOCKS_TRANSACTIONS_SIGNING_CONFIG_SECURITY_PARAMETER, AGGREGATOR_CARDANO_TRANSACTIONS_DATABASE_CONNECTION_POOL_SIZE, AGGREGATOR_CARDANO_TRANSACTIONS_PROVER_CACHE_POOL_SIZE, AGGREGATOR_CARDANO_TRANSACTIONS_SIGNING_CONFIG_SECURITY_PARAMETER, AGGREGATOR_CARDANO_TRANSACTIONS_SIGNING_CONFIG_STEP)
  • pre-release-preview
  • pre-release-2-preview
  • release-preprod
  • release-mainnet
• Activate CardanoBlocksTransactions signed entity type?:
  • release-mainnet
• Dev blog post:
  • Distribution XXXX
  • Create a dev blog post explaining the transition to dual genesis
  • Update the Multiple Aggregators testing phase dev blog post
  • TBD
• TBD

Distribution +1:

Note

• Rotate ancillary signer service account keys inrelease-mainnet
• Remove deprecated features of mithril-client
• Clean infra upgrade branches if any
• Dev blog post:
  • Distribution XXXX
  • TBD
• TBD

How

Phase 1: Prepare distribution

• Refresh NPM token for publication
• Run manual successful restoration of Cardano database + restart of the Cardano node (with ancillary)
• Run a backward compatibility test in the CI on the unstable distribution (with the unstable built e2e test)
• Run an all green aggregator stress test in the CI
• Run an all green client multi-platform test in the CI
• Prepare a PR to upgrade repository dependencies (with runbook)
• Reset the CI caches just before merging the PR to upgrade repository dependencies
• Merge the PR to upgrade repository dependencies
• Check if Cardano minimum supported versions in networks.json must be updated following the runbook
• Create a draft PR for rotating the documentation
• Create a draft PR for updating networks.json file
• Create a PR for rotating the CHANGELOG.md (set the distribution version)
• Update the PR for rotating the CHANGELOG.md (if needed)
• Merge the PR for rotating the CHANGELOG.md
• Prepare announcements and highlights
• Prepare dev blog posts

Phase 2: Rollout pre-release distribution

• Remove CardanoImmutableFilesFull from AGGREGATOR_SIGNED_ENTITY_TYPES in:
  • pre-release-preview
• Tag and deploy a 26XX.0 pre-release distribution on the pre-release-preview network
• Fill information in GitHub release:
  • Release notes
  • Highlights
• Make sure pre-release-preview is working as expected
• Create fix deployment jpraynaud/fix-warp-huge-backtraces branch
• Activate CardanoBlocksTransactions signed entity type:
  • pre-release-preview
• Re-deploy pre-release-preview with jpraynaud/fix-warp-huge-backtraces branch:
  • First with previous distribution image 2617.0-2478748
  • Then with new distribution 26XX.0-23e124d
• Make sure pre-release-preview is working as expected
• Merge fix deployment jpraynaud/fix-warp-huge-backtraces branch on main
• Make sure Grafana dashboards are fetching data for pre-release-preview
• Make sure libraries version next have been published on npm registry
• Run an all green client multi-platform test in the CI
• Run a Cardano transaction prover benchmark:
  • Store results in the issue
  • Make sure that there is no performance regression since last release
• Announce the pre-release on discord channel
• Announce any breaking change in:
  • #cardano-scaling slack channel for pre-release
  • #blockfrost-scaling slack channel for pre-release
  • #ddz-iog-daedalus slack channel for pre-release
• Wait for at least 2 epochs and check that:
  • Previous signers have registered in the current epoch
  • New signers have registered in the current epoch
  • Previous signers have signed in the 2 epochs
  • New signers have signed in the 2 epochs
  • Certificates are created for the CardanoBlocksTransactions

Phase 3: Rollout release distribution

• Rotate KMS credentials:
  • Destroy the previous key for the ancillary signer service account of release-mainnet
  • Create a new key for the ancillary signer service account of release-mainnet
  • Configure release-mainnet environment with new service account credentials
  • Check that KMS credentials are working with the creation of the next Cardano database snapshot
• Release the 26XX.0 distribution
• Fill information in GitHub release:
  • Cardano node compatibility
  • Networks compatibility
• Make sure release-mainnet is working as expected
• Make sure release-preprod is working as expected
• Remove CardanoImmutableFilesFull from AGGREGATOR_SIGNED_ENTITY_TYPES in:
  • release-preprod
  • release-mainnet
• Activate CardanoBlocksTransactions signed entity type:
  • release-preprod
• Update CardanoBlocksTransactions configuration parameter AGGREGATOR_CARDANO_BLOCKS_TRANSACTIONS_SIGNING_CONFIG_SECURITY_PARAMETER to 5:
  • release-preprod
• Re-deploy release-mainnet and release-preprod with jpraynaud/fix-warp-huge-backtraces branch
  • First with previous distribution image 2617.0-2478748
  • Then with new distribution 26XX.0-23e124d
• Make sure release-mainnet is working as expected
• Make sure release-preprod is working as expected
• Make sure libraries have been published on crates.io
• Make sure libraries version latest have been published on npm registry
• Make sure Grafana dashboards are fetching data for release-mainnet
• Make sure Grafana dashboards are fetching data for release-preprod
• Merge the PR for rotating the documentation
• Merge the PR for updating networks.json file
• Merge the PR for rotating the CHANGELOG.md
• Announce the release on discord channel (with breaking-change, upcoming features activation, ...)
• Announce any breaking/new feature change in:
  • #cardano-scaling slack channel for release
  • #blockfrost-scaling slack channel for release
  • #ddz-iog-daedalus slack channel for release
• Update Hydra tutorial if client breaking change for database restoration
• Dev blog post:
  • Distribution 26XX
  • Update the Switching to Cardano database incremental certification dev blog post
• Update roadmap with new distribution
• Update the signer version in the next Cardano node release