Release 0.33

[lidel]

Part of ongoing maintenance of key libraries and services ipshipyard/roadmaps#1

Meta

• Release owner: @lidel
• Release reviewer: @gammazero
• Expected RC1 date: Tue 17th or Fri 20th
• Expected RC2 date: Wed 22nd
• Expected RC3 date: Thu 23rc
• 🚢 Expected final release date: Wed 22nd or 29th
• Release PR: Release 0.33.0 #10636
• Accompanying PR for improving the release process: docs: RELEASE_CHECKLIST.md update for 0.33 #10674
• Internal Staging tests by Shipyard: Kubo 0.32.1 vs 0.33.x @ Notion
• Changelog: https://github.com/ipfs/kubo/blob/master/docs/changelogs/v0.33.md

Items In Scope

Required

• feat(libp2p): shared TCP listeners and AutoTLS.AutoWSS #10565
  • for RC1, have this in go-libp2p@main: fix(sampledconn): Correctly handle slow bytes and closed conns libp2p/go-libp2p#3080
• fix: increase provider sample size #10589
• chore: update to boxo without goprocess #10567
• Update to boxo with refactored providerQueryManager. #10595
• fix: go-libp2p v0.37.2 #10603
• docs(changelog/v0.33): bitswap reprovide changes #10604
• chore: p2p-forge/client v0.1.0 #10605
• refactor(rcmgr): use default libp2p rcmgr metrics #9947
• fix: ipfs-webui v4.4.1 #10608
• refactor(cmds): do not return errors embedded in result type #10527
• chore: boxo v0.25.0 #10619
• Upgrade to Boxo v0.26.0 #10631
• new go-libp2p release with fixes
• go-libp2p-kad-dht with accelerated client fix
• datastore perf fixes / saner defaults
  • feat: expose BlockKeyCacheSize and enable WriteThrough datastore options #10614
  • cmd/files: flush parent folders #10630
  • feat(flatfs): default to sync=false #10632
  • mfs: clean cache on sync boxo#751
• fix: ipfs-webui v4.4.2 #10635 with fix from fix: ipld-explorer-components@8.1.3 ipfs-webui#2310
• [skip changelog] pinmfs: mitigate slow mfs writes when it triggers #10623

Found during RC1:

• TCP connection multiplexing (tcpreuse package) doesn't work with pnet libp2p/go-libp2p#3119
  • added explicit check to daemon.go, no need to bump go-libp2p
• goroutine leak on kubo staging 02 and potential side effects
  • "autonat" and "autonatv2/autonat.go-L132" is a slow consumer of event.EvtPeer* libp2p/go-libp2p#3120
  • go-libp2p-kad-dht/subscriber_notifee.go-L37" is a slow consumer of event.EvtPeer*  libp2p/go-libp2p-kad-dht#1007
• gc not engaging on kubo staging 02 (tbd if same or separate issue)
• fix(dns): update default DNSLink resolvers #10655
• https://github.com/libp2p/go-libp2p-kad-dht/releases/tag/v0.28.2
• quicreuse: make it possible to use an application-constructed quic.Transport libp2p/go-libp2p#3122 (fine to just bump quic-go, and not go-libp2p)
• chore: p2p-forge v0.2.2 + go-libp2p-kad-dht v0.28.2 #10663
• do not send cancel message to peer that sent block boxo#784
• fix(client): race in cert renewal ipshipyard/p2p-forge#42
• fix(bitswap): negative wantlist metrics #10670

Nice To Have (Optional)

• Agree if we use /ipX or /dnsX
  • For now we keep using /ipX
  • Will revisit once this is done: feat: opt-in WithShortForgeAddrs(true|false) to produce /dnsX addrs ipshipyard/p2p-forge#40
• fix(cli): support HTTPS in ipfs --api #10659
• accelerated-dht: cleanup peer from message sender on disconnection libp2p/go-libp2p-kad-dht#1009
• mfs: clean cache on sync boxo#751
• fix(cmds/add): disallow --wrap with --to-files #10612
• Broken CID info for identity multihashes ipld-explorer-components#464
• feat(bootstrap): add JS-based va1.bootstrap.libp2p.io #10575
• Boxo
  • feat(routing/http/server): add routing timeout boxo#720
  • feat(gateway): allow localhost http:// DoH resolvers boxo#645
• AutoTLS
  • acme_client "got renewal info" (logger does not respect go-log config) ipshipyard/p2p-forge#8 (comment)
  • AutoTLS: attempts to get cert when not publicly diallable #10570
  • add flag for adding catch-all listener without touching swarm listeners in config
• Denylist support fixes and improvements
  • Support legacy double-hash entries for IPNS CIDs and DNSLink ipfs-shipyard/nopfs#40
  • ipfs blockservice is incompatible with sessions ipfs-shipyard/nopfs#34 (NOfs content filtering is not applied to bitswap)
  • Support consuming https://badbits.dwebops.pub/badbits.deny rainbow#126 (same applies to Kubo)
    • denylist: ?format=car does not skip directly blocked CIDs #10361
    • HTTPSubscriber: detect when list is not append-only ipfs-shipyard/nopfs#39
    • HTTPSubscriber: check with If-None-Match or If-Modified-Since ipfs-shipyard/nopfs#38
• Update pubsub and add default validator #9684 + update to v0.12.0 with GossipSub v1.2
• Add basic UnixFS sanity check to 'ipfs files cp' #10331
• feat(filestore): add mmap reader option boxo#665 disabled by default, with opt-in variable similar to this or this
• Prevent multiple instances of "ipfs bitswap reprovide" running at the same time #10513
• gateway/blocks-backend: GetBlock should not perform IPLD decoding boxo#673

[lidel]

Team is working on multiple high impact perf. fixes, making a slight delay worth it.

We are pushing RC1 to the next week, and final release to January (after holiday break).

[lidel]

✅ Release Checklist (v0.33.0-rc1)

This is a rare case where we make RC on Friday, but we want to ship RC1 before holidays, so folks can test improved AutoTLS, MFS and other features (changelog) over holidays :)

• Prepare the release branch and update version numbers accordingly
  using ./kuboreleaser --skip-check-before release --version vX.Y.Z(-rcN) prepare-branch or ...

  • create a new branch release-vX.Y.Z
    • use master as base if Z == 0
    • use release as base if Z > 0
  • update the CurrentVersionNumber in version.go in the master branch to vX.Y+1.0-dev
    • example
  • update the CurrentVersionNumber in version.go in the release-vX.Y branch to vX.Y.Z(-rcN)
    • example
  • create a draft PR from release-vX.Y to release
    • example
  • Cherry-pick commits from master to the release-vX.Y.Z using git cherry-pick -x <commit>
  • Add full changelog and contributors to the changelog
    • Replace the Changelog and Contributors sections of the changelog with the stdout of ./bin/mkreleaselog. Note that the command expects your $GOPATH/src/github.com/ipfs/kubo to include latest commits from release-vX.Y
      • do NOT copy the stderr
  • verify all CI checks on the PR from release-vX.Y to release are passing
  • Merge the PR from release-vX.Y to release using the Create a merge commit
    • do NOT use Squash and merge nor Rebase and merge because we need to be able to sign the merge commit
    • do NOT delete the release-vX.Y branch
• Create the release tag
  using ./kuboreleaser release --version vX.Y.Z(-rcN) tag or ...

  • This is a dangerous operation! Go and Docker publishing are difficult to reverse! Have the release reviewer verify all the commands marked with ⚠️!
  • ⚠️ tag the HEAD commit using git tag -s vX.Y.Z(-rcN) -m 'Prerelease X.Y.Z(-rcN)'
  • ⚠️ tag the HEAD commit of the release branch using git tag -s vX.Y.Z -m 'Release X.Y.Z'
  • ⚠️ verify the tag is signed and tied to the correct commit using git show vX.Y.Z(-rcN)
  • ⚠️ push the tag to GitHub using git push origin vX.Y.Z(-rcN)
    • do NOT use git push --tags because it pushes all your local tags
• Verify ipfs/distributions's .tool-versions's golang entry is set to the latest go release on the major go branch Kubo is being tested on (see go-version:).
• Publish to Dockerhub, NPM, and dist.ipfs.tech and GitHub using ./kuboreleaser --skip-check-before --skip-run release --version vX.Y.Z(-rcN) publish-to-all or follow each step below:
  • Publish the release to DockerHub
    using ./kuboreleaser --skip-check-before --skip-run release --version vX.Y.Z(-rcN) publish-to-dockerhub or ...

    • Wait for Publish docker image workflow run initiated by the tag push to finish
    • verify the image is available on Docker Hub
  • Publish the release to dist.ipfs.tech
    using ./kuboreleaser release --version vX.Y.Z(-rcN) publish-to-distributions or ...

    • check out ipfs/distributions
    • create new branch: run git checkout -b release-kubo-X.Y.Z(-rcN)
    • run ./dist.sh add-version kubo vX.Y.Z(-rcN) to add the new version to the versions file
      • usage
    • create and merge the PR which updates dists/kubo/versions and dists/go-ipfs/versions ( and dists/kubo/current and dists/go-ipfs/current)
      • example
    • wait for the CI workflow run initiated by the merge to master to finish
    • verify the release is available on dist.ipfs.tech
  • Publish the release to NPM
    using ./kuboreleaser release --version vX.Y.Z(-rcN) publish-to-npm (⚠️ you might need to run the command a couple of times because GHA might not be able to see the new distribution straight away due to caching) or ...

    • run the Release to npm workflow
    • check Release to npm workflow run logs to verify it discovered the new release
    • verify the release is available on NPM
  • Publish the release to GitHub
    using ./kuboreleaser release --version vX.Y.Z(-rcN) publish-to-github or ...

    • create a new release on GitHub
      • RC example
      • FINAL example
      • use the vX.Y.Z(-rcN) tag
      • link to the release issue
      • link to the changelog in the description
      • check the This is a pre-release checkbox
      • copy the changelog (without the header) in the description
      • do NOT check the This is a pre-release checkbox
    • run the sync-release-assets workflow
    • wait for the sync-release-assets workflow run to finish
    • verify the release assets are present in the GitHub release
• Update Kubo staging environment, see the Running Kubo tests on staging for details.
  • Test last release against the current RC
  • Test last release against the current one
• Promote the release
  using ./kuboreleaser release --version vX.Y.Z(-rcN) promote or ...

  • create an IPFS Discourse topic
    • prerelease example
    • release example
    • use Kubo vX.Y.Z(-rcN) is out! as the title
    • use kubo and go-ipfs as topics
    • repeat the title as a heading (##) in the description
    • link to the GitHub Release, binaries on IPNS, docker pull command and release notes in the description
  • pin the IPFS Discourse topic globally
    • you can make the topic a banner if there is no banner already
  • verify the IPFS Discourse topic was copied to:
    • #ipfs-chatter in IPFS Discord
    • #ipfs-chatter in FIL Slack
    • #ipfs-chatter:ipfs.io in Matrix
  • Add the link to the IPFS Discourse topic to the GitHub Release description
    • example
  • create an issue comment mentioning early testers on the release issue
    • example
  • create an issue comment linking to the release on the release issue
    • example
  • ask the marketing team to tweet about the release in #shared-pl-marketing-requests in FIL Slack
    • example
  • post the link to the GitHub Release to Reddit
    • example
• Test the new version with ipfs-companion (currently skipped)
  using ./kuboreleaser release --version vX.Y.Z(-rcN) test-ipfs-companion or ...

  • run the e2e
    • use vX.Y.Z(-rcN) as the Kubo image version
  • wait for the e2e workflow run to finish
• Update Kubo in ipfs-desktop
  using ./kuboreleaser release --version vX.Y.Z(-rcN) update-ipfs-desktop or ...

  • check out ipfs/ipfs-desktop
  • run npm install
  • create a PR which updates package.json and package-lock.json
  • add @SgtPooki as reviewer
• Update Kubo docs
  using ./kuboreleaser release --version vX.Y.Z(-rcN) update-ipfs-docs or ...

  • run the update-on-new-ipfs-tag.yml workflow
  • merge the PR created by the update-on-new-ipfs-tag.yml workflow run
• Create a blog entry on blog.ipfs.tech
  using ./kuboreleaser release --version vX.Y.Z(-rcN) update-ipfs-blog --date YYYY-MM-DD or ...

  • create a PR which adds a release note for the new Kubo version
    • example
  • merge the PR
  • verify the blog entry was published
• Merge the release branch back into master, ignoring the changes to version.go (keep the -dev) version,
  using ./kuboreleaser release --version vX.Y.Z(-rcN) merge-branch or ...

  • create a new branch merge-release-vX.Y.Z from release
  • create and merge a PR from merge-release-vX.Y.Z to master
• Prepare for the next release
  using ./kuboreleaser release --version vX.Y.Z(-rcN) prepare-next or ...

  • Create the next changelog
  • Link to the new changelog in the CHANGELOG.md file
  • Create the next release issue
• Create a dependency update PR
  • check out ipfs/kubo
  • go over direct dependencies from go.mod in the root directory (NOTE: do not run go get -u as it will upgrade indirect dependencies which may cause problems)
  • run make mod_tidy
  • create a PR which updates go.mod and go.sum
  • add the PR to the next release milestone
• Close the release issue

[lidel]

Kubo v0.33.0-rc1 is out! ✨

Early testers ping for RC1 testing:

• Charity Engine (@Rytiss, @tristanolive)
• Fission (@bmann)
• Infura (@MichaelMure)
• OrbitDB (@haydenyoung)
• Pinata (@obo20)
• Shipyard (@cewood, @ns4plabs)
• Siderus (@koalalorenzo)
• Textile (@sanderpick)
• @RubenKelevra

You're getting this message because you're listed here. Please update this list if you no longer want to be included.