Releases: filecoin-project/boost
v2.2.0
Overview
We're excited to introduce Boost v2.2.0. This release enables Direct Data Onboarding (DDO) for Boost.
Compatibility
This release is compatible with Lotus versions v1.26.0 and v1.26.1. For the latest compatibility details, refer to the Boost and Lotus compatibility matrix.
⬆️ Upgrade
The upgrade guide/tutorial for upgrading to v2.2.0 can be found in the Boost documentation.
⚠️ Golang version
Boost v2.2.0 required Golang version v1.21.7 or higher.
👉 ⚠️ ❗ ❗ Warning: This release has breaking changes to configuration structure ❗ ❗
- Please save you current configuration to a file with
boostd config updated --diff
and note down your pricing configuration from UI. - Start
boostd
after upgrade and shutdown after a successful start - Review the
<boost repo>/config.toml
for configuration parameters saved in step 1 and update them if they have been reset to default values. You should also restore your pricing configuration back from UI.
⚠️ ❗ ❗ Aggregate proving must be enabled in Lotus Miner for verified DDO deals to work properly❗ ❗
⭐ Highlights
DDO integration
- Boost now support onboarding verified direct deals.
boostd import-direct --help
NAME:
boostd import-direct - Import data for direct onboarding flow with Boost
USAGE:
boostd import-direct [command options] <piececid> <file>
OPTIONS:
--delete-after-import whether to delete the data for the import after the data has been added to a sector (default: false)
--client-addr value
--allocation-id value (default: 0)
--remove-unsealed-copy (default: false)
--skip-ipni-announce indicates that deal index should not be announced to the IPNI(Network Indexer) (default: false)
--start-epoch value start epoch by when the deal should be proved by provider on-chain (default: 2 days from now) (default: 0)
--help, -h show help
- Boost UI has a new deals page for direct deals.
- The Boost client now offers extended functionality for handling data caps. Users can leverage the boostd client to create new data cap allocations, in addition to viewing a list of existing allocations. This streamlines the process of managing storage quotas for your direct deals.
LID clean up
- A new process now runs every 6 hours to automatically clean up old deals from the LID. It reduces the storage footprint of the LID, improving overall performance and efficiency.
By removing unnecessary data, the LID becomes more streamlined and easier to manage. - Deals that have been completed or have been penalized for violating storage commitments are no longer stored in the LID. Deals associated with the legacy system are removed once completed. Direct deals that have reached their maximum term are purged from the LID.
- The cleanup process also removes any orphaned deals from the LID. These are deals that don't have a corresponding entry in the SQL database but are still attributed to a particular miner.
🗄️ Curio compatibility
This Boost release will be fully compatible with upcoming Curio release. Curio is the new improved Lotus-miner alternative with a distributed scheduling. You can read more about Curio here.
🏗️ Improvements
- Users can advertise new deals and removal of old deals manually to IPNI.
- Boost no longer uses
go-fil-markets
repo as a dependency. All the code for handling legacy deals had been purged from Boost. - Fixed a bug that resulted in an unexpected "End of File" (EOF) error during the parsing of segments for PODSI deals.
- Boost client can be used to extend the existing verified claims for old and new verified deal.
- Boost UI page is updated to show all the relevant information for DDO deals.
- Some minor fixes to improve the DDO deal workflow in Boost.
What's Changed
- fix: remove on-the-fly indexing by @LexLuthr in #1875
- fix: unexpected EOF when parsing segments for PODSI deals by @LexLuthr in #1874
- feat: add deal IPNI announce command by @LexLuthr in #1877
- feat: manual legacy deal IPNI announcement by @LexLuthr in #1879
- fix: remove boost-gfm and go-data-transfer by @LexLuthr in #1740
- fix: remove MakeBoostDeal test method by @LexLuthr in #1883
- integrate direct data onboarding (nv22) by @nonsense in #1847
- feat: LID clean up by @LexLuthr in #1881
- chore: release v2.2.0-rc1 by @LexLuthr in #1884
- chore: update devnet versions by @LexLuthr in #1885
- fix: refactor LID clean up by @LexLuthr in #1886
- fix: panic when no claim is found by @LexLuthr in #1888
- fix: allocate should not compare tmin and tmax with head by @LexLuthr in #1890
- fix: update DDO UI by @LexLuthr in #1889
- chore(deps): bump follow-redirects from 1.15.0 to 1.15.6 in /react by @dependabot in #1894
- chore(deps): bump github.com/filecoin-project/lotus from 1.26.0-rc1 to 1.26.0-rc3 by @dependabot in #1893
- feat: add claim-extend command by @LexLuthr in #1891
- fix: account for DDO in UI by @LexLuthr in #1892
- chore(deps): bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /react by @dependabot in #1896
- chore: bump lotus to v1.26.0 by @LexLuthr in #1898
- chore(deps): bump express from 4.18.1 to 4.19.2 in /react by @dependabot in #1899
- chore: bump lotus to v1.26.1 by @LexLuthr in #1904
- fix: enable on-the-fly-reindexing by @LexLuthr in #1903
- fix: extend-claim cli DC batching by @LexLuthr in #1900
- fix: handle not found errors in pdcleaner by @LexLuthr in #1905
- fix: lid recovery by @LexLuthr in #1906
- feat: GetAsk API by @LexLuthr in #1907
Full Changelog: v2.1.2...v2.2.0
v2.2.0-rc2
Overview
We're excited to introduce Boost v2.2.0-rc2. This release enables Direct Data Onboarding (DDO) for Boost.
Compatibility
This release is compatible with Lotus versions v1.26.0. For the latest compatibility details, refer to the Boost and Lotus compatibility matrix.
⬆️ Upgrade
The upgrade guide/tutorial for upgrading to v2.2.0-rc2 can be found in the Boost documentation.
⚠️ Golang version
Boost v2.2.0-rc2 required Golang version v1.21.7 or higher.
👉 ⚠️ ❗ ❗ Warning: This release has breaking changes to configuration structure ❗ ❗
- Please save you current configuration to a file with
boostd config update --diff
and note down your pricing configuration from UI. - Start
boostd
after upgrade and shutdown after a successful start - Review the
<boost repo>/config.toml
for configuration parameters saved in step 1 and update them if they have been reset to default values. You should also restore your pricing configuration back from UI.
⚠️ ❗ ❗ Aggregate proving must be enabled in Lotus Miner for verified DDO deals to work properly❗ ❗
🏗️ Improvements
- Boost client can be used to extend the existing verified claims for old and new verified deal.
- Boost UI page is updated to show all the relevant information for DDO deals.
- Some minor fixes to improve the DDO deal workflow in Boost.
What's Changed
- fix: panic when no claim is found by @LexLuthr in #1888
- fix: allocate should not compare tmin and tmax with head by @LexLuthr in #1890
- fix: update DDO UI by @LexLuthr in #1889
- chore(deps): bump follow-redirects from 1.15.0 to 1.15.6 in /react by @dependabot in #1894
- chore(deps): bump github.com/filecoin-project/lotus from 1.26.0-rc1 to 1.26.0-rc3 by @dependabot in #1893
- feat: add claim-extend command by @LexLuthr in #1891
- fix: account for DDO in UI by @LexLuthr in #1892
- chore(deps): bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /react by @dependabot in #1896
- chore: bump lotus to v1.26.0 by @LexLuthr in #1898
- chore(deps): bump express from 4.18.1 to 4.19.2 in /react by @dependabot in #1899
Full Changelog: v2.2.0-rc1...v2.2.0-rc2
v2.2.0-rc1
Overview
We're excited to introduce Boost v2.2.0-rc1. This release enables Direct Data Onboarding (DDO) for Boost.
Compatibility
This release is compatible with Lotus versions v1.26.0-rc1 and v1.26.0-rc2. For the latest compatibility details, refer to the Boost and Lotus compatibility matrix.
⬆️ Upgrade
The upgrade guide/tutorial for upgrading to v2.2.0-rc1 can be found in the Boost documentation.
⚠️ Golang version
Boost v2.2.0-rc1 required Golang version v1.21.7 or higher.
👉 ⚠️ ❗ ❗ Warning: This release has breaking changes to configuration structure ❗ ❗
- Please save you current configuration to a file with
boostd config update --diff
and note down your pricing configuration from UI. - Start
boostd
after upgrade and shutdown after a successful start - Review the
<boost repo>/config.toml
for configuration parameters saved in step 1 and update them if they have been reset to default values. You should also restore your pricing configuration back from UI.
⚠️ ❗ ❗ Aggregate proving must be enabled in Lotus Miner for verified DDO deals to work properly❗ ❗
⭐ Highlights
DDO integration
- Boost now support onboarding verified direct deals.
boostd import-direct --help
NAME:
boostd import-direct - Import data for direct onboarding flow with Boost
USAGE:
boostd import-direct [command options] <piececid> <file>
OPTIONS:
--delete-after-import whether to delete the data for the import after the data has been added to a sector (default: false)
--client-addr value
--allocation-id value (default: 0)
--remove-unsealed-copy (default: false)
--skip-ipni-announce indicates that deal index should not be announced to the IPNI(Network Indexer) (default: false)
--start-epoch value start epoch by when the deal should be proved by provider on-chain (default: 2 days from now) (default: 0)
--help, -h show help
- Boost UI has a new deals page for direct deals.
- The Boost client now offers extended functionality for handling data caps. Users can leverage the boostd client to create new data cap allocations, in addition to viewing a list of existing allocations. This streamlines the process of managing storage quotas for your direct deals.
LID clean up
- A new process now runs every 6 hours to automatically clean up old deals from the LID. It reduces the storage footprint of the LID, improving overall performance and efficiency.
By removing unnecessary data, the LID becomes more streamlined and easier to manage. - Deals that have been completed or have been penalized for violating storage commitments are no longer stored in the LID. Deals associated with the legacy system are removed once completed. Direct deals that have reached their maximum term are purged from the LID.
- The cleanup process also removes any orphaned deals from the LID. These are deals that don't have a corresponding entry in the SQL database but are still attributed to a particular miner.
🏗️ Improvements
- Users can advertise new deals and removal of old deals manually to IPNI.
- Boost no longer uses
go-fil-markets
repo as a dependency. All the code for handling legacy deals had been purged from Boost. - Fixed a bug that resulted in an unexpected "End of File" (EOF) error during the parsing of segments for PODSI deals.
- We've eliminated the practice of on-the-fly indexing for deals. This approach involved indexing deals whenever encountered through HTTP requests or Bitswap retrievals to reduce inefficiencies that could arise when dealing with unsealed sectors or inaccurate IPNI information.
What's Changed
- fix: remove on-the-fly indexing by @LexLuthr in #1875
- fix: unexpected EOF when parsing segments for PODSI deals by @LexLuthr in #1874
- feat: add deal IPNI announce command by @LexLuthr in #1877
- feat: manual legacy deal IPNI announcement by @LexLuthr in #1879
- fix: remove boost-gfm and go-data-transfer by @LexLuthr in #1740
- fix: remove MakeBoostDeal test method by @LexLuthr in #1883
- integrate direct data onboarding (nv22) by @nonsense in #1847
- feat: LID clean up by @LexLuthr in #1881
- chore: release v2.2.0-rc1 by @LexLuthr in #1884
- chore: update devnet versions by @LexLuthr in #1885
- fix: refactor LID clean up by @LexLuthr in #1886
Full Changelog: v2.1.2...v2.2.0-rc1
v2.1.2
Overview
We're excited to introduce Boost v2.1.2. This release focuses on reducing the resource usage utilisation for boostd-data
service.
This is a recommended release for everyone on Boost v2.x
Compatibility
This release is compatible with Lotus versions v1.24.0 and v1.25.0. For the latest compatibility details, refer to the Boost and Lotus compatibility matrix.
⭐ Highlights
- A new command to send a removal advertisement to IPNI. This can be used to clean up any old records from network indexers.
- Performance improvements in piece doctor.
- Piece doctor can now be disabled in configuration.
- UI enhancements for deal lifecycle and sector lifecycle separation.
What's Changed
- feat: data segment indexing by @LexLuthr in #1739
- Update deal list page by @LexLuthr in #1837
- fix: handle complete checkpoint in UI deal list by @LexLuthr in #1841
- fix: config to disable piece doctor by @LexLuthr in #1849
- Prefetch state claims to change O(n) calls to Lotus State to O(1) by @RobQuistNL in #1855
- Add sectorID index to deal table by @RobQuistNL in #1853
- add indexes to deals DB by @LexLuthr in #1857
- feat: cleanup older fund manager logs by @LexLuthr in #1856
- doc: fix flag info and error message for block serving by @rvagg in #1825
- feat: index ad removal command by @LexLuthr in #1859
- fix: UI deal error messages by @LexLuthr in #1863
- bump up cbor-gen utility by @nonsense in #1865
- safe handlers by @nonsense in #1866
- Upgrade dependencies by @masih in #1867
- fix: handle panic in podsi indexing by @ischasny in #1868
- fix: fallback to regular indexing if podsi failed by @ischasny in #1870
- chore: release v2.1.2 by @LexLuthr in #1871
Full Changelog: v2.1.1...v2.1.2
v2.1.1
Overview
We're excited to introduce Boost v2.1.1. This release focuses on reducing the resource usage utilisation for boostd-data
service.
Compatibility
This release is compatible with Lotus versions v1.24.0 and v1.25.0. For the latest compatibility details, refer to the Boost and Lotus compatibility matrix.
⚠️ Important Upgrade Notice
👉 Before updating, carefully read the release notes and documentation. Storage providers will need to set up a database service and undergo a database migration process. Incorrect procedures can lead to permanent data loss.
⭐ Highlights
- The high CPU usage by boostd-data service is now fixed
- Deal indexing has been improved to avoid deals getting stuck in "Announcing" state
- Boost UI should not display the momentary
searching message: %!w(<nil>)
message anymore
What's Changed
- fix: parametrise add index parallelism by @ischasny in #1827
- piece-directory: make sure we process recs correctly by @nonsense in #1828
- fix: set default add index concurrency to 8 by @ischasny in #1829
- skip flaky TestDealsRejectedForFunds test by @nonsense in #1830
- booster-http: add pprof-port flag by @nonsense in #1831
- fix: better formatting for error messages by @ischasny in #1833
- boostx: print warning instead of exiting on api mismatch by @nonsense in #1832
- chore: release v2.1.1 by @LexLuthr in #1835
Full Changelog: v2.1.0...v2.1.1
v2.1.0
Overview
We're excited to introduce Boost v2.1.0, a significant update that transforms how Boost manages indices for user data. This version phases out the DAG store in favor of the new Local Index Directory (LID). Alongside this, we've added new features and improvements, and squashed some bugs to enhance your experience.
Compatibility
This release is compatible with Lotus versions v1.24.0 and v1.25.0. For the latest compatibility details, refer to the Boost and Lotus compatibility matrix.
⚠️ Important Upgrade Notice
👉 Before updating, carefully read the release notes and documentation. Storage providers will need to set up a database service and undergo a database migration process. Incorrect procedures can lead to permanent data loss.
Highlights
- New HTTP Gateway (Frisbii):
booster-http
now usesFrisbii
, a trustless gateway. It no longer supports servingunixfs
files, requiring an additionalbifrost-gateway
binary for such files. - Boost UI Enhancements: The UI now allows SPs to check payload CIDs for pieceCIDs, aiding in retrieval testing and debugging.
- Boostd-data Metrics: Emission of metrics for each API method, with updated dashboards to monitor
boostd-data
performance and service uptime. - Index Provider Upgrades: Switch from Graphsync to HTTP-libp2p protocol and UI improvements for better retrieval and indexing information display.
- Multiple Miners to Single LID: Connect multiple miners to a single LID instance for unified retrieval and deal direction.
- Manual PSD (Publish Storage Deal): Manual control over PSD message publishing, with a configuration variable for enabling it.
- Graphsync Improvements: Better connection termination to resolve stalled retrievals.
- New IPNI UI Page: Dedicated page for IPNI retrievals with detailed information and sync status.
Improvements
- Performance Enhancements: Reduced devnet build time, parallelization in index migration, batched write operations, and optimized deal filter mechanism.
- UI and Config File Tweaks: Improved UI elements, added CORS handler for
booster-http
, and generated commented config file onboostd
init. - Database Support: Interfaces for LevelDB and YugabyteDB, catering to different data storage scales.
- Extended Deal Durations: Support for making deals up to 3.5 years long.
- Timeout Configurations: Updated and customizable CQL client timeouts.
Notable Changes
- Graphsync Server and Client Updates: Enhanced response handling for identity CID requests and improved multipart downloads for online deals.
- Security Enhancements: Critical security issue resolved by upgrading to grpc v1.56.3.
- Boostd-data Command Improvements: Added CQL timeout configuration.
- Monitoring Stack Update: Revised dashboards shipped with the Docker-based monitoring stack.
- Database Migration Support: Added support for parallel processing in database migration.
Upgrading to Boost v2.1.0
For a smooth transition to Boost v2.1.0, follow the detailed instructions available in the Boost Documentation Tutorials - How to Migrate to Boost v2.
We're confident that these updates will enhance your experience with Boost, and we look forward to your feedback! 🚀
What's Changed
- chore(deps): bump tough-cookie from 4.0.0 to 4.1.3 in /react by @dependabot in #1558
- Boost release/v2 by @nonsense in #1520
- Update README.md's TOC by @nonsense in #1582
- chore(deps): bump word-wrap from 1.2.3 to 1.2.4 in /react by @dependabot in #1571
- chore(deps): bump semver from 6.3.0 to 6.3.1 in /react by @dependabot in #1581
- fix: BitswapPublicAddresses default by @LexLuthr in #1588
- feat: add corshandler to IPFS gateway by @LexLuthr in #1589
- fix version flag for booster-http and booster-bitswap by @nonsense in #1590
- feat: manual publish storage deal message by @LexLuthr in #1585
- Use shared piece reader for blockstore get (fixed conflicts) by @nonsense in #1584
- feat: show ad chain head in UI by @LexLuthr in #1592
- feat: utility to get miner address from datastore by @LexLuthr in #1595
- fix: migration config bug by @LexLuthr in #1596
- Fix startup with disabled index provider by @dirkmc in #1604
- Create CONTRIBUTING.md by @brendalee in #1603
- fix: update deal filter to use
ExpectedSealDuration
andMaxDealStartDelay
by @LexLuthr in #1609 - migrate-lid: fix return err msg by @nonsense in #1612
- feat: add docker to containerize a standalone boost-client by @alvin-reyes in #1616
- timeout for migrate-lid by @nonsense in #1631
- chore: bump boost-graphsync to v0.13.7 by @LexLuthr in #1647
- feat: add multi-miner to single LID support by @LexLuthr in #1656
- chore: release v2.1.0-rc1 by @LexLuthr in #1657
- fix(docker): no inline
sed
commands by @hannahhoward in #1658 - Build and publish boost devnet containers via CI by @masih in #1661
- Allow container build job to push images by @masih in #1663
- feat: add InUnsealed utility to boostx by @LexLuthr in #1680
- Disable flaky TestConcurrentTransfers by @dirkmc in #1682
- fix: disable leveldb as LID default by @LexLuthr in #1681
- fix: init with commented config by @LexLuthr in #1694
- fix: add parallelisation to migrate-lid and improve logging by @LexLuthr in #1689
- fix missing icons by @dirkmc in #1698
- chore(deps): upgrade graphsync for identity CID fix by @rvagg in #1704
- feat: update docker devnet build by @LexLuthr in #1702
- fix config.toml for devnet by @nonsense in #1708
- fix: close piece reader after generating indexes by @LexLuthr in #1707
- fix: docker devnet entrypoint by @LexLuthr in #1712
- recover from panics in go-routine in client-side io.Reader encoder by @nonsense in #1706
- fix: GS query panic by @LexLuthr in #1714
- fix: Rename deal data root cid in Boost UI by @LexLuthr in #1719
- fix: remove payloadRootCid lookup by @LexLuthr in #1721
- ci: explicitly switch macos build to silicon and upgrade xcode by @galargh in #1720
- feat: show payload CIDs in UI by @LexLuthr in #1722
- Ivan/http transport upgrades by @ischasny in #1732
- feat: Ipnisync by @LexLuthr in #1731
- ci: switch macos builds to apple silicon by @galargh in #1726
- direct data onboarding - disabled by @nonsense in #1627
- Fix boostd-data go module path and remove replace directives by @masih in #1741
- Fix static check issues by @masih in #1743
- fix LID web UI by @nonsense in #1745
- use explicit repo location for boostd-data by @nonsense in #1748
- add support for yugabytedb for devnet by @nonsense in #1756
- feat: run deal filters in parallel by @LexLuthr in #1746
- chore: release v2.1.0-rc2 by @LexLuthr in #1764
- chore(deps): bump @babel/traverse from 7.18.0 to 7.23.2 in /react by @dependabot in #1759
- fix: empty IPNI UI due to lack if latest Ad CID by @LexLuthr in #1766
- fix: handle isexist error for IsDirectDeal column in migration by @LexLuthr in #1761
- Create support ticket template by @LexLuthr in #1749
- feat: spans and debug lines for piece directory by @nonsense in #1773
- fix: Remove li...
v2.1.0-rc3
Overview
We're happy to announce the Boost v2.1.0-rc3 release. Boost v2 revamps the way Boost handles indices for user data, introducing the Local Index Directory (LID) and removing the DAG store. In addition, there are a number of new features including bug fixes, increased flexibility on LID setup, and more.
This release is compatible with the Lotus v1.24.0-rc1 and v1.25.0-rc1 release. See the Boost and Lotus compatibility matrix for up to date information to plan your upgrade.
👉 ⚠️ ❗ ❗ Before updating, please read the release notes and documentation carefully, as storage providers will need to setup a database service and go through a database migration process. Incorrect process can result in permanent data loss ❗ ❗ ⚠️
⭐ Highlights
New HTTP Gateway (Frisbii)
- The
booster-http
binary has been updated to useFrisbii
as a trustless gateway. - Serving
unixfs
file is no longer supported bybooster-http
. Users would need to run an additionalbifrost-gateway
binary to serveunixfs
formatted files. - The new gateway implementation should allow retrieval verifiability by default.
Boostd-data metrics
- The
boostd-data
service emits metrics for each API method. - The dashboards shipped with the docker based monitoring stack now include these metrics.
- The updated dashboards can be used to understand the
boostd-data
performance, as well as service uptime for all Boost services. - Documentation on how to deploy monitoring has been updated to include all the new changes.
🏗️ Improvements
- CQL client timeout has been updated to 60 seconds from the original default of 6 seconds.
- Users can now tweak the CQL timeout further using the --CQLtimeout
flag when starting the
boostd-data` command. - Boost now supports making 3.5 year long deals.
- Write operations for index insertion in YugabyteDB are now batched with the size of 10k entries per batch, which will improve performance for large write operations.
What's Changed
- chore(deps): bump @babel/traverse from 7.18.0 to 7.23.2 in /react by @dependabot in #1759
- fix: empty IPNI UI due to lack if latest Ad CID by @LexLuthr in #1766
- fix: handle isexist error for IsDirectDeal column in migration by @LexLuthr in #1761
- Create support ticket template by @LexLuthr in #1749
- feat: spans and debug lines for piece directory by @nonsense in #1773
- fix: Remove line forcing for logs in issue template by @LexLuthr in #1770
- feat: add cql timeout to boostd-data command by @LexLuthr in #1772
- booster-http refactor - trustless only, improved compression, logging & e2e testing w/ trusted by @rvagg in #1676
- fix: awaiting publish confirmation count by @LexLuthr in #1778
- fix: update deal filters to use new values by @LexLuthr in #1779
- fix: add CQL timeout to lid-migrate command by @LexLuthr in #1783
- Resolve critical security issue by Upgrading to grpc v1.56.3 by @masih in #1785
- boostx: add venus to stats cmd by @nonsense in #1786
- fix: use new MaxSectorExpirationExtension by @LexLuthr in #1788
- boostd-data: yugabyte migrate fix missing cql timeout by default by @nonsense in #1790
- feat: add boostd-data metrics by @LexLuthr in #1784
- chore: release v2.1.0-rc3 by @LexLuthr in #1794
- fix: speed up ydb insert by @ischasny in #1793
- yugabyte/service: parallel batch inserts for car indices by @nonsense in #1795
Full Changelog: v2.1.0-rc2...v2.1.0-rc3
v2.1.0-rc2
Overview
We're happy to announce the Boost v2.1.0-rc2 release. Boost v2 revamps the way Boost handles indices for user data, introducing the Local Index Directory (LID) and removing the DAG store. In addition, there are a number of new features including bug fixes, increased flexibility on LID setup, and more.
This release is compatible with the Lotus v1.24.0-rc1 and v1.25.0-rc1 release. See the Boost and Lotus compatibility matrix for up to date information to plan your upgrade.
👉 ⚠️ ❗ ❗ Before updating, please read the release notes and documentation carefully, as storage providers will need to setup a database service and go through a database migration process. Incorrect process can result in permanent data loss ❗ ❗ ⚠️
⭐ Highlights
Boost UI will allow SPs to check the payload CIDs for a pieceCID. These CID are useful for retrieval testing and debugging.

Index Provider
- The index-provider has switched from Graphsync data transfer protocol to HTTP-libp2p protocol.
- The Retrieval page in the UI has been split up so that only retrieval deal information is shown on the existing page, and all indexing information is shown on the Indexer page.
- The indexer UI in Boost displays the lag between the latest local advertisement and the sync status of the network indexers.

🏗️ Improvements
- The devnet build time has been reduced from 40+ minutes to <5 minutes.
- The
migrate-lid
command now support parallelisation of index migration to speed up the over all migration time required for switching from Boost v1 to v2. boostd
init generates a commented out config file- The deal filter mechanism has been redesigned to run in parallel. This should speed up the deal acceptance time and reduce client timeouts.
- Graphsync server has been updated to respond correctly for identity CID requests to reduce the overall retrieval failure rates.
- The deal download has been redesigned to perform multi-part downloads to reduce the download time for online deals(like aria2c).
- The pieceReader is closed correctly after generating index to fix the issue of dangling deleted files on Lotus AP workers.
What's Changed
- fix(docker): no inline
sed
commands by @hannahhoward in #1658 - Build and publish boost devnet containers via CI by @masih in #1661
- Allow container build job to push images by @masih in #1663
- feat: add InUnsealed utility to boostx by @LexLuthr in #1680
- Disable flaky TestConcurrentTransfers by @dirkmc in #1682
- fix: disable leveldb as LID default by @LexLuthr in #1681
- fix: init with commented config by @LexLuthr in #1694
- fix: add parallelisation to migrate-lid and improve logging by @LexLuthr in #1689
- fix missing icons by @dirkmc in #1698
- chore(deps): upgrade graphsync for identity CID fix by @rvagg in #1704
- feat: update docker devnet build by @LexLuthr in #1702
- fix config.toml for devnet by @nonsense in #1708
- fix: close piece reader after generating indexes by @LexLuthr in #1707
- fix: docker devnet entrypoint by @LexLuthr in #1712
- recover from panics in go-routine in client-side io.Reader encoder by @nonsense in #1706
- fix: GS query panic by @LexLuthr in #1714
- fix: Rename deal data root cid in Boost UI by @LexLuthr in #1719
- fix: remove payloadRootCid lookup by @LexLuthr in #1721
- ci: explicitly switch macos build to silicon and upgrade xcode by @galargh in #1720
- feat: show payload CIDs in UI by @LexLuthr in #1722
- Ivan/http transport upgrades by @ischasny in #1732
- feat: Ipnisync by @LexLuthr in #1731
- ci: switch macos builds to apple silicon by @galargh in #1726
- direct data onboarding - disabled by @nonsense in #1627
- Fix boostd-data go module path and remove replace directives by @masih in #1741
- Fix static check issues by @masih in #1743
- fix LID web UI by @nonsense in #1745
- use explicit repo location for boostd-data by @nonsense in #1748
- add support for yugabytedb for devnet by @nonsense in #1756
- feat: run deal filters in parallel by @LexLuthr in #1746
New Contributors
Full Changelog: v2.1.0-rc1...v2.1.0-rc2
v2.1.0-rc1
Overview
We're happy to announce the Boost v2.1.0-rc1 release. Boost v2 revamps the way Boost handles indices for user data, introducing the Local Index Directory (LID) and removing the DAG store. In addition, there are a number of new features including bug fixes, increased flexibility on LID setup, and more.
The Local Index Directory requires a database for persistence. After reviewing various options and consulting the community, we are providing interfaces to two databases - LevelDB (for SPs holding less than 1 PiB of user data) and YugabyteDB (for SPs holding more than 1 PiB of user data). ❗ Please take your future growth into account before choosing a database ❗
👉 ⚠️ ❗ ❗ Before updating, please read the release notes and documentation carefully, as storage providers will need to setup a database service and go through a database migration process. Incorrect process can result in permanent data loss ❗ ❗ ⚠️
Highlights
Multiple Miners to Single LID
- Storage providers running multiple miners can connect all of their
boostd
,booster-http
andbooster-bitswap
instances to the same LID instance. - All
boostd
,booster-http
andbooster-bitswap
instances can serve retrievals from any miner connected to the LID instance.
- New storage deals coming to a
boostd
will be directed to its dedicated miner.
Graphsync Improvements
The Graphsync library should now terminate the connection properly. This should permanently fix the stalled Graphsync retrievals issue that some storage providers have experienced.
Manual PSD (Publish Storage Deal)
This new feature allows SPs to manually send PSD messages. Once this feature is turned on, Boost will no longer automatically send PSD messages unless explicitly prompted by the user. The feature can be turned on with the following config variable:
[Dealmaking]
// When set to true, the user is responsible for publishing deals manually.
// The values of MaxDealsPerPublishMsg and PublishMsgPeriod will be
// ignored, and deals will remain in the pending state until manually published.
ManualDealPublish bool
More details about how to use this feature can be found in the Boost docs.
New UI page for IPNI interactions
- IPNI retrievals are now separated out from the data retrievals page.
- Latest local and remote advertisement CIDs can be used to check the sync status.
- Publisher details and other config can be found in the UI.

⚠️ Users will now see Boost v2 documentation by default. To refer to Boost v1 documentation, you can select v1 in the dropdown on the header.
⚠️ We will no longer be backporting any fixes to v1 releases.
🏗️ Additional Improvements
- The miner ID is now displayed on the top of all UI pages.
- The boost binaries will now show the network name to avoid confusion.
./boost --version
boost version 2.0.0-rc1+calibnet+git.8f88bde.dirty
- Disabled index-provider will not crash
boostd
process Booster-http
now has a CORS handler for IPFS gateway as well as full piece download path.- Cached pieceReader allows faster retrievals speeds for the same content.
- Boost now uses the Lotus pieceReader enhancements, resulting in a 3x improvement in read performance.
Upgrading to Boost v2
You can find detailed instructions on how to upgrade to Boost v2 via the Boost Documentation Tutorials - How to Migrate to Boost v2.
What's Changed
- Update lotus and boxo versions by @dirkmc in #1466
- backport #1523 to main by @LexLuthr in #1556
- chore(deps): bump tough-cookie from 4.0.0 to 4.1.3 in /react by @dependabot in #1558
- Boost release/v2 by @nonsense in #1520
- Update README.md's TOC by @nonsense in #1582
- chore(deps): bump word-wrap from 1.2.3 to 1.2.4 in /react by @dependabot in #1571
- chore(deps): bump semver from 6.3.0 to 6.3.1 in /react by @dependabot in #1581
- fix: BitswapPublicAddresses default by @LexLuthr in #1588
- feat: add corshandler to IPFS gateway by @LexLuthr in #1589
- fix version flag for booster-http and booster-bitswap by @nonsense in #1590
- feat: manual publish storage deal message by @LexLuthr in #1585
- Use shared piece reader for blockstore get (fixed conflicts) by @nonsense in #1584
- feat: show ad chain head in UI by @LexLuthr in #1592
- feat: utility to get miner address from datastore by @LexLuthr in #1595
- fix: migration config bug by @LexLuthr in #1596
- Fix startup with disabled index provider by @dirkmc in #1604
- Create CONTRIBUTING.md by @brendalee in #1603
- fix: update deal filter to use
ExpectedSealDuration
andMaxDealStartDelay
by @LexLuthr in #1609 - migrate-lid: fix return err msg by @nonsense in #1612
- feat: add docker to containerize a standalone boost-client by @alvin-reyes in #1616
- timeout for migrate-lid by @nonsense in #1631
- chore: bump boost-graphsync to v0.13.7 by @LexLuthr in #1647
- feat: add multi-miner to single LID support by @LexLuthr in #1656
New Contributors
- @brendalee made their first contribution in #1603
- @alvin-reyes made their first contribution in #1616
Full Changelog: v2.0.0...v2.1.0-rc1
v1.7.5
Overview
We're happy to announce the Boost v1.7.5 release.
This release is compatible with the Lotus v1.23.0 release. See the Boost and Lotus compatibility matrix for up to date information to plan your upgrade.
Improvements
- Fixes the
boostd
crash when index-provider is disabled in the configuration - Boost deals will not be filtered out based on
ExpectedSealDuration
andMaxDealStartDelay
values MaxDealStartDelay
will now only be applicable for online deals- ❗
StartEpochSealingBuffer
will now be used from[Dealmaking]
section for boost deals instead of[LotusDealmaking]
section ❗
Full Changelog: v1.7.4...v1.7.5