v0.5.0-rc4

What's Changed

• chore(deps): bump go.opentelemetry.io/otel/metric from 0.32.1 to 0.33.0 by @dependabot in #1267
• Update Docker image build pipelines by @jbowen93 in #1337
• chore!(nodebuilder/p2p): increment the chain-id for arabica by @evan-forbes in #1370

Full Changelog: v0.5.0-rc3...v0.5.0-rc4

v0.5.0-rc3

What's Changed

• docs(adr): Public API by @Wondertan in #506
• fix(header): Extend ValidateBasic on ExtendedHeader to include data root check by @renaynay in #1364
• fix(das): Correct WithMetrics fx Invocation by @derrandz in #1366
• refactor(nodebuilder/p2p): Allow specifying network alias with --p2p.network flag by @renaynay in #1357
• fix(coreaccessor): pfd submission nil check by @distractedm1nd in #1368
• chore(deps): bump golangci/golangci-lint-action from 3.3.0 to 3.3.1 by @dependabot in #1360
• fix(eds/byzantine): trim NMTWrapper's namespace during BEFP validation by @evan-forbes in #1354
• chore!: bump celestia-app to v0.10.0-rc1 by @evan-forbes in #1338

Full Changelog: v0.5.0-rc2...v0.5.0-rc3

v0.5.0-rc2

What's Changed

• feat(share/availability/light): Add some debug logs for sampling by @renaynay in #1294
• fix(gateway): switch to separate Listen/Serve by @distractedm1nd in #1314
• feat(api/gateway): Implement context cancellation middleware by @renaynay in #1321
• feat(.github): add labels on issues workflow by @renaynay in #1315
• refactor(nodebuilder): Logger names in subpackages to constructor/<subpkg> by @renaynay in #1324
• refactor: linewrapping comments to 100 width by @distractedm1nd in #1274
• fix(.github): Point at correct action by @renaynay in #1335
• fix(.github/workflows): Implement working label automation by @renaynay in #1346
• improvement(header/server): implement timeout for GetRangeByHeight request by @vgonkivs in #1318
• fix(rpc): fraud.Proof (un)marshalling and subscription as chan by @distractedm1nd in #1307
• fix(share): incorrect TestGetSharesByNamespace test by @walldiss in #1342
• fix(nodebuilder): Use test node for tests instead of New by @renaynay in #1350
• improvement(header/p2p): get bestHead if timeout is reached by @vgonkivs in #1319
• refactor(ipld): use Set/GetCell API from rstm2d by @Wondertan in #1173

Full Changelog: v0.5.0-rc1...v0.5.0-rc2

v0.5.0-rc1

Release candidate for v0.5.0

What's Changed

BREAKING

• refactor!(params): Moving params to nodebuilder and adding config by @distractedm1nd in #1168
• feat!(api/rpc): OpenRPC scaffolding, migrating OpenAPI gateway to api/gateway by @distractedm1nd in #1175
• feat!(gateway): reenabling gateway, adding flag to enable by @distractedm1nd in #1199
• feat(rpc): Add client wrapper by @distractedm1nd in #1195

FEATURE

• feat(blocksync): share.WriteEDS by @distractedm1nd in #1139
• feat(blocksync): share.ReadEDS by @distractedm1nd in #1158
• feat(blocksync): Benchmarks for ReadEDS and WriteEDS by @distractedm1nd in #1197
• feat(ipld): integration test for namespace hasher by @distractedm1nd in #1256

IMPROVEMENT

• fix: removing protocolIDs from global namespace in p2p and fraud by @distractedm1nd in #1268
• refactor(nodebuilder): Allow custom networks to work over already-initialised store by @renaynay in #1270
• refactor(share): move bad encoding fraud proof to share pkg by @walldiss in #1241
• refactor(daser): use functional options pattern to configure daser by @derrandz in #1225
• feat(rpc): adding daser stub for friendly API error by @distractedm1nd in #1308
• chore(share/eds): adding test utility for generating embedded test data by @distractedm1nd in #1320

BUG FIX

• fix: restricting PFD metrics to successful PFDs by @distractedm1nd in #1255
• fix(das): Only debug log sampled header if it was actually successful by @renaynay in #1295
• fix(rpc): custom JSON ExtendedHeader marshal/unmarshaling using amino wrapper by @distractedm1nd in #1292

MISCELLANEOUS

• chore: gen less acounts to reduce test execution times by @Wondertan in #1253
• chore(share/ipld): remove dead code by @Wondertan in #1257
• refactor(core/fetcher): Improve error messages for fetcher methods by @renaynay in #1263
• refactor(nodebuilder/header): Clarify error for init store failure by @renaynay in #1272
• add semver tags to docker image builds by @jbowen93 in #1277
• Makefile: update go-acc version due to errors and new release by @MSevey in #1281
• chore(nodebuilder/p2p): Update to new arabica chain-id by @evan-forbes in #1284
• refactor(share/ipld): Use NamespaceSize var for consistency by @renaynay in #1297
• chore(deps): bump golangci/golangci-lint-action from 3.2.0 to 3.3.0 by @dependabot in #1264
• chore: bump celestia app v0.9.0 by @evan-forbes in #1300

New Contributors

• @derrandz made their first contribution in #1225

Full Changelog: v0.4.1...v0.5.0-rc1

v0.4.2 - Upgrade network ID to `arabica-1`

This patch release upgrades the network ID to arabica-1.

Full Changelog: v0.4.1...v0.4.2

v0.4.1 - Bug fixes for `namespaceHasher` and `CoreAccessor`

This patch release contains a two bug fixes for issues found during testing on v0.4.0.

1. namespaceHasher.Write's error is not properly handled for, so in the case that data comes in that is of an unexpected length, the error is not caught and causes a panic. To remedy that, namespaceHasher.Sum now returns nil in the case that the data is empty such that multihash.encodeHash will catch it and an error will be returned.

2. CoreAccessor used the context passed into it from construction to manage its lifecycle which was incorrect as that context should only be used for the start operation rather than the management of its lifecycle. CoreAccessor now manages its own context internally to the component.

What's Changed

IMPROVEMENTS

• refactor(rpc): move sanity check logic from rpc handlers to implementations. by @HoytRen in #1213

BUG FIXES

• fix: returning nil sum when hasher receives data with incorrect length by @distractedm1nd in #1245
• fix(state): CoreAccessor manages its own context by @renaynay in #1247

MISC

• .github: add triggering on versioned tags by @MSevey in #1237
• refactor(share/ipld): Use consts for node type determination by @renaynay in #1244

Full Changelog: v0.4.0...v0.4.1

v0.4.0

v0.4.0 | 2022-10-17

This release includes large refactorings, breaking changes and improvements related to four primary work-streams: building out the public API, hardening HeaderExchange, refactoring blocksync architecture and increasing metrics coverage.

Most notably, this release contains:

• 2 breaking changes, in addition to other improvements, to HeaderExchange as part of our work to harden and optimise it
• A large-scale refactoring to the structure of the new nodebuilder package (formerly node) as well as the way components are constructed/managed, in an effort to clarify the boundaries between components and create a cleaner foundation for building out the node's API
• Upgrade to the latest breakingcelestia-app dependency, including using instance of app instead of kvstore in our integration tests (located in nodebuilder/tests)
• Metrics coverage for das and fraud packages, in addition to increased coverage for other key celestia-node behaviours
• Consolidation of ipld and share packages into a cleaner, better defined structure, as well as general improvements to ipld to remove redundant/unnecessary behaviours

What's Changed

BREAKING

• feat!(header/p2p): add deadlines on stream and rework proto by @vgonkivs in #1038
• feat!(header/p2p): extend header response with status code by @vgonkivs in #1135
• chore!: upgrade to celestia-app v0.7.0 by @rootulp in #1147

FEATURES

• feat(metrics): metrics for PFD transactions, sampling, and broadcasts by @distractedm1nd in #1083
• feat(das): add metrics to DASer by @walldiss in #1125
• feat(fraud): add metrics to fraud package by @vgonkivs in #1047
• feat(nodebuilder): add AccountAddress to state's Module by @Bidon15 in #1209
• feat(header/p2p): implement ErrHeadersLimitExceeded error by @vgonkivs in #1153

IMPROVEMENTS

• refactor(node/state): Refactor node pkg -> nodebuilder and split services into modules by @distractedm1nd @renaynay in #997
• refactor(nodebuilder): Move parse methods and flag definitions to nodebuilder subpackages by @distractedm1nd in #1161
• refactor(nodebuilder): moving service/ services to respective node sub-packages by @distractedm1nd in #1056
• fix(state): returning CoreAccessor from nodebuilder for state.WithMetrics by @distractedm1nd in #1200
• refactor(header/p2p): request head from multiple peers by @vgonkivs in #1046
• refactor(swamp): use celestia-app instead of the kvstore by @evan-forbes in #1160
• refactor(share): Consolidate share pkg by @walldiss in #1177
• refactor(share): IPLD/NMT improvements by @walldiss in #1223

BUG FIXES

• bugfix(fraud): close stream properly by @vgonkivs in #1152
• bugfix(nodebuilder/fraud): remove fraud.Subscriber from module by @vgonkivs in #1169
• bugfix: fix flakiness in TestFullReconstructFromLights by @vgonkivs in #1171
• fix(ipld/plugin): don't truncate a type byte when it's not in the data by @Wondertan in #1196
• fix(nodebuider/header): initialize Store during start lifecycle instead of fx.Invoke by @Wondertan in #1218
• fix(nodebuilder): fix broken shutdown for services on fraud proofs receival by @distractedm1nd in #1220

MISC

• log(header): Add detailed error for non-adjacent header attempt by @renaynay in #1123
• log(das): add extra debug logs for sampled headers by @walldiss in #1157
• chore(deps): bump go.opentelemetry.io/otel/metric from 0.32.0 to 0.32.1 by @dependabot in #1167
• chore(deps): bump codecov/codecov-action from 3.1.0 to 3.1.1 by @dependabot in #1164
• chore(deps): bump go.uber.org/fx from 1.18.1 to 1.18.2 by @dependabot in #1179
• Disable verbose log on Debug Level for badger, watchdog, and basichost. by @HoytRen in #1180
• Fix markdown lint workflow to run on directories by @adlerjohn in #1170
• fix (.github/workflows): Skip over integration tests in unit test action by @renaynay in #1193
• fix: README link for header doc by @rootulp in #1192
• nodebuilder: Remove current borked RPC tests in favour of implementing better ones with #962 by @renaynay in #1190
• docs/adr: add daser parallelization adr by @walldiss in #1096
• chore: disable debug log in Makefile by @Wondertan in #1210
• fix(nodebuilder/header): Only provide p2p.Subscriber as p2p.Broadcaster for bridge node by @renaynay in #1203
• chore(nodebuilder/header): remove dead metrics code by @Wondertan in #1217
• Remove watchdog since go 1.19 don't need it anymore. by @HoytRen in #1212
• chore(deps): bump github.com/spf13/cobra from 1.5.0 to 1.6.0 by @dependabot in #1222
• .github: add new ci_release workflow to unify CI by @MSevey in #1191

New Contributors

• @HoytRen made their first contribution in #1180
• @MSevey made their first contribution in #1191

Full Changelog: v0.3.1...v0.4.0

v0.3.1

v0.3.1 - DASer parallelisation and quick fix to cel-key

This patch release notably includes an optimisation to the DASer in order to provide faster sampling over past headers and a fix to the cel-key utility to read from the correct node directory (which was broken in v0.3.0).

What's Changed

FEATURES

• feat(rpc): implementing staking reads and replace sdk.Address by @distractedm1nd in #1060

IMPROVEMENTS

• refactor(service/state): CoreAccessor relies on header.Head instead of header.Getter by @renaynay in #1088
• das: Parallelise DASer by @walldiss in #988
• header/store: use an atomic.Pointer instead of Lock by @Martichou in #1079

BUG FIXES

• fix(cel-key): reading from correct directory by @distractedm1nd in #1087
• refactor(service/state): Fix log to return correct height of balance request by @renaynay in #1089

DEPENDENCIES

• chore(deps): bump github.com/multiformats/go-multiaddr from 0.6.0 to 0.7.0 by @dependabot in #1071
• chore(deps): bump go.opentelemetry.io/otel/sdk from 1.9.0 to 1.10.0 by @dependabot in #1094
• chore(deps): bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp from 1.9.0 to 1.10.0 by @dependabot in #1098
• chore(deps): Upgrading to go 1.19 by @distractedm1nd in #1067
• chore(deps): bump go.opentelemetry.io/otel/metric from 0.31.0 to 0.32.0 by @dependabot in #1136
• refactor(service/state): Use DefaultProofRuntime instead of ibc-go VerifyMembership endpoint by @renaynay in #1131
• chore: remove deprecated linters by @rootulp in #1150
• chore(deps): bump github.com/ipfs/go-cid from 0.3.0 to 0.3.2 by @dependabot in #1053

MISC

• typo: Fix the cmd grpc port in the docs by @sweexordious in #1097
• README: ✍️ added Table of Contents by @nitinmewar in #1082
• Remove Changelog section from CONTRIBUTING.md by @rootulp in #1148
• docs(adr): BlockSync Overhaul ADR #11 - Part 1 by @Wondertan in #1037
• Remove @liamsi from global CODEOWNERS by @Wondertan in #1095
• header/p2p: remove ExtendedHeaderRequest struct by @vgonkivs in #1141
• Add Protobuf section to CONTRIBUTING.md by @rootulp in #1145

New Contributors

• @sweexordious made their first contribution in #1097
• @walldiss made their first contribution in #988
• @Martichou made their first contribution in #1079

Full Changelog: v0.3.0...v0.3.1

v0.3.0

v0.3.0 | 2022-09-06

Arabica Testnet Release

It has been half a year since our last minor release, and, as a result, our third minor release of celestia-node introduces a large range of new and hardened functionality. Some key highlights can be found below:

Highlights

• Re-introduction of full node type
  • Operates solely on the p2p network and independently from the Core network
  • Able to sync and reconstruct block data from light nodes only
  • Secures light node by generating and broadcasting fraud proofs
  • Light nodes discover Full Nodes over p2p network via DHT
• Implementation of state interaction via a celestia-core connection (ability to read and verify balances, as well as submit transactions to the celestia network), specifically the ability to submit a PayForData transaction
• Data availability sampling (DASing) over headers in the past and local caching of successful sampling results
• Introduction of FraudService
• • Bad Encoding Fraud Proofs (BEFPs)
• Alpha node API
• Key management utility via cel-key
• Introduction of telemetry via opentelemetry
• Parallelized GetSharesByNamespace resulting in faster application data retrieval
• Removes Core Node embedding for Bridge Node

Migration

The Node's Store default directory path has changed, and If you have been running any of the past two release candidates against mamaki and want to retain your data, please run the Node with --node.store ~/.celestia-<node_type> flag or migrate your store from $HOME/.celestia-<node_type> to $HOME/.celestia-<node_type>-mamaki.

BREAKING CHANGES

• feat!(header/p2p/exchange): Upgrade and reformat header exchange protocol ID
• refactor!(cmd/cel-key): Extract cel-key utility into separate binary
• refactor!(cmd|node): Refactor core flags

FEATURES

• feat(node): Introduce full node type
• feat(service/state): Introduce StateService and CoreAccessor
• feat(das): Implement DASing of past headers
• feat(fraud): Implement Bad Encoding Fraud Proofs (BEFPs)
• feat(ipld|shares): Rework data retrieval
• feat(service/state): Add /submit_pfd endpoint
• feat(service/rpc): Add rpc service to gather all available endpoints | alpha API
• feat(node/state): Automatic key generation if no key found in keystore
• feat(service/share): Implement CacheAvailability
• feat(service/share): Implement full node discovery
• feat(fraud): Implement fraud service
• feat(node): Integrate fraud service
• feat(fraud): Implement fraud store
• feat(das): Implement DASState
• feat(cmd): Integration of OpenTelemetry with OTLP exporter
• feat(service/state): Implement balance verification against AppHash
• feat(service/state): Transaction proxy for staking module tx types
• feat(fraud): implement fraud sync

IMPROVEMENTS

• refactor(service/share): No error if namespace not found in GetSharesByNamespace
• refactor(header): Move header sub-services to different directories
• test(service/share): Add full node reconstruction tests
• dep(ipld): Move from DAGService to BlockService
• refactor(ipld): Rework retriever
• refactor(fraud): Extend proof structure
• feat(service/share): Handle disconnection of discovered peer
• feat(ipld): Concurrent GetSharesByNamespace
• refactor(cmd): Use context.WithValue instead of custom Env implementation
• feat(cmd): Provide ability to specify network to user
• feat|refactor(header/sync): network head determination
• params: Switch default network to arabica

BUG FIXES

• fix(service/header): Unlock heightSub lock on rare cases
• fix(core): Prevent spamming of old headers during bridge node sync
• fix(service/share): Fix GetSharesByNamespace ordering
• fix(ipld): Concurrent GetShares to prevent blocking of share fetching in case of one unavailable share
• fix(ipld): Fix race in rare case when Retriever fetches from both row and col
• fix(ipld): Rely on rsmt2d.Codec constructor to avoid races
• fix(node): Invoke function to ensure empty block data is stored at least once
• fix(das): Return from sampling routine upon failure
• fix(das): Fix checkpoint in catchUp manager
• fix(service/header): Fix header store design issues with Batch
• fix(ipld): Re-enable re-importing of rsmt2d.ExtendedDataSquare
• fix(ipld): Fix races in doRequest method
• fix(service/share): Protect datastore autobatch with a lock
• fix(node/p2p): Add condition to restore routing table refresh period
• fix(node): Change applied Availability for bridge node
• fix(fraud): Blocklist peer if sent invalid fraud proof

MISCELLANEOUS

• refactor(core): Remove embedded core node option
• chore(ipld|shares): cleanup Share types, global default rsmt2d codec, various cleanups
• refactor(header): Exchange interface implements Getter
• feat(service/share): Make discovery constants configurable

New Contributors

• @rootulp made their first contribution in #479
• @toanngosy made their first contribution in #496
• @adlerjohn made their first contribution in #605
• @liamsi made their first contribution in #607
• @0xyicheng made their first contribution in #767
• @distractedm1nd made their first contribution in #852
• @iofq made their first contribution in #769

Full Changelog: https://github.com/celestiaorg/celestia-node/compare/v0.2.0...v0.3.0## v0.3.0 | 2022-09-06

v0.3.0-rc2

Another release candidate for the almost complete Full Node feature.