2.0.0

Introducing Clio version 2.0.0

Version 2.0.0 of Clio, an XRP Ledger API server optimized for HTTP and WebSocket API calls, is now available. This release adds new features, bug fixes, and amendment support.

Amendment Support

The following amendments have been introduced since Clio 1.0.4 and have transaction model changes. Clio 2.0.0 is built with libxrpl 1.12.0, which supports these amendments.

• AMM (https://xrpl.org/known-amendments.html#amm)
  • Note: the amm_info API method is not yet supported and will be introduced in a later release
• Clawback (https://xrpl.org/known-amendments.html#clawback)
• XRPFees (https://xrpl.org/known-amendments.html#xrpfees)

If these amendments are enabled and you have not upgraded Clio to 2.0.0 or newer, the ETL will be amendment blocked and new ledgers will not be processed.

The current voting status of these amendments in mainnet is available here: https://xrpscan.com/amendments

Database Migration

If you are currently running Clio on a previous version and upgrading to 2.0.0, you must perform a database migration to properly support NFT data. Instructions for the migration are described https://github.com/XRPLF/clio/tree/clio_migrator%402.0.0

What's Changed

• Implement logging abstraction by @godexsoft in #371
• Fix pre-commit to only check files changed by clang-format by @godexsoft in #398
• Fix account ownership check by @godexsoft in #383
• include searched_all in error response of tx by @cjcobb23 in #407
• helper function for subscribe to ensure cleanup by @cjcobb23 in #402
• Add closed to header for all paths by @godexsoft in #416
• Add custom error for malformed owner and request by @godexsoft in #417
• Use custom malformedAddress error in ledger_entry by @godexsoft in #419
• Return lgrIdxsInvalid error for ledger_max_index less than ledger_min_index by @officialfrancismendoza in #339
• Update headers to use #pragma once by @godexsoft in #422
• Add custom error for malformed request by @godexsoft in #414
• Return srcCurMalformed on invalid taker_pays in book_offers by @godexsoft in #413
• Add workaround for source_location for AppleClang by @godexsoft in #431
• Always add git ref to version string by @legleux in #430
• add connection counting by @cjcobb23 in #433
• Fix malformed output format over ws rpc by @godexsoft in #426
• Remove branch name from version string by @legleux in #437
• Implement cli parsing using boost::po by @godexsoft in #436
• Update documentation and config with ssl_cert_file and ssl_key_file by @officialfrancismendoza in #443
• Fix gateway balances to match rippled output by @godexsoft in #441
• Update documentation and config with start_sequence by @officialfrancismendoza in #438
• Add copyright to top of each source file by @godexsoft in #444
• Increase file descriptor limit by @legleux in #449
• Update readme with more log configurations by @godexsoft in #447
• Document dos_guard in example config. Log when client surpasses rate limit by @cjcobb23 in #451
• Unit-tests for slightly modified DOSGuard by @godexsoft in #453
• Build Clio on macOS and Ubuntu 22.04 by @legleux in #456
• Feature/profiler by @cindyyan317 in #458
• Add hotfix to old account_tx ledger index max out-of-bounds PR by @officialfrancismendoza in #461
• Fix LGR_IDXS_INVALID error code mismatch by @officialfrancismendoza in #463
• Feature/dosguard : add request limit by @cindyyan317 in #462
• Write Clio version file from template by @legleux in #457
• replace unique_lock with scoped_lock by @cindyyan317 in #467
• Demote confusing errors that are not really errors by @godexsoft in #471
• Add Unittests Coverage by @cindyyan317 in #472
• Fix bug with ClioVersion to prevent crash at runtime by @godexsoft in #473
• add code coverage job by @cindyyan317 in #477
• Update gitignore and readme by @cindyyan317 in #481
• Fix clio-server link issue by @cindyyan317 in #485
• Eliminate extraneous bypass in ETLSource.cpp by @officialfrancismendoza in #484
• Run test on mac by @cindyyan317 in #490
• Adjust DosGuard default cfg by @cindyyan317 in #496
• Implement basics of RPC subsystem by @godexsoft in #487
• Improve sweephandler test flakiness by @godexsoft in #499
• Add unittests for subscription by @cindyyan317 in #488
• Document RPC framework by @godexsoft in #501
• Fix backend test fail by @cindyyan317 in #502
• Rpc: support rpc call without parameter by @cindyyan317 in #503
• Update CONTRIBUTING.md by @intelliot in #504
• Fix mac build failure by @cindyyan317 in #509
• Exit 1 on failed experimental builds so build step fails by @legleux in #507
• Add common validator by @cindyyan317 in #510
• Add libfmt by @godexsoft in #514
• Change loglines severity and channel by @godexsoft in #517
• Fix marker issue by @cindyyan317 in #518
• Update CONTRIBUTING.md by @intelliot in #520
• Provide coroutine process interface for handler by @cindyyan317 in #521
• Account channel by @cindyyan317 in #519
• Handler account_currencies by @cindyyan317 in #524
• Tx handler in new RPC framework by @cindyyan317 in #526
• Fix ledger_entry API : can not get ticket entry by @cindyyan317 in #528
• Add IfType requirement to RPC framework by @cindyyan317 in #530
• Bug fix: ledger_entry responds incorrectly when the expected type mismatch the actual type by @cindyyan317 in #532
• Gateway balance fix by @cindyyan317 in #535
• Gateway balance by @cindyyan317 in #536
• Ledger entry in new RPC framework by @cindyyan317 in #534
• Custom error validator by @cindyyan317 in #540
• Make NFT URI lookup O(1) by @ledhed2222 in #313
• Implement book_offers in new RPC framework by @cindyyan317 in #542
• Implement ledger_range rpc handler by @godexsoft in #548
• Fix subscription forward issue by @cindyyan317 in #544
• Implement account_lines with new RPC framework by @godexsoft in #551
• Transaction entry with new RPC framework by @cindyyan317 in #553
• Fix spawn by @cindyyan317 in #556
• Add codecov.io steps by @godexsoft in #546
• Implement nextgen nft_info handler by @godexsoft in #558
• noripple_check implementation of new RPC system by @cindyyan317 in #554
• Use JSS string by @cindyyan317 in #563
• account_tx of new RPC framework by @cindyyan317 in #562
• Implement nft_buy_offers and tests by @godexsoft in #568
• account_offer in new RPC framework by @cindyyan317 in #567
• Cassandra backend revamp by @godexsoft in #537
• Implement nextgen nft_sell_offers handler by @godexsoft in #571
• account_info implementation in new RPC framework by @cindyyan317 in #573
• Fix compile error on clang14.0.3 by @cindyyan317 in #577
• Implement nextgen random handler and tests by @godexsoft in #576
• Create ngContext by @cindyyan317 in #579
• Implement nft_history nextgen handler by @godexsoft in #581
• Reformat codebase with 120 char limit by @godexsoft in #583
• Implement better error handling and datastax unauthorized workaround by @g...

2.0

The exact version number is 2.0.0. This 2.0 alias is provided in case of inadvertent linking. Please continue to the 2.0.0 release.

1.0.4

What's Changed

• add nft_history by @ledhed2222 in #255
• Database read throttle by @cjcobb23 in #242
• add work queue output to server_info by @cjcobb23 in #322
• Implement subscription for book_changes by @godexsoft in #315
• Remove useless mutex from BackendInterface and its usage by @godexsoft in #326
• Make ProbingETL toJson serialize underlying source states while disconnected or probing by @godexsoft in #325
• Throw error if server bind or listen fails by @cjcobb23 in #309
• Implement unique taging of incoming requests by @godexsoft in #311
• Missing lgrIdxMalformed error message for account_tx API call by @officialfrancismendoza in #336
• Return dstAmtMalformed in book offers by @officialfrancismendoza in #333
• Mark package release's version string by @legleux in #317
• bugfix/add-default-limit-to-noripple_check by @officialfrancismendoza in #324
• Return dstIsrMalformed for book_offers by @officialfrancismendoza in #341
• bugfix/account_tx-returns-marker by @officialfrancismendoza in #282
• Allow server to download cache from another clio server by @cjcobb23 in #246
• Remove postgres support from clio by @godexsoft in #327
• Return malformedAddress in ledger_entry by @officialfrancismendoza in #343
• Return malformedOwner in ticket.owner for ledger_entry by @officialfrancismendoza in #344
• Return malformedOwner for deposit_preauth.owner in ledger_entry by @officialfrancismendoza in #345
• Return badMarket for same currency in taker_gets and taker_pays in book_offers by @officialfrancismendoza in #357
• fix bug on cache download from peer when ledger not found by @cjcobb23 in #370
• Abstraction for config by @godexsoft in #358
• Change index name for nft buy/sell offers api by @shawnxie999 in #377
• Fix nft_sell_offers/nft_buy_offers limit and marker by @godexsoft in #342
• Add offers to the response regardless of it being empty by @godexsoft in #389
• Return account malformed error for invalid accounts by @godexsoft in #388
• Add checks for empty array in accounts/accounts_proposed subscriptions by @godexsoft in #387
• Implement a simple check to suppress 'validated' flag output by @godexsoft in #393
• Remove checks for a valid subscription in subscribe/unsubscribe rpc by @godexsoft in #386
• Port ignore_default support for account_lines rpc by @godexsoft in #391
• Return actNotFound for non-existent account in account iteration methods by @cjcobb23 in #382
• put peers in correct spot in example config by @cjcobb23 in #376
• Return account malformed error from account_tx when account is malformed by @cjcobb23 in #319
• Return correct error on subscription to non-existing stream by @godexsoft in #390
• return error on negative limit by @cjcobb23 in #394
• Add clang-format git hook by @godexsoft in #395
• Return limit for assorted account methods by @officialfrancismendoza in #328
• Add doxygen documentation by @officialfrancismendoza in #307
• remove accountFromSeed by @cjcobb23 in #399
• handle invalidHotWallet in gateway_balances by @cjcobb23 in #384
• Better handle markers in nft_buy_offers and nft_sell_offers by @cjcobb23 in #400
• Add rpcINVALID_PARAMS error code by @officialfrancismendoza in #365
• Add rpcBAD_ISSUER to subscribe by @officialfrancismendoza in #379
• Return srcIsrMalformed for taker_gets issuer in book_offers by @officialfrancismendoza in #355
• Introduce support for custom error codes by @godexsoft in #396
• Return noPermission in ledger admin call by @officialfrancismendoza in #360
• 1.0.4 Release Candidate 2 by @legleux in #465

New Contributors

• @shawnxie999 made their first contribution in #377

Full Changelog: 1.0.3...1.0.4

1.0.3

What's Changed

• added cache hit rate to server info by @ethanlabelle in #220
• refactor main.cpp into an application class by @natenichols in #151
• set grpc max message size to unlimited by @cjcobb23 in #249
• Add CONTRIBUTING documentation (#293) by @godexsoft in #296
• Fix bug causing some ledgers to not be published by @cjcobb23 in #281
• Timeout tests by @legleux in #257
• insert delivered_amount based on close time by @cjcobb23 in #252
• cleanup README and example config by @cjcobb23 in #247
• Implement book_changes RPC by @godexsoft in #300
• Reconnect after timeout in ETLSource websockets by @godexsoft in #297
• Fix compiler warnings by @godexsoft in #306
• Probing ETL Source by @godexsoft in #292
• Fixes issue with assigning values to offers objects by @ledhed2222 in #301
• Proposed 1.0.3 by @manojsdoshi in #409

Full Changelog: 1.0.2...1.0.3

1.0.2

What's Changed

• report ledger when no marker exists in ledger_data by @natenichols in #203
• Fixed 503 response code by @Shoukozumi in #214
• moved warnings array to outermost level of response json by @ethanlabelle in #208
• ensure lgrInfo is in context.range by @natenichols in #226
• remove assert in fetchLedgerPage by @cjcobb23 in #227
• Fine tune cache download by @cjcobb23 in #215
• NFTInfo RPC route by @ledhed2222 in #98
• change id to object_id in diff response to ledger command by @cjcobb23 in #218
• Updated backend README.md with the latest Cassandra schemas by @Shoukozumi in #170
• wrap atomics in shared_ptr for cache download by @cjcobb23 in #230
• Fixed warning message to be XRPL standard compliant by @Shoukozumi in #229
• server_info improvements by @cjcobb23 in #240
• Guard for GCC < 11 and update readme by @legleux in #243
• update rippled to 1.9.2 by @cjcobb23 in #228
• Update version to 1.0.2 by @cjcobb23 in #245

New Contributors

• @ledhed2222 made their first contribution in #98

Full Changelog: 1.0.1...1.0.2

1.0.1

What's Changed

• Use ledger close times for stale data warning by @ethanlabelle in #194
• removed unused LayeredCache by @ethanlabelle in #199
• remove "this software is in a beta version" from readme by @natenichols in #204
• Make sure to extract only validated ledgers from rippled
• Set version to 1.0.1 by @cjcobb23 in #216

Full Changelog: 1.0.0...1.0.1

1.0.0

What's Changed

• disable cache when CacheLoadStyle::NONE by @natenichols in #152
• send messages to subscribers w/ shared_ptr by @natenichols in #147
• pin-dependency-versions by @legleux in #157
• handle string ledger_index values in doAccountTx by @natenichols in #162
• report transactions as validated in account_tx by @natenichols in #165
• dont report error.what() when returning rpcINTERNAL by @natenichols in #163
• add clio version to repository by @natenichols in #156
• cleanup subscriptions when session errors or is closed by @natenichols in #146
• remove unused file in src/clio by @natenichols in #169
• Added log rotation feature and console/file logging config options by @Shoukozumi in #161
• append warning for stale data (#46) by @ethanlabelle in #175
• Use separate IO context for socket IO by @cjcobb23 in #168
• Iterate account nft pages without using successor by @cjcobb23 in #177
• cache commands that dont take parameters by @natenichols in #153
• error when marker does not exist in ledger_data by @natenichols in #167
• fix ledger_index_min/max in account_tx response by @natenichols in #172
• return no offers when an owner directory is not found by @natenichols in #176
• specify limit hi, low, and defaults in HandlerTable by @natenichols in #135
• set version to 0.3.0-b1 by @natenichols in #178
• Fixed previous log rotation commit rebase issue by @Shoukozumi in #181
• bypass forwarding cache if ledger_index is current or closed by @cjcobb23 in #185
• log every request and duration at info by @cjcobb23 in #183
• append warning to the base level of the response by @natenichols in #182
• always append clio warning by @cjcobb23 in #186
• send id in sendError by @natenichols in #184
• set version to 0.3.0-b2 by @natenichols in #188
• append warnings to response instead of result by @natenichols in #192
• allow user to specify no peer in doAccountLines by @natenichols in #193
• forward channel_verify and channel_authorize by @natenichols in #196
• set version to 0.3.0-b3 by @natenichols in #197
• reserve correctly when limit is numeric_limits::max() by @natenichols in #198
• Add headers for building with gcc-12 by @legleux in #201
• set version to 1.0.0 by @natenichols in #202

New Contributors

• @ethanlabelle made their first contribution in #175

Full Changelog: 0.2.0...1.0.0

0.2.0

What's Changed

Numerous stability improvements and addition of NFT RPCs

• fix validated_ledger location in server_info by @natenichols in #144
• return entryNotFound instead of ledgerNotFound by @natenichols in #149
• Log cleanup by @cjcobb23 in #150
• add support for account_nfts, nft_buy_offers, nft_sell_offers
• Fix message queueing in WsSession
• Fix for offer directory iteration
• Fix crasher in ledger data
• Properly catch in ip() when peer has disconnected
• Fixed server_info to match rippled
• Round age to zero when negative
• use 64 bit counters to prevent overflow

0.1.0

0.1.0

clio is an XRP Ledger API server. clio is optimized for RPC calls, over websocket or JSON-RPC. Validated historical ledger and transaction data is stored in a more space efficient format, using up to 4 times less space than rippled.

clio uses Cassandra or ScyllaDB, allowing for scalable read throughput. Multiple clio nodes can share access to the same dataset, allowing for a highly available cluster of clio nodes, without the need for redundant data storage or computation.

0.1.0 is the first beta of Project Clio. It contains:

• ./src/backend is the BackendInterface. This provides an abstraction for reading and writing information to a database.
• ./src/etl is the ReportingETL. The classes in this folder are used to extract information from the P2P network and write it to a database, either locally or over the network.
• ./src/rpc contains RPC handlers that are called by clients. These handlers should expose the same API as rippled.
• ./src/subscriptions contains the SubscriptionManager. This manages publishing to clients subscribing to streams or accounts.
• ./src/webserver contains a flex server that handles both http/s and ws/s traffic on a single port.
• ./unittests simple unit tests that write to and read from a database to verify that the ETL works.