regular node mode #399
Closed
regular node mode #399
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
khorolets
reviewed
Apr 29, 2025
kobayurii
force-pushed
the
regular_node_mode/rpc_database
branch
from
4311670 to
9f46355
Compare
khorolets
reviewed
May 26, 2025
Member
khorolets
left a comment
khorolets
left a comment
There was a problem hiding this comment.
Really good job, I've left a few comments mostly about naming to polish. Overall, looks great!
| /// 2. 261 * 5 = 1305 | ||
| /// | ||
| /// Output: 1305 | ||
| pub async fn get_data_range_id( |
Member
There was a problem hiding this comment.
Consider get_data_range_suffix, this is not a strong opinion just an idea
Member
Author
There was a problem hiding this comment.
I would keep the name get_data_range_id because the usage of this function is not limited to the database crate.
kobayurii
force-pushed
the
regular_node_mode/rpc_database
branch
2 times, most recently
from
e724f0d to
21350a1
Compare
block garbage collected
2. rewrite indexer to save data into specific data_range by block_height
kobayurii
force-pushed
the
regular_node_mode/rpc_database
branch
from
7812fd3 to
9d66715
Compare
Member
Author
|
Since we changed the database schema, it is not up-to-date. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant changes to enhance the database's handling of state changes by implementing data range partitioning, adding configuration options for archival mode and data ranges, and updating database schema and logic accordingly. These changes improve scalability, allow for more efficient data storage, and provide flexibility in managing historical data.
Configuration Enhancements:
keep_data_ranges_numberandarchival_modefields toGeneralRpcServerConfigandCommonGeneralRpcServerConfigto allow configuration of data ranges and archival mode. Default values and validation logic were also implemented. [1] [2] [3] [4] [5]default_env_configs.rsto include environment variables forKEEP_DATA_RANGES_NUMBERandARCHIVAL_MODE.Utility Functions:
get_data_range_id,get_earliest_available_block_height, andcheck_block_height_availabilityutility functions inutils.rsto calculate data range IDs, determine earliest available blocks, and validate block height availability.Database Schema and Partitioning:
create_new_range_tablesto dynamically create partitioned tables for state changes based on data range IDs.block_hashcolumn from state changes tables to simplify the schema.Database Logic Updates:
StateIndexerDbManagerto include a method for creating new range tables and removed theblock_hashparameter from state change methods.PostgresDBManagerto calculatedata_range_iddynamically and use it in SQL queries to access partitioned tables. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]Documentation and Comments:
data_range_idis calculated and used.