Some updates to Lighthouse book (#6995)

* #6447


  - Move some deprecated pages to a new section under `Archived`

- Remove fallback log in mev as the log will not be present after VC using `/eth/v3/validator/blocks` endpoint by default
- Add warning against using Btrfs file system (thank you @ChosunOne for the report)
- Add data shared by @mcdee  on tree states API queries time
- Rename partial withdrawals to validator sweep to differentiate it from the upcoming execution layer partial withdrawals
- Update NAT API response
- Update docs on IPv6

- Rename .md files to follow a standard prefix section name, e.g., installation_*.md, advanced_*.md
- Standardise .md files using underscore `_` instead of hyphen `-` to be consistent with other files naming conventions.

Author: chong-he

book/.markdownlint.yml

@@ -8,7 +8,7 @@ MD010:
 MD013: false
 
 # MD028: set to false to allow blank line between blockquote: https://github.com/DavidAnson/markdownlint/blob/main/doc/md028.md
-# This is because the blockquotes are shown separatedly (a deisred outcome) when having a blank line in between
+# This is because the blockquotes are shown separately (a desired outcome) when having a blank line in between
 MD028: false
 
 # MD024: set siblings_only to true so that same headings with different parent headings are allowed

book/src/SUMMARY.md

@@ -2,58 +2,54 @@
 
 * [Introduction](./intro.md)
 * [Installation](./installation.md)
-  * [Pre-Built Binaries](./installation-binaries.md)
-  * [Docker](./docker.md)
-  * [Build from Source](./installation-source.md)
-  * [Raspberry Pi 4](./pi.md)
-  * [Cross-Compiling](./cross-compiling.md)
-  * [Homebrew](./homebrew.md)
-  * [Update Priorities](./installation-priorities.md)
+  * [Pre-Built Binaries](./installation_binaries.md)
+  * [Docker](./installation_docker.md)
+  * [Build from Source](./installation_source.md)
+  * [Cross-Compiling](./installation_cross_compiling.md)
+  * [Homebrew](./installation_homebrew.md)
+  * [Update Priorities](./installation_priorities.md)
 * [Run a Node](./run_a_node.md)
-* [Become a Validator](./mainnet-validator.md)
-* [Validator Management](./validator-management.md)
-  * [The `validator-manager` Command](./validator-manager.md)
-    * [Creating validators](./validator-manager-create.md)
-    * [Moving validators](./validator-manager-move.md)
-    * [Managing validators](./validator-manager-api.md)
-  * [Slashing Protection](./slashing-protection.md)
-  * [Voluntary Exits](./voluntary-exit.md)
-  * [Partial Withdrawals](./partial-withdrawal.md)
-  * [Validator Monitoring](./validator-monitoring.md)
-  * [Doppelganger Protection](./validator-doppelganger.md)
-  * [Suggested Fee Recipient](./suggested-fee-recipient.md)
-  * [Validator Graffiti](./graffiti.md)
+* [Become a Validator](./mainnet_validator.md)
+* [Validator Management](./validator_management.md)
+  * [The `validator-manager` Command](./validator_manager.md)
+    * [Creating validators](./validator_manager_create.md)
+    * [Moving validators](./validator_manager_move.md)
+    * [Managing validators](./validator_manager_api.md)
+  * [Slashing Protection](./validator_slashing_protection.md)
+  * [Voluntary Exits](./validator_voluntary_exit.md)
+  * [Validator Sweep](./validator_sweep.md)
+  * [Validator Monitoring](./validator_monitoring.md)
+  * [Doppelganger Protection](./validator_doppelganger.md)
+  * [Suggested Fee Recipient](./validator_fee_recipient.md)
+  * [Validator Graffiti](./validator_graffiti.md)
 * [APIs](./api.md)
-  * [Beacon Node API](./api-bn.md)
-    * [Lighthouse API](./api-lighthouse.md)
-    * [Validator Inclusion APIs](./validator-inclusion.md)
-  * [Validator Client API](./api-vc.md)
-    * [Endpoints](./api-vc-endpoints.md)
-    * [Authorization Header](./api-vc-auth-header.md)
-    * [Signature Header](./api-vc-sig-header.md)
-  * [Prometheus Metrics](./advanced_metrics.md)
-* [Lighthouse UI (Siren)](./lighthouse-ui.md)
-  * [Configuration](./ui-configuration.md)
-  * [Authentication](./ui-authentication.md)
-  * [Usage](./ui-usage.md)
-  * [FAQs](./ui-faqs.md)
+  * [Beacon Node API](./api_bn.md)
+    * [Lighthouse API](./api_lighthouse.md)
+    * [Validator Inclusion APIs](./api_validator_inclusion.md)
+  * [Validator Client API](./api_vc.md)
+    * [Endpoints](./api_vc_endpoints.md)
+    * [Authorization Header](./api_vc_auth_header.md)
+  * [Prometheus Metrics](./api_metrics.md)
+* [Lighthouse UI (Siren)](./ui.md)
+  * [Configuration](./ui_configuration.md)
+  * [Authentication](./ui_authentication.md)
+  * [Usage](./ui_usage.md)
+  * [FAQs](./ui_faqs.md)
 * [Advanced Usage](./advanced.md)
-  * [Checkpoint Sync](./checkpoint-sync.md)
-  * [Custom Data Directories](./advanced-datadir.md)
-  * [Proposer Only Beacon Nodes](./advanced-proposer-only.md)
-  * [Remote Signing with Web3Signer](./validator-web3signer.md)
+  * [Checkpoint Sync](./advanced_checkpoint_sync.md)
+  * [Custom Data Directories](./advanced_datadir.md)
+  * [Proposer Only Beacon Nodes](./advanced_proposer_only.md)
+  * [Remote Signing with Web3Signer](./advanced_web3signer.md)
   * [Database Configuration](./advanced_database.md)
-  * [Database Migrations](./database-migrations.md)
-  * [Key Management (Deprecated)](./key-management.md)
-  * [Key Recovery](./key-recovery.md)
+  * [Database Migrations](./advanced_database_migrations.md)
+  * [Key Recovery](./advanced_key_recovery.md)
   * [Advanced Networking](./advanced_networking.md)
-  * [Running a Slasher](./slasher.md)
-  * [Redundancy](./redundancy.md)
-  * [Release Candidates](./advanced-release-candidates.md)
-  * [MEV](./builders.md)
-  * [Merge Migration](./merge-migration.md)
-  * [Late Block Re-orgs](./late-block-re-orgs.md)
-  * [Blobs](./advanced-blobs.md)
+  * [Running a Slasher](./advanced_slasher.md)
+  * [Redundancy](./advanced_redundancy.md)
+  * [Release Candidates](./advanced_release_candidates.md)
+  * [MEV](./advanced_builders.md)
+  * [Late Block Re-orgs](./advanced_re-orgs.md)
+  * [Blobs](./advanced_blobs.md)
 * [Command Line Reference (CLI)](./help_general.md)
   * [Beacon Node](./help_bn.md)
   * [Validator Client](./help_vc.md)
@@ -62,7 +58,11 @@
     * [Import](./help_vm_import.md)
     * [Move](./help_vm_move.md)
 * [Contributing](./contributing.md)
-  * [Development Environment](./setup.md)
+  * [Development Environment](./contributing_setup.md)
 * [FAQs](./faq.md)
 * [Protocol Developers](./developers.md)
 * [Security Researchers](./security.md)
+* [Archived](./archived.md)
+  * [Merge Migration](./archived_merge_migration.md)
+  * [Raspberry Pi 4](./archived_pi.md)
+  * [Key Management](./archived_key_management.md)

book/src/advanced.md

@@ -6,19 +6,17 @@ elsewhere?
 This section provides detailed information about configuring Lighthouse for specific use cases, and
 tips about how things work under the hood.
 
-* [Checkpoint Sync](./checkpoint-sync.md): quickly sync the beacon chain to perform validator duties.
-* [Custom Data Directories](./advanced-datadir.md): modify the data directory to your preferred location.
-* [Proposer Only Beacon Nodes](./advanced-proposer-only.md): beacon node only for proposer duty for increased anonymity.
-* [Remote Signing with Web3Signer](./validator-web3signer.md): don't want to store your keystore in local node? Use web3signer.
+* [Checkpoint Sync](./advanced_checkpoint_sync.md): quickly sync the beacon chain to perform validator duties.
+* [Custom Data Directories](./advanced_datadir.md): modify the data directory to your preferred location.
+* [Proposer Only Beacon Nodes](./advanced_proposer_only.md): beacon node only for proposer duty for increased anonymity.
+* [Remote Signing with Web3Signer](./advanced_web3signer.md): don't want to store your keystore in local node? Use web3signer.
 * [Database Configuration](./advanced_database.md): understanding space-time trade-offs in the database.
-* [Database Migrations](./database-migrations.md): have a look at all previous Lighthouse database scheme versions.
-* [Key Management](./key-management.md): explore how to generate wallet with Lighthouse.
-* [Key Recovery](./key-recovery.md): explore how to recover wallet and validator with Lighthouse.
+* [Database Migrations](./advanced_database_migrations.md): have a look at all previous Lighthouse database scheme versions.
+* [Key Recovery](./advanced_key_recovery.md): explore how to recover wallet and validator with Lighthouse.
 * [Advanced Networking](./advanced_networking.md): open your ports to have a diverse and healthy set of peers.
-* [Running a Slasher](./slasher.md): contribute to the health of the network by running a slasher.
-* [Redundancy](./redundancy.md): want to have more than one beacon node as backup? This is for you.
-* [Release Candidates](./advanced-release-candidates.md): latest release of Lighthouse to get feedback from users.
-* [Maximal Extractable Value](./builders.md): use external builders for a potential higher rewards during block proposals
-* [Merge Migration](./merge-migration.md): look at what you need to do during a significant network upgrade: The Merge
-* [Late Block Re-orgs](./late-block-re-orgs.md): read information about Lighthouse late block re-orgs.
-* [Blobs](./advanced-blobs.md): information about blobs in Deneb upgrade
+* [Running a Slasher](./advanced_slasher.md): contribute to the health of the network by running a slasher.
+* [Redundancy](./advanced_redundancy.md): want to have more than one beacon node as backup? This is for you.
+* [Release Candidates](./advanced_release_candidates.md): latest release of Lighthouse to get feedback from users.
+* [Maximal Extractable Value](./advanced_builders.md): use external builders for a potential higher rewards during block proposals
+* [Late Block Re-orgs](./advanced_re-orgs.md): read information about Lighthouse late block re-orgs.
+* [Blobs](./advanced_blobs.md): information about blobs in Deneb upgrade

book/src/advanced-blobs.md book/src/advanced_blobs.mdbook/src/advanced-blobs.md renamed to book/src/advanced_blobs.md

@@ -38,4 +38,4 @@ In the Deneb network upgrade, one of the changes is the implementation of EIP-48
    curl "http://localhost:5052/lighthouse/database/info" | jq
    ```
 
-   Refer to [Lighthouse API](./api-lighthouse.md#lighthousedatabaseinfo) for an example response.
+   Refer to [Lighthouse API](./api_lighthouse.md#lighthousedatabaseinfo) for an example response.

book/src/builders.md book/src/advanced_builders.mdbook/src/builders.md renamed to book/src/advanced_builders.md

@@ -83,11 +83,11 @@ is something afoot.
 
 To update gas limit per-validator you can use the [standard key manager API][gas-limit-api].
 
-Alternatively, you can use the [lighthouse API](api-vc-endpoints.md). See below for an example.
+Alternatively, you can use the [lighthouse API](api_vc_endpoints.md). See below for an example.
 
 ### Enable/Disable builder proposals via HTTP
 
-Use the [lighthouse API](api-vc-endpoints.md) to enable/disable use of the builder API on a per-validator basis.
+Use the [lighthouse API](api_vc_endpoints.md) to enable/disable use of the builder API on a per-validator basis.
 You can also update the configured gas limit with these requests.
 
 #### `PATCH /lighthouse/validators/:voting_pubkey`
@@ -98,7 +98,7 @@ You can also update the configured gas limit with these requests.
 |-------------------|--------------------------------------------|
 | Path              | `/lighthouse/validators/:voting_pubkey`    |
 | Method            | PATCH                                      |
-| Required Headers  | [`Authorization`](./api-vc-auth-header.md) |
+| Required Headers  | [`Authorization`](./api_vc_auth_header.md) |
 | Typical Responses | 200, 400                                   |
 
 #### Example Path
@@ -147,7 +147,7 @@ INFO Published validator registrations to the builder network, count: 3, service
 
 ### Fee Recipient
 
-Refer to [suggested fee recipient](suggested-fee-recipient.md) documentation.
+Refer to [suggested fee recipient](validator_fee_recipient.md) documentation.
 
 ### Validator definitions example
 
@@ -244,16 +244,9 @@ INFO Builder payload ignored
 INFO Chain is unhealthy, using local payload
 ```
 
-In case of fallback you should see a log indicating that the locally produced payload was
-used in place of one from the builder:
-
-```text
-INFO Reconstructing a full block using a local payload
-```
-
 ## Information for block builders and relays
 
-Block builders and relays can query beacon node events from the [Events API](https://ethereum.github.io/beacon-APIs/#/Events/eventstream). An example of querying the payload attributes in the Events API is outlined in [Beacon node API - Events API](./api-bn.md#events-api)
+Block builders and relays can query beacon node events from the [Events API](https://ethereum.github.io/beacon-APIs/#/Events/eventstream). An example of querying the payload attributes in the Events API is outlined in [Beacon node API - Events API](./api_bn.md#events-api)
 
 [mev-rs]: https://github.com/ralexstokes/mev-rs
 [mev-boost]: https://github.com/flashbots/mev-boost

book/src/checkpoint-sync.md book/src/advanced_checkpoint_sync.mdbook/src/checkpoint-sync.md renamed to book/src/advanced_checkpoint_sync.md

@@ -134,7 +134,7 @@ Important information to be aware of:
 * It is safe to interrupt state reconstruction by gracefully terminating the node – it will pick up
   from where it left off when it restarts.
 * You can start reconstruction from the HTTP API, and view its progress. See the
-  [`/lighthouse/database`](./api-lighthouse.md) APIs.
+  [`/lighthouse/database`](./api_lighthouse.md) APIs.
 
 For more information on historic state storage see the
 [Database Configuration](./advanced_database.md) page.

book/src/advanced_database.md

@@ -61,6 +61,26 @@ that we have observed are:
   to apply. We observed no significant performance benefit from `--hierarchy-exponents 5,7,11`, and
   a substantial increase in space consumed.
 
+The following table lists the data for different configurations. Note that the disk space requirement is for the `chain_db` and `freezer_db`, excluding the `blobs_db`.
+
+| Hierarchy Exponents | Storage Requirement | Sequential Slot Query | Uncached Query | Time to Sync |
+|---|---|---|---|---|
+| 5,9,11,13,16,18,21 (default) | 418 GiB | 250-700 ms | up to 10 s | 1 week |
+| 5,7,11 (frequent snapshots) | 589 GiB | 250-700 ms | up to 6 s | 1 week |
+| 0,5,7,11 (per-slot diffs) | 2500 GiB | 250-700 ms | up to 4 s | 7 weeks |
+
+[Jim](https://github.com/mcdee) has done some experiments to study the response time of querying random slots (uncached query) for `--hierarchy-exponents 0,5,7,11` (per-slot diffs) and `--hierarchy-exponents 5,9,11,13,17,21` (per-epoch diffs), as show in the figures below. From the figures, two points can be concluded:
+
+- response time (y-axis) increases with slot number (x-axis) due to state growth.
+- response time for per-slot configuration in general is 2x faster than that of per-epoch.
+
+In short, setting different configurations is a trade-off between disk space requirement, sync time and response time. The data presented here is useful to help users choosing the configuration that suit their needs.
+
+_We acknowledge the data provided by [Jim](https://github.com/mcdee) and his consent for us to share it here._
+
+![Response time for per-epoch archive](./imgs/per-epoch.png)
+![Response time for per-slot archive](./imgs/per-slot.png)
+
 If in doubt, we recommend running with the default configuration! It takes a long time to
 reconstruct states in any given configuration, so it might be some time before the optimal
 configuration is determined.

book/src/database-migrations.md book/src/advanced_database_migrations.mdbook/src/database-migrations.md renamed to book/src/advanced_database_migrations.md

@@ -12,7 +12,7 @@ lighthouse --network mainnet --datadir /var/lib/my-custom-dir bn --staking
 lighthouse --network mainnet --datadir /var/lib/my-custom-dir vc
 ```
 
-The first step creates a `validators` directory under `/var/lib/my-custom-dir` which contains the imported keys and [`validator_definitions.yml`](./validator-management.md).
+The first step creates a `validators` directory under `/var/lib/my-custom-dir` which contains the imported keys and [`validator_definitions.yml`](./validator_management.md).
 After that, we simply run the beacon chain and validator client with the custom dir path.
 
 ## Relative Paths