You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Make the Complete History Archive section a practical operator guide
Rework the archivist docs around what an operator is actually trying to
do — install the tool, decide where to run it, pick a command, then
scan / repair / mirror:
- Drop --verify from the default examples and give it its own section
covering what it checks and what it costs.
- Add a command-selection table, and document --low/--high.
- Note that stellar-archivist is separate from stellar-core: run it off
the validator host, and never stop the validator to work on its
archive.
- Remove the stray `systemctl start stellar-core` at the end of the
section, which paired with nothing and implied Core had been stopped.
- Fix an invalid ':::attention' admonition and copyedit throughout.
Also establish that a Full Validator's archive is a complete one.
Neither the Full Validator definition nor the Tier 1 requirements
mentioned archive depth, so nothing said an archive should reach back to
the first ledger. Tier 1 delegates its archive requirement to the Full
Validator definition, so stating completeness there covers both. The
archives page now frames publishing as two stages, so the Backfilling
section no longer implies you are finished when the archive holds only
recent checkpoints.
Copy file name to clipboardExpand all lines: docs/validators/README.mdx
+3-3Lines changed: 3 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -49,7 +49,7 @@ The basic flow, which you can navigate through using the "Admin Guide" on the le
49
49
50
50
## Types of validator nodes {/* #types-of-nodes */}
51
51
52
-
There are two types of validator nodes, and they perform the same basic functions: they run Stellar Core, connect to peers, submit transactions, and store the state of the ledger. The difference is this: a **Basic Validator** does not publish a history archive; a **Full Validator**does.
52
+
There are two types of validator nodes, and they perform the same basic functions: they run Stellar Core, connect to peers, submit transactions, and store the state of the ledger. The difference is this: a **Basic Validator** does not publish a history archive; a **Full Validator**publishes one covering the network's complete history.
53
53
54
54
:::info
55
55
@@ -67,9 +67,9 @@ The advantage: signatures can serve as official endorsements of specific ledgers
67
67
68
68
### Full Validator
69
69
70
-
#### Validating, offers public archive
70
+
#### Validating, offers a complete public archive
71
71
72
-
A Full Validator is the same as a Basic Validator except that it also publishes a [History Archive](./admin-guide/environment-preparation.mdx)containing snapshots of the ledger, including all transactions and their results. A Full Validator writes to an internet-facing blob store — such as AWS or Azure — so it's a bit more expensive and complex to run, but it also does the most to support the network’s resilience and decentralization.
72
+
A Full Validator is the same as a Basic Validator except that it also publishes a [History Archive](./admin-guide/environment-preparation.mdx)covering the network's **complete history** — snapshots of the ledger, including all transactions and their results. A Full Validator writes to an internet-facing blob store — such as AWS or Azure — so it's a bit more expensive and complex to run, but it also does the most to support the network’s resilience and decentralization.
73
73
74
74
When other nodes join the network — or experience difficulty and temporarily fall out of sync — they can consult archives offered by Full Validators to catch up on the history of the network. Redundant archives prevent a single point of failure, and allow network participants to verify the veracity of a given history.
Copy file name to clipboardExpand all lines: docs/validators/admin-guide/publishing-history-archives.mdx
+59-21Lines changed: 59 additions & 21 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,7 +3,12 @@ title: Publishing History Archives
3
3
sidebar_position: 50
4
4
---
5
5
6
-
If you want to run a [Full Validator](../README.mdx#full-validator), you need to set up your node to publish a history archive. You can host an archive using a blob store such as Amazon's S3 or Digital Ocean's spaces, or you can simply serve a local archive directly via an HTTP server such as Nginx or Apache. If you're setting up a [Basic Validator](../README.mdx#basic-validator), you can skip this section. No matter what kind of node you're planning to run, make sure to set it up to `get` history, which is covered in [Environment Preparation](./environment-preparation.mdx).
6
+
If you want to run a [Full Validator](../README.mdx#full-validator), you need to set up your node to publish a history archive covering the network's complete history. You can host an archive using a blob store such as Amazon's S3 or Digital Ocean's spaces, or you can simply serve a local archive directly via an HTTP server such as Nginx or Apache. If you're setting up a [Basic Validator](../README.mdx#basic-validator), you can skip this section. No matter what kind of node you're planning to run, make sure to set it up to `get` history, which is covered in [Environment Preparation](./environment-preparation.mdx).
7
+
8
+
Getting to a complete archive takes two stages:
9
+
10
+
- Configuring your node to publish, described in the sections below. This fills the archive forward from the moment your node starts.
11
+
-[Completing the archive](#complete-history-archive), which backfills everything from before that point.
7
12
8
13
:::caution[One archive per node]
9
14
@@ -160,7 +165,7 @@ server {
160
165
161
166
Given the choice, it's best to configure your history archive _prior to_ your node's initial sync with an existing network. That way your validator's history publishes as you join, and subsequently sync with, the network.
162
167
163
-
However, if you have not published an archive during the node's initial sync, the steps required to create a history archive for an existing validator — in other words, to upgrade a Basic Validator to a Full Validator — are quite straightforward. First, you'll need to stop your `stellar-core` instance:
168
+
However, if you have not published an archive during the node's initial sync, the steps required to start publishing from an existing validator are quite straightforward. This is the first of the two stages on the way to a Full Validator; [completing the archive](#complete-history-archive) follows. First, you'll need to stop your `stellar-core` instance:
164
169
165
170
```bash
166
171
systemctl stop stellar-core # modify this if not using systemctl
@@ -193,11 +198,15 @@ As you allow your node to join the network again, you can watch it start publish
At this stage your validator is successfully publishing its history, which enables other users to join the network using your archive.
201
+
At this stage your validator is publishing each new checkpoint as the network advances, which lets other nodes catch up from your archive as far back as the point where you started publishing. Everything before that point is still missing, and filling it in is the second stage.
197
202
198
203
## Complete History Archive
199
204
200
-
The [stellar-archivist](https://github.com/stellar/rs-stellar-archivist) command line tool scans, repairs, and mirrors history archives. Using the [SDF package repositories](https://github.com/stellar/packages), you can install it by running:
205
+
This is the second stage. A [Full Validator](../README.mdx#full-validator) publishes an archive reaching back to the network's first ledger. The [stellar-archivist](https://github.com/stellar/rs-stellar-archivist) command line tool gives you what you need to close the gap.
206
+
207
+
### Installing stellar-archivist
208
+
209
+
Using the [SDF package repositories](https://github.com/stellar/packages), you can install it by running:
201
210
202
211
```bash
203
212
apt-get install stellar-archivist-rs
@@ -207,12 +216,40 @@ The tool is also published on [crates.io](https://crates.io/crates/stellar-archi
207
216
208
217
For detailed usage, please run `stellar-archivist-rs --help` (or `stellar-archivist --help` if installed from crates.io).
209
218
219
+
### Where to run stellar-archivist
220
+
221
+
stellar-archivist is a standalone tool. It does not link against or communicate with `stellar-core`; all it needs is read and write access to the archive itself. That access can be a local filesystem path (`file://`), an HTTP(S) source, or a cloud object store (`s3://`, `gcs://`, `azblob://`, `b2://`), so you can run it from any machine that can reach your archive.
222
+
223
+
:::caution
224
+
225
+
Run stellar-archivist on a separate host from your validator to avoid resource contention.
226
+
227
+
Do **not** stop your validator to work on its archive. stellar-archivist operates on the archive files directly and does not interfere with live publishing. Your node should stay online and keep publishing checkpoints throughout.
228
+
229
+
:::
230
+
231
+
### Choosing a command
232
+
233
+
| Your goal | Command |
234
+
| --- | --- |
235
+
| Find out whether an archive is complete, and where the gaps are |[`scan`](#scanning-an-archive) — read-only |
236
+
| Fix missing or corrupt files in an archive you already have |[`repair`](#repairing-an-archive)|
237
+
| Build an archive from scratch, or backfill a range of history |[`mirror`](#mirroring-an-archive)|
238
+
239
+
All three accept `--low` and `--high` to restrict the work to a ledger range.
240
+
241
+
### Checking file contents with `--verify`
242
+
243
+
By default, every command checks only that files exist. Adding `--verify` also checks that their contents are sound, which requires significantly more CPU and memory.
244
+
245
+
Leave it off for routine completeness checks and for fixing missing files. Turn it on to find and fix corruption, where a file exists but its contents are wrong.
246
+
210
247
### Scanning an archive
211
248
212
-
`scan` is read-only. It reports files that are missing, and when `--verify` is passed, it also checks file content integrity.
249
+
`scan` is read-only. It walks the archive and reports the files that are missing, which is how you find out whether your archive has a gap and exactly where it is.[^1]
`repair`scans and fixes an archive in place: it identifies missing or broken files at the destination and re-fetches them from a known-good source — such as the SDF public history archive. `--verify` validates file integrity at both the destination and the source.
314
+
Once a scan has told you what is broken, `repair` fixes it in place: it identifies missing or broken files at the destination and re-fetches them from a known-good source — such as the SDF public history archive.
@@ -296,21 +332,21 @@ Repair runs in stages and reports each one separately: a main pass that audits t
296
332
297
333
:::tip
298
334
299
-
A report written by `scan` can be fed straight back in as a repair plan (with the `--plan` flag). A repair with a pre-existing plan skips the main pass, because the plan already says what is broken.
335
+
The JSON report written by `scan` can be handed straight back to `repair`as a plan, using `--plan`. Repair then re-fetches exactly those files.
@@ -328,13 +364,15 @@ Add `--dry-run` to report what would be repaired without writing anything. Combi
328
364
329
365
### Mirroring an archive
330
366
331
-
`mirror` copies a source archive to a destination. Use it to onboard a fresh archive, or to backfill a range of history your validator did not publish itself. For example, run the following command against an empty destination:
367
+
`mirror` copies a source archive to a destination. It is the command to reach for when backfilling: run it once to build a complete archive from scratch, or scope it with `--low` and `--high` to fill in just the range of history your validator did not publish itself.
368
+
369
+
For example, run the following command against an empty destination:
2026-08-20T17:09:23.851784Z INFO Updated destination .well-known to checkpoint 1023 (0x000003ff)
348
386
```
349
387
350
-
:::tip
388
+
Afterwards, the destination contains a new archive covering history up to checkpoint 1023.
351
389
352
-
Mirroring is resumable. Running it again against the same destination with a higher `--high` picks up from the destination's current checkpoint and copies only what is missing, unless you pass `--overwrite`.
390
+
:::tip
353
391
354
-
:::
392
+
Mirroring is resumable. Running it again against the same destination with a higher `--high` picks up from the destination's current checkpoint and copies only what is missing.
355
393
356
-
Once the archive is complete, start your Stellar Core instance again.
394
+
Files already present at the destination are skipped rather than re-fetched. Use `--overwrite` to re-fetch and replace them instead.
357
395
358
-
```bash
359
-
systemctl start stellar-core
360
-
```
396
+
:::
361
397
362
398
You should now have a complete history archive being written by your full validator. Congratulations!
399
+
400
+
[^1]: The sample output on this page was captured with `--verify`, which is why it reports hash mismatches alongside the missing files.
0 commit comments