Publishing History Archives

If you want to run a Full Validator, you need to set up your node to publish a history archive. You can host an archive using a blob store such as Amazon's S3 or Digital Ocean's spaces, or you can simply serve a local archive directly via an HTTP server such as Nginx or Apache. If you're setting up a Basic Validator, you can skip this section. No matter what kind of node you're planning to run, make sure to set it up to get history, which is covered in Environment Preparation.

One archive per node

Each node must publish to its own dedicated archive. Writing to the same archive from multiple nodes is not supported and will result in undefined behavior, potentially including data loss. If you're running multiple Full Validators (as Tier 1 organizations do), configure a separate archive for each — for example, history-a.example.com, history-b.example.com, and history-c.example.com.

Caching and History Archives

The primary cost of running a validator will very likely be egress bandwidth. A crucial part of your strategy to manage those costs should be caching. You can significantly reduce these data transfer costs by using common caching techniques or a CDN. Three simple rules apply to caching the History archives:

Do not cache the archive state file .well-known/stellar-history.json (Cache-Control: no-cache)
Do not cache HTTP 4xx responses (Cache-Control: no-cache)
Cache everything else for as long as possible (> 1 day)

Local History Archive Using nginx

Here, we'll demonstrate how you can configure your node to store its history files in the local filesystem, and then publish that history using nginx for our webserver software.

First, you must add a history configuration stanza to your /etc/stellar/stellar-core.cfg configuration file.

[HISTORY.local]
get="cp /mnt/xvdf/stellar-core-archive/node_001/{0} {1}"
put="cp {0} /mnt/xvdf/stellar-core-archive/node_001/{1}"
mkdir="mkdir -p /mnt/xvdf/stellar-core-archive/node_001/{0}"

Then, you must run Stellar Core's new-hist command to create the local history archive.

sudo -u stellar stellar-core --conf /etc/stellar/stellar-core.cfg new-hist local

This command creates the history archive structure:

$ tree -a /mnt/xvdf/stellar-core-archive/
/mnt/xvdf/stellar-core-archive
└── node_001
    ├── history
    │   └── 00
    │       └── 00
    │           └── 00
    │               └── history-00000000.json
    └── .well-known
        └── stellar-history.json

6 directories, 2 files

Now that the history archive's file structure is ready, you can configure a virtual host in nginx to serve the local archive.

server {
  listen 80;
  root /mnt/xvdf/stellar-core-archive/node_001/;

  server_name history.example.com;

  # do not cache 404 errors
  error_page 404 /404.html;
  location = /404.html {
    add_header Cache-Control "no-cache" always;
  }

  # do not cache history state file
  location ~ ^/.well-known/stellar-history.json$ {
    add_header Cache-Control "no-cache" always;
    try_files $uri =404;
  }

  # cache entire history archive for 1 day
  location / {
    add_header Cache-Control "max-age=86400";
    try_files $uri =404;
  }
}

Amazon S3 History Archive

Now, let's demonstrate a configuration where your node stores its history files using Amazon's S3 service. You can then publish that history using an Amazon S3 static site, or again use nginx for your webserver software. This time, using nginx, we'll include some proxy and CDN configuration, as well.

Start by adding a history configuration stanza to your /etc/stellar/stellar-core.cfg configuration file.

[HISTORY.s3]
get='curl -sf http://history.example.com/{0} -o {1}' # Cached HTTP endpoint
put='aws s3 cp --region us-east-1 {0} s3://bucket.name/{1}' # Direct S3 access

Then, you must run Stellar Core's new-hist command to create and initialize the S3 archive.

sudo -u stellar stellar-core --conf /etc/stellar/stellar-core.cfg new-hist s3

These S3 history files can be served with something as simple as an Amazon S3 static site.

Optionally, you may want to place a reverse proxy and CDN in front of the S3 static site (we'll use nginx for this example).

server {
  listen 80;
  root /srv/nginx/history.example.com;
  index index.html index.htm;

  server_name history.example.com;

  # use google nameservers for lookups
  resolver 8.8.8.8 8.8.4.4;

  # bucket.name s3 static site endpoint
  set $s3_bucket "bucket.name.s3-website-us-east-1.amazonaws.com";

  # do not cache 404 errors
  error_page 404 /404.html;
  location = /404.html {
    add_header Cache-Control "no-cache" always;
  }

  # do not cache history state file
  location ~ ^/.well-known/stellar-history.json$ {
    add_header Cache-Control "no-cache" always;
    proxy_intercept_errors on;
    proxy_pass  http://$s3_bucket;
    proxy_read_timeout 120s;
    proxy_redirect off;
    proxy_buffering off;
    proxy_set_header        Host            $s3_bucket;
    proxy_set_header        X-Real-IP       $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
  }

  # cache history archive for 1 day
  location / {
    add_header Cache-Control "max-age=86400";
    proxy_intercept_errors on;
    proxy_pass  http://$s3_bucket;
    proxy_read_timeout 120s;
    proxy_redirect off;
    proxy_buffering off;
    proxy_set_header        Host            $s3_bucket;
    proxy_set_header        X-Real-IP       $remote_addr;
    proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
  }
}

Backfilling a History Archive

Given the choice, it's best to configure your history archive prior to your node's initial sync with an existing network. That way your validator's history publishes as you join, and subsequently sync with, the network.

However, if you have not published an archive during the node's initial sync, the steps required to create a history archive for an existing validator — in other words, to upgrade a Basic Validator to a Full Validator — are quite straightforward. First, you'll need to stop your stellar-core instance:

systemctl stop stellar-core # modify this if not using systemctl

Then, add the history archive configuration to your node's /etc/stellar/stellar-core.cfg configuration file.

[HISTORY.local]
get="cp /mnt/xvdf/stellar-core-archive/node_001/{0} {1}"
put="cp {0} /mnt/xvdf/stellar-core-archive/node_001/{1}"
mkdir="mkdir -p /mnt/xvdf/stellar-core-archive/node_001/{0}"

Next, you must run Stellar Core's new-hist command to create and initialize the local archive. (This is done as the stellar user.)

sudo -u stellar stellar-core --conf /etc/stellar/stellar-core.cfg new-hist local

Now you can start your Stellar Core instance again:

systemctl start stellar-core # modify this if not using systemctl

As you allow your node to join the network again, you can watch it start publishing a few checkpoints to the newly created archive.

2019-04-25T12:30:43.275 GDUQJ [History INFO] Publishing 1 queued checkpoints [16895-16895]: Awaiting 0/0 prerequisites of: publish-000041ff

At this stage your validator is successfully publishing its history, which enables other users to join the network using your archive.

Complete History Archive

The stellar-archivist command line tool scans, repairs, and mirrors history archives. Using the SDF package repositories, you can install it by running:

apt-get install stellar-archivist-rs

The tool is also published on crates.io.

For detailed usage, please run stellar-archivist-rs --help (or stellar-archivist --help if installed from crates.io).

Scanning an archive

scan is read-only. It reports files that are missing, and when --verify is passed, it also checks file content integrity.

stellar-archivist-rs scan file:///mnt/xvdf/stellar-core-archive/node_001 --verify --report report.json

2026-08-21T22:05:55.175197Z  INFO Starting scan of file:///mnt/xvdf/stellar-core-archive/node_001
2026-08-21T22:05:55.177412Z  INFO Processing 32 checkpoints from 63 (0x0000003f) to 2047 (0x000007ff)
2026-08-21T22:05:55.179433Z ERROR Missing results/00/00/00/results-000000bf.xdr.gz
2026-08-21T22:05:55.190684Z ERROR Failed to process bucket/8c/e5/7e/bucket-8ce57e5f8c2ee574af879dde28cf87a0d936a5a1ea905b377409f275a4e73a1c.xdr.gz: Hash mismatch: got 49ec4c32c208f6eb53d0e847bc8a4819544a71ca04c20f47737ba0842f133370
2026-08-21T22:05:55.191706Z ERROR Missing transactions/00/00/05/transactions-000005bf.xdr.gz
2026-08-21T22:05:55.195509Z ERROR Missing scp/00/00/03/scp-0000037f.xdr.gz
2026-08-21T22:05:55.206930Z ERROR Missing scp/00/00/06/scp-0000067f.xdr.gz
2026-08-21T22:05:55.209163Z ERROR Missing history/00/00/06/history-000006ff.json
2026-08-21T22:05:55.221592Z ERROR Failed to process bucket/d6/2e/37/bucket-d62e37771e7ae305d981317d710cd9f6bdf254d16caa315955217deddaaaab05.xdr.gz: Hash mismatch: got 49ec4c32c208f6eb53d0e847bc8a4819544a71ca04c20f47737ba0842f133370
...
2026-08-21T22:05:55.237827Z  INFO Scan complete: 378 files found, 11 missing or corrupt
2026-08-21T22:05:55.237835Z ERROR 1 history file(s) missing or corrupt
2026-08-21T22:05:55.237840Z ERROR 1 transactions file(s) missing or corrupt
2026-08-21T22:05:55.237845Z ERROR 1 results file(s) missing or corrupt
2026-08-21T22:05:55.237849Z ERROR 6 bucket file(s) missing or corrupt
2026-08-21T22:05:55.237854Z  WARN 2 optional scp file(s) missing or corrupt
2026-08-21T22:05:55.238022Z  WARN Archive is incomplete: 11 issue(s) found
error: Archive issues found

--report writes the same findings as JSON:

{
  "version": 1,
  "well_known": null,
  "files": {
    "1471": ["transactions"],
    "1663": ["scp"],
    "1791": ["history"],
    "191": ["results"],
    "895": ["scp"]
  },
  "buckets": [
    "0e3f43118dc8761a94102e2f043f7994a0a12078220b2873dce162fd1d195a50",
    "344dd4abedf73ee3d65d07702e0a1a065b710fcfabc9202821ac50c16e5db898",
    "8ce57e5f8c2ee574af879dde28cf87a0d936a5a1ea905b377409f275a4e73a1c",
    "8e0eb1c3e797336973e5e7d3591b5bf6385de2d6d990d81a9732d76d61d62cf9",
    "d62e37771e7ae305d981317d710cd9f6bdf254d16caa315955217deddaaaab05",
    "dc1031942e58d472b8f7f9630250a296426c767afdca92fb687450adb87e1700"
  ],
  "checkpoints": [],
  "summary": {
    "succeeded": 378,
    "skipped": 0,
    "failed": 11,
    "retries": 0
  }
}

files maps a checkpoint ledger to the file types broken at that checkpoint.
buckets lists bucket hashes that are missing or failed their SHA-256 check.
checkpoints lists cross-file and hash chain failures, which are re-fetched whole.
well_known is the checkpoint to restore .well-known/stellar-history.json from, or null when it is healthy.

Repairing an archive

repair scans and fixes an archive in place: it identifies missing or broken files at the destination and re-fetches them from a known-good source — such as the SDF public history archive. --verify validates file integrity at both the destination and the source.

stellar-archivist-rs repair \
  https://history.stellar.org/prd/core-testnet/core_testnet_001 \
  file:///mnt/xvdf/stellar-core-archive/node_001 \
  --verify

2026-08-21T22:05:55.831816Z  INFO Starting repair from https://history.stellar.org/prd/core-testnet/core_testnet_001 to file:///mnt/xvdf/stellar-core-archive/node_001 with 32 workers
2026-08-21T22:05:56.036395Z  INFO Processing 32 checkpoints from 63 (0x0000003f) to 2047 (0x000007ff)
2026-08-21T22:05:56.103469Z  INFO Progress: 32/32 checkpoints processed
2026-08-21T22:05:56.103510Z  INFO Repair main pass completed: 395 files processed, 0 failed
2026-08-21T22:05:56.103520Z  INFO Repair file retry completed: 0 files processed, 0 failed
2026-08-21T22:05:56.103526Z  INFO Repair checkpoint retry completed: 0 files processed, 0 failed

Repair runs in stages and reports each one separately: a main pass that audits the destination, a file retry that re-fetches everything recorded as broken, and a checkpoint retry for cross-file and chain failures.

tip

A report written by scan can be fed straight back in as a repair plan (with the --plan flag). A repair with a pre-existing plan skips the main pass, because the plan already says what is broken.

stellar-archivist-rs repair \
  https://history.stellar.org/prd/core-testnet/core_testnet_001 \
  file:///mnt/xvdf/stellar-core-archive/node_001 \
  --verify --plan report.json

A final scan confirms the archive has been repaired:

stellar-archivist-rs scan file:///mnt/xvdf/stellar-core-archive/node_001 --verify

2026-08-21T22:05:57.627359Z  INFO Starting scan of file:///mnt/xvdf/stellar-core-archive/node_001
2026-08-21T22:05:57.629405Z  INFO Processing 32 checkpoints from 63 (0x0000003f) to 2047 (0x000007ff)
2026-08-21T22:05:57.686713Z  INFO Progress: 32/32 checkpoints processed
2026-08-21T22:05:57.686747Z  INFO Scan complete: 395 files found, 0 missing or corrupt

tip

Add --dry-run to report what would be repaired without writing anything. Combined with --report, this writes a plan you can review and apply later.

Mirroring an archive

mirror copies a source archive to a destination. Use it to onboard a fresh archive, or to backfill a range of history your validator did not publish itself. For example, run the following command against an empty destination:

stellar-archivist-rs mirror \
  https://history.stellar.org/prd/core-testnet/core_testnet_001 \
  file:///mnt/xvdf/stellar-core-archive/node_001 \
  --high 1023 --verify

2026-08-20T17:09:23.266746Z  INFO Starting mirror from https://history.stellar.org/prd/core-testnet/core_testnet_001 to file:///mnt/xvdf/stellar-core-archive/node_001 with 32 workers
2026-08-20T17:09:23.664272Z  INFO No existing archive at destination; starting a fresh mirror
2026-08-20T17:09:23.664300Z  INFO Processing 16 checkpoints from 63 (0x0000003f) to 1023 (0x000003ff)
2026-08-20T17:09:23.850821Z  INFO Progress: 16/16 checkpoints processed
2026-08-20T17:09:23.850848Z  INFO Mirror completed: 196 files copied, 0 failed, 0 skipped
2026-08-20T17:09:23.850855Z  INFO No existing .well-known, creating new one at checkpoint 1023 (0x000003ff)
2026-08-20T17:09:23.851784Z  INFO Updated destination .well-known to checkpoint 1023 (0x000003ff)

tip

Mirroring is resumable. Running it again against the same destination with a higher --high picks up from the destination's current checkpoint and copies only what is missing, unless you pass --overwrite.

Once the archive is complete, start your Stellar Core instance again.

systemctl start stellar-core

You should now have a complete history archive being written by your full validator. Congratulations!

Publishing History Archives

Caching and History Archives

Local History Archive Using nginx

Amazon S3 History Archive

Backfilling a History Archive

Complete History Archive

Scanning an archive

Repairing an archive

Mirroring an archive

Raven MCP Server

Stellar Skills

Caching and History Archives​

Local History Archive Using nginx​

Amazon S3 History Archive​

Backfilling a History Archive​

Complete History Archive​

Scanning an archive​

Repairing an archive​

Mirroring an archive​

Caching and History Archives

Local History Archive Using nginx

Amazon S3 History Archive

Backfilling a History Archive

Complete History Archive

Scanning an archive

Repairing an archive

Mirroring an archive