Skip to main content

Maintenance

Maintenance here refers to anything involving taking your validator temporarily out of the network (to apply security patches, system upgrade, etc).

As an administrator of a validator, you must ensure that the maintenance you are about to apply to the validator is safe for the overall network and for your validator.

Safe means that the other validators that depend on yours will not be affected too much when you turn off your validator for maintenance and that your validator will continue to operate as part of the network when it comes back up.

If you are changing some settings that may impact network wide settings such as protocol version, review the network upgrades page.

If you're changing your quorum set configuration, also read the section on what to do.

We recommend performing the following steps in order (repeat sequentially as needed if you run multiple nodes).

  1. Advertise your intention to others that may depend on you. Some coordination is required to avoid situations where too many nodes go down at the same time.
  2. Dependencies should assess the health of their quorum
  3. If there is no objection, take your instance down
  4. When done, start your instance that should rejoin the network
  5. The instance will be completely caught up when it's both Synced and there is no backlog in uploading history.

Community Notice: Action Required Before Stellar Core Upgrade​

We have identified a specific scenario on Ubuntu where upgrading the Stellar Core package can inadvertently trigger an automatic, unplanned upgrade of PostgreSQL. This can occur if a newer version of PostgreSQL is available in your repositories. This is because the Stellar Core package lists it as a dependency, "apt" automatically installs the newer PostgreSQL package within the same transaction as the Stellar Core upgrade, which can lead to an unexpected database restart and can cause the Core process to crash.

Action: To prevent service disruption and ensure data integrity, the node operators must manually apply PostgreSQL patch BEFORE executing the Stellar Core package upgrade.

Pre-Upgrade Steps​

Before running an apt command that includes the stellar-core package, follow these steps to manage the PostgreSQL upgrade safely:

  1. Gracefully shut down stellar-core: sudo systemctl stop stellar-core
  2. apt list --upgradable | grep postgresql
  3. sudo apt-get install postgresql-16 postgresql-client-16 # or the specific version package
  4. Restart stellar-core: sudo systemctl start stellar-core
  5. Wait for core to get in sync: stellar-core-cmd info

Then, you may proceed to deploy the newer version of stellar-core package in your customary way (apt, config management tool, etc)

Special Considerations During Quorum Set Updates​

When you join the ranks of node operators, it's also important to join the conversation. The best way to do that: follow the#validators channel on the Stellar Developer Discord. If you can't do that for some reason, sign up for the Stellar Validators Google Group.

Sometimes an organization needs to make changes that will impact the quorum sets of others:

  • taking a validator down for long period of time
  • adding new validators to their pool

In both cases, it's crucial to stage the changes to preserve quorum intersection and general good health of the network:

  • Be careful about removing too many nodes from your quorum set before the nodes are taken down. If different people remove different sets the remaining sets may not overlap between nodes and may cause network splits.
  • Watch out for adding too many nodes in your quorum set at the same time. If not done carefully, this can cause those nodes to overpower your configuration.

Recommended steps are for the entity that adds/removes nodes to do so first between their own nodes, and then have people reflect those changes gradually (over several rounds) in their respective quorum configurations.