The validator nodes are in direct contact with the sentry nodes. The IP of validator nodes is private, they have a carefully crafted firewall and whitelist IP connection range of the sentry nodes. Also the tendermint instance is only aware of the sentry nodes IP addresses. The validator node holds a private key generated by a KMS of our choice, where the passphrase is held in a hardware wallet of our choice in a location that is known only to a few people. A minimum of 1 validator node should always be online.

Relayer nodes are still experimental in design and also for us. It should be carefully examined if they actually make the system more resilient and provide a reliable anti-DDOS protection or rather can be an additional fail-system where we have to be careful.

Backup nodes, for us usually limited to 1-2 per network, have a higher disk capacity than the other servers and act as creating backups. They shut down the gaiad for the time of taking the backup in an interval between 48 and 72 hours. So we always have a good state to use in case we need to deploy new servers and are quickly up-to-date with the blockchain.

With building this infrastructure, we keep in mind to distribute our servers splitted with own managed servers and servers hosted by different cloud providers. This enables us to scale geographically as well as in storage, computing and bandwith. Cost of a provider is not always the first indicator for us what to choose. But security, control, location and tools will always play a role in deciding which provider to choose.