Notes on multiple running nodes (testnet + mainnet)

[sbellem]

Creating groups for the testnet and mainnet

We'll use secrettest for the testnet and secretmain for the mainnet.

Testnet

groupadd secrettest

mkdir /opt/secrettest
chown -R root:secrettest /opt/secrettest
chmod -R g+rw /opt/secrettest

Using mia as a username example, and secrettest as a group:

adduser mia
usermod mia -aG sudo
usermod -g secrettest mia

[Unit]
Description=Secret node service
After=network.target

[Service]
Type=simple
Environment=SCRT_ENCLAVE_DIR=/home/mia/lib
Environment=SCRT_SGX_STORAGE=/opt/secrettest/.sgx_secrets
WorkingDirectory=/home/mia
Environment=LD_LIBRARY_PATH=/home/mia/lib:$LD_LIBRARY_PATH
ExecStart=/usr/local/bin/secretz start
User=mia
Restart=on-failure
StartLimitInterval=0
RestartSec=3
LimitNOFILE=65535
LimitMEMLOCK=209715200

[Install]
WantedBy=multi-user.target

State-sync of a (mainnet) node

May apply to a testnet node as well.

Configuration

There are 2 files controlling containing configuration parameters for the state sync, both located under the .secretd/config by default:

• app.toml
• config.toml

app.toml

Make sure that iavl-disable-fastnode = true. Even though the comments in the file mentions that the default is true the field is actually set to false.

# IAVLDisableFastNode enables or disables the fast node feature of IAVL. 
# Default is true.
iavl-disable-fastnode = true

config.toml

Clean-up data of previous state-syncs

Clear tendermint state sync files from /tmp:

note: marsellus is the username

cd /tmp && ls -l /tmp | awk '$3=="marsellus" { print $9 }' | xargs rm -r

[sbellem]

Swap file

A minimum of 8G of swap space is recommended per node.

To add a swap file of 24 G:

sudo dd if=/dev/zero of=/swapfilename count=24 bs=1G
mkswap /swapfilename
sudo chown root:root /swapfilename
sudo chmod 600 /swapfilename
sudo swapon /swapfilename

make changes permanent

echo '/swapfilename none swap sw 0 0' | sudo tee -a /etc/fstab

references

https://askubuntu.com/a/927909/1088806
https://askubuntu.com/a/1177620/1088806

[sbellem]

to reset everything and get statesync started in a fresh state

sudo systemctl stop secret-node 

sudo umount -l /tmp && sudo mount -t tmpfs -o size=8G,mode=1777 overflow /tmp

sudo rm -rf ~/.secretd/.compute \
&& sudo rm -rf ~/.secretd/data \
&& mkdir ~/.secretd/data \
&& secretd7 tendermint unsafe-reset-all

mkdir ~/.secretd/data/snapshots/

sudo systemctl restart secret-node
journalctl -f -u secret-node

[sbellem]

https://askubuntu.com/questions/199565/not-enough-space-on-tmp