sync.md

State Sync on Osmosis

State sync can greatly speed up the time it takes to bring a node online, and the smaller state size will make processing the epoch much faster. There is however a bug that affects osmosisd v5.0.0-v6.0.1 which causes the replay to fail after syncing

This walks through the steps to bring up a new node using state sync on osmosis v6.0.1, this will likely be outdated pretty quickly (Jan 2, 2022) once the bug is fixed.

Install Osmosis

sudo addgroup --system osmo && sudo adduser --ingroup osmo --system --home /var/lib/osmosis osmo
sudo chmod 0700 /var/lib/osmosis

sudo apt install build-essential snapd
sudo snap install go --classic

sudo -su osmo
cd ~
git clone https://github.com/osmosis-labs/osmosis.git
cd osmosis
git checkout v6.0.1
make install

cd ~
mkdir -p ~/bin
cp ~/go/bin/osmosisd ~/bin

echo 'export PATH=$PATH:~/bin' >> ~/.bashrc
. ~/.bashrc

osmosisd init osmosis --chain-id=osmosis-1
wget -O .osmosisd/config/genesis.json https://github.com/osmosis-labs/networks/raw/main/osmosis-1/genesis.json

Configure state sync

First get a trust height:

I've brought a public RPC node online specifically for syncing, it does not have much history or a tx index, but should suffice for statesync.

curl -s http://osmo-sync.blockpane.com:26657/block?height=$( \
  curl -s http://osmo-sync.blockpane.com:26657/status | jq -r .result.sync_info.latest_block_height |sed 's/[0-9]\{3\}$/000/') | \
  jq -r '"trust_height = " + .result.block.header.height + "\ntrust_hash = \"" + .result.block_id.hash + "\""'

Edit the config.toml and enable statesync, update the rpc_servers, trust_height, and trust_hash. For example it will look like this (though you'll likely want to use a more recent height.)

Note: I don't know of another RPC node, so the example below needs to be fixed by replacing FIXME with another RPC node

[statesync]
  enable = true
  rpc_servers = "osmo-sync.blockpane.com:26657,FIXME:26657"
  trust_height = 2641273
  trust_hash = "E564B6CBE4C9CD39D5BCBDC431E8F9D362082D7031F27EEFBEA7B8BF378FB2F7"
  trust_period = "168h0m0s"
  discovery_time = "15s"
  temp_dir = ""

Create the systemd unit and start the node:

exit # back to normal user
sudo cat > /etc/systemd/system/osmosisd.service << EOF
[Unit]
Description=Osmosis Daemon
After=network-online.target

[Service]
User=osmo
ExecStart=/var/lib/osmosis/bin/osmosisd start
Restart=always
RestartSec=3
LimitNOFILE=infinity

[Install]
WantedBy=multi-user.target

EOF

sudo systemctl daemon-reload
sudo systemctl enable osmosisd
sudo systemctl start osmosisd
sudo journalctl -fu osmosisd

Once the statesync is done, the node will attempt to restart, and will fail with an error:

wrong Block.Header.Version. Expected {11 0}, got {11 1}

Now implement the workaround provided by Callum Waters in this github issue

sudo -su osmo
cd ~
git clone https://github.com/tendermint/tendermint
cd tendermint
git checkout callum/app-version
make install
~/go/bin/tendermint set-app-version 1 --home ~/.osmosisd
exit
sudo systemctl restart osmosisd && sudo journalctl -fu osmosisd