eosio benchmarking notes with txn_test_gen_plugin

eosiobench.md

This general procedure was used when doing pre-dawn 3.0 performance testing as mentioned in EOSIO/eos#2078.

Network layout: 2 nodes connected to each other on same box, 1 producer, 1 generator
Hardware: AMD Ryzen at 3.9Ghz
Software: Linux 4.15; eosio compiled with Clang6 -O3
Tested on Apr25 2018 on the DAWN-2018-04-23-ALPHA branch

Make an empty directory for our configs & data, mkdir ~/eos.data, and define a logging.json that doesn't print debug information (which occurs for each txn) to the console

cat << EOF > ~/eos.data/logging.json
{
  "includes": [],
  "appenders": [{
      "name": "consoleout",
      "type": "console",
      "args": {
        "stream": "std_out",
        "level_colors": [{
            "level": "debug",
            "color": "green"
          },{
            "level": "warn",
            "color": "brown"
          },{
            "level": "error",
            "color": "red"
          }
        ]
      },
      "enabled": true
    }
  ],

  "loggers": [{
      "name": "default",
      "level": "info",
      "enabled": true,
      "additivity": false,
      "appenders": [
        "consoleout"
      ]
    }
  ]
}
EOF

Launch producer

$ ./nodeos -d ~/eos.data/producer_node --config-dir ~/eos.data/producer_node -l ~/eos.data/logging.json --http-server-address "" -p eosio -e

Launch non-producer that will generate transactions

$ ./nodeos -d ~/eos.data/generator_node --config-dir ~/eos.data/generator_node -l ~/eos.data/logging.json --plugin eosio::txn_test_gen_plugin --plugin eosio::wallet_api_plugin --plugin eosio::chain_api_plugin --p2p-peer-address localhost:9876 --p2p-listen-endpoint localhost:5555

Create a wallet on the non-producer and set bios contract

$ ./cleos wallet create
$ ./cleos set contract eosio ~/eos/build.release/contracts/eosio.bios/ 

Initialize the accounts txn_test_gen_plugin uses

$ curl --data-binary '["eosio", "5KQwrPbwdL6PhXujxW37FSSQZ1JiwsST4cqQzDeyXtP79zkvFD3"]' http://localhost:8888/v1/txn_test_gen/create_test_accounts

Start transaction generation, this will submit 20 transactions evey 20ms (total of 1000TPS)

$ curl --data-binary '["", 20, 20]' http://localhost:8888/v1/txn_test_gen/start_generation

Note the producer console prints

eosio generated block 9b8b851d... #3219 @ 2018-04-25T16:07:47.000 with 500 trxs, lib: 3218
eosio generated block e5b3cd5d... #3220 @ 2018-04-25T16:07:47.500 with 500 trxs, lib: 3219
eosio generated block b243aeaa... #3221 @ 2018-04-25T16:07:48.000 with 500 trxs, lib: 3220

showing 1000TPS. Producer CPU usage is only ~48%

Hi @spoonincode , I tried this on lastest eos (commit id: 503e0c8e97e02d586f30fb36467e32673ce9ad9b). It shows 800tps(400trxs / 500ms).
See screenshot for detail: Top one is producer, bottom one is generator.

What makes me confused is that i tested dawn-v3.0.0 on this machine, it easily reach 1000tps, and not only that, when i used EOSBenchTool it can even reach 1900tps.

any ideas about this?

@spartucus can you share the config.ini and your cluster configuration. I was trying 3 node setup (1 local + 2 Azure) but was able to achieve max 300 TPS by sending txns using cleos while eosjs was giving 10 TPS, can you share your thoughts on how can I achieve the 1000 TPS you were able to achieve ?

@spartucus - How can I achieve the same throughput without the plugin if I send the transactions using the cleos ?