innodb-tokudb-sysbench-pure-random-writes

===testing model===
a. 1 master and 2 slave
   semi-sync is enabled
   sysbench testing in production not a mock
b. PS5.7: https://github.com/BohuTANG/tokudb
c. 16-CPUS 16GB-Memory 100GB-SSD ECS
d. sysbench: 128threads, random-writes 3600s
   8 tables which prepared by sysbench(oltp.lua)

===semi-sync====
rpl_semi_sync_master_enabled=ON
rpl_semi_sync_master_timeout=1000000000000000000
rpl_semi_sync_master_wait_point=AFTER_SYNC

===tokudb.cnf===
tokudb_directio=ON
tokudb_fsync_log_period=1000
tokudb_cache_size=4G
binlog_sync=0

===innodb.cnf===
innodb_flush_log_at_trx_commit=2
innodb_buffer_pool_size=4G
binlog_sync=0

===writes.sh=
line1=$(head -n 1 ips.txt)
line2=($(tail -n 1 ips.txt))
echo $line1
echo $line2

TEST=./tests/db/insert.lua
HOST=$line1
USER=${line2[0]}
PWD=${line2[1]}
PORT=3306
CNT=8
SIZE=1000000000000
#!/bin/sh
#cat ips.txt
#{ip}
#{user} {pwd}

line1=$(head -n 1 ips.txt)
line2=($(tail -n 1 ips.txt))
echo $line1
echo $line2

FILE="tokudb_write.log"
TEST=./tests/db/insert.lua
HOST=$line1
USER=${line2[0]}
PWD=${line2[1]}
PORT=3306
CNT=8
SIZE=10000000000
TIME=600
THD=128

ENGINE=tokudb
#ENGINE=innodb
./sysbench\
▸   --test=$TEST\
▸   --mysql-host=$HOST \
▸   --mysql-port=$PORT \
▸   --mysql-user=$USER\
▸   --mysql-password=$PWD\
▸   --mysql-db=sbtest\
▸   --mysql-table-engine=$ENGINE\
▸   --oltp-tables-count=$CNT\
▸   --oltp-table-size=$SIZE\
▸   --max-time=$TIME\
▸   --num-threads=$THD\
▸   --report-interval=1\
▸   --oltp-auto-inc=off\
▸   --oltp-read-only=off\
▸   --oltp-test-mode=complex\
▸   --oltp-reconnect=on\
▸   --max-requests=0\
▸   run



====tokudb random writes====
OLTP test statistics:
    queries performed:
        read:                            0
        write:                           37648698
        other:                           0
        total:                           37648698
    transactions:                        0      (0.00 per sec.)
    read/write requests:                 37648698 (10457.96 per sec.)
    other operations:                    0      (0.00 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          3600.0027s
    total number of events:              37648698
    total time taken by event execution: 460738.3588s
    response time:
         min:                                  0.19ms
         avg:                                 12.24ms
         max:                               3254.48ms
         approx.  95 percentile:              22.41ms

Threads fairness:
    events (avg/stddev):           294130.4531/339.55
    execution time (avg/stddev):   3599.5184/0.03


===innodb random writes===
OLTP test statistics:
    queries performed:
        read:                            0
        write:                           23067868
        other:                           0
        total:                           23067868
    transactions:                        0      (0.00 per sec.)
    read/write requests:                 23067868 (6407.22 per sec.)
    other operations:                    0      (0.00 per sec.)
    ignored errors:                      0      (0.00 per sec.)
    reconnects:                          0      (0.00 per sec.)

General statistics:
    total time:                          3600.2917s
    total number of events:              23067868
    total time taken by event execution: 460775.8693s
    response time:
         min:                                  0.41ms
         avg:                                 19.97ms
         max:                               2062.22ms
         approx.  95 percentile:              62.37ms

Threads fairness:
    events (avg/stddev):           180217.7188/517.64
    execution time (avg/stddev):   3599.8115/0.08

By using the above configuration(*.cnf), summary:
1. TokuDB is 1.6X faster than InnoDB in pure-random-writes mode
   I have no idea why InnoDB does not provide better performance(ECS? But TokuDB does...hmm)

2. TokuDB needs to implement the group commit in flush_logs, and here is the patch:
   https://github.com/BohuTANG/tokudb/commit/58e61b8aaf9c5cc4d1c5fceda475afde661280e8
   With this patch, random-writes will be better
   
3. There is a big performance drop caused by MYSQL_BIN_LOG::ordered_commit
   it should not be used without good reason for TokuDB.
   If we set global binlog_order_commits=off, random-writes will be the best.
   
4. Percona throw the 'INSERT IGNORE' optimization away just so some tests would work 'correctly'(DB-937)
   Insert ignore speedup: https://github.com/percona/tokudb-engine/wiki/Insert-ignore-speedup
   DB-937: https://github.com/percona/percona-server/commit/c66efd0e564c0b4c9203e1075576e71908cfa687#diff-743d21825d8dd1f43251f51c8e62f656R3895