Skip to content

Instantly share code, notes, and snippets.

@fo40225
Created September 13, 2019 05:20
Show Gist options
  • Save fo40225/78025b40e233867cbf3be02fb9d64292 to your computer and use it in GitHub Desktop.
Save fo40225/78025b40e233867cbf3be02fb9d64292 to your computer and use it in GitHub Desktop.
MG07ACA14TE on LSI SAS 3108 JBOD mode fio test
ubuntu@t21p-4u-001:/sda$ ~/fio/fio --loops=5 --size=1g --runtime=10 --stonewall --direct=1 --group_reporting \
> --name=SeqQ32T1read --bs=128k --iodepth=32 --rw=read \
> --name=SeqQ32T1write --bs=128k --iodepth=32 --rw=write \
> --name=4kQ8T8read --bs=4k --iodepth=8 --numjobs=8 --rw=randread \
> --name=4kQ8T8write --bs=4k --iodepth=8 --numjobs=8 --rw=randwrite \
> --name=4kQ32T1read --bs=4k --iodepth=32 --rw=randread \
> --name=4kQ32T1write --bs=4k --iodepth=32 --rw=randwrite \
--name=4kQ1T1read --bs=4k --iodepth=1 --rw=randread \
> --name=4kQ1T1read --bs=4k --iodepth=1 --rw=randread \
> --name=4kQ1T1write --bs=4k --iodepth=1 --rw=randwrite
SeqQ32T1read: (g=0): rw=read, bs=(R) 128KiB-128KiB, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=psync, iodepth=32
SeqQ32T1write: (g=1): rw=write, bs=(R) 128KiB-128KiB, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=psync, iodepth=32
4kQ8T8read: (g=2): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=8
...
4kQ8T8write: (g=3): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=8
...
4kQ32T1read: (g=4): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=32
4kQ32T1write: (g=5): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=32
4kQ1T1read: (g=6): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
4kQ1T1write: (g=7): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
fio-3.15
Starting 22 processes
SeqQ32T1read: Laying out IO file (1 file / 1024MiB)
SeqQ32T1write: Laying out IO file (1 file / 1024MiB)
4kQ8T8read: Laying out IO file (1 file / 1024MiB)
4kQ8T8read: Laying out IO file (1 file / 1024MiB)
4kQ8T8read: Laying out IO file (1 file / 1024MiB)
4kQ8T8read: Laying out IO file (1 file / 1024MiB)
4kQ8T8read: Laying out IO file (1 file / 1024MiB)
4kQ8T8read: Laying out IO file (1 file / 1024MiB)
4kQ8T8read: Laying out IO file (1 file / 1024MiB)
4kQ8T8read: Laying out IO file (1 file / 1024MiB)
4kQ8T8write: Laying out IO file (1 file / 1024MiB)
4kQ8T8write: Laying out IO file (1 file / 1024MiB)
4kQ8T8write: Laying out IO file (1 file / 1024MiB)
4kQ8T8write: Laying out IO file (1 file / 1024MiB)
4kQ8T8write: Laying out IO file (1 file / 1024MiB)
4kQ8T8write: Laying out IO file (1 file / 1024MiB)
4kQ8T8write: Laying out IO file (1 file / 1024MiB)
4kQ8T8write: Laying out IO file (1 file / 1024MiB)
4kQ32T1read: Laying out IO file (1 file / 1024MiB)
4kQ32T1write: Laying out IO file (1 file / 1024MiB)
4kQ1T1read: Laying out IO file (1 file / 1024MiB)
4kQ1T1write: Laying out IO file (1 file / 1024MiB)
Jobs: 1 (f=1): [_(21),w(1)][28.1%][w=1793KiB/s][w=448 IOPS][eta 03m:30s]
SeqQ32T1read: (groupid=0, jobs=1): err= 0: pid=8623: Thu Sep 12 15:38:06 2019
read: IOPS=1974, BW=247MiB/s (259MB/s)(2468MiB/10001msec)
clat (usec): min=374, max=13712, avg=501.56, stdev=255.19
lat (usec): min=375, max=13713, avg=502.08, stdev=255.19
clat percentiles (usec):
| 1.00th=[ 408], 5.00th=[ 416], 10.00th=[ 420], 20.00th=[ 441],
| 30.00th=[ 441], 40.00th=[ 445], 50.00th=[ 465], 60.00th=[ 486],
| 70.00th=[ 519], 80.00th=[ 523], 90.00th=[ 545], 95.00th=[ 848],
| 99.00th=[ 938], 99.50th=[ 955], 99.90th=[ 979], 99.95th=[ 9241],
| 99.99th=[ 9241]
bw ( KiB/s): min=155628, max=258622, per=69.88%, avg=176574.42, stdev=23002.70, samples=19
iops : min= 1215, max= 2020, avg=1378.95, stdev=179.73, samples=19
lat (usec) : 500=64.56%, 750=29.71%, 1000=5.64%
lat (msec) : 4=0.02%, 10=0.07%, 20=0.01%
cpu : usr=2.10%, sys=6.27%, ctx=19747, majf=0, minf=43
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=19743,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
SeqQ32T1write: (groupid=1, jobs=1): err= 0: pid=8684: Thu Sep 12 15:38:06 2019
write: IOPS=1972, BW=247MiB/s (258MB/s)(2465MiB/10001msec); 0 zone resets
clat (usec): min=370, max=53141, avg=498.77, stdev=544.87
lat (usec): min=373, max=53145, avg=502.49, stdev=544.84
clat percentiles (usec):
| 1.00th=[ 404], 5.00th=[ 416], 10.00th=[ 424], 20.00th=[ 445],
| 30.00th=[ 445], 40.00th=[ 453], 50.00th=[ 474], 60.00th=[ 482],
| 70.00th=[ 494], 80.00th=[ 498], 90.00th=[ 523], 95.00th=[ 840],
| 99.00th=[ 914], 99.50th=[ 930], 99.90th=[ 963], 99.95th=[ 5407],
| 99.99th=[43254]
bw ( KiB/s): min=200715, max=266729, per=97.34%, avg=245702.95, stdev=15908.33, samples=19
iops : min= 1568, max= 2083, avg=1919.05, stdev=124.08, samples=19
lat (usec) : 500=80.79%, 750=13.80%, 1000=5.31%
lat (msec) : 2=0.01%, 4=0.03%, 10=0.04%, 20=0.01%, 50=0.01%
lat (msec) : 100=0.01%
cpu : usr=2.19%, sys=8.30%, ctx=19728, majf=0, minf=52
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,19722,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
4kQ8T8read: (groupid=2, jobs=8): err= 0: pid=8745: Thu Sep 12 15:38:06 2019
read: IOPS=421, BW=1685KiB/s (1725kB/s)(16.5MiB/10021msec)
clat (usec): min=187, max=267522, avg=18963.73, stdev=21124.27
lat (usec): min=187, max=267522, avg=18964.26, stdev=21124.30
clat percentiles (usec):
| 1.00th=[ 1811], 5.00th=[ 2999], 10.00th=[ 3818], 20.00th=[ 5342],
| 30.00th=[ 7046], 40.00th=[ 8586], 50.00th=[ 10814], 60.00th=[ 15139],
| 70.00th=[ 20055], 80.00th=[ 28705], 90.00th=[ 43779], 95.00th=[ 59507],
| 99.00th=[ 96994], 99.50th=[116917], 99.90th=[204473], 99.95th=[231736],
| 99.99th=[267387]
bw ( KiB/s): min= 947, max= 2066, per=90.87%, avg=1530.32, stdev=37.11, samples=152
iops : min= 233, max= 514, avg=379.79, stdev= 9.31, samples=152
lat (usec) : 250=0.02%
lat (msec) : 2=1.56%, 4=9.64%, 10=34.68%, 20=24.07%, 50=22.20%
lat (msec) : 100=6.94%, 250=0.85%, 500=0.02%
cpu : usr=0.03%, sys=0.22%, ctx=4252, majf=0, minf=98
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=4221,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
4kQ8T8write: (groupid=3, jobs=8): err= 0: pid=8819: Thu Sep 12 15:38:06 2019
write: IOPS=434, BW=1739KiB/s (1780kB/s)(17.1MiB/10079msec); 0 zone resets
clat (usec): min=152, max=188083, avg=18396.14, stdev=36382.25
lat (usec): min=153, max=188084, avg=18396.81, stdev=36382.23
clat percentiles (usec):
| 1.00th=[ 269], 5.00th=[ 1680], 10.00th=[ 1893], 20.00th=[ 2409],
| 30.00th=[ 2507], 40.00th=[ 2606], 50.00th=[ 2704], 60.00th=[ 2802],
| 70.00th=[ 2900], 80.00th=[ 2999], 90.00th=[ 93848], 95.00th=[100140],
| 99.00th=[109577], 99.50th=[149947], 99.90th=[187696], 99.95th=[187696],
| 99.99th=[187696]
bw ( KiB/s): min= 1030, max= 2123, per=69.85%, avg=1214.05, stdev=37.59, samples=152
iops : min= 253, max= 529, avg=300.11, stdev= 9.49, samples=152
lat (usec) : 250=0.87%, 500=1.26%, 1000=0.14%
lat (msec) : 2=9.97%, 4=71.22%, 10=0.11%, 100=11.82%, 250=4.61%
cpu : usr=0.07%, sys=0.35%, ctx=4443, majf=0, minf=80
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,4381,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8
4kQ32T1read: (groupid=4, jobs=1): err= 0: pid=8887: Thu Sep 12 15:38:06 2019
read: IOPS=170, BW=683KiB/s (700kB/s)(6836KiB/10003msec)
clat (usec): min=802, max=53149, avg=5846.64, stdev=2871.48
lat (usec): min=803, max=53149, avg=5847.18, stdev=2871.49
clat percentiles (usec):
| 1.00th=[ 1500], 5.00th=[ 1827], 10.00th=[ 2245], 20.00th=[ 3130],
| 30.00th=[ 4015], 40.00th=[ 4883], 50.00th=[ 5866], 60.00th=[ 6783],
| 70.00th=[ 7504], 80.00th=[ 8291], 90.00th=[ 9241], 95.00th=[ 9634],
| 99.00th=[12256], 99.50th=[13304], 99.90th=[14091], 99.95th=[53216],
| 99.99th=[53216]
bw ( KiB/s): min= 329, max= 700, per=75.97%, avg=518.89, stdev=74.42, samples=19
iops : min= 82, max= 175, avg=129.26, stdev=18.65, samples=19
lat (usec) : 1000=0.06%
lat (msec) : 2=7.20%, 4=22.59%, 10=66.82%, 20=3.28%, 100=0.06%
cpu : usr=0.19%, sys=0.51%, ctx=1712, majf=0, minf=12
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1709,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
4kQ32T1write: (groupid=5, jobs=1): err= 0: pid=8948: Thu Sep 12 15:38:06 2019
write: IOPS=450, BW=1800KiB/s (1843kB/s)(17.6MiB/10001msec); 0 zone resets
clat (usec): min=162, max=175117, avg=2217.62, stdev=13108.67
lat (usec): min=163, max=175118, avg=2218.13, stdev=13108.67
clat percentiles (usec):
| 1.00th=[ 243], 5.00th=[ 273], 10.00th=[ 281], 20.00th=[ 297],
| 30.00th=[ 322], 40.00th=[ 343], 50.00th=[ 371], 60.00th=[ 396],
| 70.00th=[ 412], 80.00th=[ 433], 90.00th=[ 478], 95.00th=[ 545],
| 99.00th=[ 88605], 99.50th=[ 93848], 99.90th=[112722], 99.95th=[149947],
| 99.99th=[175113]
bw ( KiB/s): min= 1113, max= 1968, per=72.58%, avg=1306.47, stdev=200.43, samples=19
iops : min= 278, max= 492, avg=326.32, stdev=50.21, samples=19
lat (usec) : 250=1.40%, 500=90.85%, 750=5.73%, 1000=0.02%
lat (msec) : 100=1.73%, 250=0.27%
cpu : usr=0.30%, sys=2.12%, ctx=4555, majf=0, minf=10
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,4501,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=32
4kQ1T1read: (groupid=6, jobs=1): err= 0: pid=9009: Thu Sep 12 15:38:06 2019
read: IOPS=175, BW=701KiB/s (718kB/s)(7012KiB/10002msec)
clat (usec): min=187, max=14151, avg=5699.42, stdev=2538.14
lat (usec): min=188, max=14152, avg=5699.95, stdev=2538.14
clat percentiles (usec):
| 1.00th=[ 1450], 5.00th=[ 1745], 10.00th=[ 2212], 20.00th=[ 3064],
| 30.00th=[ 4047], 40.00th=[ 4883], 50.00th=[ 5669], 60.00th=[ 6521],
| 70.00th=[ 7373], 80.00th=[ 8225], 90.00th=[ 8979], 95.00th=[ 9503],
| 99.00th=[11338], 99.50th=[12256], 99.90th=[13829], 99.95th=[14091],
| 99.99th=[14091]
bw ( KiB/s): min= 361, max= 631, per=69.21%, avg=485.16, stdev=55.34, samples=19
iops : min= 90, max= 157, avg=120.89, stdev=13.74, samples=19
lat (usec) : 250=0.06%
lat (msec) : 2=8.04%, 4=21.33%, 10=68.85%, 20=1.71%
cpu : usr=0.12%, sys=0.61%, ctx=1756, majf=0, minf=12
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1753,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
4kQ1T1write: (groupid=7, jobs=1): err= 0: pid=9070: Thu Sep 12 15:38:06 2019
write: IOPS=443, BW=1773KiB/s (1816kB/s)(17.3MiB/10001msec); 0 zone resets
clat (usec): min=142, max=155973, avg=2251.71, stdev=13258.15
lat (usec): min=142, max=155973, avg=2252.23, stdev=13258.16
clat percentiles (usec):
| 1.00th=[ 235], 5.00th=[ 265], 10.00th=[ 273], 20.00th=[ 289],
| 30.00th=[ 306], 40.00th=[ 334], 50.00th=[ 363], 60.00th=[ 396],
| 70.00th=[ 416], 80.00th=[ 433], 90.00th=[ 469], 95.00th=[ 537],
| 99.00th=[ 92799], 99.50th=[ 98042], 99.90th=[109577], 99.95th=[112722],
| 99.99th=[156238]
bw ( KiB/s): min= 1568, max= 2376, per=99.84%, avg=1770.11, stdev=218.01, samples=19
iops : min= 392, max= 594, avg=442.53, stdev=54.50, samples=19
lat (usec) : 250=2.44%, 500=90.62%, 750=4.76%, 1000=0.14%
lat (msec) : 4=0.02%, 50=0.05%, 100=1.65%, 250=0.34%
cpu : usr=0.47%, sys=1.85%, ctx=4484, majf=0, minf=10
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,4433,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=247MiB/s (259MB/s), 247MiB/s-247MiB/s (259MB/s-259MB/s), io=2468MiB (2588MB), run=10001-10001msec
Run status group 1 (all jobs):
WRITE: bw=247MiB/s (258MB/s), 247MiB/s-247MiB/s (258MB/s-258MB/s), io=2465MiB (2585MB), run=10001-10001msec
Run status group 2 (all jobs):
READ: bw=1685KiB/s (1725kB/s), 1685KiB/s-1685KiB/s (1725kB/s-1725kB/s), io=16.5MiB (17.3MB), run=10021-10021msec
Run status group 3 (all jobs):
WRITE: bw=1739KiB/s (1780kB/s), 1739KiB/s-1739KiB/s (1780kB/s-1780kB/s), io=17.1MiB (17.9MB), run=10079-10079msec
Run status group 4 (all jobs):
READ: bw=683KiB/s (700kB/s), 683KiB/s-683KiB/s (700kB/s-700kB/s), io=6836KiB (7000kB), run=10003-10003msec
Run status group 5 (all jobs):
WRITE: bw=1800KiB/s (1843kB/s), 1800KiB/s-1800KiB/s (1843kB/s-1843kB/s), io=17.6MiB (18.4MB), run=10001-10001msec
Run status group 6 (all jobs):
READ: bw=701KiB/s (718kB/s), 701KiB/s-701KiB/s (718kB/s-718kB/s), io=7012KiB (7180kB), run=10002-10002msec
Run status group 7 (all jobs):
WRITE: bw=1773KiB/s (1816kB/s), 1773KiB/s-1773KiB/s (1816kB/s-1816kB/s), io=17.3MiB (18.2MB), run=10001-10001msec
Disk stats (read/write):
sda: ios=27426/32928, merge=0/45, ticks=108412/110092, in_queue=218536, util=93.89%
ubuntu@t21p-4u-001:/sda$
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment