hdd_vs_ssd_vs_cached_hdd

hdd_bench.log

Jobs: 1 (f=1): [_(5),w(1)][61.7%][r=0KiB/s,w=0KiB/s][r=0,w=0 IOPS][eta 01h:04m:36s]
Seqread: (groupid=0, jobs=1): err= 0: pid=3334: Fri Dec 20 20:30:30 2019
   read: IOPS=125, BW=125MiB/s (132MB/s)(5000MiB/39841msec)
    slat (usec): min=35, max=782, avg=154.22, stdev=27.76
    clat (msec): min=2, max=421, avg= 7.81, stdev= 8.56
     lat (msec): min=2, max=421, avg= 7.96, stdev= 8.56
    clat percentiles (msec):
     |  1.00th=[    6],  5.00th=[    6], 10.00th=[    7], 20.00th=[    7],
     | 30.00th=[    7], 40.00th=[    7], 50.00th=[    8], 60.00th=[    8],
     | 70.00th=[    8], 80.00th=[    8], 90.00th=[    8], 95.00th=[    8],
     | 99.00th=[   28], 99.50th=[   43], 99.90th=[   60], 99.95th=[  129],
     | 99.99th=[  422]
   bw (  KiB/s): min=17808, max=136651, per=87.19%, avg=112047.01, stdev=15892.69, samples=79
   iops        : min=   17, max=  133, avg=108.92, stdev=15.51, samples=79
  lat (msec)   : 4=0.02%, 10=96.80%, 20=2.00%, 50=0.80%, 100=0.32%
  lat (msec)   : 250=0.02%, 500=0.04%
  cpu          : usr=0.18%, sys=2.18%, ctx=5012, majf=0, minf=269
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=5000,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1
Seqwrite: (groupid=1, jobs=1): err= 0: pid=3342: Fri Dec 20 20:30:30 2019
  write: IOPS=106, BW=107MiB/s (112MB/s)(5000MiB/46849msec)
    slat (usec): min=60, max=443, avg=218.18, stdev=32.38
    clat (msec): min=2, max=407, avg= 9.14, stdev=17.92
     lat (msec): min=2, max=408, avg= 9.36, stdev=17.92
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    7], 10.00th=[    7], 20.00th=[    7],
     | 30.00th=[    7], 40.00th=[    7], 50.00th=[    8], 60.00th=[    8],
     | 70.00th=[    8], 80.00th=[    8], 90.00th=[    8], 95.00th=[    8],
     | 99.00th=[   99], 99.50th=[  144], 99.90th=[  220], 99.95th=[  359],
     | 99.99th=[  409]
   bw (  KiB/s): min= 1568, max=122384, per=76.25%, avg=83331.54, stdev=26605.98, samples=93
   iops        : min=    1, max=  119, avg=80.84, stdev=25.96, samples=93
  lat (msec)   : 4=3.16%, 10=93.54%, 20=1.10%, 50=0.80%, 100=0.44%
  lat (msec)   : 250=0.86%, 500=0.10%
  cpu          : usr=1.03%, sys=1.66%, ctx=5017, majf=0, minf=12
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,5000,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1
512Kread: (groupid=2, jobs=1): err= 0: pid=3348: Fri Dec 20 20:30:30 2019
   read: IOPS=83, BW=41.5MiB/s (43.5MB/s)(5000MiB/120471msec)
    slat (usec): min=24, max=548, avg=98.19, stdev=14.14
    clat (usec): min=1097, max=192024, avg=11938.84, stdev=8770.81
     lat (usec): min=1201, max=192120, avg=12038.13, stdev=8770.85
    clat percentiles (usec):
     |  1.00th=[  1532],  5.00th=[  1795], 10.00th=[  1909], 20.00th=[  7767],
     | 30.00th=[  9241], 40.00th=[ 10552], 50.00th=[ 11863], 60.00th=[ 13042],
     | 70.00th=[ 14615], 80.00th=[ 15926], 90.00th=[ 17695], 95.00th=[ 18744],
     | 99.00th=[ 33817], 99.50th=[ 65799], 99.90th=[121111], 99.95th=[149947],
     | 99.99th=[189793]
   bw (  KiB/s): min= 9739, max=43905, per=69.07%, avg=29353.75, stdev=4622.13, samples=240
   iops        : min=   19, max=   85, avg=56.81, stdev= 9.03, samples=240
  lat (msec)   : 2=11.48%, 4=2.05%, 10=21.65%, 20=62.50%, 50=1.76%
  lat (msec)   : 100=0.40%, 250=0.16%
  cpu          : usr=0.14%, sys=0.99%, ctx=10018, majf=0, minf=138
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=10000,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1
512Kwrite: (groupid=3, jobs=1): err= 0: pid=3355: Fri Dec 20 20:30:30 2019
  write: IOPS=32, BW=16.5MiB/s (17.3MB/s)(5000MiB/303576msec)
    slat (usec): min=32, max=1101.0k, avg=711.22, stdev=23828.72
    clat (usec): min=1224, max=4507.0k, avg=29636.24, stdev=187912.74
     lat (usec): min=1357, max=4508.1k, avg=30348.53, stdev=192551.30
    clat percentiles (usec):
     |  1.00th=[   1287],  5.00th=[   1369], 10.00th=[   1418],
     | 20.00th=[   1598], 30.00th=[   1696], 40.00th=[   1844],
     | 50.00th=[   1958], 60.00th=[   2073], 70.00th=[   2180],
     | 80.00th=[   2311], 90.00th=[   5211], 95.00th=[   6783],
     | 99.00th=[1098908], 99.50th=[1166017], 99.90th=[2332034],
     | 99.95th=[3338666], 99.99th=[3741320]
   bw (  KiB/s): min=  749, max=153600, per=100.00%, avg=23804.54, stdev=33067.86, samples=312
   iops        : min=    1, max=  300, avg=45.96, stdev=64.60, samples=312
  lat (msec)   : 2=53.40%, 4=32.23%, 10=10.45%, 20=0.18%, 50=0.22%
  lat (msec)   : 100=0.20%, 250=0.92%, 500=0.31%, 750=0.21%, 1000=0.16%
  lat (msec)   : 2000=1.57%, >=2000=0.15%
  cpu          : usr=0.19%, sys=0.42%, ctx=10045, majf=0, minf=11
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,10000,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1
4kQD32read: (groupid=4, jobs=1): err= 0: pid=3392: Fri Dec 20 20:30:30 2019
   read: IOPS=257, BW=1029KiB/s (1054kB/s)(5000MiB/4973676msec)
    slat (usec): min=2, max=267, avg=40.77, stdev= 9.47
    clat (usec): min=92, max=3418.7k, avg=124291.13, stdev=150832.31
     lat (usec): min=158, max=3418.8k, avg=124332.86, stdev=150832.18
    clat percentiles (usec):
     |  1.00th=[    865],  5.00th=[   5604], 10.00th=[  10028],
     | 20.00th=[  19006], 30.00th=[  31589], 40.00th=[  47449],
     | 50.00th=[  68682], 60.00th=[  96994], 70.00th=[ 137364],
     | 80.00th=[ 198181], 90.00th=[ 312476], 95.00th=[ 450888],
     | 99.00th=[ 717226], 99.50th=[ 725615], 99.90th=[ 792724],
     | 99.95th=[ 884999], 99.99th=[1753220]
   bw (  KiB/s): min=    7, max= 3108, per=84.58%, avg=870.28, stdev=209.48, samples=9924
   iops        : min=    1, max=  777, avg=217.20, stdev=52.38, samples=9924
  lat (usec)   : 100=0.01%, 250=0.03%, 500=0.38%, 750=0.49%, 1000=0.42%
  lat (msec)   : 2=1.95%, 4=0.72%, 10=5.96%, 20=10.91%, 50=20.48%
  lat (msec)   : 100=19.47%, 250=24.87%, 500=10.32%, 750=3.89%, 1000=0.09%
  lat (msec)   : 2000=0.03%, >=2000=0.01%
  cpu          : usr=0.58%, sys=1.62%, ctx=1264515, majf=0, minf=42
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued rwt: total=1280000,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32
4kQD32write: (groupid=5, jobs=1): err= 0: pid=3877: Fri Dec 20 20:30:30 2019
  write: IOPS=276, BW=1106KiB/s (1133kB/s)(812MiB/751185msec)
    slat (usec): min=2, max=2658.5k, avg=119.84, stdev=10834.23
    clat (usec): min=348, max=6023.7k, avg=115560.95, stdev=370833.99
     lat (usec): min=457, max=6023.7k, avg=115681.66, stdev=371162.66
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    6], 10.00th=[    9], 20.00th=[   16],
     | 30.00th=[   22], 40.00th=[   27], 50.00th=[   31], 60.00th=[   35],
     | 70.00th=[   42], 80.00th=[   59], 90.00th=[  194], 95.00th=[  334],
     | 99.00th=[ 2232], 99.50th=[ 2500], 99.90th=[ 3943], 99.95th=[ 4212],
     | 99.99th=[ 4463]
   bw (  KiB/s): min=    8, max= 5568, per=100.00%, avg=1812.56, stdev=1606.27, samples=917
   iops        : min=    2, max= 1392, avg=453.12, stdev=401.58, samples=917
  lat (usec)   : 500=0.01%, 750=0.02%, 1000=0.05%
  lat (msec)   : 2=0.63%, 4=2.25%, 10=8.69%, 20=15.56%, 50=49.61%
  lat (msec)   : 100=8.42%, 250=5.69%, 500=5.97%, 750=0.19%, 1000=0.07%
  lat (msec)   : 2000=1.43%, >=2000=1.41%
  cpu          : usr=0.75%, sys=1.89%, ctx=203051, majf=0, minf=10
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,207778,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=125MiB/s (132MB/s), 125MiB/s-125MiB/s (132MB/s-132MB/s), io=5000MiB (5243MB), run=39841-39841msec

Run status group 1 (all jobs):
  WRITE: bw=107MiB/s (112MB/s), 107MiB/s-107MiB/s (112MB/s-112MB/s), io=5000MiB (5243MB), run=46849-46849msec

Run status group 2 (all jobs):
   READ: bw=41.5MiB/s (43.5MB/s), 41.5MiB/s-41.5MiB/s (43.5MB/s-43.5MB/s), io=5000MiB (5243MB), run=120471-120471msec

Run status group 3 (all jobs):
  WRITE: bw=16.5MiB/s (17.3MB/s), 16.5MiB/s-16.5MiB/s (17.3MB/s-17.3MB/s), io=5000MiB (5243MB), run=303576-303576msec

Run status group 4 (all jobs):
   READ: bw=1029KiB/s (1054kB/s), 1029KiB/s-1029KiB/s (1054kB/s-1054kB/s), io=5000MiB (5243MB), run=4973676-4973676msec

Run status group 5 (all jobs):
  WRITE: bw=1106KiB/s (1133kB/s), 1106KiB/s-1106KiB/s (1133kB/s-1133kB/s), io=812MiB (851MB), run=751185-751185msec

Disk stats (read/write):
  sda: ios=1302802/230618, merge=892/2613, ticks=160700876/26571550, in_queue=184225816, util=70.27%