fio randread 4k 6disks

gistfile1.txt

# fio --filename=/dev/sdb:/dev/sdc:/dev/sdd:/dev/sde:/dev/sdf:/dev/sdg --direct=1 --rw=randread --bs=4k --numjobs=6 --iodepth=32 --runtime=60 --time_based --group_reporting --name=-randread --ioengine=libaio
-sync-write: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=32
...
fio-3.25
Starting 6 processes
Jobs: 6 (f=36): [r(6)][100.0%][r=1153MiB/s][r=295k IOPS][eta 00m:00s]
-sync-write: (groupid=0, jobs=6): err= 0: pid=10337: Wed Mar  8 13:17:58 2023
  read: IOPS=307k, BW=1198MiB/s (1256MB/s)(70.2GiB/60003msec)
    slat (usec): min=2, max=4415, avg=10.05, stdev= 8.46
    clat (usec): min=86, max=79754, avg=614.91, stdev=430.05
     lat (usec): min=178, max=79775, avg=625.08, stdev=429.91
    clat percentiles (usec):
     |  1.00th=[  285],  5.00th=[  330], 10.00th=[  363], 20.00th=[  404],
     | 30.00th=[  445], 40.00th=[  482], 50.00th=[  523], 60.00th=[  570],
     | 70.00th=[  635], 80.00th=[  725], 90.00th=[  906], 95.00th=[ 1156],
     | 99.00th=[ 2180], 99.50th=[ 2900], 99.90th=[ 5080], 99.95th=[ 5932],
     | 99.99th=[ 7439]
   bw (  MiB/s): min=  884, max= 1361, per=100.00%, avg=1199.01, stdev=11.57, samples=714
   iops        : min=226474, max=348596, avg=306945.87, stdev=2961.89, samples=714
  lat (usec)   : 100=0.01%, 250=0.13%, 500=44.64%, 750=37.29%, 1000=10.47%
  lat (msec)   : 2=6.23%, 4=1.03%, 10=0.21%, 20=0.01%, 50=0.01%
  lat (msec)   : 100=0.01%
  cpu          : usr=8.55%, sys=69.12%, ctx=3007574, majf=0, minf=68109
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued rwts: total=18397937,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=1198MiB/s (1256MB/s), 1198MiB/s-1198MiB/s (1256MB/s-1256MB/s), io=70.2GiB (75.4GB), run=60003-60003msec

Disk stats (read/write):
  sdb: ios=3059614/0, merge=0/0, ticks=1802669/0, in_queue=1802669, util=99.98%
  sdc: ios=3059610/0, merge=0/0, ticks=1714480/0, in_queue=1714480, util=99.99%
  sdd: ios=3059613/0, merge=0/0, ticks=1953145/0, in_queue=1953145, util=99.96%
  sde: ios=3059618/0, merge=0/0, ticks=1742313/0, in_queue=1742313, util=99.91%
  sdf: ios=3059625/0, merge=0/0, ticks=1558406/0, in_queue=1558406, util=99.96%
  sdg: ios=3059626/0, merge=0/0, ticks=1948507/0, in_queue=1948507, util=99.98%