Fusion-io ioDrive2 1205GB MLC, Product Number:DG1MH

gistfile1.txt

# fio-status -a

Found 1 ioMemory device in this system
Driver version: 3.2.15 build 1699

Adapter: Single Controller Adapter
        Fusion-io ioDrive2 1205GB MLC, Product Number:DG1MH, SN:US0DG1MH760513530009
        ioDrive2 Adapter Controller, PN:1XF66
        External Power: NOT connected
        PCIe Bus voltage: avg 11.82V
        PCIe Bus current: avg 0.95A
        PCIe Bus power: avg 13.53W
        PCIe Power limit threshold: 24.75W
        PCIe slot available power: unavailable
        Connected ioMemory modules:
          fct0: Product Number:DG1MH, SN:1308D026B

fct0    Attached
        ioDrive2 Adapter Controller, Product Number:DG1MH, SN:1308D026B
!! ---> There are active errors or warnings on this device!  Read below for details.
        ioDrive2 Adapter Controller, PN:1XF66
        SMP(AVR) Versions: App Version: 1.0.25.0, Boot Version: 0.0.8.1
        Located in slot 0 Center of ioDrive2 Adapter Controller SN:1308D026B
        Powerloss protection: protected
        PCI:01:00.0, Slot Number:2
        Vendor:1aed, Device:2001, Sub vendor:1028, Sub device:1f70
        Firmware v7.1.15, rev 110356 Public
        1205.00 GBytes device size
        Format: v500, 2353515625 sectors of 512 bytes
        PCIe slot available power: 25.00W
        PCIe negotiated link: 4 lanes at 2.5 Gt/sec each, 1000.00 MBytes/sec total
        Internal temperature: 49.71 degC, max 50.20 degC
        Internal voltage: avg 1.01V, max 1.02V
        Aux voltage: avg 2.48V, max 2.48V
        Reserve space status: Healthy; Reserves: 100.00%, warn at 10.00%
        Active media: 100.00%
        Rated PBW: 17.00 PB, 99.34% remaining
        Lifetime data volumes:
           Physical bytes written: 112,131,554,756,312
           Physical bytes read   : 97,996,578,177,888
        RAM usage:
           Current: 855,402,880 bytes
           Peak   : 855,402,880 bytes
        Contained VSUs:
          fct0: ID:0, UUID:b6e672be-c8f7-4253-80b7-1e435b57e27c

        ACTIVE WARNINGS:
            The bandwidth of the PCI slot is not optimal for the ioMemory.

fct0    State: Online, Type: block device
        ID:0, UUID:b6e672be-c8f7-4253-80b7-1e435b57e27c
        1205.00 GBytes device size
        Format: 2353515625 sectors of 512 bytes

root@pxe-client:/tmp/1# fio --rw=write --ioengine=sync --fdatasync=1 --directory=test-data --size=22m --bs=2300 --name=mytest
mytest: (g=0): rw=write, bs=(R) 2300B-2300B, (W) 2300B-2300B, (T) 2300B-2300B, ioengine=sync, iodepth=1
fio-3.39
Starting 1 process
mytest: Laying out IO file (1 file / 22MiB)
Jobs: 1 (f=1)
mytest: (groupid=0, jobs=1): err= 0: pid=11864: Fri Oct 10 14:55:43 2025
  write: IOPS=7072, BW=15.5MiB/s (16.3MB/s)(22.0MiB/1418msec); 0 zone resets
    clat (usec): min=3, max=271, avg= 6.40, stdev= 4.33
     lat (usec): min=3, max=271, avg= 6.57, stdev= 4.34
    clat percentiles (usec):
     |  1.00th=[    4],  5.00th=[    4], 10.00th=[    5], 20.00th=[    5],
     | 30.00th=[    5], 40.00th=[    5], 50.00th=[    7], 60.00th=[    8],
     | 70.00th=[    8], 80.00th=[    8], 90.00th=[    9], 95.00th=[    9],
     | 99.00th=[   13], 99.50th=[   17], 99.90th=[   35], 99.95th=[   50],
     | 99.99th=[  215]
   bw (  KiB/s): min=15349, max=17002, per=100.00%, avg=16175.50, stdev=1168.85, samples=2
   iops        : min= 6834, max= 7570, avg=7202.00, stdev=520.43, samples=2
  lat (usec)   : 4=5.94%, 10=92.06%, 20=1.67%, 50=0.27%, 100=0.03%
  lat (usec)   : 250=0.02%, 500=0.01%
  fsync/fdatasync/sync_file_range:
    sync (usec): min=37, max=671, avg=133.32, stdev=71.66
    sync percentiles (usec):
     |  1.00th=[   39],  5.00th=[   42], 10.00th=[   55], 20.00th=[   58],
     | 30.00th=[   58], 40.00th=[   59], 50.00th=[  147], 60.00th=[  198],
     | 70.00th=[  200], 80.00th=[  202], 90.00th=[  204], 95.00th=[  210],
     | 99.00th=[  235], 99.50th=[  241], 99.90th=[  265], 99.95th=[  273],
     | 99.99th=[  359]
  cpu          : usr=2.61%, sys=31.05%, ctx=26914, majf=0, minf=9
  IO depths    : 1=200.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwts: total=0,10029,0,0 short=10029,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=15.5MiB/s (16.3MB/s), 15.5MiB/s-15.5MiB/s (16.3MB/s-16.3MB/s), io=22.0MiB (23.1MB), run=1418-1418msec

Disk stats (read/write):
  fioa: ios=0/20960, sectors=0/139954, merge=0/0, ticks=0/888, in_queue=888, util=0.00%


# https://gist.github.com/superboum/aaa45d305700a7873a8ebbab1abddf2b
root@pxe-client:/tmp/1# ./benchmark-custom.sh test
/bin/bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8): No such file or directory
write_throughput: (g=0): rw=write, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.39
Starting 4 processes
write_throughput: Laying out IO file (1 file / 100MiB)
write_throughput: Laying out IO file (1 file / 100MiB)
write_throughput: Laying out IO file (1 file / 100MiB)
write_throughput: Laying out IO file (1 file / 100MiB)
Jobs: 4 (f=4): [W(4)][100.0%][w=1061MiB/s][w=1061 IOPS][eta 00m:00s]          
write_throughput: (groupid=0, jobs=4): err= 0: pid=11942: Fri Oct 10 14:57:33 2025
  write: IOPS=1074, BW=1079MiB/s (1131MB/s)(63.5GiB/60241msec); 0 zone resets
    slat (usec): min=102, max=493, avg=174.04, stdev=27.81
    clat (msec): min=234, max=477, avg=237.35, stdev= 6.57
     lat (msec): min=234, max=477, avg=237.53, stdev= 6.57
    clat percentiles (msec):
     |  1.00th=[  236],  5.00th=[  236], 10.00th=[  236], 20.00th=[  236],
     | 30.00th=[  236], 40.00th=[  236], 50.00th=[  239], 60.00th=[  239],
     | 70.00th=[  239], 80.00th=[  239], 90.00th=[  239], 95.00th=[  239],
     | 99.00th=[  239], 99.50th=[  239], 99.90th=[  338], 99.95th=[  439],
     | 99.99th=[  472]
   bw (  MiB/s): min=  921, max= 1189, per=100.00%, avg=1080.04, stdev=14.82, samples=480
   iops        : min=  921, max= 1188, avg=1079.16, stdev=14.81, samples=480
  lat (msec)   : 250=100.03%, 500=0.36%
  cpu          : usr=3.17%, sys=2.59%, ctx=64770, majf=0, minf=149
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=0,64752,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=1079MiB/s (1131MB/s), 1079MiB/s-1079MiB/s (1131MB/s-1131MB/s), io=63.5GiB (68.2GB), run=60241-60241msec

Disk stats (read/write):
  fioa: ios=0/66955, sectors=0/137085163, merge=0/0, ticks=0/15832028, in_queue=15832028, util=0.00%
write_iops: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.39
Starting 1 process
write_iops: Laying out IO file (1 file / 100MiB)
Jobs: 1 (f=1): [w(1)][100.0%][w=328MiB/s][w=83.9k IOPS][eta 00m:00s]
write_iops: (groupid=0, jobs=1): err= 0: pid=12014: Fri Oct 10 14:58:35 2025
  write: IOPS=85.1k, BW=332MiB/s (349MB/s)(19.5GiB/60001msec); 0 zone resets
    slat (usec): min=4, max=637, avg= 9.78, stdev= 2.06
    clat (usec): min=35, max=4833, avg=741.66, stdev=42.38
     lat (usec): min=44, max=4841, avg=751.43, stdev=42.83
    clat percentiles (usec):
     |  1.00th=[  685],  5.00th=[  693], 10.00th=[  693], 20.00th=[  709],
     | 30.00th=[  717], 40.00th=[  734], 50.00th=[  742], 60.00th=[  750],
     | 70.00th=[  758], 80.00th=[  766], 90.00th=[  783], 95.00th=[  807],
     | 99.00th=[  857], 99.50th=[  873], 99.90th=[ 1057], 99.95th=[ 1303],
     | 99.99th=[ 1532]
   bw (  KiB/s): min=326376, max=357170, per=100.00%, avg=340661.78, stdev=7295.58, samples=120
   iops        : min=81594, max=89292, avg=85165.34, stdev=1823.88, samples=120
  lat (usec)   : 50=0.01%, 100=0.01%, 250=0.01%, 500=0.01%, 750=61.15%
  lat (usec)   : 1000=38.72%
  lat (msec)   : 2=0.12%, 4=0.01%, 10=0.01%
  cpu          : usr=14.14%, sys=85.51%, ctx=3439, majf=0, minf=37
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=0,5105378,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
  WRITE: bw=332MiB/s (349MB/s), 332MiB/s-332MiB/s (349MB/s-349MB/s), io=19.5GiB (20.9GB), run=60001-60001msec

Disk stats (read/write):
  fioa: ios=0/5143517, sectors=0/41148181, merge=0/0, ticks=0/143856, in_queue=143856, util=0.00%
read_throughput: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=64
...
fio-3.39
Starting 4 processes
read_throughput: Laying out IO file (1 file / 100MiB)
read_throughput: Laying out IO file (1 file / 100MiB)
read_throughput: Laying out IO file (1 file / 100MiB)
read_throughput: Laying out IO file (1 file / 100MiB)
Jobs: 4 (f=4): [R(4)][100.0%][r=1388MiB/s][r=1388 IOPS][eta 00m:00s]          
read_throughput: (groupid=0, jobs=4): err= 0: pid=12145: Fri Oct 10 14:59:39 2025
  read: IOPS=1384, BW=1388MiB/s (1456MB/s)(81.6GiB/60179msec)
    slat (usec): min=44, max=192, avg=72.86, stdev= 9.73
    clat (msec): min=9, max=1149, avg=184.97, stdev=246.63
     lat (msec): min=10, max=1149, avg=185.04, stdev=246.63
    clat percentiles (msec):
     |  1.00th=[   14],  5.00th=[   19], 10.00th=[   26], 20.00th=[   36],
     | 30.00th=[   47], 40.00th=[   59], 50.00th=[   73], 60.00th=[   92],
     | 70.00th=[  132], 80.00th=[  284], 90.00th=[  609], 95.00th=[  785],
     | 99.00th=[ 1003], 99.50th=[ 1028], 99.90th=[ 1053], 99.95th=[ 1070],
     | 99.99th=[ 1116]
   bw (  MiB/s): min= 1288, max= 1446, per=100.00%, avg=1389.18, stdev= 7.31, samples=480
   iops        : min= 1288, max= 1446, avg=1388.40, stdev= 7.34, samples=480
  lat (msec)   : 10=0.01%, 20=5.81%, 50=27.21%, 100=30.35%, 250=14.40%
  lat (msec)   : 500=8.50%, 750=8.67%, 1000=4.24%, 2000=1.13%
  cpu          : usr=0.51%, sys=3.08%, ctx=82706, majf=0, minf=149
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=83304,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=1388MiB/s (1456MB/s), 1388MiB/s-1388MiB/s (1456MB/s-1456MB/s), io=81.6GiB (87.6GB), run=60179-60179msec

Disk stats (read/write):
  fioa: ios=86067/5, sectors=176265216/110, merge=0/0, ticks=15762104/24, in_queue=15762128, util=0.00%
read_iops: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=libaio, iodepth=64
fio-3.39
Starting 1 process
read_iops: Laying out IO file (1 file / 100MiB)
Jobs: 1 (f=1): [r(1)][100.0%][r=259MiB/s][r=66.3k IOPS][eta 00m:00s]
read_iops: (groupid=0, jobs=1): err= 0: pid=12216: Fri Oct 10 15:00:42 2025
  read: IOPS=65.5k, BW=256MiB/s (268MB/s)(15.0GiB/60001msec)
    slat (usec): min=4, max=286, avg=12.25, stdev=11.13
    clat (usec): min=200, max=2408, avg=963.31, stdev=215.58
     lat (usec): min=223, max=2421, avg=975.56, stdev=215.12
    clat percentiles (usec):
     |  1.00th=[  578],  5.00th=[  635], 10.00th=[  676], 20.00th=[  758],
     | 30.00th=[  840], 40.00th=[  922], 50.00th=[  971], 60.00th=[ 1020],
     | 70.00th=[ 1057], 80.00th=[ 1123], 90.00th=[ 1237], 95.00th=[ 1336],
     | 99.00th=[ 1532], 99.50th=[ 1614], 99.90th=[ 1795], 99.95th=[ 1860],
     | 99.99th=[ 2024]
   bw (  KiB/s): min=231216, max=277152, per=100.00%, avg=262357.28, stdev=11266.07, samples=120
   iops        : min=57804, max=69288, avg=65589.24, stdev=2816.60, samples=120
  lat (usec)   : 250=0.01%, 500=0.04%, 750=19.52%, 1000=36.58%
  lat (msec)   : 2=43.85%, 4=0.01%
  cpu          : usr=10.19%, sys=57.53%, ctx=1226371, majf=0, minf=37
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=100.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, >=64=0.0%
     issued rwts: total=3932044,0,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=256MiB/s (268MB/s), 256MiB/s-256MiB/s (268MB/s-268MB/s), io=15.0GiB (16.1GB), run=60001-60001msec

Disk stats (read/write):
  fioa: ios=4068314/5, sectors=32546512/108, merge=0/0, ticks=1410900/0, in_queue=1410900, util=0.00%