Skip to content

Instantly share code, notes, and snippets.

@jacobkahn

jacobkahn/gist:3510a154e50b18defd236eeea461e3fb

Created July 24, 2019 00:35
Show Gist options
  • Save jacobkahn/3510a154e50b18defd236eeea461e3fb to your computer and use it in GitHub Desktop.
Save jacobkahn/3510a154e50b18defd236eeea461e3fb to your computer and use it in GitHub Desktop.
Download ZIP
NCCL Tests on 2 node p3dn.24xlarge + ethernet - all_reduce_perf -b 8 -e 1G -f 2 -g 1 -c 1 -n 100
Raw
gistfile1.txt
#
# out-of-place in-place
# size count type redop time algbw busbw error time algbw busbw error
# (B) (elements) (us) (GB/s) (GB/s) (us) (GB/s) (GB/s)
8 2 float sum 112.7 0.00 0.00 2e-07 110.9 0.00 0.00 1e-07
16 4 float sum 110.5 0.00 0.00 0e+00 111.3 0.00 0.00 1e-07
32 8 float sum 110.6 0.00 0.00 1e-07 111.1 0.00 0.00 1e-07
64 16 float sum 112.2 0.00 0.00 1e-07 112.1 0.00 0.00 6e-08
128 32 float sum 113.1 0.00 0.00 6e-08 113.3 0.00 0.00 6e-08
256 64 float sum 112.5 0.00 0.00 6e-08 113.8 0.00 0.00 6e-08
512 128 float sum 111.8 0.00 0.01 6e-08 113.1 0.00 0.01 6e-08
1024 256 float sum 123.1 0.01 0.02 2e-07 124.0 0.01 0.02 2e-07
2048 512 float sum 159.1 0.01 0.02 2e-07 158.2 0.01 0.02 2e-07
4096 1024 float sum 183.4 0.02 0.04 5e-07 183.1 0.02 0.04 5e-07
8192 2048 float sum 202.7 0.04 0.08 5e-07 204.1 0.04 0.08 5e-07
16384 4096 float sum 212.7 0.08 0.14 5e-07 211.4 0.08 0.15 5e-07
32768 8192 float sum 236.5 0.14 0.26 5e-07 233.5 0.14 0.26 5e-07
65536 16384 float sum 333.6 0.20 0.37 5e-07 333.0 0.20 0.37 5e-07
131072 32768 float sum 391.1 0.34 0.63 5e-07 390.5 0.34 0.63 5e-07
262144 65536 float sum 531.0 0.49 0.93 5e-07 531.7 0.49 0.92 5e-07
524288 131072 float sum 828.6 0.63 1.19 5e-07 830.2 0.63 1.18 5e-07
1048576 262144 float sum 1668.0 0.63 1.18 5e-07 1668.6 0.63 1.18 5e-07
2097152 524288 float sum 1662.5 1.26 2.37 5e-07 1662.4 1.26 2.37 5e-07
4194304 1048576 float sum 3337.1 1.26 2.36 5e-07 3335.4 1.26 2.36 5e-07
8388608 2097152 float sum 6674.4 1.26 2.36 5e-07 6675.4 1.26 2.36 5e-07
16777216 4194304 float sum 13350 1.26 2.36 5e-07 13350 1.26 2.36 5e-07
33554432 8388608 float sum 26701 1.26 2.36 5e-07 26700 1.26 2.36 5e-07
67108864 16777216 float sum 53401 1.26 2.36 5e-07 53402 1.26 2.36 5e-07
134217728 33554432 float sum 106805 1.26 2.36 5e-07 106805 1.26 2.36 5e-07
268435456 67108864 float sum 213660 1.26 2.36 5e-07 213610 1.26 2.36 5e-07
536870912 134217728 float sum 427272 1.26 2.36 5e-07 427278 1.26 2.36 5e-07
1073741824 268435456 float sum 854497 1.26 2.36 5e-07 854543 1.26 2.36 5e-07
# Out of bounds values : 0 OK
# Avg bus bandwidth : 1.01567
#
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment