Skip to content

Instantly share code, notes, and snippets.

@takagi

takagi/cifar_fp16

Last active June 10, 2019 06:54
Show Gist options
  • Save takagi/c40565033391e3486e3cb6094c79796f to your computer and use it in GitHub Desktop.
Save takagi/c40565033391e3486e3cb6094c79796f to your computer and use it in GitHub Desktop.
Download ZIP
Comparison of Chainer's cifar example between in FP32 mode and in FP16 mode
Raw
cifar_fp16
$ CHAIENR_DTYPE=float16 python train_cifar.py -d 0
Device: @cupy:0
# Minibatch-size: 64
# epoch: 300
Using CIFAR10 dataset.
epoch main/loss validation/main/loss main/accuracy validation/main/accuracy elapsed_time
1 2.32253 2.12119 0.175971 0.192178 23.2918
2 1.7554 1.87858 0.304497 0.302747 49.5816
3 1.46396 1.61379 0.450664 0.416202 75.7544
4 1.19319 1.09644 0.576685 0.613455 101.872
5 1.00631 0.950184 0.652274 0.66869 128.084
6 0.90759 0.947134 0.693002 0.673766 154.2
7 0.849793 0.886227 0.71899 0.709594 180.264
8 0.808017 0.721276 0.735635 0.757962 206.306
9 0.769259 0.840263 0.75034 0.723229 232.388
10 0.746859 0.934423 0.760783 0.696557 258.461
11 0.722813 0.770808 0.768826 0.753384 284.479
12 0.699017 0.856111 0.776929 0.723627 310.542
13 0.681315 0.708714 0.784707 0.772293 336.578
14 0.671638 0.697229 0.787532 0.773189 362.652
15 0.657078 0.677996 0.790913 0.774881 408.185
16 0.638679 0.709751 0.796595 0.763236 456.475
17 0.63258 0.746751 0.800571 0.771795 504.841
18 0.622672 0.793111 0.801096 0.751791 553.377
19 0.620495 0.737358 0.803577 0.759256 601.851
20 0.610699 0.673288 0.807518 0.782842 650.327
21 0.602079 0.863245 0.808983 0.727707 698.866
22 0.598414 0.685147 0.8116 0.772293 747.455
23 0.594778 0.713615 0.81104 0.773189 795.879
24 0.593061 0.678424 0.8128 0.78125 844.278
25 0.581764 0.788939 0.818135 0.745721 892.811
26 0.442184 0.431781 0.860455 0.857086 941.176
27 0.41406 0.441371 0.867338 0.85629 989.666
28 0.409392 0.545269 0.870699 0.824144 1038.19
29 0.417326 0.443872 0.866508 0.85201 1086.63
30 0.40924 0.581538 0.869038 0.819765 1135.03
...
Raw
cifar_fp32
$ python train_cifar.py -d 0
Device: @cupy:0
# Minibatch-size: 64
# epoch: 300
Using CIFAR10 dataset.
epoch main/loss validation/main/loss main/accuracy validation/main/accuracy elapsed_time
1 2.35414 2.12122 0.159427 0.184216 48.0756
2 1.77024 1.78773 0.296355 0.298766 96.6433
3 1.40901 1.32299 0.46953 0.522094 145.224
4 1.15733 1.1619 0.585807 0.589371 193.851
5 1.00747 1.11056 0.653912 0.634654 242.54
6 0.920811 0.930986 0.68794 0.689889 291.118
7 0.85315 0.785346 0.71813 0.743929 339.523
8 0.8059 1.00068 0.737036 0.676951 388.061
9 0.77168 0.869913 0.751299 0.715764 436.809
10 0.750303 0.788927 0.758483 0.746915 485.336
11 0.717459 0.768602 0.771187 0.745621 533.788
12 0.703678 0.750112 0.776689 0.750299 582.262
13 0.69506 0.693186 0.778752 0.777667 630.841
14 0.666868 0.685967 0.786852 0.773885 679.337
15 0.66427 0.683768 0.790193 0.773686 727.825
16 0.649588 0.780172 0.791593 0.74174 776.204
17 0.637906 0.699203 0.796355 0.775975 824.655
18 0.630894 0.694679 0.799956 0.770402 873.202
19 0.623075 0.815673 0.802137 0.742138 921.717
20 0.61615 0.956148 0.803717 0.686903 970.312
21 0.609601 0.629242 0.806066 0.797572 1018.77
22 0.603919 0.657678 0.810319 0.786525 1067.28
23 0.59922 0.709533 0.810539 0.772791 1115.87
24 0.594129 0.832633 0.809939 0.73338 1164.36
25 0.584432 0.695485 0.816536 0.761445 1212.81
26 0.445156 0.488616 0.859795 0.840764 1261.28
27 0.41883 0.501832 0.866897 0.840068 1309.77
28 0.424226 0.507571 0.864637 0.842954 1358.23
29 0.418762 0.506568 0.867927 0.830414 1406.78
30 0.416286 0.463812 0.867998 0.848328 1455.3
...
Raw
runtime_info
$ python
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import chainer
>>> chainer.print_runtime_info()
Platform: Linux-4.15.0-50-generic-x86_64-with-debian-buster-sid
Chainer: 6.0.0
NumPy: 1.16.4
CuPy:
CuPy Version : 6.0.0
CUDA Root : /usr/local/cuda
CUDA Build Version : 10000
CUDA Driver Version : 10000
CUDA Runtime Version : 10000
cuDNN Build Version : 7500
cuDNN Version : 7500
NCCL Build Version : 2402
NCCL Runtime Version : 2402
iDeep: 2.0.0.post3
>>>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment