Skip to content

Instantly share code, notes, and snippets.

@takagi

takagi/mnist_fp16

Created June 10, 2019 08:41
Show Gist options
  • Save takagi/7a6917c44a1c288b15bb7975e27b916f to your computer and use it in GitHub Desktop.
Save takagi/7a6917c44a1c288b15bb7975e27b916f to your computer and use it in GitHub Desktop.
Download ZIP
Comparison of Chainer's mnist example between in FP32 mode and in FP16 mode
Raw
mnist_fp16
$ CHAINER_DTYPE=float16 python train_mnist.py -d 0
Device: @cupy:0
# unit: 1000
# Minibatch-size: 100
# epoch: 20
epoch main/loss validation/main/loss main/accuracy validation/main/accuracy elapsed_time
1 nan nan 0.0994271 0.0980225 3.91818
2 nan nan 0.0997917 0.0980225 6.22553
3 nan nan 0.0995833 0.0980225 8.72424
4 nan nan 0.099375 0.0980225 11.3325
5 nan nan 0.0997396 0.0980225 13.7157
6 nan nan 0.0995312 0.0980225 16.2174
7 nan nan 0.0994271 0.0980225 18.8041
8 nan nan 0.0996354 0.0980225 21.3289
9 nan nan 0.0996875 0.0980225 23.8468
10 nan nan 0.099375 0.0980225 26.3605
11 nan nan 0.0995833 0.0980225 28.8567
12 nan nan 0.0993229 0.0980225 31.3275
13 nan nan 0.0996875 0.0980225 33.886
14 nan nan 0.0996875 0.0980225 36.3637
15 nan nan 0.0996875 0.0980225 38.6991
16 nan nan 0.0994792 0.0980225 40.981
17 nan nan 0.0991667 0.0980225 43.2793
18 nan nan 0.0992187 0.0980225 45.6158
19 nan nan 0.0997917 0.0980225 47.8968
20 nan nan 0.0998437 0.0980225 50.1865
Raw
mnist_fp32
$ python train_mnist.py -d 0
Device: @cupy:0
# unit: 1000
# Minibatch-size: 100
# epoch: 20
Downloading from http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz...
Downloading from http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz...
Downloading from http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz...
Downloading from http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz...
epoch main/loss validation/main/loss main/accuracy validation/main/accuracy elapsed_time
1 0.191721 0.103644 0.942867 0.966 3.72396
2 0.0727625 0.0803097 0.977783 0.9745 6.0102
3 0.0488638 0.0803536 0.984132 0.974 8.31127
4 0.0353504 0.0675346 0.988282 0.9814 10.617
5 0.0268242 0.0791875 0.991048 0.9788 12.9174
6 0.0236898 0.0881506 0.992232 0.9778 15.253
7 0.0250604 0.0709666 0.991898 0.9806 17.5925
8 0.0176728 0.082598 0.994231 0.9785 19.9265
9 0.0146536 0.0784669 0.995482 0.9802 22.207
10 0.015635 0.0938949 0.994982 0.9796 24.4942
11 0.0143474 0.0830709 0.995849 0.9803 26.7645
12 0.0112423 0.097842 0.996365 0.9798 29.0801
13 0.0117989 0.0894112 0.996432 0.9828 31.4166
14 0.0116941 0.112915 0.996582 0.9801 33.6981
15 0.0136293 0.0840091 0.995899 0.9823 35.9836
16 0.00918777 0.0957944 0.997099 0.9824 38.2678
17 0.00915884 0.120189 0.997132 0.9762 40.564
18 0.0102279 0.0965553 0.997149 0.9837 42.8879
19 0.00831765 0.100924 0.997699 0.9816 45.2091
20 0.0102448 0.111097 0.997066 0.9817 47.5284
Raw
runtime_info
$ python
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import chainer
>>> chainer.print_runtime_info()
Platform: Linux-4.15.0-50-generic-x86_64-with-debian-buster-sid
Chainer: 6.0.0
NumPy: 1.16.4
CuPy:
CuPy Version : 6.0.0
CUDA Root : /usr/local/cuda
CUDA Build Version : 10000
CUDA Driver Version : 10000
CUDA Runtime Version : 10000
cuDNN Build Version : 7500
cuDNN Version : 7500
NCCL Build Version : 2402
NCCL Runtime Version : 2402
iDeep: 2.0.0.post3
>>>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment