Skip to content

Instantly share code, notes, and snippets.

@takagi
Last active June 14, 2019 02:14
Show Gist options
  • Save takagi/133c27c7c1edfa6917a595cafc000423 to your computer and use it in GitHub Desktop.
Save takagi/133c27c7c1edfa6917a595cafc000423 to your computer and use it in GitHub Desktop.
Comparison of Chainer's memnn example between in FP32 mode and in FP16 mode
$ CHAINER_DTYPE=float16 python train_memnn.py tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_train.txt tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_test.txt -d 0
Training data: tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_train.txt: 2000
Test data: tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_test.txt: 200
epoch main/loss validation/main/loss main/accuracy validation/main/accuracy
1 nan nan 0.0017004 0
2 nan nan 0 0
3 nan nan 0 0
4 nan nan 0 0
5 nan nan 0 0
6 nan nan 0 0
7 nan nan 0 0
8 nan nan 0 0
9 nan nan 0 0
10 nan nan 0 0
11 nan nan 0 0
12 nan nan 0 0
13 nan nan 0 0
14 nan nan 0 0
15 nan nan 0 0
16 nan nan 0 0
17 nan nan 0 0
18 nan nan 0 0
19 nan nan 0 0
20 nan nan 0 0
21 nan nan 0 0
22 nan nan 0 0
23 nan nan 0 0
24 nan nan 0 0
25 nan nan 0 0
26 nan nan 0 0
27 nan nan 0 0
28 nan nan 0 0
29 nan nan 0 0
30 nan nan 0 0
...
$ python train_memnn.py tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_train.txt tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_test.txt -d 0
Training data: tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_train.txt: 2000
Test data: tasks_1-20_v1-2/en-10k/qa1_single-supporting-fact_test.txt: 200
epoch main/loss validation/main/loss main/accuracy validation/main/accuracy
1 2.41833 1.80678 0.2083 0.225
2 1.76084 1.71291 0.2972 0.381
3 1.62661 1.52482 0.4552 0.521
4 1.40628 1.29272 0.5494 0.573
5 0.937755 0.539432 0.728 0.863
6 0.230902 0.09883 0.9655 0.994
7 0.0569344 0.0417305 0.9993 0.997
8 0.0273535 0.0242202 0.9999 0.999
9 0.0167176 0.0162068 1 1
10 0.011597 0.011871 1 1
11 0.00858905 0.00896583 1 1
12 0.00666347 0.00704912 1 1
13 0.00533772 0.00574273 1 1
14 0.00437632 0.00473799 1 1
15 0.00365677 0.00395577 1 1
16 0.00309958 0.00336608 1 1
17 0.00266046 0.00289834 1 1
18 0.00230534 0.00251321 1 1
19 0.00201576 0.00220578 1 1
20 0.00177504 0.00193605 1 1
21 0.00157303 0.00172307 1 1
22 0.00140205 0.00153442 1 1
23 0.00125536 0.00137551 1 1
24 0.00112903 0.00123596 1 1
25 0.00101942 0.00111762 1 1
26 0.00092357 0.00101332 1 1
27 0.000839527 0.000921253 1 1
28 0.000765088 0.000840034 1 1
29 0.000699283 0.000767484 1 1
30 0.000640476 0.000703148 1 1
...
$ python
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import chainer
>>> chainer.print_runtime_info()
Platform: Linux-4.15.0-50-generic-x86_64-with-debian-buster-sid
Chainer: 6.0.0
NumPy: 1.16.4
CuPy:
CuPy Version : 6.0.0
CUDA Root : /usr/local/cuda
CUDA Build Version : 10000
CUDA Driver Version : 10000
CUDA Runtime Version : 10000
cuDNN Build Version : 7500
cuDNN Version : 7500
NCCL Build Version : 2402
NCCL Runtime Version : 2402
iDeep: 2.0.0.post3
>>>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment