Skip to content

Instantly share code, notes, and snippets.

@takagi

takagi/ptb_fp16

Last active July 1, 2019 03:15
Show Gist options
  • Save takagi/8a1d4f064087a4ca551e04f2fe04343d to your computer and use it in GitHub Desktop.
Save takagi/8a1d4f064087a4ca551e04f2fe04343d to your computer and use it in GitHub Desktop.
Download ZIP
Raw
ptb_fp16
$ CHAINER_DTYPE=float16 python train_ptb.py -d 0 -e 10
#vocab = 10000
epoch iteration perplexity val_perplexity
0 500 326440
0 1000 301342
1 1500 298940 inf
1 2000 334369
1 2500 334369
2 3000 306202 inf
2 3500 339762
3 4000 321258 inf
3 4500 334369
3 5000 316159
4 5500 353628 inf
4 6000 313640
4 6500 318698
5 7000 339762 inf
5 7500 316159
6 8000 294196 inf
6 8500 326440
6 9000 371016
7 9500 316159 inf
7 10000 348015
7 10500 318698
8 11000 306202 inf
8 11500 326440
9 12000 318698 inf
9 12500 323838
9 13000 291851
test
test perplexity: inf
Raw
ptb_fp32
$ python train_ptb.py -d 0 -e 10
#vocab = 10000
epoch iteration perplexity val_perplexity............] 3.69%
0 500 696.825
0 1000 300.379
1 1500 227.869 207.567
1 2000 202.66
1 2500 169.194
2 3000 156.06 152.122
2 3500 149.424
3 4000 137.197 129.369
3 4500 130.502
3 5000 119.299
4 5500 117.775 118.105
4 6000 112.313
4 6500 104.091
5 7000 107.539 109.152
5 7500 100.775
6 8000 92.93 102.855
6 8500 95.7028
6 9000 99.9391
7 9500 83.7397 99.7991
7 10000 93.0974
7 10500 83.7269
8 11000 86.057 96.3761
8 11500 82.3142
9 12000 83.6467 94.6426
9 12500 79.6013
9 13000 77.0567
test
test perplexity: 90.019997959895
Raw
runtime_info
$ python
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import chainer
>>> chainer.print_runtime_info()
Platform: Linux-4.15.0-50-generic-x86_64-with-debian-buster-sid
Chainer: 7.0.0a1
NumPy: 1.16.4
CuPy:
CuPy Version : 7.0.0a1
CUDA Root : /usr/local/cuda
CUDA Build Version : 10000
CUDA Driver Version : 10000
CUDA Runtime Version : 10000
cuDNN Build Version : 7500
cuDNN Version : 7500
NCCL Build Version : 2402
NCCL Runtime Version : 2402
iDeep: 2.0.0.post3
>>>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment