Created
October 21, 2019 20:47
-
-
Save neomatrix369/8125f18f2a1d6da236ab5bd26831e69d to your computer and use it in GitHub Desktop.
Logs of the GPU version of dl4j-nlp failing due to OOME (dl4j-nlp-cuda)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
openjdk version "1.8.0_192" | |
OpenJDK Runtime Environment (build 1.8.0_192-20181024121959.buildslave.jdk8u-src-tar--b12) | |
GraalVM 1.0.0-rc9 (build 25.192-b12-jvmci-0.49, mixed mode) | |
Data (.tar.gz file) already exists at /tmp/dl4j_w2vSentiment/aclImdb_v1.tar.gz | |
Data (extracted) already exists at /tmp/dl4j_w2vSentiment/aclImdb | |
2019-10-21 21:42:44,309ate - 21:42:44,309 INFO ~ Loaded [JCublasBackend] backend | |
2019-10-21 21:42:46,256ate - 21:42:46,256 INFO ~ Number of threads used for NativeOps: 32 | |
2019-10-21 21:42:47,140ate - 21:42:47,140 INFO ~ Number of threads used for BLAS: 0 | |
2019-10-21 21:42:47,145ate - 21:42:47,145 INFO ~ Backend used: [CUDA]; OS: [Linux] | |
2019-10-21 21:42:47,145ate - 21:42:47,145 INFO ~ Cores: [8]; Memory: [3.4GB]; | |
2019-10-21 21:42:47,145ate - 21:42:47,145 INFO ~ Blas vendor: [CUBLAS] | |
2019-10-21 21:42:47,145ate - 21:42:47,145 INFO ~ Device Name: [GeForce GTX 1050]; CC: [6.1]; Total/free memory: [4238737408] | |
2019-10-21 21:42:47,207ate - 21:42:47,207 INFO ~ Starting ComputationGraph with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE] | |
2019-10-21 21:42:47,265ate - 21:42:47,265 INFO ~ cuDNN not found: use cuDNN for better GPU performance by including the deeplearning4j-cuda module. For more information, please refer to: https://deeplearning4j.org/docs/latest/deeplearning4j-config-cudnn | |
java.lang.ClassNotFoundException: org.deeplearning4j.nn.layers.convolution.CudnnConvolutionHelper | |
at java.net.URLClassLoader.findClass(URLClassLoader.java:382) | |
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) | |
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) | |
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) | |
at java.lang.Class.forName0(Native Method) | |
at java.lang.Class.forName(Class.java:264) | |
at org.deeplearning4j.nn.layers.convolution.ConvolutionLayer.initializeHelper(ConvolutionLayer.java:75) | |
at org.deeplearning4j.nn.layers.convolution.ConvolutionLayer.<init>(ConvolutionLayer.java:67) | |
at org.deeplearning4j.nn.conf.layers.ConvolutionLayer.instantiate(ConvolutionLayer.java:168) | |
at org.deeplearning4j.nn.conf.graph.LayerVertex.instantiate(LayerVertex.java:107) | |
at org.deeplearning4j.nn.graph.ComputationGraph.init(ComputationGraph.java:570) | |
at org.deeplearning4j.nn.graph.ComputationGraph.init(ComputationGraph.java:439) | |
at org.deeplearning4j.examples.convolution.sentenceclassification.CnnSentenceClassificationExample.main(CnnSentenceClassificationExample.java:132) | |
Number of parameters by layer: | |
cnn3 90100 | |
cnn4 120100 | |
cnn5 150100 | |
globalPool 0 | |
out 602 | |
Loading word vectors and creating DataSetIterators | |
Exception in thread "main" java.lang.OutOfMemoryError: Cannot allocate new FloatPointer(300): totalBytes = 2194M, physicalBytes = 7013M | |
at org.bytedeco.javacpp.FloatPointer.<init>(FloatPointer.java:76) | |
at org.bytedeco.javacpp.FloatPointer.<init>(FloatPointer.java:41) | |
at org.nd4j.compression.impl.AbstractCompressor.compress(AbstractCompressor.java:159) | |
at org.nd4j.compression.impl.AbstractCompressor.compress(AbstractCompressor.java:134) | |
at org.nd4j.storage.CompressedRamStorage.store(CompressedRamStorage.java:84) | |
at org.deeplearning4j.models.embeddings.loader.WordVectorSerializer.loadStaticModel(WordVectorSerializer.java:2785) | |
at org.deeplearning4j.examples.convolution.sentenceclassification.CnnSentenceClassificationExample.main(CnnSentenceClassificationExample.java:141) | |
Caused by: java.lang.OutOfMemoryError: Physical memory usage is too high: physicalBytes (7013M) > maxPhysicalBytes (6891M) | |
at org.bytedeco.javacpp.Pointer.deallocator(Pointer.java:585) | |
at org.bytedeco.javacpp.Pointer.init(Pointer.java:125) | |
at org.bytedeco.javacpp.FloatPointer.allocateArray(Native Method) | |
at org.bytedeco.javacpp.FloatPointer.<init>(FloatPointer.java:68) | |
... 6 more |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment