Created
July 18, 2016 05:47
-
-
Save ganeshchand/b4887f0e09e81f510952630985acef5c to your computer and use it in GitHub Desktop.
IntelliJ Out-of-memory Troubleshooting
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Check Memory usage: | |
println(sys.runtime.totalMemory()) | |
println(sys.runtime.maxMemory()) | |
println(sys.runtime.freeMemory()) | |
Often, Scala or Spark program will throw Spark driver and executor will be entirely running inside the JVM that is running your code shown here that creates the SparkContext. | |
By that time, it's too late to obtain more Java heap than was allocated when the JVM started. You need to add that -Xmx1G arg to the | |
command IntelliJ uses to launch the JVM that runs your code. | |
VM Options: -Xmx4G |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment