ebuildy/get_memory.scala

Created April 7, 2017 08:42

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/ebuildy/4fb26a225e5e9109e74c193fe2ea438b.js"></script>
Save ebuildy/4fb26a225e5e9109e74c193fe2ea438b to your computer and use it in GitHub Desktop.

Download ZIP

Get total memory used by all executors on Apache Spark

Raw

get_memory.scala

sc.getExecutorMemoryStatus.map(a => (a._2._1 - a._2._2)/(1024.0*1024*1024)).sum

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment