Skip to content

Instantly share code, notes, and snippets.

@agibsonccc
Created September 15, 2016 13:08
Show Gist options
  • Save agibsonccc/e2376ffdf5764bddb896ddb19a5fcbb6 to your computer and use it in GitHub Desktop.
Save agibsonccc/e2376ffdf5764bddb896ddb19a5fcbb6 to your computer and use it in GitHub Desktop.
bin/spark-submit --class io.skymind.TestGPUInvocation \
--conf spark.mesos.gpus.max=1 \
--master mesos://"$MASTER:7077" \
--deploy-mode cluster \
--conf spark.mesos.role=spark \
--conf spark.driver.memory=2G \
--driver-java-options "-Dlog4j.configuration=file:///home/centos/dist/log4j.properties" \
--executor-memory 3G \
--conf spark.mesos.gpu.enabled=true \
--conf spark.mesos.coarse=true \
--total-executor-cores 3 \
--conf spark.eventLog.enabled=true \
--conf spark.executorEnv.MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so \
--conf spark.mesos.gpus.max=1 \
--conf spark.executor.uri=https://some/spark-distro.tgz \
/path/to/jar
~
16/09/15 13:08:40 DEBUG CoarseMesosSchedulerBackend: !slaveIdsWithExecutors.contains(slaveId) is true
16/09/15 13:08:40 DEBUG CoarseMesosSchedulerBackend: Declining offer max gpus: 1 gpus 1
16/09/15 13:08:40 DEBUG CoarseMesosSchedulerBackend: Declining offer: b861f18f-64c5-4b2d-8142-39651737d19e-O43712 with attributes: Map() mem: 6144.0 cpu: 15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment