Skip to content

Instantly share code, notes, and snippets.

@emaxerrno
Created April 13, 2016 20:04
Show Gist options
  • Select an option

  • Save emaxerrno/cb7259ba9523f5f7d4a773624be5852f to your computer and use it in GitHub Desktop.

Select an option

Save emaxerrno/cb7259ba9523f5f7d4a773624be5852f to your computer and use it in GitHub Desktop.
aprk.log
I0413 19:47:42.202162 31355 exec.cpp:143] Version: 0.28.0
I0413 19:47:42.204401 31361 exec.cpp:217] Executor registered on slave df110eaf-9d50-4727-818f-6021a9946d5b-S4
+++ echo FOOBAR
+++ export LD_LIBRARY_PATH=:/usr/lib/hadoop/lib/native
+++ LD_LIBRARY_PATH=:/usr/lib/hadoop/lib/native
+++ export HADOOP_CONF_DIR=/etc/hadoop/conf
+++ HADOOP_CONF_DIR=/etc/hadoop/conf
+++ export SPARK_LOG_DIR=/var/log/spark
+++ SPARK_LOG_DIR=/var/log/spark
+++ export SPARK_PID_DIR=/run/spark
+++ SPARK_PID_DIR=/run/spark
+++ export JAVA_HOME=/usr/lib/jvm/java-8-oracle
+++ JAVA_HOME=/usr/lib/jvm/java-8-oracle
+++ export MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so
+++ MESOS_NATIVE_JAVA_LIBRARY=/usr/lib/libmesos.so
+++ export SPARK_CONF_DIR=/etc/spark/conf/
+++ SPARK_CONF_DIR=/etc/spark/conf/
+++ export SPARK_LOCAL_IP=10.240.0.16
+++ SPARK_LOCAL_IP=10.240.0.16
++ set +a
++ '[' -z '' ']'
++ ASSEMBLY_DIR2=/usr/lib/spark-1.6.1-bin-hadoop2.6/assembly/target/scala-2.11
++ ASSEMBLY_DIR1=/usr/lib/spark-1.6.1-bin-hadoop2.6/assembly/target/scala-2.10
++ [[ -d /usr/lib/spark-1.6.1-bin-hadoop2.6/assembly/target/scala-2.11 ]]
++ '[' -d /usr/lib/spark-1.6.1-bin-hadoop2.6/assembly/target/scala-2.11 ']'
++ export SPARK_SCALA_VERSION=2.10
++ SPARK_SCALA_VERSION=2.10
+ '[' -n /usr/lib/jvm/java-8-oracle ']'
+ RUNNER=/usr/lib/jvm/java-8-oracle/bin/java
+ SPARK_ASSEMBLY_JAR=
+ '[' -f /usr/lib/spark-1.6.1-bin-hadoop2.6/RELEASE ']'
+ ASSEMBLY_DIR=/usr/lib/spark-1.6.1-bin-hadoop2.6/lib
+ GREP_OPTIONS=
++ ls -1 /usr/lib/spark-1.6.1-bin-hadoop2.6/lib
++ grep '^spark-assembly.*hadoop.*\.jar$'
++ wc -l
+ num_jars=1
+ '[' 1 -eq 0 -a -z '' -a '' '!=' 1 ']'
+ '[' -d /usr/lib/spark-1.6.1-bin-hadoop2.6/lib ']'
++ ls -1 /usr/lib/spark-1.6.1-bin-hadoop2.6/lib
++ grep '^spark-assembly.*hadoop.*\.jar$'
+ ASSEMBLY_JARS=spark-assembly-1.6.1-hadoop2.6.0.jar
+ '[' 1 -gt 1 ']'
+ SPARK_ASSEMBLY_JAR=/usr/lib/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar
+ LAUNCH_CLASSPATH=/usr/lib/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar
+ '[' -n '' ']'
+ export _SPARK_ASSEMBLY=/usr/lib/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar
+ _SPARK_ASSEMBLY=/usr/lib/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar
+ [[ -n '' ]]
+ CMD=()
+ IFS=
+ read -d '' -r ARG
++ /usr/lib/jvm/java-8-oracle/bin/java -cp /usr/lib/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar org.apache.spark.launcher.Main org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.240.0.12:36785 --executor-id df110eaf-9d50-4727-818f-6021a9946d5b-S4 --hostname 104.154.100.212 --cores 2 --app-id 41cddcef-2898-4a05-9c87-4b49b638cbef-0023
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ CMD+=("$ARG")
+ IFS=
+ read -d '' -r ARG
+ exec /usr/lib/jvm/java-8-oracle/bin/java -cp /etc/spark/conf/:/usr/lib/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar:/usr/lib/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-api-jdo-3.2.6.jar:/usr/lib/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-core-3.2.10.jar:/usr/lib/spark-1.6.1-bin-hadoop2.6/lib/datanucleus-rdbms-3.2.9.jar:/etc/hadoop/conf -Xms2048m -Xmx2048m org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@10.240.0.12:36785 --executor-id df110eaf-9d50-4727-818f-6021a9946d5b-S4 --hostname 104.154.100.212 --cores 2 --app-id 41cddcef-2898-4a05-9c87-4b49b638cbef-0023
16/04/13 19:47:42 INFO CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT]
16/04/13 19:47:42 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/04/13 19:47:43 INFO SecurityManager: Changing view acls to: root
16/04/13 19:47:43 INFO SecurityManager: Changing modify acls to: root
16/04/13 19:47:43 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
16/04/13 19:47:43 INFO SecurityManager: Changing view acls to: root
16/04/13 19:47:43 INFO SecurityManager: Changing modify acls to: root
16/04/13 19:47:43 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); users with modify permissions: Set(root)
16/04/13 19:47:43 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:43 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
16/04/13 19:47:44 WARN Utils: Service 'sparkExecutorActorSystem' could not bind on port 0. Attempting port 1.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.
16/04/13 19:47:44 ERROR Remoting: Remoting system has been terminated abrubtly. Attempting to shut down transports
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.
16/04/13 19:47:44 INFO Slf4jLogger: Slf4jLogger started
16/04/13 19:47:44 INFO Remoting: Starting remoting
16/04/13 19:47:44 ERROR NettyTransport: failed to bind to /104.154.100.212:0, shutting down Netty transport
Exception in thread "main" java.net.BindException: Failed to bind to: /104.154.100.212:0: Service 'sparkExecutorActorSystem' failed after 16 retries!
at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:393)
at akka.remote.transport.netty.NettyTransport$$anonfun$listen$1.apply(NettyTransport.scala:389)
at scala.util.Success$$anonfun$map$1.apply(Try.scala:206)
at scala.util.Try$.apply(Try.scala:161)
at scala.util.Success.map(Try.scala:206)
at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
at scala.concurrent.Future$$anonfun$map$1.apply(Future.scala:235)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)
at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
16/04/13 19:47:44 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment