Created
January 31, 2017 17:07
-
-
Save kyrozetera/0cadd1ffb094c1f55810c2758cf294bb to your computer and use it in GitHub Desktop.
Log of Spark JobServer spinning up a context on marathon
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [2017-01-31 16:59:42,330] INFO ocalContextSupervisorActor [] [akka://JobServer/user/context-supervisor] - Creating a SparkContext named test-context | |
| [2017-01-31 16:59:42,336] INFO .jobserver.JobManagerActor [] [akka://JobServer/user/context-supervisor/test-context] - Starting actor spark.jobserver.JobManagerActor | |
| [2017-01-31 16:59:42,382] INFO k.jobserver.JobStatusActor [] [akka://JobServer/user/context-supervisor/test-context/$a] - Starting actor spark.jobserver.JobStatusActor | |
| [2017-01-31 16:59:42,382] INFO k.jobserver.JobResultActor [] [akka://JobServer/user/context-supervisor/test-context/$b] - Starting actor spark.jobserver.JobResultActor | |
| [2017-01-31 16:59:42,441] WARN org.apache.spark.SparkConf [] [akka://JobServer/user/context-supervisor/test-context] - The configuration key spark.akka.threads is not supported any more because Spark doesn't use Akka since 2.0 | |
| [2017-01-31 16:59:42,483] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Running Spark version 2.0.1 | |
| [2017-01-31 16:59:42,784] WARN doop.util.NativeCodeLoader [] [akka://JobServer/user/context-supervisor/test-context] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable | |
| [2017-01-31 16:59:43,023] INFO ache.spark.SecurityManager [] [akka://JobServer/user/context-supervisor/test-context] - Changing view acls to: root | |
| [2017-01-31 16:59:43,024] INFO ache.spark.SecurityManager [] [akka://JobServer/user/context-supervisor/test-context] - Changing modify acls to: root | |
| [2017-01-31 16:59:43,027] INFO ache.spark.SecurityManager [] [akka://JobServer/user/context-supervisor/test-context] - Changing view acls groups to: | |
| [2017-01-31 16:59:43,028] INFO ache.spark.SecurityManager [] [akka://JobServer/user/context-supervisor/test-context] - Changing modify acls groups to: | |
| [2017-01-31 16:59:43,029] INFO ache.spark.SecurityManager [] [akka://JobServer/user/context-supervisor/test-context] - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(root); groups with view permissions: Set(); users with modify permissions: Set(root); groups with modify permissions: Set() | |
| [2017-01-31 16:59:43,342] INFO rg.apache.spark.util.Utils [] [akka://JobServer/user/context-supervisor/test-context] - Successfully started service 'sparkDriver' on port 45783. | |
| [2017-01-31 16:59:43,370] INFO org.apache.spark.SparkEnv [] [akka://JobServer/user/context-supervisor/test-context] - Registering MapOutputTracker | |
| [2017-01-31 16:59:43,399] INFO org.apache.spark.SparkEnv [] [akka://JobServer/user/context-supervisor/test-context] - Registering BlockManagerMaster | |
| [2017-01-31 16:59:43,424] INFO k.storage.DiskBlockManager [] [akka://JobServer/user/context-supervisor/test-context] - Created local directory at /tmp/blockmgr-2a672e30-21b1-45e8-b88b-f52bb496ef9a | |
| [2017-01-31 16:59:43,449] INFO storage.memory.MemoryStore [] [akka://JobServer/user/context-supervisor/test-context] - MemoryStore started with capacity 414.4 MB | |
| [2017-01-31 16:59:43,514] INFO org.apache.spark.SparkEnv [] [akka://JobServer/user/context-supervisor/test-context] - Registering OutputCommitCoordinator | |
| [2017-01-31 16:59:43,674] INFO ark_project.jetty.util.log [] [akka://JobServer/user/context-supervisor/test-context] - Logging initialized @114467ms | |
| [2017-01-31 16:59:43,795] INFO roject.jetty.server.Server [] [akka://JobServer/user/context-supervisor/test-context] - jetty-9.2.z-SNAPSHOT | |
| [2017-01-31 16:59:43,812] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@745d1de2{/jobs,null,AVAILABLE} | |
| [2017-01-31 16:59:43,813] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@1549a835{/jobs/json,null,AVAILABLE} | |
| [2017-01-31 16:59:43,813] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@4c8454b4{/jobs/job,null,AVAILABLE} | |
| [2017-01-31 16:59:43,813] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@64b5659{/jobs/job/json,null,AVAILABLE} | |
| [2017-01-31 16:59:43,813] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@1a65f97f{/stages,null,AVAILABLE} | |
| [2017-01-31 16:59:43,813] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@42b5f206{/stages/json,null,AVAILABLE} | |
| [2017-01-31 16:59:43,813] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@2cfa3f10{/stages/stage,null,AVAILABLE} | |
| [2017-01-31 16:59:43,813] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@70df4d48{/stages/stage/json,null,AVAILABLE} | |
| [2017-01-31 16:59:43,814] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@4544ec2d{/stages/pool,null,AVAILABLE} | |
| [2017-01-31 16:59:43,814] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@280c0341{/stages/pool/json,null,AVAILABLE} | |
| [2017-01-31 16:59:43,814] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@5573bbef{/storage,null,AVAILABLE} | |
| [2017-01-31 16:59:43,814] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@5f4c0904{/storage/json,null,AVAILABLE} | |
| [2017-01-31 16:59:43,814] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@3d0bb344{/storage/rdd,null,AVAILABLE} | |
| [2017-01-31 16:59:43,814] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@7555b9af{/storage/rdd/json,null,AVAILABLE} | |
| [2017-01-31 16:59:43,814] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@3d31c829{/environment,null,AVAILABLE} | |
| [2017-01-31 16:59:43,815] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@28ae2a2{/environment/json,null,AVAILABLE} | |
| [2017-01-31 16:59:43,815] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@62fbf024{/executors,null,AVAILABLE} | |
| [2017-01-31 16:59:43,815] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@3a3ba515{/executors/json,null,AVAILABLE} | |
| [2017-01-31 16:59:43,815] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@9f852fb{/executors/threadDump,null,AVAILABLE} | |
| [2017-01-31 16:59:43,815] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@5cb22e94{/executors/threadDump/json,null,AVAILABLE} | |
| [2017-01-31 16:59:43,823] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@61758ff6{/static,null,AVAILABLE} | |
| [2017-01-31 16:59:43,824] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@6818dc0{/,null,AVAILABLE} | |
| [2017-01-31 16:59:43,824] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@4da56861{/api,null,AVAILABLE} | |
| [2017-01-31 16:59:43,824] INFO ver.handler.ContextHandler [] [akka://JobServer/user/context-supervisor/test-context] - Started o.s.j.s.ServletContextHandler@ae29cf{/stages/stage/kill,null,AVAILABLE} | |
| [2017-01-31 16:59:43,829] INFO tty.server.ServerConnector [] [akka://JobServer/user/context-supervisor/test-context] - Started ServerConnector@2198a9a0{HTTP/1.1}{0.0.0.0:42123} | |
| [2017-01-31 16:59:43,829] INFO roject.jetty.server.Server [] [akka://JobServer/user/context-supervisor/test-context] - Started @114621ms | |
| [2017-01-31 16:59:43,829] INFO rg.apache.spark.util.Utils [] [akka://JobServer/user/context-supervisor/test-context] - Successfully started service 'SparkUI' on port 42123. | |
| [2017-01-31 16:59:43,831] INFO rg.apache.spark.ui.SparkUI [] [akka://JobServer/user/context-supervisor/test-context] - Bound SparkUI to 0.0.0.0, and started at http://172.17.0.2:42123 | |
| [2017-01-31 16:59:43,854] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.datastax.spark_spark-cassandra-connector_2.11-2.0.0-M3.jar at spark://172.17.0.2:45783/jars/com.datastax.spark_spark-cassandra-connector_2.11-2.0.0-M3.jar with timestamp 1485881983853 | |
| [2017-01-31 16:59:43,854] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.github.sstone_amqp-client_2.11-1.5.jar at spark://172.17.0.2:45783/jars/com.github.sstone_amqp-client_2.11-1.5.jar with timestamp 1485881983854 | |
| [2017-01-31 16:59:43,854] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.rabbitmq_amqp-client-4.0.1.jar at spark://172.17.0.2:45783/jars/com.rabbitmq_amqp-client-4.0.1.jar with timestamp 1485881983854 | |
| [2017-01-31 16:59:43,854] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.typesafe.akka_akka-actor_2.11-2.4.16.jar at spark://172.17.0.2:45783/jars/com.typesafe.akka_akka-actor_2.11-2.4.16.jar with timestamp 1485881983854 | |
| [2017-01-31 16:59:43,854] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.github.nscala-time_nscala-time_2.11-2.16.0.jar at spark://172.17.0.2:45783/jars/com.github.nscala-time_nscala-time_2.11-2.16.0.jar with timestamp 1485881983854 | |
| [2017-01-31 16:59:43,854] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.fasterxml.jackson.core_jackson-core-2.8.6.jar at spark://172.17.0.2:45783/jars/com.fasterxml.jackson.core_jackson-core-2.8.6.jar with timestamp 1485881983854 | |
| [2017-01-31 16:59:43,855] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.fasterxml.jackson.core_jackson-databind-2.8.6.jar at spark://172.17.0.2:45783/jars/com.fasterxml.jackson.core_jackson-databind-2.8.6.jar with timestamp 1485881983854 | |
| [2017-01-31 16:59:43,855] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.fasterxml.jackson.module_jackson-module-scala_2.11-2.8.6.jar at spark://172.17.0.2:45783/jars/com.fasterxml.jackson.module_jackson-module-scala_2.11-2.8.6.jar with timestamp 1485881983855 | |
| [2017-01-31 16:59:43,855] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/org.scalaj_scalaj-http_2.11-2.3.0.jar at spark://172.17.0.2:45783/jars/org.scalaj_scalaj-http_2.11-2.3.0.jar with timestamp 1485881983855 | |
| [2017-01-31 16:59:43,855] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/org.elasticsearch_elasticsearch-spark_2.11-2.4.4.jar at spark://172.17.0.2:45783/jars/org.elasticsearch_elasticsearch-spark_2.11-2.4.4.jar with timestamp 1485881983855 | |
| [2017-01-31 16:59:43,855] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/io.netty_netty-all-4.0.33.Final.jar at spark://172.17.0.2:45783/jars/io.netty_netty-all-4.0.33.Final.jar with timestamp 1485881983855 | |
| [2017-01-31 16:59:43,855] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/commons-beanutils_commons-beanutils-1.8.0.jar at spark://172.17.0.2:45783/jars/commons-beanutils_commons-beanutils-1.8.0.jar with timestamp 1485881983855 | |
| [2017-01-31 16:59:43,855] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.twitter_jsr166e-1.1.0.jar at spark://172.17.0.2:45783/jars/com.twitter_jsr166e-1.1.0.jar with timestamp 1485881983855 | |
| [2017-01-31 16:59:43,855] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/org.joda_joda-convert-1.2.jar at spark://172.17.0.2:45783/jars/org.joda_joda-convert-1.2.jar with timestamp 1485881983855 | |
| [2017-01-31 16:59:43,855] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/org.scala-lang_scala-reflect-2.11.8.jar at spark://172.17.0.2:45783/jars/org.scala-lang_scala-reflect-2.11.8.jar with timestamp 1485881983855 | |
| [2017-01-31 16:59:43,855] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/org.slf4j_slf4j-api-1.7.21.jar at spark://172.17.0.2:45783/jars/org.slf4j_slf4j-api-1.7.21.jar with timestamp 1485881983855 | |
| [2017-01-31 16:59:43,856] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.typesafe_config-1.3.0.jar at spark://172.17.0.2:45783/jars/com.typesafe_config-1.3.0.jar with timestamp 1485881983855 | |
| [2017-01-31 16:59:43,856] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/org.scala-lang.modules_scala-java8-compat_2.11-0.7.0.jar at spark://172.17.0.2:45783/jars/org.scala-lang.modules_scala-java8-compat_2.11-0.7.0.jar with timestamp 1485881983856 | |
| [2017-01-31 16:59:43,856] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/joda-time_joda-time-2.9.7.jar at spark://172.17.0.2:45783/jars/joda-time_joda-time-2.9.7.jar with timestamp 1485881983856 | |
| [2017-01-31 16:59:43,856] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.fasterxml.jackson.core_jackson-annotations-2.8.6.jar at spark://172.17.0.2:45783/jars/com.fasterxml.jackson.core_jackson-annotations-2.8.6.jar with timestamp 1485881983856 | |
| [2017-01-31 16:59:43,856] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.fasterxml.jackson.module_jackson-module-paranamer-2.8.6.jar at spark://172.17.0.2:45783/jars/com.fasterxml.jackson.module_jackson-module-paranamer-2.8.6.jar with timestamp 1485881983856 | |
| [2017-01-31 16:59:43,856] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/root/.ivy2/jars/com.thoughtworks.paranamer_paranamer-2.8.jar at spark://172.17.0.2:45783/jars/com.thoughtworks.paranamer_paranamer-2.8.jar with timestamp 1485881983856 | |
| [2017-01-31 16:59:43,856] INFO parkContextFactory$$anon$1 [] [akka://JobServer/user/context-supervisor/test-context] - Added JAR file:/app/spark-job-server.jar at spark://172.17.0.2:45783/jars/spark-job-server.jar with timestamp 1485881983856 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment