Created
June 11, 2014 14:40
-
-
Save libratiger/e5f0ea20efb02052cdef to your computer and use it in GitHub Desktop.
Kmeans algorithm driver.log
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 14/06/11 14:34:46 INFO SparkContext: Job finished: collect at KMeans.scala:232, took 9.079383951 s | |
| 14/06/11 14:34:46 INFO SparkContext: Starting job: collectAsMap at KMeans.scala:224 | |
| 14/06/11 14:34:46 INFO DAGScheduler: Registering RDD 15 (reduceByKey at KMeans.scala:224) | |
| 14/06/11 14:34:46 INFO DAGScheduler: Got job 4 (collectAsMap at KMeans.scala:224) with 9000 output partitions (allowLocal=false) | |
| 14/06/11 14:34:46 INFO DAGScheduler: Final stage: Stage 9 (collectAsMap at KMeans.scala:224) | |
| 14/06/11 14:34:46 INFO DAGScheduler: Parents of final stage: List(Stage 10) | |
| 14/06/11 14:34:46 INFO DAGScheduler: Missing parents: List(Stage 10) | |
| 14/06/11 14:34:46 INFO DAGScheduler: Submitting Stage 10 (MapPartitionsRDD[15] at reduceByKey at KMeans.scala:224), which has no missing parents | |
| 14/06/11 14:34:50 INFO DAGScheduler: Submitting 1200 missing tasks from Stage 10 (MapPartitionsRDD[15] at reduceByKey at KMeans.scala:224) | |
| 14/06/11 14:34:50 INFO TaskSchedulerImpl: Adding task set 10.0 with 1200 tasks | |
| 14/06/11 14:34:50 INFO TaskSetManager: Starting task 10.0:0 as TID 13900 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:51 INFO TaskSetManager: Serialized task 10.0:0 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:51 INFO TaskSetManager: Starting task 10.0:1 as TID 13901 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:51 INFO TaskSetManager: Serialized task 10.0:1 as 52128357 bytes in 108 ms | |
| 14/06/11 14:34:51 INFO TaskSetManager: Starting task 10.0:2 as TID 13902 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:51 INFO TaskSetManager: Serialized task 10.0:2 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:51 INFO TaskSetManager: Starting task 10.0:3 as TID 13903 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:51 INFO TaskSetManager: Serialized task 10.0:3 as 52128357 bytes in 107 ms | |
| 14/06/11 14:34:51 INFO TaskSetManager: Starting task 10.0:4 as TID 13904 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:51 INFO TaskSetManager: Serialized task 10.0:4 as 52128357 bytes in 269 ms | |
| 14/06/11 14:34:51 INFO TaskSetManager: Starting task 10.0:5 as TID 13905 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:51 INFO TaskSetManager: Serialized task 10.0:5 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:51 INFO TaskSetManager: Starting task 10.0:6 as TID 13906 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:51 INFO TaskSetManager: Serialized task 10.0:6 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:51 INFO TaskSetManager: Starting task 10.0:7 as TID 13907 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:52 INFO TaskSetManager: Serialized task 10.0:7 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:52 INFO TaskSetManager: Starting task 10.0:8 as TID 13908 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:52 INFO TaskSetManager: Serialized task 10.0:8 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:52 INFO TaskSetManager: Starting task 10.0:9 as TID 13909 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:52 INFO TaskSetManager: Serialized task 10.0:9 as 52128357 bytes in 111 ms | |
| 14/06/11 14:34:52 INFO TaskSetManager: Starting task 10.0:10 as TID 13910 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:52 INFO TaskSetManager: Serialized task 10.0:10 as 52128357 bytes in 108 ms | |
| 14/06/11 14:34:52 INFO TaskSetManager: Starting task 10.0:11 as TID 13911 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:52 INFO TaskSetManager: Serialized task 10.0:11 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:52 INFO TaskSetManager: Starting task 10.0:12 as TID 13912 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:52 INFO TaskSetManager: Serialized task 10.0:12 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:52 INFO TaskSetManager: Starting task 10.0:13 as TID 13913 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:52 INFO TaskSetManager: Serialized task 10.0:13 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:52 INFO TaskSetManager: Starting task 10.0:14 as TID 13914 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:52 INFO TaskSetManager: Serialized task 10.0:14 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:52 INFO TaskSetManager: Starting task 10.0:15 as TID 13915 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:52 INFO TaskSetManager: Serialized task 10.0:15 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:52 INFO TaskSetManager: Starting task 10.0:16 as TID 13916 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:53 INFO TaskSetManager: Serialized task 10.0:16 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:53 INFO TaskSetManager: Starting task 10.0:17 as TID 13917 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:53 INFO TaskSetManager: Serialized task 10.0:17 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:53 INFO TaskSetManager: Starting task 10.0:18 as TID 13918 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:53 INFO TaskSetManager: Serialized task 10.0:18 as 52128357 bytes in 108 ms | |
| 14/06/11 14:34:53 INFO TaskSetManager: Starting task 10.0:19 as TID 13919 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:53 INFO TaskSetManager: Serialized task 10.0:19 as 52128357 bytes in 108 ms | |
| 14/06/11 14:34:53 INFO TaskSetManager: Starting task 10.0:20 as TID 13920 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:53 INFO TaskSetManager: Serialized task 10.0:20 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:53 INFO TaskSetManager: Starting task 10.0:21 as TID 13921 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:53 INFO TaskSetManager: Serialized task 10.0:21 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:53 INFO TaskSetManager: Starting task 10.0:22 as TID 13922 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:53 INFO TaskSetManager: Serialized task 10.0:22 as 52128357 bytes in 326 ms | |
| 14/06/11 14:34:53 INFO TaskSetManager: Starting task 10.0:23 as TID 13923 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:53 INFO TaskSetManager: Serialized task 10.0:23 as 52128357 bytes in 108 ms | |
| 14/06/11 14:34:53 INFO TaskSetManager: Starting task 10.0:24 as TID 13924 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:54 INFO TaskSetManager: Serialized task 10.0:24 as 52128357 bytes in 102 ms | |
| 14/06/11 14:34:54 INFO TaskSetManager: Starting task 10.0:25 as TID 13925 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:54 INFO TaskSetManager: Serialized task 10.0:25 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:54 INFO TaskSetManager: Starting task 10.0:26 as TID 13926 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:54 INFO TaskSetManager: Serialized task 10.0:26 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:54 INFO TaskSetManager: Starting task 10.0:27 as TID 13927 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:54 INFO TaskSetManager: Serialized task 10.0:27 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:54 INFO TaskSetManager: Starting task 10.0:28 as TID 13928 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:54 INFO TaskSetManager: Serialized task 10.0:28 as 52128357 bytes in 109 ms | |
| 14/06/11 14:34:54 INFO TaskSetManager: Starting task 10.0:29 as TID 13929 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:54 INFO TaskSetManager: Serialized task 10.0:29 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:54 INFO TaskSetManager: Starting task 10.0:30 as TID 13930 on executor 2: husky004 (PROCESS_LOCAL) | |
| 14/06/11 14:34:54 INFO TaskSetManager: Serialized task 10.0:30 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:54 INFO TaskSetManager: Starting task 10.0:31 as TID 13931 on executor 1: husky003 (PROCESS_LOCAL) | |
| 14/06/11 14:34:54 INFO TaskSetManager: Serialized task 10.0:31 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:54 INFO TaskSetManager: Starting task 10.0:32 as TID 13932 on executor 0: husky002 (PROCESS_LOCAL) | |
| 14/06/11 14:34:54 INFO TaskSetManager: Serialized task 10.0:32 as 52128357 bytes in 110 ms | |
| 14/06/11 14:34:55 INFO SparkDeploySchedulerBackend: Executor 2 disconnected, so removing it | |
| 14/06/11 14:34:55 ERROR TaskSchedulerImpl: Lost executor 2 on husky004: remote Akka client disassociated | |
| 14/06/11 14:34:55 INFO TaskSetManager: Re-queueing tasks for 2 from TaskSet 10.0 | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13927 (task 10.0:27) | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13918 (task 10.0:18) | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13909 (task 10.0:9) | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13900 (task 10.0:0) | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13930 (task 10.0:30) | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13912 (task 10.0:12) | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13903 (task 10.0:3) | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13921 (task 10.0:21) | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13906 (task 10.0:6) | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13924 (task 10.0:24) | |
| 14/06/11 14:34:55 WARN TaskSetManager: Lost TID 13915 (task 10.0:15) | |
| 14/06/11 14:34:55 INFO SparkDeploySchedulerBackend: Executor 0 disconnected, so removing it | |
| 14/06/11 14:34:55 ERROR TaskSchedulerImpl: Lost executor 0 on husky002: remote Akka client disassociated | |
| 14/06/11 14:34:55 INFO TaskSetManager: Re-queueing tasks for 0 from TaskSet 10.0 | |
| 14/06/11 14:34:55 INFO DAGScheduler: Executor lost: 2 (epoch 2) | |
| 14/06/11 14:34:55 INFO Stage: Stage 11 is now unavailable on executor 2 (65/100, false) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment