Skip to content

Instantly share code, notes, and snippets.

@wyukawa
Created March 13, 2016 11:21
Show Gist options
  • Select an option

  • Save wyukawa/b55be7a5619654e9c1ae to your computer and use it in GitHub Desktop.

Select an option

Save wyukawa/b55be7a5619654e9c1ae to your computer and use it in GitHub Desktop.
2016-03-13 13:15:58,362 INFO [HiveServer2-Handler-Pool: Thread-98838]: mapreduce.JobSubmitter (JobSubmitter.java:printTokens(287)) - Submitting tokens for job: job_1455848418011_44042
2016-03-13 13:15:58,568 INFO [HiveServer2-Handler-Pool: Thread-98838]: impl.YarnClientImpl (YarnClientImpl.java:submitApplication(274)) - Submitted application application_1455848418011_44042
2016-03-13 13:15:58,569 INFO [HiveServer2-Handler-Pool: Thread-98838]: mapreduce.Job (Job.java:submit(1294)) - The url to track the job: http://resourcemanager:port/proxy/application_1455848418011_44042/
2016-03-13 13:15:58,569 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - Starting Job = job_1455848418011_44042, Tracking URL = http://resourcemanager:port/proxy/application_1455848418011_44042/
2016-03-13 13:15:58,569 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - Kill Command = /usr/hdp/2.3.4.0-3485/hadoop/bin/hadoop job -kill job_1455848418011_44042
2016-03-13 13:16:02,725 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - Hadoop job information for Stage-6: number of mappers: 4; number of reducers: 0
2016-03-13 13:16:02,745 WARN [HiveServer2-Handler-Pool: Thread-98838]: mapreduce.Counters (AbstractCounters.java:getGroup(234)) - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2016-03-13 13:16:02,745 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:02,745 Stage-6 map = 0%, reduce = 0%
2016-03-13 13:16:07,846 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:07,846 Stage-6 map = 25%, reduce = 0%, Cumulative CPU 3.47 sec
2016-03-13 13:16:12,943 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:12,943 Stage-6 map = 28%, reduce = 0%, Cumulative CPU 38.74 sec
2016-03-13 13:16:15,999 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:15,999 Stage-6 map = 30%, reduce = 0%, Cumulative CPU 49.18 sec
2016-03-13 13:16:19,067 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:19,067 Stage-6 map = 32%, reduce = 0%, Cumulative CPU 59.48 sec
2016-03-13 13:16:19,099 INFO [HiveServer2-Handler-Pool: Thread-61]: thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(294)) - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V8
2016-03-13 13:16:19,101 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created local directory: /tmp/9411433c-2c09-41b9-b289-ed5539d39ca7_resources
2016-03-13 13:16:19,111 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created HDFS directory: /tmp/hive/hive/9411433c-2c09-41b9-b289-ed5539d39ca7
2016-03-13 13:16:19,111 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created local directory: /tmp/hive/9411433c-2c09-41b9-b289-ed5539d39ca7
2016-03-13 13:16:19,113 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created HDFS directory: /tmp/hive/hive/9411433c-2c09-41b9-b289-ed5539d39ca7/_tmp_space.db
2016-03-13 13:16:19,113 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:setOperationLogSessionDir(248)) - Operation log session directory is created: /tmp/hive/operation_logs/9411433c-2c09-41b9-b289-ed5539d39ca7
2016-03-13 13:16:19,178 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:acquire(295)) - We are setting the hadoop caller context to 9411433c-2c09-41b9-b289-ed5539d39ca7 for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:16:19,178 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:release(309)) - We are resetting the hadoop caller context for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:16:19,186 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:acquire(295)) - We are setting the hadoop caller context to 9411433c-2c09-41b9-b289-ed5539d39ca7 for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:16:19,186 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:release(309)) - We are resetting the hadoop caller context for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:16:19,202 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:acquire(295)) - We are setting the hadoop caller context to 9411433c-2c09-41b9-b289-ed5539d39ca7 for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:16:19,216 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:release(309)) - We are resetting the hadoop caller context for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:16:22,124 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:22,124 Stage-6 map = 34%, reduce = 0%, Cumulative CPU 69.47 sec
2016-03-13 13:16:25,178 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:25,178 Stage-6 map = 36%, reduce = 0%, Cumulative CPU 79.5 sec
2016-03-13 13:16:28,231 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:28,231 Stage-6 map = 38%, reduce = 0%, Cumulative CPU 89.76 sec
2016-03-13 13:16:31,285 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:31,285 Stage-6 map = 40%, reduce = 0%, Cumulative CPU 99.69 sec
2016-03-13 13:16:34,337 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:34,337 Stage-6 map = 42%, reduce = 0%, Cumulative CPU 109.67 sec
2016-03-13 13:16:37,389 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:37,389 Stage-6 map = 44%, reduce = 0%, Cumulative CPU 120.24 sec
2016-03-13 13:16:40,443 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:40,443 Stage-6 map = 47%, reduce = 0%, Cumulative CPU 130.08 sec
2016-03-13 13:16:43,495 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:43,495 Stage-6 map = 49%, reduce = 0%, Cumulative CPU 139.82 sec
2016-03-13 13:16:46,547 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:46,547 Stage-6 map = 51%, reduce = 0%, Cumulative CPU 149.61 sec
2016-03-13 13:16:49,598 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:49,598 Stage-6 map = 53%, reduce = 0%, Cumulative CPU 159.48 sec
2016-03-13 13:16:52,651 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:52,651 Stage-6 map = 55%, reduce = 0%, Cumulative CPU 169.21 sec
2016-03-13 13:16:55,703 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:55,703 Stage-6 map = 57%, reduce = 0%, Cumulative CPU 178.92 sec
2016-03-13 13:16:58,758 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:16:58,758 Stage-6 map = 59%, reduce = 0%, Cumulative CPU 188.67 sec
2016-03-13 13:17:01,811 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:01,811 Stage-6 map = 61%, reduce = 0%, Cumulative CPU 198.39 sec
2016-03-13 13:17:04,862 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:04,862 Stage-6 map = 63%, reduce = 0%, Cumulative CPU 208.2 sec
2016-03-13 13:17:07,912 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:07,912 Stage-6 map = 65%, reduce = 0%, Cumulative CPU 218.1 sec
2016-03-13 13:17:10,965 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:10,965 Stage-6 map = 67%, reduce = 0%, Cumulative CPU 228.1 sec
2016-03-13 13:17:12,998 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:12,998 Stage-6 map = 68%, reduce = 0%, Cumulative CPU 230.45 sec
2016-03-13 13:17:17,066 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:17,066 Stage-6 map = 69%, reduce = 0%, Cumulative CPU 243.47 sec
2016-03-13 13:17:19,089 INFO [HiveServer2-Handler-Pool: Thread-61]: thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(294)) - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V8
2016-03-13 13:17:19,204 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created local directory: /tmp/fca12264-09b7-4746-b963-d37136151278_resources
2016-03-13 13:17:19,206 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created HDFS directory: /tmp/hive/hive/fca12264-09b7-4746-b963-d37136151278
2016-03-13 13:17:19,206 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created local directory: /tmp/hive/fca12264-09b7-4746-b963-d37136151278
2016-03-13 13:17:19,208 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created HDFS directory: /tmp/hive/hive/fca12264-09b7-4746-b963-d37136151278/_tmp_space.db
2016-03-13 13:17:19,208 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:setOperationLogSessionDir(248)) - Operation log session directory is created: /tmp/hive/operation_logs/fca12264-09b7-4746-b963-d37136151278
2016-03-13 13:17:19,215 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:19,215 Stage-6 map = 70%, reduce = 0%, Cumulative CPU 246.75 sec
2016-03-13 13:17:19,271 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:acquire(295)) - We are setting the hadoop caller context to fca12264-09b7-4746-b963-d37136151278 for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:17:19,271 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:release(309)) - We are resetting the hadoop caller context for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:17:19,278 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:acquire(295)) - We are setting the hadoop caller context to fca12264-09b7-4746-b963-d37136151278 for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:17:19,278 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:release(309)) - We are resetting the hadoop caller context for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:17:19,294 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:acquire(295)) - We are setting the hadoop caller context to fca12264-09b7-4746-b963-d37136151278 for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:17:19,295 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:release(309)) - We are resetting the hadoop caller context for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:17:22,271 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:22,271 Stage-6 map = 71%, reduce = 0%, Cumulative CPU 253.29 sec
2016-03-13 13:17:26,338 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:26,338 Stage-6 map = 72%, reduce = 0%, Cumulative CPU 263.01 sec
2016-03-13 13:17:29,387 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:29,387 Stage-6 map = 73%, reduce = 0%, Cumulative CPU 269.54 sec
2016-03-13 13:17:34,471 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:34,471 Stage-6 map = 74%, reduce = 0%, Cumulative CPU 279.38 sec
2016-03-13 13:17:37,521 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:37,521 Stage-6 map = 75%, reduce = 0%, Cumulative CPU 285.89 sec
2016-03-13 13:17:40,572 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:40,572 Stage-6 map = 76%, reduce = 0%, Cumulative CPU 295.65 sec
2016-03-13 13:17:43,621 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:43,621 Stage-6 map = 77%, reduce = 0%, Cumulative CPU 302.13 sec
2016-03-13 13:17:46,671 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:46,671 Stage-6 map = 78%, reduce = 0%, Cumulative CPU 308.64 sec
2016-03-13 13:17:49,720 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:49,720 Stage-6 map = 79%, reduce = 0%, Cumulative CPU 315.19 sec
2016-03-13 13:17:55,819 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:55,819 Stage-6 map = 80%, reduce = 0%, Cumulative CPU 329.65 sec
2016-03-13 13:17:58,868 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:17:58,868 Stage-6 map = 81%, reduce = 0%, Cumulative CPU 336.37 sec
2016-03-13 13:18:01,918 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:01,918 Stage-6 map = 82%, reduce = 0%, Cumulative CPU 343.02 sec
2016-03-13 13:18:04,967 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:04,967 Stage-6 map = 83%, reduce = 0%, Cumulative CPU 349.61 sec
2016-03-13 13:18:08,016 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:08,016 Stage-6 map = 84%, reduce = 0%, Cumulative CPU 356.14 sec
2016-03-13 13:18:11,069 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:11,069 Stage-6 map = 85%, reduce = 0%, Cumulative CPU 362.82 sec
2016-03-13 13:18:17,220 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:17,220 Stage-6 map = 86%, reduce = 0%, Cumulative CPU 376.14 sec
2016-03-13 13:18:19,097 INFO [HiveServer2-Handler-Pool: Thread-61]: thrift.ThriftCLIService (ThriftCLIService.java:OpenSession(294)) - Client protocol version: HIVE_CLI_SERVICE_PROTOCOL_V8
2016-03-13 13:18:19,217 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created local directory: /tmp/27d18ae9-e5ae-4d48-ac28-6f957f0b4420_resources
2016-03-13 13:18:19,218 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created HDFS directory: /tmp/hive/hive/27d18ae9-e5ae-4d48-ac28-6f957f0b4420
2016-03-13 13:18:19,218 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created local directory: /tmp/hive/27d18ae9-e5ae-4d48-ac28-6f957f0b4420
2016-03-13 13:18:19,219 INFO [HiveServer2-Handler-Pool: Thread-61]: session.SessionState (SessionState.java:createPath(613)) - Created HDFS directory: /tmp/hive/hive/27d18ae9-e5ae-4d48-ac28-6f957f0b4420/_tmp_space.db
2016-03-13 13:18:19,220 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:setOperationLogSessionDir(248)) - Operation log session directory is created: /tmp/hive/operation_logs/27d18ae9-e5ae-4d48-ac28-6f957f0b4420
2016-03-13 13:18:19,283 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:acquire(295)) - We are setting the hadoop caller context to 27d18ae9-e5ae-4d48-ac28-6f957f0b4420 for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:18:19,283 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:release(309)) - We are resetting the hadoop caller context for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:18:19,290 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:acquire(295)) - We are setting the hadoop caller context to 27d18ae9-e5ae-4d48-ac28-6f957f0b4420 for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:18:19,290 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:release(309)) - We are resetting the hadoop caller context for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:18:19,306 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:acquire(295)) - We are setting the hadoop caller context to 27d18ae9-e5ae-4d48-ac28-6f957f0b4420 for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:18:19,308 INFO [HiveServer2-Handler-Pool: Thread-61]: session.HiveSessionImpl (HiveSessionImpl.java:release(309)) - We are resetting the hadoop caller context for thread HiveServer2-Handler-Pool: Thread-61
2016-03-13 13:18:20,272 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:20,272 Stage-6 map = 87%, reduce = 0%, Cumulative CPU 382.71 sec
2016-03-13 13:18:23,323 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:23,323 Stage-6 map = 88%, reduce = 0%, Cumulative CPU 389.21 sec
2016-03-13 13:18:26,371 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:26,371 Stage-6 map = 89%, reduce = 0%, Cumulative CPU 395.73 sec
2016-03-13 13:18:29,420 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:29,420 Stage-6 map = 90%, reduce = 0%, Cumulative CPU 402.29 sec
2016-03-13 13:18:32,469 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:32,469 Stage-6 map = 91%, reduce = 0%, Cumulative CPU 408.9 sec
2016-03-13 13:18:38,566 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:38,566 Stage-6 map = 92%, reduce = 0%, Cumulative CPU 422.1 sec
2016-03-13 13:18:41,616 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:41,616 Stage-6 map = 93%, reduce = 0%, Cumulative CPU 428.63 sec
2016-03-13 13:18:44,664 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:44,664 Stage-6 map = 94%, reduce = 0%, Cumulative CPU 435.18 sec
2016-03-13 13:18:47,712 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:47,712 Stage-6 map = 95%, reduce = 0%, Cumulative CPU 441.78 sec
2016-03-13 13:18:50,761 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:50,761 Stage-6 map = 96%, reduce = 0%, Cumulative CPU 448.35 sec
2016-03-13 13:18:53,811 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:53,811 Stage-6 map = 97%, reduce = 0%, Cumulative CPU 454.95 sec
2016-03-13 13:18:59,909 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:18:59,909 Stage-6 map = 98%, reduce = 0%, Cumulative CPU 468.14 sec
2016-03-13 13:19:02,958 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:19:02,958 Stage-6 map = 99%, reduce = 0%, Cumulative CPU 474.67 sec
2016-03-13 13:19:08,040 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - 2016-03-13 13:19:08,040 Stage-6 map = 100%, reduce = 0%, Cumulative CPU 482.54 sec
2016-03-13 13:19:09,061 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - MapReduce Total cumulative CPU time: 8 minutes 2 seconds 540 msec
2016-03-13 13:19:09,069 INFO [HiveServer2-Handler-Pool: Thread-98838]: exec.Task (SessionState.java:printInfo(923)) - Ended Job = job_1455848418011_44042
2016-03-13 13:19:09,936 INFO [HiveServer2-Handler-Pool: Thread-98838]: retry.RetryInvocationHandler (RetryInvocationHandler.java:invoke(144)) - Exception while invoking getFileInfo of class ClientNamenodeProtocolTranslatorPB over namenode.host:8020. Trying to fail over immediately.
java.io.IOException: Failed on local exception: java.io.IOException: Couldn't set up IO streams; Host Details : local host is: "hiveserver.host"; destination host is: "namenode.host":8020;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:773)
at org.apache.hadoop.ipc.Client.call(Client.java:1431)
at org.apache.hadoop.ipc.Client.call(Client.java:1358)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
at com.sun.proxy.$Proxy15.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771)
at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:252)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
at com.sun.proxy.$Proxy16.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2116)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1315)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1311)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1311)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1424)
at org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:268)
at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:455)
at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:89)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1703)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1460)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1237)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1101)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1096)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:154)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:183)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:257)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:410)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(HiveSessionImpl.java:391)
at org.apache.hive.service.cli.CLIService.executeStatement(CLIService.java:261)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:486)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1313)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1298)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Couldn't set up IO streams
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:791)
at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:373)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1493)
at org.apache.hadoop.ipc.Client.call(Client.java:1397)
... 42 more
Caused by: java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:784)
... 45 more
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment