Skip to content

Instantly share code, notes, and snippets.

@marians
Last active December 17, 2015 14:09
Show Gist options
  • Select an option

  • Save marians/5622706 to your computer and use it in GitHub Desktop.

Select an option

Save marians/5622706 to your computer and use it in GitHub Desktop.
Hadoop problem logs as of 2013-05-21
2013-05-21 21:32:20,681 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50144, bytes: 1777, op: HDFS_READ, cliID: DFSClient_attempt_201305152304_0004_m_000000_1_722979034_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_-4766979604280382827_1040, duration: 240000
2013-05-21 21:32:20,804 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50145, bytes: 7517, op: HDFS_READ, cliID: DFSClient_attempt_201305152304_0004_m_000000_1_722979034_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_3289312915423223722_1003, duration: 736000
2013-05-21 21:35:59,387 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_-4719156887958776590_1064 src: /127.0.0.1:50215 dest: /127.0.0.1:50010
2013-05-21 21:35:59,486 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50215, dest: /127.0.0.1:50010, bytes: 6755373, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_12778723_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_-4719156887958776590_1064, duration: 97388000
2013-05-21 21:35:59,486 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_-4719156887958776590_1064 terminating
2013-05-21 21:35:59,517 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_7352355141257109301_1065 src: /127.0.0.1:50216 dest: /127.0.0.1:50010
2013-05-21 21:35:59,519 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50216, dest: /127.0.0.1:50010, bytes: 371, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_12778723_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_7352355141257109301_1065, duration: 490000
2013-05-21 21:35:59,519 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_7352355141257109301_1065 terminating
2013-05-21 21:35:59,524 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_-5649321702225883188_1066 src: /127.0.0.1:50217 dest: /127.0.0.1:50010
2013-05-21 21:35:59,526 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50217, dest: /127.0.0.1:50010, bytes: 30, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_12778723_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_-5649321702225883188_1066, duration: 407000
2013-05-21 21:35:59,526 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_-5649321702225883188_1066 terminating
2013-05-21 21:35:59,584 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_1680627007369647451_1067 src: /127.0.0.1:50218 dest: /127.0.0.1:50010
2013-05-21 21:35:59,591 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50218, dest: /127.0.0.1:50010, bytes: 80973, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_12778723_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_1680627007369647451_1067, duration: 5922000
2013-05-21 21:35:59,591 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_1680627007369647451_1067 terminating
2013-05-21 21:35:59,599 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50220, bytes: 81609, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-187738396_35, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_1680627007369647451_1067, duration: 352000
2013-05-21 21:35:59,630 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_-5625365488277704265_1068 src: /127.0.0.1:50222 dest: /127.0.0.1:50010
2013-05-21 21:35:59,632 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50222, dest: /127.0.0.1:50010, bytes: 122, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-12013881_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_-5625365488277704265_1068, duration: 546000
2013-05-21 21:35:59,632 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_-5625365488277704265_1068 terminating
2013-05-21 21:35:59,681 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_-1450040174609345042_1070 src: /127.0.0.1:50223 dest: /127.0.0.1:50010
2013-05-21 21:35:59,685 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50223, dest: /127.0.0.1:50010, bytes: 103958, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-187738396_35, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_-1450040174609345042_1070, duration: 3020000
2013-05-21 21:35:59,685 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_-1450040174609345042_1070 terminating
2013-05-21 21:35:59,690 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_9100288043730152482_1071 src: /127.0.0.1:50224 dest: /127.0.0.1:50010
2013-05-21 21:35:59,692 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50224, dest: /127.0.0.1:50010, bytes: 106, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-12013881_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_9100288043730152482_1071, duration: 572000
2013-05-21 21:35:59,692 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: PacketResponder 0 for block blk_9100288043730152482_1071 terminating
2013-05-21 21:35:59,696 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50225, bytes: 34, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-187738396_35, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_-5649321702225883188_1066, duration: 171000
2013-05-21 21:35:59,891 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50227, bytes: 110, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-883482191_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_9100288043730152482_1071, duration: 215000
2013-05-21 21:35:59,899 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50229, bytes: 81609, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1042326125_941, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_1680627007369647451_1067, duration: 463000
2013-05-21 21:36:00,010 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50230, bytes: 6808153, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_-1042326125_941, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_-4719156887958776590_1064, duration: 39913000
2013-05-21 21:36:03,820 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50235, bytes: 375, op: HDFS_READ, cliID: DFSClient_attempt_201305152304_0012_m_000000_0_-81035529_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_7352355141257109301_1065, duration: 165000
2013-05-21 21:36:05,123 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50236, bytes: 11553594, op: HDFS_READ, cliID: DFSClient_attempt_201305152304_0012_m_000000_0_-81035529_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_-2668867479214123552_1063, duration: 1058988000
2013-05-21 21:37:19,309 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50109, bytes: 1849344, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_12778723_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_-2668867479214123552_1063, duration: 490609943000
2013-05-21 21:37:19,310 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(127.0.0.1:50010, storageID=DS-2043951618-192.168.0.102-50010-1368635777558, infoPort=50075, ipcPort=50020):Got exception while serving blk_-2668867479214123552_1063 to /127.0.0.1:
java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/127.0.0.1:50010 remote=/127.0.0.1:50109]
at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246)
at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:159)
at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:198)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks(BlockSender.java:392)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:490)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:202)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:104)
at java.lang.Thread.run(Thread.java:680)
2013-05-21 21:37:19,312 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(127.0.0.1:50010, storageID=DS-2043951618-192.168.0.102-50010-1368635777558, infoPort=50075, ipcPort=50020):DataXceiver
java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/127.0.0.1:50010 remote=/127.0.0.1:50109]
at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246)
at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:159)
at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:198)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks(BlockSender.java:392)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:490)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:202)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:104)
at java.lang.Thread.run(Thread.java:680)
2013-05-21 21:38:50,441 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /127.0.0.1:50010, dest: /127.0.0.1:50125, bytes: 1849344, op: HDFS_READ, cliID: DFSClient_NONMAPREDUCE_12778723_1, offset: 0, srvID: DS-2043951618-192.168.0.102-50010-1368635777558, blockid: blk_-2668867479214123552_1063, duration: 490606655000
2013-05-21 21:38:50,442 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(127.0.0.1:50010, storageID=DS-2043951618-192.168.0.102-50010-1368635777558, infoPort=50075, ipcPort=50020):Got exception while serving blk_-2668867479214123552_1063 to /127.0.0.1:
java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/127.0.0.1:50010 remote=/127.0.0.1:50125]
at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246)
at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:159)
at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:198)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks(BlockSender.java:392)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:490)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:202)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:104)
at java.lang.Thread.run(Thread.java:680)
2013-05-21 21:38:50,442 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(127.0.0.1:50010, storageID=DS-2043951618-192.168.0.102-50010-1368635777558, infoPort=50075, ipcPort=50020):DataXceiver
java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/127.0.0.1:50010 remote=/127.0.0.1:50125]
at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246)
at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:159)
at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:198)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks(BlockSender.java:392)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:490)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:202)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:104)
at java.lang.Thread.run(Thread.java:680)
2013-05-21 21:35:59,624 INFO org.apache.hadoop.mapred.JobInProgress: job_201305152304_0012: nMaps=1 nReduces=1 max=-1
2013-05-21 21:35:59,636 INFO org.apache.hadoop.mapred.JobTracker: Job job_201305152304_0012 added successfully for user 'hadoop' to queue 'default'
2013-05-21 21:35:59,636 INFO org.apache.hadoop.mapred.JobTracker: Initializing job_201305152304_0012
2013-05-21 21:35:59,636 INFO org.apache.hadoop.mapred.AuditLogger: USER=hadoop IP=127.0.0.1 OPERATION=SUBMIT_JOB TARGET=job_201305152304_0012 RESULT=SUCCESS
2013-05-21 21:35:59,636 INFO org.apache.hadoop.mapred.JobInProgress: Initializing job_201305152304_0012
2013-05-21 21:35:59,694 INFO org.apache.hadoop.mapred.JobInProgress: jobToken generated and stored with users keys in /Users/hadoop/tmpdir/mapred/system/job_201305152304_0012/jobToken
2013-05-21 21:35:59,696 INFO org.apache.hadoop.mapred.JobInProgress: Input size for job job_201305152304_0012 = 11464030. Number of splits = 1
2013-05-21 21:35:59,697 INFO org.apache.hadoop.mapred.JobInProgress: tip:task_201305152304_0012_m_000000 has split on node:/default-rack/192.168.0.102
2013-05-21 21:35:59,697 INFO org.apache.hadoop.mapred.JobInProgress: job_201305152304_0012 LOCALITY_WAIT_FACTOR=1.0
2013-05-21 21:35:59,697 INFO org.apache.hadoop.mapred.JobInProgress: Job job_201305152304_0012 initialized successfully with 1 map tasks and 1 reduce tasks.
2013-05-21 21:35:59,877 INFO org.apache.hadoop.mapred.JobTracker: Adding task (JOB_SETUP) 'attempt_201305152304_0012_m_000002_0' to tip task_201305152304_0012_m_000002, for tracker 'tracker_192.168.0.102:localhost/127.0.0.1:60527'
2013-05-21 21:36:02,594 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201305152304_0012_m_000002_0' has completed task_201305152304_0012_m_000002 successfully.
2013-05-21 21:36:02,595 INFO org.apache.hadoop.mapred.JobTracker: Adding task (MAP) 'attempt_201305152304_0012_m_000000_0' to tip task_201305152304_0012_m_000000, for tracker 'tracker_192.168.0.102:localhost/127.0.0.1:60527'
2013-05-21 21:36:02,596 INFO org.apache.hadoop.mapred.JobInProgress: Choosing data-local task task_201305152304_0012_m_000000
2013-05-21 21:36:05,612 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201305152304_0012_m_000000_0' has completed task_201305152304_0012_m_000000 successfully.
2013-05-21 21:36:05,613 INFO org.apache.hadoop.mapred.JobTracker: Adding task (REDUCE) 'attempt_201305152304_0012_r_000000_0' to tip task_201305152304_0012_r_000000, for tracker 'tracker_192.168.0.102:localhost/127.0.0.1:60527'
2013-05-21 21:51:32,565 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201305152304_0004_r_000000_2: Task attempt_201305152304_0004_r_000000_2 failed to report status for 603 seconds. Killing!
2013-05-21 21:51:32,565 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201305152304_0004_r_000000_2'
2013-05-21 21:51:32,565 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201305152304_0012_r_000000_0: Task attempt_201305152304_0012_r_000000_0 failed to report status for 603 seconds. Killing!
2013-05-21 21:35:56,148 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Number of transactions: 4 Total time for transactions(ms): 1Number of transact
ions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0
2013-05-21 21:35:59,375 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock: /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_201305
152304_0012/job.jar. blk_-4719156887958776590_1064
2013-05-21 21:35:59,486 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:50010 is added to blk_-4719156887
958776590_1064 size 6755373
2013-05-21 21:35:59,487 INFO org.apache.hadoop.hdfs.StateChange: Removing lease on file /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_201305152304_00
12/job.jar from client DFSClient_NONMAPREDUCE_12778723_1
2013-05-21 21:35:59,487 INFO org.apache.hadoop.hdfs.StateChange: DIR* NameSystem.completeFile: file /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_2013
05152304_0012/job.jar is closed by DFSClient_NONMAPREDUCE_12778723_1
2013-05-21 21:35:59,495 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Increasing replication for file /Users/hadoop/tmpdir/mapred/staging/hadoop/.st
aging/job_201305152304_0012/job.jar. New replication is 10
2013-05-21 21:35:59,514 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Increasing replication for file /Users/hadoop/tmpdir/mapred/staging/hadoop/.st
aging/job_201305152304_0012/job.split. New replication is 10
2013-05-21 21:35:59,516 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock: /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_201305
152304_0012/job.split. blk_7352355141257109301_1065
2013-05-21 21:35:59,519 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:50010 is added to blk_73523551412
57109301_1065 size 371
2013-05-21 21:35:59,519 INFO org.apache.hadoop.hdfs.StateChange: Removing lease on file /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_201305152304_00
12/job.split from client DFSClient_NONMAPREDUCE_12778723_1
2013-05-21 21:35:59,520 INFO org.apache.hadoop.hdfs.StateChange: DIR* NameSystem.completeFile: file /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_2013
05152304_0012/job.split is closed by DFSClient_NONMAPREDUCE_12778723_1
2013-05-21 21:35:59,523 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock: /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_201305
152304_0012/job.splitmetainfo. blk_-5649321702225883188_1066
2013-05-21 21:35:59,526 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:50010 is added to blk_-5649321702
225883188_1066 size 30
2013-05-21 21:35:59,526 INFO org.apache.hadoop.hdfs.StateChange: Removing lease on file /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_201305152304_00
12/job.splitmetainfo from client DFSClient_NONMAPREDUCE_12778723_1
2013-05-21 21:35:59,527 INFO org.apache.hadoop.hdfs.StateChange: DIR* NameSystem.completeFile: file /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_2013
05152304_0012/job.splitmetainfo is closed by DFSClient_NONMAPREDUCE_12778723_1
2013-05-21 21:35:59,583 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock: /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_201305
152304_0012/job.xml. blk_1680627007369647451_1067
2013-05-21 21:35:59,591 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:50010 is added to blk_16806270073
69647451_1067 size 80973
2013-05-21 21:35:59,592 INFO org.apache.hadoop.hdfs.StateChange: Removing lease on file /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_201305152304_00
12/job.xml from client DFSClient_NONMAPREDUCE_12778723_1
2013-05-21 21:35:59,592 INFO org.apache.hadoop.hdfs.StateChange: DIR* NameSystem.completeFile: file /Users/hadoop/tmpdir/mapred/staging/hadoop/.staging/job_201305152304_0012/job.xml is closed by DFSClient_NONMAPREDUCE_12778723_1
2013-05-21 21:35:59,629 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock: /Users/hadoop/tmpdir/mapred/system/job_201305152304_0012/job-info. blk_-5625365488277704265_1068
2013-05-21 21:35:59,632 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:50010 is added to blk_-5625365488277704265_1068 size 122
2013-05-21 21:35:59,635 INFO org.apache.hadoop.hdfs.StateChange: Removing lease on file /Users/hadoop/tmpdir/mapred/system/job_201305152304_0012/job-info from client DFSClient_NONMAPREDUCE_-12013881_1
2013-05-21 21:35:59,636 INFO org.apache.hadoop.hdfs.StateChange: DIR* NameSystem.completeFile: file /Users/hadoop/tmpdir/mapred/system/job_201305152304_0012/job-info is closed by DFSClient_NONMAPREDUCE_-12013881_1
2013-05-21 21:35:59,680 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock: /tmp/temp878257021/tmp-1023539707/_logs/history/job_201305152304_0012_conf.xml. blk_-1450040174609345042_1070
2013-05-21 21:35:59,686 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:50010 is added to blk_-1450040174609345042_1070 size 103958
2013-05-21 21:35:59,687 INFO org.apache.hadoop.hdfs.StateChange: Removing lease on file /tmp/temp878257021/tmp-1023539707/_logs/history/job_201305152304_0012_conf.xml from client DFSClient_NONMAPREDUCE_-187738396_35
2013-05-21 21:35:59,687 INFO org.apache.hadoop.hdfs.StateChange: DIR* NameSystem.completeFile: file /tmp/temp878257021/tmp-1023539707/_logs/history/job_201305152304_0012_conf.xml is closed by DFSClient_NONMAPREDUCE_-187738396_35
2013-05-21 21:35:59,689 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock: /Users/hadoop/tmpdir/mapred/system/job_201305152304_0012/jobToken. blk_9100288043730152482_1071
2013-05-21 21:35:59,692 INFO org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:50010 is added to blk_9100288043730152482_1071 size 106
2013-05-21 21:35:59,693 INFO org.apache.hadoop.hdfs.StateChange: Removing lease on file /Users/hadoop/tmpdir/mapred/system/job_201305152304_0012/jobToken from client DFSClient_NONMAPREDUCE_-12013881_1
2013-05-21 21:35:59,693 INFO org.apache.hadoop.hdfs.StateChange: DIR* NameSystem.completeFile: file /Users/hadoop/tmpdir/mapred/system/job_201305152304_0012/jobToken is closed by DFSClient_NONMAPREDUCE_-12013881_1
2013-05-21 21:32:19,667 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201305152304_0004_m_000000_1 task's state:UNASSIGNED
2013-05-21 21:32:19,668 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201305152304_0004_m_000000_1 which needs 1 slots
2013-05-21 21:32:19,668 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201305152304_0004_m_000000_1 which needs 1 slots
2013-05-21 21:32:19,682 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201305152304_0004_m_-1436610021
2013-05-21 21:32:19,683 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201305152304_0004_m_-1436610021 spawned.
2013-05-21 21:32:19,685 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /Users/hadoop/tmpdir/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201305152304_0004/attempt_201305152304_0004_m_000000_1/taskjvm.sh
2013-05-21 21:32:20,289 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201305152304_0004_m_-1436610021 given task: attempt_201305152304_0004_m_000000_1
2013-05-21 21:32:20,821 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_m_000000_1 1.0% hdfs://localhost:8020/user/hadoop/input/capacity-scheduler.xml:0+7457
2013-05-21 21:32:20,822 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201305152304_0004_m_000000_1 is done.
2013-05-21 21:32:20,823 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201305152304_0004_m_000000_1 was 6
2013-05-21 21:32:20,823 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
2013-05-21 21:32:21,009 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201305152304_0004_m_-1436610021 exited with exit code 0. Number of tasks it ran: 1
2013-05-21 21:32:22,551 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:32:28,557 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:32:34,562 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:32:37,563 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:32:43,568 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:32:49,572 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:32:52,574 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:32:58,578 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:04,582 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:07,585 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:13,589 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:19,592 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:22,594 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:28,597 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:34,602 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:37,604 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:43,608 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:49,611 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:52,613 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:33:58,618 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:04,622 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:07,624 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:13,627 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:19,630 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:22,632 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:28,637 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:34,641 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:37,644 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:43,647 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:49,651 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:52,654 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:34:58,658 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:04,663 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:07,664 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:13,669 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:19,673 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:22,676 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:28,680 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:34,685 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:37,687 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:43,692 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:49,698 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:52,701 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:58,705 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:35:59,881 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201305152304_0012_m_000002_0 task's state:UNASSIGNED
2013-05-21 21:35:59,882 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201305152304_0012_m_000002_0 which needs 1 slots
2013-05-21 21:35:59,882 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201305152304_0012_m_000002_0 which needs 1 slots
2013-05-21 21:35:59,939 INFO org.apache.hadoop.mapred.JobLocalizer: Initializing user hadoop on this TT.
2013-05-21 21:36:01,257 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201305152304_0012_m_-2138510030
2013-05-21 21:36:01,258 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201305152304_0012_m_-2138510030 spawned.
2013-05-21 21:36:01,259 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /Users/hadoop/tmpdir/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201305152304_0012/attempt_201305152304_0012_m_000002_0/taskjvm.sh
2013-05-21 21:36:01,874 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201305152304_0012_m_-2138510030 given task: attempt_201305152304_0012_m_000002_0
2013-05-21 21:36:02,471 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_m_000002_0 0.0% setup
2013-05-21 21:36:02,472 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201305152304_0012_m_000002_0 is done.
2013-05-21 21:36:02,472 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201305152304_0012_m_000002_0 was -1
2013-05-21 21:36:02,472 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
2013-05-21 21:36:02,596 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201305152304_0012_m_000000_0 task's state:UNASSIGNED
2013-05-21 21:36:02,597 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201305152304_0012_m_000000_0 which needs 1 slots
2013-05-21 21:36:02,597 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201305152304_0012_m_000000_0 which needs 1 slots
2013-05-21 21:36:02,614 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201305152304_0012_m_-335494615
2013-05-21 21:36:02,616 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201305152304_0012_m_-335494615 spawned.
2013-05-21 21:36:02,618 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /Users/hadoop/tmpdir/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201305152304_0012/attempt_201305152304_0012_m_000000_0/taskjvm.sh
2013-05-21 21:36:02,662 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201305152304_0012_m_-2138510030 exited with exit code 0. Number of tasks it ran: 1
2013-05-21 21:36:03,224 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201305152304_0012_m_-335494615 given task: attempt_201305152304_0012_m_000000_0
2013-05-21 21:36:04,206 INFO org.apache.hadoop.mapred.TaskTracker: Received KillTaskAction for task: attempt_201305152304_0012_m_000002_0
2013-05-21 21:36:04,206 INFO org.apache.hadoop.mapred.TaskTracker: About to purge task: attempt_201305152304_0012_m_000002_0
2013-05-21 21:36:04,206 INFO org.apache.hadoop.mapred.IndexCache: Map ID attempt_201305152304_0012_m_000002_0 not found in cache
2013-05-21 21:36:04,709 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:05,465 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_m_000000_0 1.0%
2013-05-21 21:36:05,466 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201305152304_0012_m_000000_0 is done.
2013-05-21 21:36:05,466 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201305152304_0012_m_000000_0 was 3252
2013-05-21 21:36:05,466 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
2013-05-21 21:36:05,614 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201305152304_0012_r_000000_0 task's state:UNASSIGNED
2013-05-21 21:36:05,614 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201305152304_0012_r_000000_0 which needs 1 slots
2013-05-21 21:36:05,614 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 1 and trying to launch attempt_201305152304_0012_r_000000_0 which needs 1 slots
2013-05-21 21:36:05,630 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201305152304_0012_r_-796843957
2013-05-21 21:36:05,631 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201305152304_0012_r_-796843957 spawned.
2013-05-21 21:36:05,632 INFO org.apache.hadoop.mapred.TaskController: Writing commands to /Users/hadoop/tmpdir/mapred/local/ttprivate/taskTracker/hadoop/jobcache/job_201305152304_0012/attempt_201305152304_0012_r_000000_0/taskjvm.sh
2013-05-21 21:36:05,659 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201305152304_0012_m_-335494615 exited with exit code 0. Number of tasks it ran: 1
2013-05-21 21:36:06,246 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201305152304_0012_r_-796843957 given task: attempt_201305152304_0012_r_000000_0
2013-05-21 21:36:07,713 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:12,723 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:36:13,719 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:15,727 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:36:19,723 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:21,733 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:36:22,727 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:27,738 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:36:28,731 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:30,741 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:36:34,736 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:36,747 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:36:37,740 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:42,753 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:36:43,745 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:45,755 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:36:49,749 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:51,760 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:36:52,752 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:36:57,765 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:36:58,755 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:00,767 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:04,759 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:06,772 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:07,762 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:12,776 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:13,767 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:15,779 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:19,773 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:21,785 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:22,777 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:27,790 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:28,781 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:30,793 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:34,786 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:36,797 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:37,789 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:42,802 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:43,793 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:45,805 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:49,798 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:51,810 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:52,801 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:37:57,814 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:37:58,805 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:00,817 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:04,809 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:06,821 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:07,812 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:12,824 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:13,817 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:15,827 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:19,821 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:21,831 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:22,823 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:27,836 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:28,828 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:30,839 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:34,833 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:36,844 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:37,835 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:42,849 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:43,840 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:45,853 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:49,845 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:51,859 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:52,847 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:38:57,865 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:38:58,853 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:00,868 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:04,857 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:06,875 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:07,860 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:12,881 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:13,865 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:15,884 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:19,870 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:21,890 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:22,873 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:27,896 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:28,879 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:30,900 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:34,884 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:36,906 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:37,887 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:42,911 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:43,892 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:45,915 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:49,897 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:51,920 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:52,900 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:39:57,925 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:39:58,906 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:40:00,928 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:40:04,911 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:40:06,934 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:40:07,915 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:40:12,939 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:40:13,920 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:40:15,941 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:40:19,926 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:40:21,946 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:40:22,928 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:40:27,951 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:40:28,934 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:40:30,953 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:40:34,940 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:40:36,958 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:40:37,944 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:40:42,962 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:41:28,743 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2 0.0% reduce > copy >
2013-05-21 21:41:28,745 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0012_r_000000_0 0.0% reduce > copy >
2013-05-21 21:51:32,263 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201305152304_0004_r_000000_2: Task attempt_201305152304_0004_r_000000_2 failed to report status for 603 seconds. Killing!
2013-05-21 21:51:32,268 INFO org.apache.hadoop.mapred.TaskTracker: Process Thread Dump: lost task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment