Shingo Furuyama marblejenka
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 12/07/01 21:00:57 INFO mapreduce.Job: map 8% reduce 0% | |
| 12/07/01 21:01:05 INFO mapreduce.Job: Task Id : attempt_1340972767920_0047_m_000004_0, Status : FAILED | |
| Error: org.apache.hadoop.fs.ChecksumException: Checksum Error | |
| at org.apache.hadoop.mapred.IFileInputStream.doRead(IFileInputStream.java:173) | |
| at org.apache.hadoop.mapred.IFileInputStream.read(IFileInputStream.java:106) | |
| at org.apache.hadoop.mapred.IFileInputStream.read(IFileInputStream.java:183) | |
| at java.io.DataInputStream.readByte(DataInputStream.java:248) | |
| at org.apache.hadoop.io.WritableUtils.readVLong(WritableUtils.java:308) | |
| at org.apache.hadoop.io.WritableUtils.readVInt(WritableUtils.java:329) | |
| at org.apache.hadoop.mapred.IFile$Reader.positionToNextRecord(IFile.java:399) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 192.168.201.202 fc1 | |
| 192.168.201.203 fc2 | |
| 192.168.201.204 fc3 | |
| 192.168.201.205 fc4 | |
| 192.168.201.206 fc5 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /** | |
| * Licensed to the Apache Software Foundation (ASF) under one | |
| * or more contributor license agreements. See the NOTICE file | |
| * distributed with this work for additional information | |
| * regarding copyright ownership. The ASF licenses this file | |
| * to you under the Apache License, Version 2.0 (the | |
| * "License"); you may not use this file except in compliance | |
| * with the License. You may obtain a copy of the License at | |
| * | |
| * http://www.apache.org/licenses/LICENSE-2.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| marblejenka@ubuntu:/usr/local/lib/goldenorb$ cat input.txt | |
| 1 10 2 | |
| 2 15 1 3 | |
| 3 20 4 | |
| 4 1 3 | |
| タブ区切りでinputにする。 | |
| 意味は、[0]がvertexのid,[1]がvertexのvalue、[2]以降が伝播させる先のvertexのid | |
| データは、 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 37363 [Thread-2-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got ping response for sessionid: 0x130fdacced3000f after 1ms | |
| 340317 [Thread-2-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got notification sessionid:0x130fdacced3000f | |
| 340317 [Thread-2-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Got WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/GoldenOrb/OrbCluster/JobQueue for sessionid 0x130fdacced3000f | |
| 340317 [Thread-2-EventThread] INFO org.goldenorb.JobManager - getting jobs in queue. | |
| 340317 [Thread-2-EventThread] INFO org.goldenorb.JobManager - getting jobs in queue. | |
| 340321 [Thread-2-SendThread(localhost:2181)] DEBUG org.apache.zookeeper.ClientCnxn - Reading reply sessionid:0x130fdacced3000f, packet:: clientPath:null serverPath:null finished:false header:: 117,8 replyHeader:: 117,364,0 request:: '/GoldenOrb/OrbCluster/JobQueue,T response:: v{'Job0000000018} | |
| 340325 [Thread-2-SendThread(localhost:2181)] DEBUG org.apache. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cloudera@cloudera-vm:~$ pig | |
| 2011-06-02 07:26:54,093 [main] INFO org.apache.pig.Main - Logging error messages to: /home/cloudera/pig_1307024814091.log | |
| 2011-06-02 07:26:54,559 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost:8020 | |
| 2011-06-02 07:26:54,885 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost:8021 | |
| grunt> result = load 'test' using org.apache.pig.backend.hadoop.hbase.HBaseStorage('family:a family:b') as (a:chararray, b:chararray); | |
| grunt> dump result | |
| 2011-06-02 07:27:37,041 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN | |
| 2011-06-02 07:27:37,042 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - pig.usenewlogicalplan is set to true. New logical plan will be used. | |
| 2011-06-02 07:27:37,318 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Na |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cloudera@cloudera-vm:~/work/pig$ piglet test.rb | pig -x local | |
| 2011-05-30 20:31:30,550 [main] INFO org.apache.pig.Main - Logging error messages to: /home/cloudera/work/pig/pig_1306812690548.log | |
| 2011-05-30 20:31:30,819 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: file:/// | |
| grunt> relation_1 = LOAD 'test.rb'; | |
| grunt> STORE relation_1 INTO 'result'; | |
| 2011-05-30 20:31:31,283 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN | |
| 2011-05-30 20:31:31,283 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - pig.usenewlogicalplan is set to true. New logical plan will be used. | |
| 2011-05-30 20:31:31,454 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - (Name: relation_1: Store(file:///home/cloudera/work/pig/result:org.apache.pig.builtin.PigStorage) - scope-1 Operator Key: scope-1) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /* | |
| * Licensed to the Apache Software Foundation (ASF) under one | |
| * or more contributor license agreements. See the NOTICE file | |
| * distributed with this work for additional information | |
| * regarding copyright ownership. The ASF licenses this file | |
| * to you under the Apache License, Version 2.0 (the | |
| * "License"); you may not use this file except in compliance | |
| * with the License. You may obtain a copy of the License at | |
| * | |
| * http://www.apache.org/licenses/LICENSE-2.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| /* | |
| * Licensed to the Apache Software Foundation (ASF) under one | |
| * or more contributor license agreements. See the NOTICE file | |
| * distributed with this work for additional information | |
| * regarding copyright ownership. The ASF licenses this file | |
| * to you under the Apache License, Version 2.0 (the | |
| * "License"); you may not use this file except in compliance | |
| * with the License. You may obtain a copy of the License at | |
| * | |
| * http://www.apache.org/licenses/LICENSE-2.0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| hadoop@ip-10-150-127-73:~$ sh asakusa/batchapps/BasketAnalysisBatch/bin/experimental.sh | |
| Moving to ''asakusa/batchapps/BasketAnalysisBatch'' | |
| Processing batch BasketAnalysisBatch | |
| Processing jobflow 'BasketAnalysisJob' | |
| _EXECUTION_ID=genuuidがないらしいのでここのとこだけexperimental.shを書き変えないといけない。 | |
| _BATCH_ID=BasketAnalysisBatch | |
| _FLOW_ID=BasketAnalysisJob | |
| Deploying 'lib/jobflow-BasketAnalysisJob.jar' into '/home/hadoop/asakusa/batchapps/BasketAnalysisBatch/lib' | |
| cp: `lib/jobflow-BasketAnalysisJob.jar' and `/home/hadoop/asakusa/batchapps/BasketAnalysisBatch/lib/jobflow-BasketAnalysisJob.jar' are the same file | |
| Processing hadoop job 'BasketAnalysisBatch.BasketAnalysisJob.stage0001' |