Skip to content

Instantly share code, notes, and snippets.

@wyukawa
wyukawa / gist:5097560
Created March 6, 2013 08:09
HBaseのbulk load時のエラー
Caused by: java.lang.IllegalStateException: The value of the hbase.metrics.showTableName conf option has not been specified in SchemaMetrics
at org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.getEffectiveTableName(SchemaMetrics.java:607)
at org.apache.hadoop.hbase.regionserver.metrics.SchemaMetrics.getInstance(SchemaMetrics.java:333)
at org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured.getSchemaMetrics(SchemaConfigured.java:185)
at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:441)
at org.apache.hadoop.hbase.io.hfile.HFileReaderV2.close(HFileReaderV2.java:419)
at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.groupOrSplit(LoadIncrementalHFiles.java:410)
at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:323)
at org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles$2.call(LoadIncrementalHFiles.java:321)
at java.util.concurrent.FutureTask$S
@wyukawa
wyukawa / gist:5280805
Created March 31, 2013 14:38
result of live_soccer
$ live_soccer
31Mar 10:30 - Wolfsburg 2 x 0 Nuremberg
31Mar 12:00 - Mallorca x La Coruña
31Mar 12:00 - Brest x Lille
31Mar 12:30 - Greuther Furth x Eintracht Frankfurt
31Mar 14:00 - Espanyol x Real Sociedad
31Mar 14:15 - Independiente-AR x Boca Juniors
31Mar 15:00 - Rio Branco-ES x Espírito Santo
31Mar 15:30 - Unaí Itapuã x Luziânia
31Mar 16:00 - Operário-PR x J.Malucelli
@wyukawa
wyukawa / gist:5501630
Created May 2, 2013 11:31
how to process pull request
fork
$ git clone [email protected]:wyukawa2/ea_sports_ppi.git
$ git checkout -b ruport origin/master
vim
$ git commit -am "..."
$ git push origin ruport
pull request
$ git remote add wyukawa2 https://github.com/wyukawa2/ea_sports_ppi
@wyukawa
wyukawa / gist:5610137
Created May 20, 2013 02:34
python inherit instance variable
class A(object):
def __init__(self):
self.a = 1
def hoge(self):
if hasattr(self, "a"):
return self.a
else:
return 0
@wyukawa
wyukawa / Hive Thrift Server Error
Created October 8, 2013 01:32
Hive Thrift Server Error
2013-10-08 00:40:01,558 ERROR metadata.Hive (Hive.java:getTable(951)) - NoSuchObjectException(message:... table not found)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1369)
at sun.reflect.GeneratedMethodAccessor103.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:102)
at com.sun.proxy.$Proxy10.get_table(Unknown Source)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:838)
at sun.reflect.GeneratedMethodAccessor102.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
org.apache.hadoop.hbase.client.NoServerForRegionException: Unable to find region for … after ... tries.
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1095)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1000)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1102)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1004)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:961)
@wyukawa
wyukawa / OutOfMemoryError
Created November 15, 2013 05:03
hiveのjoinでOutOfMemoryError
Error: java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.shuffleInMemory(ReduceTask.java:1703)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.getMapOutput(ReduceTask.java:1563)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.copyOutput(ReduceTask.java:1401)
at org.apache.hadoop.mapred.ReduceTask$ReduceCopier$MapOutputCopier.run(ReduceTask.java:1333)
@wyukawa
wyukawa / fabric-bolt
Created November 16, 2013 03:48
fabric-bolt setup
pip install -r requirements/local.txt
python manage.py syncdb
python manage.py migrate
python manage.py createsuperuser
python manage.py runserver
@wyukawa
wyukawa / zookeeper error
Created November 22, 2013 10:48
zookeeper error
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:450)
at org.apache.hadoop.hbase.zookeeper.ClusterId.readClusterIdZNode(ClusterId.java:61)
at org.apache.hadoop.hbase.zookeeper.ClusterId.getId(ClusterId.java:50)
at org.apache.hadoop.hbase.zookeeper.ClusterId.hasId(ClusterId.java:44)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:615)
@wyukawa
wyukawa / gist:8092742
Created December 23, 2013 06:56
hdfs error
Caused by: java.io.IOException: Failed to get block info from any of the DN in pipeline: [...]
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.updateBlockInfo(DFSClient.java:2079)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.fetchLocatedBlocks(DFSClient.java:2020)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1975)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1967)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:735)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:165)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:436)
at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:86)
at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:55)