shrijeet’s gists

shrijeet / RenameTable.java

Created October 18, 2012 17:30

A program to rename HBase table (works for 0.92.x)


	package hbase.experiments;

	import java.io.FileNotFoundException;
	import java.io.IOException;

	import org.apache.commons.logging.Log;
	import org.apache.commons.logging.LogFactory;
	import org.apache.hadoop.conf.Configuration;
	import org.apache.hadoop.fs.FSDataOutputStream;

shrijeet / FilterRowsWithGivenColumns.java

Created October 16, 2012 23:46

FilterRowsWithGivenColumns MapReduce program

	package hbase.experiments;

	import java.io.IOException;
	import java.util.ArrayList;
	import java.util.Arrays;
	import java.util.HashSet;
	import java.util.List;

	import org.apache.hadoop.conf.Configuration;
	import org.apache.hadoop.fs.Path;

shrijeet / jt_hung.java

Created October 8, 2012 21:12

JT hung





	2012-10-05 22:51:31
	Full thread dump Java HotSpot(TM) 64-Bit Server VM (14.2-b01 mixed mode):

	"IPC Client (47) connection to master-hadoop.inw-hercules.rfiserve.net/172.22.24.45:8020 from hdfs" daemon prio=10 tid=0x00007f13b4fa5000 nid=0x48d6 in Object.wait() [0x00007f09c8855000]
	java.lang.Thread.State: TIMED_WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)

shrijeet / HBaseIncExample.java

Created September 20, 2012 23:50

[bug report] Client shutdown (without explicit flush) does not flush buffered atomic increments

	import org.hbase.async.AtomicIncrementRequest;
	import org.hbase.async.HBaseClient;

	public class HBaseIncExample {

	public static void main(String [] args) throws Exception {
	HBaseClient client = new HBaseClient("rasalghul");
	AtomicIncrementRequest req = new AtomicIncrementRequest("t0", "k1", "f1", "q1");

	for (int i = 0; i < 100; i++) {

shrijeet / master.log

Created August 24, 2012 22:16

Master Log while NPE

This file has been truncated, but you can view the full file.

	Wed Aug 22 20:24:33 EDT 2012 Starting master on ine-60.rfiserve.net
	core file size (blocks, -c) 0
	data seg size (kbytes, -d) unlimited
	scheduling priority (-e) 0
	file size (blocks, -f) unlimited
	pending signals (-i) 278528
	max locked memory (kbytes, -l) 32
	max memory size (kbytes, -m) unlimited
	open files (-n) 65536
	pipe size (512 bytes, -p) 8

shrijeet / root.log

Created August 24, 2012 22:09

Root rs log during NPE

This file has been truncated, but you can view the full file.

	Wed Aug 22 20:08:25 EDT 2012 Starting regionserver on ine-48.rfiserve.net
	core file size (blocks, -c) 0
	data seg size (kbytes, -d) unlimited
	scheduling priority (-e) 0
	file size (blocks, -f) unlimited
	pending signals (-i) 794624
	max locked memory (kbytes, -l) 32
	max memory size (kbytes, -m) unlimited
	open files (-n) 65536
	pipe size (512 bytes, -p) 8

shrijeet / npe_rs_0.92.2.log

Created August 24, 2012 20:46

Null pointer exception in region server 0.92.2

This file has been truncated, but you can view the full file.

	Wed Aug 22 20:15:03 EDT 2012 Starting regionserver on ine-51.rfiserve.net
	core file size (blocks, -c) 0
	data seg size (kbytes, -d) unlimited
	scheduling priority (-e) 0
	file size (blocks, -f) unlimited
	pending signals (-i) 794624
	max locked memory (kbytes, -l) 32
	max memory size (kbytes, -m) unlimited
	open files (-n) 65536
	pipe size (512 bytes, -p) 8

shrijeet / confs.txt

Created August 21, 2012 00:31

Hbase 0.90.3 --> 0.92.2 which configurations to tweak?

	* MSLAB - Okay to leave it turned ON 0.92.2 onwards since we have lots of memory and number of regions per region server is not too crazy high (200 regions/per RS)
	* Distributed splitting - Want it!
	* Memory accounting is different now - Okay to leave block cache's value same as it was in 0.90.0 since its already set to a liberal value (0.45)
	* Experimental off-heap cache - Leave it turned off.
	* Region max file size - Currently set to 2GB, we should go up. 4GB will cut our regions / per region server to under 100.

shrijeet / save_yourself_from_am.xml

Created August 17, 2012 23:08

You wish!

	<?xml version='1.0' encoding='UTF-8'?><feed xmlns='http://www.w3.org/2005/Atom' xmlns:apps='http://schemas.google.com/apps/2006'>
	<title>Mail Filters</title>
	<id>tag:mail.google.com,2008:filters:1345244786967</id>
	<updated>2012-08-17T23:07:52Z</updated>
	<author>
	<name>Shrijeet Paliwal</name>
	<email>[email protected]</email>
	</author>
	<entry>
	<category term='filter'></category>

shrijeet / jvm_metrics.txt

Created August 15, 2012 18:45

Namenode increase in logWarn

	jvm.metrics: hostName=inw-5.rfiserve.net, processName=NameNode, sessionId=, gcCount=37845, gcTimeMillis=6003961, logError=0, logFatal=0, logInfo=128826946, logWarn=4218, maxMemoryM=10666.6875, memHeapCommittedM=7094.8125, memHeapUsedM=5518.711, memNonHeapCommittedM=27.5, memNonHeapUsedM=27.320305, threadsBlocked=0, threadsNew=0, threadsRunnable=11, threadsTerminated=0, threadsTimedWaiting=8, threadsWaiting=44
	jvm.metrics: hostName=inw-5.rfiserve.net, processName=NameNode, sessionId=, gcCount=38113, gcTimeMillis=6041969, logError=0, logFatal=0, logInfo=129717263, logWarn=4235, maxMemoryM=10666.6875, memHeapCommittedM=7094.1875, memHeapUsedM=3677.5535, memNonHeapCommittedM=27.5, memNonHeapUsedM=27.320305, threadsBlocked=0, threadsNew=0, threadsRunnable=9, threadsTerminated=0, threadsTimedWaiting=8, threadsWaiting=46
	jvm.metrics: hostName=inw-5.rfiserve.net, processName=NameNode, sessionId=, gcCount=38379, gcTimeMillis=6089254, logError=0, logFatal=0, logInfo=130416613, logWarn=4244, maxMemoryM=10666.6875, memHe

Shrijeet shrijeet