Skip to content

Instantly share code, notes, and snippets.

@matyix
Created February 24, 2014 13:26
Show Gist options
  • Save matyix/9188384 to your computer and use it in GitHub Desktop.
Save matyix/9188384 to your computer and use it in GitHub Desktop.
Leaving the comfort zone of the JVM heap with HBase
If you are a HBase user a good experience to leave the comfort zone of the JVM heap is to start playing with the new BlockCache implementations in latest HBase versions (made me a sleepless night).
Older HBase implementations supported a single BlockCache implementation **(LruBlockCache)** but further version have introduced different cache implementations *(a block cache is a single unit of I/O, the smallest amount of data HBase writes or reads back from an HFile)*
Since the introduction of **ByteBuffers** in Java we had the chance to leverage the operating system's memory manager to move data in and out of memory in a way that's transparent to your program - and this is what exactly the new **SlabCache** and **BucketCache** implementations does, allocates memory outside the JVM heap using direct byte buffers.
``` java java.nio.ByteBuffer class
public static ByteBuffer allocateDirect(int capacity)
```
The first impression is a faster GC time using the CMS collector. Nevertheless, I think a deeper profiling is required as since the direct buffer leaves outside the JVM heap every data access is a JNI call, so it might work for a few and not for the rest.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment