Skip to content

Instantly share code, notes, and snippets.

View prasanthj's full-sized avatar

Prasanth Jayachandran prasanthj

View GitHub Profile
@prasanthj
prasanthj / native-mem-tracking.md
Last active June 5, 2024 12:48
Native memory tracking in JVM

Enable native memory tracking in JVM by specifying the following flag

-XX:NativeMemoryTracking=detail

Know the <PID> of the java process

jps

To print ps based RSS

ps -p <PID> -o pcpu,rss,size,vsize

To print native memory tracking summary

@epiphani
epiphani / CDHTez.md
Last active February 14, 2024 08:03
Getting Tez enabled on CDH5.4+

So Hive in CDH is horribly, painfully slow. Cloudera ships Hive 1.1, which is actually moderately modern. It is, however, very badly configured out of the box and patched with custom code from Cloudera. With a bit of effort, we managed to improve hive performance considerably. We really shouldn't have to do this, but Cloudera is actively working against supporting a performant Hive.

First, building Tez was fairly straightforward. Using the instructions at https://github.com/apache/tez/blob/master/docs/src/site/markdown/install.md, the only change was to use the version string "2.6.0" for the build. I believe that was the default. Don't use the CDH string, it won't work.

At the bottom of the installation instructions, there's mention of the fact that to use the local hadoop jars (rather than those packaged with tez) you must unpack the jars in HDFS rather than using the tarball. In this case, unpack the tez-minimal tarball and upload the contents to /apps/tez-0.7.0 (or whatever you prefer). Don't fo

@t3rmin4t0r
t3rmin4t0r / ats-extract-plan.py
Created June 30, 2015 19:12
ATS to Hive query plan extraction
import json
import sys
class ATSFile(object):
def __init__(self, name):
self.data = json.load(open(name))
self.name = name
def dump(self):
info = self.data["otherinfo"]
q = json.loads(info["QUERY"])
@johnynek
johnynek / AliceInAggregatorLand.scala
Last active January 24, 2024 19:38
A REPL Example of using Aggregators in scala
/**
* To get started:
* git clone https://github.com/twitter/algebird
* cd algebird
* ./sbt algebird-core/console
*/
/**
* Let's get some data. Here is Alice in Wonderland, line by line
*/
@jiaaro
jiaaro / hotkey_helpers.js
Last active December 11, 2024 17:51
Mac Automation – Javascript (JSX) Hotkey helpers
// How to use:
// 1. Open "Script Editor" (requires OS X 10.10 Yosemite)
// 2. Change the language from "AppleScript" to "JavaScript"
// 3. Paste the code below and replace the safari example.
//
// More info:
// https://developer.apple.com/library/mac/releasenotes/InterapplicationCommunication/RN-JavaScriptForAutomation/index.html
var sys_events = Application("System Events");
# Copyright (c) 2014 Lukas Fittl <[email protected]>
#
# Released in the public domain - fork as you wish.
require 'rubygems'
require 'mixlib/cli'
require 'pg'
require 'pg_query'
require 'curses'
(function () {
function c() {
var e = document.createElement("link");
e.setAttribute("type", "text/css");
e.setAttribute("rel", "stylesheet");
e.setAttribute("href", f);
e.setAttribute("class", l);
document.body.appendChild(e)
}
function h() {
@benjchristensen
benjchristensen / FuturesB.java
Last active December 19, 2024 23:11
FuturesB.java Example of using Futures for nested calls showing how it blocks inefficiently.
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Future;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
@adgedenkers
adgedenkers / vpn_connection.scpt
Created October 11, 2012 18:16
Toggle Connection to VPN on a Mac via AppleScript
tell application "System Events"
-- get current clipboard contents as a string
set CurrentClipboard to the clipboard as string
-- set the clipboad to your password
set the clipboard to "Y0urVPNPa$$w0rd"
-- start playing with the VPN
tell current location of network preferences