yashk’s gists

yashk / ex4.scala

Created June 2, 2012 13:48

scala for impatient exercise chapter 4 Maps and tuples

	/*
	Set up a map of prices for a number of gizmos that you covet. Then produce
	a second map with the same keys and the prices at a 10 percent discount.
	*/

	val gizmos = Map("Laptop" -> 100 ,"Mobile" -> 200)
	val tenPec = for((g,p) <- gizmos) yield (g,p-p*0.1)

	/*
	Write a program that reads words from a file. Use a mutable map to count

yashk / mirror_sites

Last active December 16, 2015 11:49

mirror cloudera , datastax rpm install site for offline install

wget -r --no-parent --no-host-directories --reject "mirror*,index*" http://rpm.datastax.com/community/

yashk / avro-commandline

Last active May 26, 2021 18:18

avro command line tool

	source :- http://www.michael-noll.com/blog/2013/03/17/reading-and-writing-avro-files-from-the-command-line/
	download tools jar - http://mirror.catn.com/pub/apache/avro/avro-1.7.5/java/avro-tools-1.7.5.jar

	add following to ~/.bashrc

	alias avro='java -jar /home/yash/software/avro/avro-tools-1.7.5.jar'

	now use just as avro command

yashk / WordCountAvroWrite.java

Last active February 22, 2018 16:16

CVS to Avro using cascading



	import java.io.File;
	import java.util.Properties;

	import org.apache.avro.Schema;

	import cascading.avro.AvroScheme;
	import cascading.flow.Flow;
	import cascading.flow.FlowDef;

yashk / WordCountAvroWrite.java

Last active January 4, 2016 16:29

cascading.avro - supply types via Fields class

	public class
	WordCountAvroWrite
	{
	public static void
	main( String[] args ) throws Exception
	{
	String docPath = "sample/data/WC/input.txt";
	String wcPath = "sample/data/WC/out";
	String schemaFile = "sample/data/WC/input.avsc";

yashk / run.sh

Created February 3, 2014 09:40

Linux - Run a command as diff user

sudo -u <user> <command>;

yashk / install.sh

Created February 3, 2014 18:09

install jdk 7 on suse

	su su -
	mkdir ~/tmp
	cd ~/tmp
	wget http://10.13.130.62:9999/jdk-7u51-linux-x64.rpm
	rpm -ev jdk
	rpm -ihv jdk-7u51-linux-x64.rpm
	update-alternatives --remove-all java
	update-alternatives --install "/usr/bin/java" "java" "/usr/java/latest/bin/java" 1
	update-alternatives --set java /usr/java/latest/bin/java
	source /usr/lib64/cmf/service/common/cloudera-config.sh

yashk / du.sh

Created February 5, 2014 14:27

Avg folder size under linux dir

du -s /data/sub_dir/*/* |  awk '{sum=sum+$1} END {print ((sum / NR) /1024)/1024 " GB"}'

yashk / port.sh

Created February 18, 2014 15:59

find used port on linux

netstat -antu |egrep ':8006|:8007|:8008|:8010'

yashk / log_file

Created February 26, 2014 09:36

Google cloud java api Error Log

	Feb 26, 2014 9:28:05 AM com.google.api.client.http.HttpRequest execute
	CONFIG: -------------- REQUEST --------------
	POST https://accounts.google.com/o/oauth2/token
	Accept-Encoding: gzip
	User-Agent: Google-HTTP-Java-Client/1.17.0-rc (gzip)
	Content-Type: application/x-www-form-urlencoded; charset=UTF-8
	Content-Length: 208

	Feb 26, 2014 9:28:05 AM com.google.api.client.http.HttpRequest execute
	CONFIG: curl -v --compressed -X POST -H 'Accept-Encoding: gzip' -H 'User-Agent: Google-HTTP-Java-Client/1.17.0-rc (gzip)' -H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8' -d '@-' -- 'https://accounts.google.com/o/oauth2/token' << $$$

yash yashk