zsxwing’s gists

zsxwing / argument.sh

Last active December 15, 2015 21:58

Set the $1 $2 $3, ..., and $# in the current terminal. Avoid to open a new file when you just want to test a small function.

	set -- a b c
	echo "$1 $2 $3"

zsxwing / iter_lines.sh

Last active December 15, 2015 21:58

How to iterate lines in a file or generated by a command correctly.

	while read line; do
	echo $line;
	done < <(cat test.txt)

zsxwing / long_string.sh

Last active December 15, 2015 21:58

How to create a long string in bash shell

	# output "I am an exceptionally long string."
	long_string="I am an exceptionally
	long string."
	echo ${long_string}

zsxwing / error_log.sh

Last active December 15, 2015 21:58

A error function to output the message with time.

	err() {
	echo "[$(date +'%Y-%m-%dT%H:%M:%S%z')]: $@" >&2
	}

zsxwing / usage.sh

Last active December 15, 2015 21:58

Convenient way to write a usage function.

	usage() {
	cat <<END;
	ez_mail TITLE CONTENT FROM_ADDRESS RECIEVER_ADDRESS
	EXAMPLE
	ez_mail 'Hello World!' 'Hi!' [email protected] [email protected]
	END
	}

zsxwing / comment.sh

Created April 7, 2013 12:22

How to use comments in shell.

	# one line comment
	:<<test
	Multiple lines comment.
	test

zsxwing / CurryTest.scala

Created April 8, 2013 02:43

Currying example.

	object CurryTest {

	def modN(n: Int)(x: Int) = x % n == 0 //> modN: (n: Int)(x: Int)Boolean

	val nums = List(1, 2, 3, 4, 5, 6, 7, 8) //> nums : List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8)

	println(nums.filter(modN(2))) //> List(2, 4, 6, 8)
	println(nums.filter(modN(3))) //> List(3, 6)
	}

zsxwing / Fibonacci.scala

Created April 10, 2013 07:16

Fibonacci Numbers in Scala

	import scala.math.BigInt
	lazy val fibs: Stream[BigInt] = BigInt(0) #::
	BigInt(1) #::
	fibs.zip(fibs.tail).map { n => n._1 + n._2 }

zsxwing / gist:5924191

Last active April 3, 2017 04:45

Pig and Hive source codes overview. Small hint: use "explain" to find the operation keywords, and search them in the source codes to find the corresponding class.

	Pig:
	org.apache.pig.newplan.logical.relational.LogicalPlan
	org.apache.pig.backend.hadoop.executionengine.physicalLayer.plans.PhysicalPlan
	org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.plans.MROperPlan
	org.apache.pig.parser.QueryParserDriver.parse(String)
	org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(LogicalPlan,Properties)
	org.apache.pig.PigServer.launchPlan(PhysicalPlan,String)
	org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.compile(PhysicalPlan,PigContext)
	org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(MROperPlan,MapReduceOper,Configuration,PigContext)
	org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(MROperPlan,String)

zsxwing / notes.txt

Created August 5, 2013 06:58 — forked from nipra/notes.txt

	# Installing CDH4 on a Single Linux Node in Pseudo-distributed Mode
	# https://ccp.cloudera.com/display/CDH4DOC/Installing+CDH4+on+a+Single+Linux+Node+in+Pseudo-distributed+Mode

	# Installing CDH4 with MRv1 on a Single Linux Node in Pseudo-distributed mode
	# On Ubuntu and other Debian systems
	nipra@lambda:Downloads$ wget -cv http://archive.cloudera.com/cdh4/one-click-install/precise/amd64/cdh4-repository_1.0_all.deb
	nipra@lambda:Downloads$ sudo dpkg -i cdh4-repository_1.0_all.deb # Adds /etc/apt/sources.list.d/cloudera-cdh4.list ??
	nipra@lambda:Downloads$ dpkg -L cdh4-repository # To view the files on Ubuntu systems

	# Install CDH4

Shixiong Zhu zsxwing