SUMANTAL slgithub

GIT Everyday Command Brushup

http://aayushtuladhar.wordpress.com/2012/01/19/git-brushup/

####Git Setup - Configuring Username/Email

git config --global user.name "Aayush Tuladhar"'
git config --global user.email "[email protected]"'

####Creating an empty GIT repository

Mac Shortcut	Windows Shortcut	Description
⌘ + ⇧ + R	Ctrl + Shift + R	Open / Search for Resources
⌘ + ⇧ + T	Ctrl + Shift + T	Open / Search for types (Useful in finding classes)
⌘ + O	Ctrl + O	Shows Quick Outline of the Java Class
⌘ + T	CTRL-T	Show type hierarchy
ALT – ↑ or ↓	ALT – ↑ or ↓	Move line/block
⌃ + Space	Ctrl + Space	Content assist and Code completion
⌘ + ⇧ + F	Ctrl + Shift + F	Format source code
⌘ + L	Ctrl + L	Go to Line Number

Introduction to Kafka

Kafka acts as a kind of write-ahead log (WAL) that records messages to a persistent store (disk) and allows subscribers to read and apply these changes to their own stores in a system appropriate time-frame.

Terminology:

Producers send messages to brokers
Consumers read messages from brokers
Messages are sent to a topic

Storm

Storm is a distributed and fault-tolerant realtime computation system

###Features of storm:

Scalable and robust
Fault-tolrant (automatic reassigning of tasks)
Reliable (all messages are processed at least once)
Fast

#Mongo UseCase:

Installing mongodb on 5 machines with the following deamon configurations:

Host	Mongo Role
router.mongo.cw.com	Router(mongos), Application Server, Arbiter (Shard1), Arbiter (Shard2), Config1, Config2, Config3
shard1r1.mongo.cw.com	shard1 replica primary
shard1r2.mongo.cw.com	shard1 replica secondary
shard2r1.mongo.cw.com	shard2 replica primary

##Using yarn as the resource manager you can deploy spark application in two modes:

yarn-standalone mode, in which your driver program is running as a thread of the yarn application master, which itself runs on one of the node managers in the cluster. The Yarn client just pulls status from the application master. This mode is same as a mapreduce job, where the MR application master coordinates the containers to run the map/reduce tasks.

With this mode, your application is actually run on the remote machine where the Application Master is run upon. Thus application that involve local interaction will not work well, e.g. spark-shell.

yarn-client mode, in which your driver program is running on the yarn client where you type the command to submit the spark application (may not be a machine in the yarn cluster). In this mode, although the drive program is running on the client machine, the tasks are executed on the executors in the node managers of the YARN cluster.

Simply putting to gether:

	DAEMON_PATH=/opt/kafka/bin
	DAEMON_NAME=kafka
	# Check that networking is up.
	#[ ${NETWORKING} = "no" ] && exit 0

	PATH=$PATH:$DAEMON_PATH

	# See how we were called.
	case "$1" in
	start)

	#!/bin/bash
	#
	# Licensed to the Apache Software Foundation (ASF) under one or more
	# contributor license agreements. See the NOTICE file distributed with
	# this work for additional information regarding copyright ownership.
	# The ASF licenses this file to You under the Apache License, Version 2.0
	# (the "License"); you may not use this file except in compliance with
	# the License. You may obtain a copy of the License at
	#
	# http://www.apache.org/licenses/LICENSE-2.0

	#!/bin/sh
	exec scala -savecompiled "$0" "$@"

	!#
	import scala.collection.mutable.Map
	import scala.util.Random
	import scala.collection.mutable.ArrayBuffer
	import java.io._

	class IPGenerator(var sessionCount: Int, var sessionLength: Int) {

	import org.apache.http.HttpEntity;
	import org.apache.http.HttpResponse;
	import org.apache.http.StatusLine;
	import org.apache.http.client.ClientProtocolException;
	import org.apache.http.client.HttpClient;
	import org.apache.http.client.methods.HttpGet;
	import org.apache.http.impl.client.DefaultHttpClient;
	import org.json.simple.JSONArray;
	import org.json.simple.JSONObject;
	import org.json.simple.JSONValue;