colin b. erdman cerdman

TO IMPORT & EXPORT DATA FROM RDBMS (MYSQL,ORACLE, etc) INTO HDFS / HIVE / HBASE

Pre-requisite

Apache Hadoop

Apache Sqoop (compatible with Hadoop version)

Apache Hive (optional)

Best Google Chrome Extensions

List compiled by Amit Agarwal

Vimium — Power users can browse the web using keyboard shortcuts. No mouse required.
Buffer — Share links to multiple social websites with a go.
PushBullet — Send web links, text notes and even push files from computer to your phone.
Clip Better — Don't send raw links over email, send previews that suggest what a link is all about.
Streamus — A YouTube music play for Chrome that also includes a radio.
Mighty Text — Send and receive SMS text messages from your desktop

Workflows (States) in Django

I'm going to cover a simple, but effective, utility for managing state and transitions (aka workflow). We often need to store the state (status) of a model and it should only be in one state at a time.

Common Software Uses

Publishing (Draft->Approved->Published->Expired->Deleted)

On master node

wget http://archive.cloudera.com/cdh5/one-click-install/redhat/6/x86_64/cloudera-cdh-5-0.x86_64.rpm
sudo yum --nogpgcheck localinstall cloudera-cdh-5-0.x86_64.rpm
sudo yum clean all
sudo yum install hadoop-hdfs-namenode
sudo yum install R git
sudo yum install spark-core spark-master spark-python

cd

	#
	# ------------------------------------------
	# CLOUDWATCH LOGS AGENT CONFIGURATION FILE
	# ------------------------------------------
	#
	# --- DESCRIPTION ---
	# This file is used by the CloudWatch Logs Agent to specify what log data to send to the service and how.
	# You can modify this file at any time to add, remove or change configuration.
	#
	# NOTE: A running agent must be stopped and restarted for configuration changes to take effect.

	package main

	import (
	"flag"
	"fmt"
	"strings"
	)

	const DB_REDSHIFT = "redshift"
	const DB_POSTGRES = "postgres"

	# hadoop install
	# --------------
	# This installs a hadoop "worker node" on Slicify, allowing you to create a hadoop grid
	# This example requires the Slicify VPN solution, and requires that your namenode is running on a separate server.

	# config
	export SERVERIP=10.8.0.200
	export SERVERHOST=steve-ubu

	# set current date/time

	indexCtrlSpec
	----------------------------------


	describe('indexCtrl', () => {
	var scope, httpBackend;

	beforeEach(inject(($rootScope, $controller, $httpBackend) => {
	httpBackend = $httpBackend;
	// create a scope that will be provided to indexCtrl

	-- Document Revision Control Table: How to generate a partitioned analytical query in MySQL

	CREATE TABLE document_history
	(
	id int auto_increment primary key,
	document_id int,
	name varchar(100),
	modified datetime,
	user_id int
	);

	<?xml version="1.0" encoding="UTF-8"?>
	<opml version="1.0">
	<head>
	<title>AWS RSS Feeds</title>
	</head>
	<body>
	<outline title="AWS Status" text="AWS Status">
	<outline text="Amazon Mobile Analytics (N. Virginia) Service Status" title="Amazon Mobile Analytics (N. Virginia) Service Status" type="rss" xmlUrl="http://status.aws.amazon.com/rss/analytics-us-east-1.rss" htmlUrl="http://status.aws.amazon.com/" rssfr-favicon="http://status.aws.amazon.com/favicon.ico"/>
	<outline text="Amazon AppStream (N. Virginia) Service Status" title="Amazon AppStream (N. Virginia) Service Status" type="rss" xmlUrl="http://status.aws.amazon.com/rss/appstream-us-east-1.rss" htmlUrl="http://status.aws.amazon.com/" rssfr-favicon="http://status.aws.amazon.com/favicon.ico"/>
	<outline text="Auto Scaling (Tokyo) Service Status" title="Auto Scaling (Tokyo) Service Status" type="rss" xmlUrl="http://status.aws.amazon.com/rss/autoscaling-ap-northeast-1.rss" htmlUrl="http://status.a