This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python | |
import time | |
import datetime | |
import random | |
timestr = time.strftime("%Y%m%d-%H%M%S") | |
f = open('access_log_'+timestr+'.log','w') | |
ips=["123.221.14.56","16.180.70.237","10.182.189.79","218.193.16.244","198.122.118.164","114.214.178.92","233.192.62.103","244.157.45.12","81.73.150.239","237.43.24.118"] | |
referers=["-","http://www.casualcyclist.com","http://bestcyclingreviews.com/top_online_shops","http://bleater.com","http://searchengine.com"] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// Use Gists to store code you would like to remember later on | |
console.log(window); // log the "window" object to the console |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
LINKDIR=/usr/bin | |
JHOME=/usr/java/jdk1.7.0_67-cloudera/ | |
JREDIR=$JHOME/jre/bin | |
JDKDIR=$JHOME/bin | |
sudo alternatives --install $LINKDIR/java java $JREDIR/java 20000 \ | |
--slave $LINKDIR/keytool keytool $JREDIR/keytool \ | |
--slave $LINKDIR/orbd orbd $JREDIR/orbd \ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/sh | |
setup_brew () { | |
if ![-f "/usr/local/bin/brew"]; then | |
/usr/bin/ruby -e "$(/usr/bin/curl -fsSL https://raw.github.com/mxcl/homebrew/master/Library/Contributions/install_homebrew.rb)" | |
fi | |
} | |
setup_ipython () { | |
brew install readline |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
!/bin/bash | |
export tnow=$(date +"%Y-%m-%d") | |
export hnow=$(date +"%Y-%m-%d:%H") | |
export d_fname=ItemFulfillment-$tnow.csv | |
export h_fname=ItemFulfillment-$hnow.csv | |
export dirname=ItemFulfillment | |
export FTP_SERVER='IP' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python | |
import csv | |
import sys | |
import argparse | |
from string import Template | |
import subprocess | |
debug = False | |
def output(hdfspath,data): |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python | |
import csv | |
import sys | |
import argparse | |
import io | |
def genSchema(coldict): | |
ss = """ | |
{"namespace": "example.avro", |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python | |
import csv | |
import sys | |
import argparse | |
import io | |
csv.field_size_limit(sys.maxsize) | |
parser = argparse.ArgumentParser(description='Clean csv of in-line newlines') | |
parser.add_argument('infile',help='Path to input CSV file'); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<workflow-app name="sqoopSqlServerAndCreateHive" xmlns="uri:oozie:workflow:0.4"> | |
<global> | |
<configuration> | |
<property> | |
<name>table</name> | |
<value></value> | |
</property> | |
<property> | |
<name></name> | |
<value></value> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python | |
# | |
#pip install https://pyodbc.googlecode.com/files/pyodbc-2.1.11.zip | |
#Install the microsoft odbc driver for linux available here: | |
#http://www.microsoft.com/en-us/download/details.aspx?id=28160 | |
import pyodbc | |
cnxn = pyodbc.connect('DRIVER=SQL Server Native Client 11.0;SERVER=<ip>;UID=<username>;PWD=<password>') | |
cursor = cnxn.cursor() |