Skip to content

Instantly share code, notes, and snippets.

View manboubird's full-sized avatar

Toshiaki Toyama manboubird

View GitHub Profile
#!/usr/bin/ruby
class IPGenerator
public
def initialize(session_count, session_length)
@session_count = session_count
@session_length = session_length
@sessions = {}
end
# http://www.gnu.org/software/parallel/parallel_tutorial.html#controling_the_output
# issue hive queries
cat hqls.hql | parallel --line-buffer --tagstring "[ {#} - {} ]" --results parallel_std_logs --eta --progress --joblog parallel_job_logs -j0 --halt 1 --load 100% --noswap "hive -e {} > result_{#}_{}.txt"
# dry run
cat hqls.hql | parallel --dry-run --line-buffer --tagstring "[ {#} - {} ]" --results parallel_std_logs --eta --progress --joblog parallel_job_logs -j0 --halt 1 --load 100% --noswap "hive -e {} > result_{#}_{}.txt"
# transfer jar and submit job
/usr/local/bin/parallel -S hadoop@localhost --wd ... --transfer "/usr/local/hadoop-mr1/bin/hadoop jar {} scalding.marketing.JobRunner scalding.marketing.UniqueUser --hdfs --output /tmp/output --dt 2000-02-08" ::: ./target/scalding-marketing-0.0.1.jar
#!/bin/bash
PID_FILE=/usr/local/hive/pid/hiveserver2.pid
if [ -f ${PID_FILE} ]; then
echo "PID_FILE exists. ${PID_FILE}"
echo "kill `cat ${PID_FILE}`"
echo "rm ${PID_FILE}"
exit
fi
/usr/local/hive/bin/hiveserver2 >> /usr/local/hive/logs/hiveserver2.`date +%F`.log 2>&1 &
PID=$!
Personal vim command memo
- Tab
gt : go to next tab
gT : go to previous tab
{i}gt : go to tab in position i
:tab split : copy the current window to a new tab of its own
- FuzzyFinder.vim
:FufFile **/