Skip to content

Instantly share code, notes, and snippets.

@dharmeshkakadia
Last active September 20, 2017 19:06
Show Gist options
  • Select an option

  • Save dharmeshkakadia/adcc3bf0d77c3ac10114b43ced7b81b4 to your computer and use it in GitHub Desktop.

Select an option

Save dharmeshkakadia/adcc3bf0d77c3ac10114b43ced7b81b4 to your computer and use it in GitHub Desktop.
export NUM_THREADS=$1; export HADOOP_CLIENT_OPTS="-Djline.terminal=jline.UnsupportedTerminal";
for f in {12,13,15,19,21,26,31,32,34,39,3,40,42,43,45,46,48,52,54,55,56,58,60,63,66,68,71,73,79,82,83,84,85,87,89,90,91,96,98};
do
# wait till this query is submitted
while true;
do
# wait for a lot to become available
while [[ $(jobs -rp | wc -l) -ge $NUM_THREADS ]] ;
do
sleep 1;
done ;
# launch the query $f
( STARTTIME="`date +%s`"; beeline -u "jdbc:hive2://`hostname -f`:10001/tpcds_orc;transportMode=http" -f "queries/query${f}.sql" -n "" -p "" > queries/query$f.concurrent32.out 2>&1 ; SUCCESS=$? ; ENDTIME="`date +%s`"; echo "$f,1,$SUCCESS,$STARTTIME,$ENDTIME,$(($ENDTIME-$STARTTIME))" >> times_orc.concurrent$NUM_THREADS.csv ;) &
break; # Query is submitted. Go for next query.
done
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment