Skip to content

Instantly share code, notes, and snippets.

@hirobo
Last active August 6, 2020 11:49
Show Gist options
  • Save hirobo/d34a370e092c051c12e3 to your computer and use it in GitHub Desktop.
Save hirobo/d34a370e092c051c12e3 to your computer and use it in GitHub Desktop.
Shell script for mongoexport with query
#!/bin/sh
#########################################################
# usage
#do_mongoexport '2015-04-01' '2015-04-02' 'hoge'
#########################################################
get_millis()
{
if [ $# != 1 ]; then
echo $#
echo "ERROR: Too few arguments." 1>&2
return 1
fi
utc_time=$(date -d $1 -u)
time_secs=$(date +"%s" -d "$utc_time")
time_millis=`expr $time_secs \* 1000`
echo $time_millis
return 0
}
do_mongoexport()
{
if [ $# != 3 ]; then
echo $#
echo "ERROR: Too few arguments." 1>&2
return 1
fi
database_name="database_name"
collection_name="collection_name"
#out_dir=/mnt/mongoexport/working/
#log_dir=/mnt/mongoexport/working/log/
out_dir=./
log_dir=./
start_day=$1
end_day=$2
#action_name=$3
#name=$4
name=$3
end_millis=`get_millis $end_day`
start_millis=`get_millis $start_day`
echo "do mongoexport for gte $start_day lt $end_day with name $name"
log_filename="mongoexport_${name}_gte_${start_day}_lt_${end_day}.log"
out_filename="mongoexport_${name}_gte_${start_day}_lt_${end_day}.json"
out_path="${out_dir}${out_filename}"
log_path="${log_dir}${log_filename}"
query="{time:{\$gte:new Date(${start_millis}),\$lt:new Date(${end_millis})},name:\"${name}\"}"
nohup mongoexport -d ${database_name} -c ${collection_name} -q "${query}" -o ${out_path} > ${log_path} &
pid=$! # PID of the most recent background command
echo "PID is $pid"
}
@dapseen
Copy link

dapseen commented Feb 20, 2020

The only thing i picked in this script started and ended on line 55.

Rewrote it to suit my need

#!/bin/sh

sleep 1

date +"%Y  %m  %d " > date.txt

DAY=$(cat date.txt | awk '{print $3}')
YEAR=$(cat date.txt | awk '{print $1}')
MONTH=$(cat date.txt | awk '{print $2}')
TIME=T00:00:00.925Z
gte='$gte' # this ensures greater than variable is passed to the mongo query
date='$date' # the sam applies to the date variable 

query="{createdAt: { \"$gte\" : { \"$date\":  \"${YEAR}-${MONTH}-${DAY}${TIME}\" } } }"

The "" escape prevented $gte from running on cli.. so i had to make it a variable.

I hope its helpful..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment