Skip to content

Instantly share code, notes, and snippets.

@shokoe

shokoe/check_spot_price_adv.sh

Last active October 24, 2017 11:57
Show Gist options
  • Save shokoe/aeca823643733ebba7833febde1527e3 to your computer and use it in GitHub Desktop.
Save shokoe/aeca823643733ebba7833febde1527e3 to your computer and use it in GitHub Desktop.
Download ZIP
AWS spot price nagios and info script - Based on running spot instances, on-demand prices and spot prices the script calculates average spot price over a set time frame, displays lots of information and alerts (nagios style) on spot-price % on-demand-price.
Raw
check_spot_price_adv.sh
#!/bin/bash
export PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
# Requires:
# jq 1.5
# gawk
Help="Overview:
Based on running spot instances, on-demand prices and spot prices the script
calculates average spot price over a set time frame, displays lots of
information and alerts (nagios style) on spot-price % on-demand-price.
Sytanx:
Check specifc type - check_spot_price_adv.sh <single product type> <ins type> <time framae> <zone regex> <warn%>:<crit%>
example: check_spot_price_adv.sh 'Linux/UNIX (Amazon VPC)' c4.2xlarge '-24 hour' '.' 10:30
Check all running spots - check_spot_price_adv.sh Auto <time framae> <warn%>:<crit%>
example: check_spot_price_adv.sh Auto '-24 hour' 50:120
Output:
check_spot_price_adv.sh 'Linux/UNIX' c3.2xlarge '-24 hour' 'us-east-1b|us-east-1e' 10:30
Critical - Found running spots with Spot%OD > 30% (critical=1 warning=1)
Type Zone Product OD($/H) Avg($/H) Spot%OD Max($/H) Max%OD Last($/H) Last%OD OD_Total($) Total($) Time Status
c3.2xlarge us-east-1b Linux/UNIX 0.42 4.20 1000 4.20 1000 4.20 1000 11.33 113.28 26:58:16 CRIT
c3.2xlarge us-east-1e Linux/UNIX 0.42 0.10 24 0.10 24 0.10 24 10.10 2.50 24:02:57 WARN
(run-time 3s)
Requirments:
jq 1.5
gawk
working awscli
Limitation:
Only works on us-east-1
If the table printout is problematic (old nagios) remark at the script bottom"
get_avg(){
jq -r '.SpotPriceHistory[] | "\(.Timestamp)@\(.SpotPrice)@\(.AvailabilityZone)@\(.InstanceType)@\(.ProductDescription)"' |\
awk -v T=$itype -v P="$prod" 'BEGIN{FS=OFS="@"}; $4==T && $5==P;' |\
sort -n | gawk -F@ -v od="$od_price" '{
# remove spaces from product names
gsub(/ /, "_", $5)
#key=$3"@"$4"@"$5
key=$4"@"$3"@"$5
# convert $1 ISO8601 to epoch
gsub(/[-T:\.]/, " ", $1)
$1=mktime($1)
# first record
if (ptime[key]==""){
start[key]=$1
ptime[key]=$1
pprice[key]=$2
NEXT
}
total_sec[key]=total_sec[key]+$1-ptime[key]
total_price[key]=total_price[key]+($1-ptime[key])*pprice[key]
if (pprice[key] > max_price[key]) max_price[key]=pprice[key]
ptime[key]=$1
pprice[key]=$2
#print $0, total_sec[key], total_price[key]
}
END{
for (key in total_price) {
# complete stats from last change till now
if ( systime() > ptime[key] ) {
total_sec[key]=total_sec[key]+systime()-ptime[key]
total_price[key]=total_price[key]+(systime()-ptime[key])*pprice[key]
}
# output
avg=total_price[key]/total_sec[key]
max=max_price[key]
last=pprice[key]
od_total=od*total_sec[key]/3600
total=total_price[key]/3600
hour=(total_sec[key]-total_sec[key]%3600)/3600
minute=(total_sec[key]-total_sec[key]%60)%3600/60
sec=total_sec[key]%60
#printf "%s %.2f %.2f %d %.2f %.2f %.2f %.2f %d:%02d:%02d\n", key, od, avg, avg/od*100, max, last, od_total, total, hour, minute, sec
printf "%s %.2f %.2f %d %.2f %d %.2f %d %.2f %.2f %d:%02d:%02d\n", key, od, avg, avg/od*100, max, max/od*100, last, last/od*100, od_total, total, hour, minute, sec
}
}'
}
main(){
#s=`date +%s`
#echo "--- $((`date +%s`-$s))" >&2
od_price=`echo "$od_price_full" | awk -v T="$itype" '$1=="us-east-1" && $4=="linux" && $3==T {print $5}'`
out=`echo "$raw_json_all" | get_avg | sed 's#@# #g' | egrep "$zones" | sort |\
awk -v W=$warn -v C=$crit '{
S=NF+1; $S="OK";
if ($6>W) { $S="WARN"; }
if ($6>C) { $S="CRIT"; }
print $0;
}'`
echo "$out"
}
alert(){
vars="(period:'$hist' warning:$warn critical:$crit)"
msg_out="OK - All running spots Spot%OD are <${warn}% $vars"
ec=0
if echo "$info" | egrep -q 'WARN$'; then
warn_cnt=`echo "$info" | egrep -c 'WARN$'`
#warn_out="warning($warn_cnt) - `echo \"$info\" | sed 1d | egrep 'WARN$' | awk '{print $1":"$2":"$3}' | xargs`"
warn_out="warning=$warn_cnt"
msg_detail="($warn_out)"
msg_out="Warning - Found running spots with Spot%OD >${warn}% $vars"
ec=1
fi
if echo "$info" | egrep -q 'CRIT$'; then
crit_cnt=`echo "$info" | egrep -c 'CRIT$'`
#crit_out="critical($crit_cnt) - `echo \"$info\" | sed 1d | egrep 'CRIT$' | awk '{print $1":"$2":"$3}' | xargs`"
crit_out="critical=$crit_cnt"
[ -z "$msg_detail" ] && msg_detail="($crit_out)" || msg_detail="($crit_out $warn_out)"
msg_out="Critical - Found running spots with Spot%OD >${crit}% $vars"
ec=2
fi
msg="$msg_out $msg_detail"
}
s=`date +%s`
# global data retrieval
od_price_full=`ec2py_pri_regions.py`
header='Type Zone Product OD($/H) Avg($/H) Spot%OD Max($/H) Max%OD Last($/H) Last%OD OD_Total($) Total($) Time Status'
case $1 in
-h|--help|help|'')
echo "$Help"
exit 0
;;
"Auto") # this whole mess is for running describe-spot-price-history once
# this optimizes script run time by half and describe-spot-price-history by 2/3
# taking full describe-spot-price-history without limiting by running spots took forever (1m+)
raw_ins=`aws ec2 describe-instances --output json --region us-east-1 \
--filter Name=instance-lifecycle,Values=spot Name=instance-state-name,Values=running`
list_typ=`echo "$raw_ins" |\
jq -r '.Reservations[].Instances[] | "\(.InstanceType)"' |\
sort -u`
#list_plat=(`echo "$raw_ins" |\
# jq -r '.Reservations[].Instances[] | "\(.Platform)_\(.VpcId)"' |\
# sed 's#null_vpc.*#Linux/UNIX (Amazon VPC)#; s#null_null#Linux/UNIX#;' |\
# sort -u`)
list_plat=('Linux/UNIX (Amazon VPC)' 'Linux/UNIX')
raw_json_all=$(aws ec2 describe-spot-price-history --region us-east-1 \
--start-time `date +%s -d "$hist"` \
--end-time `date +%s` \
--instance-types $list_typ --product-descriptions "${list_plat[@]}" \
--output json)
[ -z "$raw_json_all" ] && echo "ERROR - Empty spot price list" && exit 3
job_list=`echo "$raw_ins" |\
jq -r '.Reservations[].Instances[] | "\(.Platform)_\(.VpcId) \(.InstanceType) \(.Placement.AvailabilityZone)"' |\
gawk '{
if($1=="null_null") $1="Linux/UNIX"
else if($1~/^null_vpc/) $1="Linux/UNIX_(Amazon_VPC)"
A[$1" "$2][$3]=1
}
END{
for (f in A){
zones=""
for(z in A[f]){
zones=zones"|"z
}
print f, zones
}
}'`
[ -z "$job_list" ] && echo "Error - No running instances found" && exit 3
thresholds=${3:-100:100}
read warn crit <<< "${thresholds/:/ }"
hist="$2"
info=`(
echo "$header"
echo "$job_list" | while read P T Z; do
prod="${P//_/ }"
itype="$T"
# host format is same as 'date -d'
# zones is a regex
zones="${Z/|/}"
main
done
) | column -t`
;;
*)
prod="$1"
itype="$2"
# host format is same as date -d
hist="$3"
# zones is a regex
zones="$4"
thresholds=${5:-100:100}
read warn crit <<< "${thresholds/:/ }"
raw_json_all=$(aws ec2 describe-spot-price-history --region us-east-1 \
--start-time `date +%s -d "$hist"` \
--end-time `date +%s` \
--instance-types $itype --product-descriptions "$prod" \
--output json)
[ -z "$raw_json_all" ] && echo "ERROR - Empty spot price list" && exit 3
info=`(
echo "$header"
main
) | column -t`
;;
esac
alert
echo $msg
echo "<pre>
$info
</pre>
(run-time: $((`date +%s`-$s))s)"
exit $ec
Raw
ec2py_pri_regions.py
#!/usr/bin/python
# curl 'http://a0.awsstatic.com/pricing/1/ec2/ri-v2/linux-unix-shared.min.js' 2>/dev/null | sed '1,5d; 6s#^callback(##; $s#);$##;' | jsonlint --format --nonstrict | json.sh -l -b | less
# WATCH THEM TABS
import urllib2
import json
import re
urls = ['http://a0.awsstatic.com/pricing/1/ec2/previous-generation/linux-od.min.js', 'http://a0.awsstatic.com/pricing/1/ec2/linux-od.min.js']
ri_url = 'http://a0.awsstatic.com/pricing/1/ec2/ri-v2/linux-unix-shared.min.js'
ru = urllib2.urlopen(ri_url).read()
ru = ru.split("(")[1].split(")")[0]
ru = re.sub(r"(\w*):", r'"\1":', ru)
rnj = json.loads(ru)
ri_url = 'http://a0.awsstatic.com/pricing/1/ec2/previous-generation/ri-v2/linux-unix-shared.min.js'
ru = urllib2.urlopen(ri_url).read()
ru = ru.split("(")[1].split(")")[0]
ru = re.sub(r"(\w*):", r'"\1":', ru)
roj = json.loads(ru)
print "region","gen","name","os","cost","stor_type","stor_count","stor_size","ecu","vcpu","mem","ri1upno_up","ri1upno_h","ri1part_up","ri1part_h","ri1all_up","ri1all_h","ri3part_up","ri3part_h","ri3all_up","ri3all_h"
gen = 1
for u in urls:
u = urllib2.urlopen(u).read()
u = u.split("(")[1].split(")")[0]
u = re.sub(r"(\w*):", r'"\1":', u)
j = json.loads(u)
#print json.dumps(j, indent=1)
for reg in j["config"]["regions"]:
## # limit region
## if reg["region"] != "us-east-1":
## continue
for fam in reg["instanceTypes"]:
for ins in fam["sizes"]:
# search for ri
ri1upno_up = "NA"
ri1upno_h = "NA"
ri1part_up = "NA"
ri1part_h = "NA"
ri1all_up = "NA"
ri1all_h = "NA"
ri3part_up = "NA"
ri3part_h = "NA"
ri3all_up = "NA"
ri3all_h = "NA"
for ri_j in [rnj, roj]:
for rreg in ri_j["config"]["regions"]:
if rreg["region"] == reg["region"]:
#print rreg["region"]
for rname in rreg["instanceTypes"]:
if rname["type"] == ins["size"]:
#print rname["type"]
#ricost = rname["terms"][0]["onDemandHourly"][0]["prices"]["USD"]
for term in rname["terms"]:
if term["term"] == "yrTerm1":
for pay in term["purchaseOptions"]:
if pay["purchaseOption"] == "noUpfront":
for xxx in pay["valueColumns"]:
if xxx["name"] == "upfront":
ri1upno_up = xxx["prices"]["USD"]
if xxx["name"] == "effectiveHourly":
ri1upno_h = xxx["prices"]["USD"]
if pay["purchaseOption"] == "partialUpfront":
for xxx in pay["valueColumns"]:
if xxx["name"] == "upfront":
ri1part_up = xxx["prices"]["USD"]
if xxx["name"] == "effectiveHourly":
ri1part_h = xxx["prices"]["USD"]
if pay["purchaseOption"] == "allUpfront":
for xxx in pay["valueColumns"]:
if xxx["name"] == "upfront":
ri1all_up = xxx["prices"]["USD"]
if xxx["name"] == "effectiveHourly":
ri1all_h = xxx["prices"]["USD"]
if term["term"] == "yrTerm3":
for pay in term["purchaseOptions"]:
if pay["purchaseOption"] == "partialUpfront":
for xxx in pay["valueColumns"]:
if xxx["name"] == "upfront":
ri3part_up = xxx["prices"]["USD"]
if xxx["name"] == "effectiveHourly":
ri3part_h = xxx["prices"]["USD"]
if pay["purchaseOption"] == "allUpfront":
for xxx in pay["valueColumns"]:
if xxx["name"] == "upfront":
ri3all_up = xxx["prices"]["USD"]
if xxx["name"] == "effectiveHourly":
ri3all_h = xxx["prices"]["USD"]
region = reg["region"]
family = fam["type"]
ecu = ins["ECU"]
if re.compile("variable").match(ecu):
ecu = "var"
mem = ins["memoryGiB"]
name = ins["size"]
storage = ins["storageGB"]
re_ssd = re.compile("SSD")
if re.compile(".*x.*SSD.*").match(storage):
stor_type = "SSD"
stor_count = storage.split(" ")[0]
stor_size = storage.split(" ")[2]
elif re.compile("ebsonly").match(storage):
stor_type = "EBS"
stor_count = "NA"
stor_size = "NA"
elif re.compile("60 SSD").match(storage):
stor_type = "SSD"
stor_count = 1
stor_size = 60
else:
stor_type = "LOC"
stor_count = storage.split(" ")[0]
stor_size = storage.split(" ")[2]
os = ins["valueColumns"][0]["name"]
cost = ins["valueColumns"][0]["prices"]["USD"]
vcpu = ins["vCPU"]
#print region,family,ecu,mem,name,storage,os,cost,vcpu
print region,gen,name,os,cost,stor_type,stor_count,stor_size,ecu,vcpu,mem,ri1upno_up,ri1upno_h,ri1part_up,ri1part_h,ri1all_up,ri1all_h,ri3part_up,ri3part_h,ri3all_up,ri3all_h
gen = gen + 1
Raw
help
Overview:
Based on running spot instances, on-demand prices and spot prices the script
calculates average spot price over a set time frame, displays lots of
information and alerts (nagios style) on spot-price % on-demand-price.
Sytanx:
Check specifc type - check_spot_price_adv.sh <single product type> <ins type> <time framae> <zone regex> <warn%>:<crit%>
example: check_spot_price_adv.sh 'Linux/UNIX (Amazon VPC)' c4.2xlarge '-24 hour' '.' 10:30
Check all running spots - check_spot_price_adv.sh Auto <time framae> <warn%>:<crit%>
example: check_spot_price_adv.sh Auto '-24 hour' 50:120
Output:
check_spot_price_adv.sh 'Linux/UNIX' c3.2xlarge '-24 hour' 'us-east-1b|us-east-1e' 10:30
Critical - Found running spots with Spot%OD > 30% (critical=1 warning=1)
Type Zone Product OD($/H) Avg($/H) Spot%OD Max($/H) Max%OD Last($/H) Last%OD OD_Total($) Total($) Time Status
c3.2xlarge us-east-1b Linux/UNIX 0.42 4.20 1000 4.20 1000 4.20 1000 11.33 113.28 26:58:16 CRIT
c3.2xlarge us-east-1e Linux/UNIX 0.42 0.10 24 0.10 24 0.10 24 10.10 2.50 24:02:57 WARN
(run-time 3s)
Requirments:
jq 1.5
gawk
working awscli
Limitation:
Only works on us-east-1
If the table printout is problematic (old nagios) remark at the script bottom
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment