send a dummy alert to prometheus-alertmanager

alert.sh

#!/bin/bash

name=$RANDOM
url='http://localhost:9093/api/v1/alerts'

echo "firing up alert $name" 

# change url o
curl -XPOST $url -d "[{ 
	\"status\": \"firing\",
	\"labels\": {
		\"alertname\": \"$name\",
		\"service\": \"my-service\",
		\"severity\":\"warning\",
		\"instance\": \"$name.example.net\"
	},
	\"annotations\": {
		\"summary\": \"High latency is high!\"
	},
	\"generatorURL\": \"http://prometheus.int.example.net/<generating_expression>\"
}]"

echo ""

echo "press enter to resolve alert"
read

echo "sending resolve"
curl -XPOST $url -d "[{ 
	\"status\": \"resolved\",
	\"labels\": {
		\"alertname\": \"$name\",
		\"service\": \"my-service\",
		\"severity\":\"warning\",
		\"instance\": \"$name.example.net\"
	},
	\"annotations\": {
		\"summary\": \"High latency is high!\"
	},
	\"generatorURL\": \"http://prometheus.int.example.net/<generating_expression>\"
}]"

echo ""

single quote is your friend..

-d '{
  "alertname": "'$name'"
}'

This is useful! Thanks for posting it.

Very useful. Thanks !

Thanks!
Original doc: https://prometheus.io/docs/alerting/clients/

Thanks! 👍

To save folks the frustration - this script actually doesn't send the resolve alert according to the v1 api spec. The prometheus contributors have point this out prometheus/alertmanager#1306. The alert only resets after a few minutes because alertmanager expects to get a list of currently firing alerts every 30s-3mins. In order to resolve the alert immediately after the curl you have to include the endsAt timestamp:

url='http://localhost:9093/api/v1/alerts'
echo "Firing up alert" 
curl -XPOST $url -d '[{"status": "firing","labels": {"alertname": "my_cool_alert","service": "curl","severity": "warning","instance": "0"},"annotations": {"summary": "This is a summary","description": "This is a description."},"generatorURL": "http://prometheus.int.example.net/<generating_expression>","startsAt": "2020-07-23T01:05:36+00:00"}]'
echo ""

echo "press enter to resolve alert"
read

echo "sending resolve"
curl -XPOST $url -d '[{"status": "resolved","labels": {"alertname": "my_cool_alert","service": "curl","severity": "warning","instance": "0"},"annotations": {"summary": "This is a summary","description": "This is a description."},"generatorURL": "http://prometheus.int.example.net/<generating_expression>","startsAt": "2020-07-23T01:05:36+00:00","endsAt": "2020-07-23T01:05:38+00:00"}]'
echo ""

Please note you'll have to update the startsAt and endsAt timestamps. I used this site to get RFC3339 compliant timestamps:
https://www.unixtimestamp.com/

So timestamp generation code as per suggestion above

startsAt=`date --iso-8601=seconds`
... # after the `read` command
endsAt=`date --iso-8601=seconds`

You just need to insert these shell variables into the curl command. This works for me. Using the --rfc-3339 instead of --iso-8601 I found T separator missing which may or may not be an issue.

I've revisited this snippet multiple times each time having to make the alterations. Here's a gist incorporating all the changes https://gist.github.com/carinadigital/fd2960fdccd77dbdabc849656c43a070

My five cents https://github.com/alexanderfefelov/docker-backpack/blob/master/prometheus/alertmanager/utils/emit-alerts.sh

Use this if you're on Mac/OSX: $(date -u '+%FT%TZ')

tks

You can pass different severity as argument to check different channels, for example:

sh test.sh 'critical'

#!/bin/bash

# Set default values
name=$RANDOM
url='https://alertmanager.local/api/v1/alerts'
summary='Testing summary!'
instance="$name.example.net"
default_severity='warning'

# Function to send alert
send_alert() {
    local status=$1
    local custom_severity=$2
    local current_severity=${custom_severity:-$default_severity}

    curl -XPOST $url -d "[
        {
            \"status\": \"$status\",
            \"labels\": {
                \"alertname\": \"$name\",
                \"service\": \"my-service\",
                \"severity\":\"$current_severity\",
                \"instance\": \"$instance\"
            },
            \"annotations\": {
                \"summary\": \"$summary\"
            },
            \"generatorURL\": \"https://prometheus.local/<generating_expression>\"
        }
    ]"
    echo ""
}

# Main script
echo "Firing up alert $name"
send_alert "firing" "$1"

read -p "Press enter to resolve alert"

echo "Sending resolve"
send_alert "resolved" "$1"

You can pass different severity as argument to check different channels, for example:

sh test.sh 'critical'

#!/bin/bash

# Set default values
name=$RANDOM
url='https://alertmanager.local/api/v1/alerts'
summary='Testing summary!'
instance="$name.example.net"
default_severity='warning'

# Function to send alert
send_alert() {
    local status=$1
    local custom_severity=$2
    local current_severity=${custom_severity:-$default_severity}

    curl -XPOST $url -d "[
        {
            \"status\": \"$status\",
            \"labels\": {
                \"alertname\": \"$name\",
                \"service\": \"my-service\",
                \"severity\":\"$current_severity\",
                \"instance\": \"$instance\"
            },
            \"annotations\": {
                \"summary\": \"$summary\"
            },
            \"generatorURL\": \"https://prometheus.local/<generating_expression>\"
        }
    ]"
    echo ""
}

# Main script
echo "Firing up alert $name"
send_alert "firing" "$1"

read -p "Press enter to resolve alert"

echo "Sending resolve"
send_alert "resolved" "$1"

should change from v1 to v2 and add an header:

    -H "Content-Type: application/json" \

Thanks, this is very helpful!
Minor changes were needed to work with newer versions:

• Change the API version of the alertmanager to v2.
• Add a header to the curl calls:

-H "Content-Type: application/json"