check_smtp.sh

#!/bin/bash

# make sure dog cli is installed first
#     sudo easy_install dogapi
# then run
#     dog event post "My first post" "Yada yada yada"
# to make sure you're prompted for credentials
# which will be stored in $HOME/.dogrc

# Now let's test SMTP on localhost
ok=$(echo QUIT | nc localhost 25)

# If the check fails, send an event to datadog
# mention (@) [email protected] to trigger an email
# hence the 2 @ signs.
if [ $ok -ne 0 ]; then
    dog event post --tags smtp "Danger, Will Robinson" "SMTP is not responding @[email protected]"
fi

If you use
dog metric post
you can also create metrics from the command line.

for instance dog metric post smtp.bytes 1234 and then create a dashboard with a graph for that metric.