Skip to content

Instantly share code, notes, and snippets.

@mattyjones
Created May 8, 2014 15:11
Show Gist options
  • Save mattyjones/d68c7768f0e102ac76dd to your computer and use it in GitHub Desktop.
Save mattyjones/d68c7768f0e102ac76dd to your computer and use it in GitHub Desktop.
This is an sec rulle file to help prevent an email storm with Icinga/Nagios caused by excessive notifications.
type=SingleWithThreshold
ptype=RegExp
pattern=dsn=(\d+)\.(\d+)\.(\d+)
context=maillog_tripped
desc=$0
action=shellcmd shutdown -h now
window=10
thresh=2
type=SingleWithThreshold
ptype=RegExp
pattern=dsn=(\d+)\.(\d+)\.(\d+)
context=icingalog_tripped
desc=$0
action=pipe 'The email threshold was tripped at %t /n Stopping the sendmil service' /bin/mail -s "Email Storm Alert" %e ; \
create mail_tripped_context ; \
shellcmd /sbin/service ido2db stop
shellcmd /sbin/service icinga stop
shellcmd /bin/sleep 30
window=10
thresh=2
type=SingleWithThreshold
ptype=RegExp
pattern=NOTIFICATION
desc=$0
action=eval %e {'email_address'} ;\
pipe 'The email threshold was tripped at %t /n Stopping the sendmil service' /bin/mail -s "Email Storm Alert" %e ; \
create icingalog_tripped ; \
shellcmd /sbin/service sendmail stop ; \
shellcmd /bin/sleep 30 ; \
spawn /usr/bin/tail -n 0 -f /var/log/maillog ; \
window=60
thresh=25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment