This is quite simple, and requires just that you add your script somewhere on the Jenkins server (you may also put shell code directly in to Jenkins, but..why). Anyway.
When failing tests, you must use:
exit
This causes Jenkins to go in to a failed-build state :-)