Real-time notifications from a task worker

gistfile1.txt

I am using Gearman to import content from a 3rd party via their API. I need the ability to notify users IN REAL TIME that an import of a specific item has failed or been blocked (we only accept MP3's and need to block video imports) or that the Gearman job itself has crashed (there is sometimes wonkiness with MySql deciding it needs to 'go away').

Thoughts and suggestions on how to accomplish this task would be greatly appreciated.

Gearman is just a job processing system. If you want application notifications, you will have to do it outside of the scope of the Gearman protocol. Somewhere way down my list of stuff I want to do when I am rich is a layer on top of Gearman that does all that stuff. It could be an extension of GearmanManager perhaps. Just a standard lib that provided job status information. But for now, everyone has to implement their own application level messaging inside their workers. And then query said messaging from other code.

I realize that Gearman can only do so much, and it really is application-level notifications. Some have suggested firing off another Gearman worker request upon success or failure, and I suppose I that is a solution but it strikes me as not using Gearman for what it really is for.

I could be wrong.

Ben Ramsey suggested that maybe Amazon's SNS might be a good fit.

chartjes. Typically I would (and have) handled this via a polling system. Regardless of how you want to handle the notifications (stick them in memcached, use a message queue (SQS, Rabbit, etc), write it to a file, whatever) ...

And just write a little javascript on the client side, that keeps hitting every second or two, an endpoint that checks for the message.