chaosq is an add-on providing a dedicated Amazon Simple Queue Service (SQS) queue. Amazon SQS is a high performance, infinitely scalable, and highly reliable message broker, built on top of Amazon Web Services.
Using a message queue like Amazon SQS is the simplest and most fault tolerant way to decouple, distribute, and scale your Heroku applications effectively, regardless of the size of your application.
Amazon SQS is supported by a large variety of programming languages and web frameworks including:
chaosq can be attached to a Heroku application via the CLI:
:::term $ heroku addons:add chaosq:test
-----> Adding chaosq to sharp-mountain-4005... done, v18 (free)
Once chaosq has been added a CHAOSQ_URL
setting will be available in the app
configuration and will contain the canonical URL used to access the newly
provisioned Amazon SQS queue. This can be confirmed using the
heroku config:get
command.
:::term $ heroku config:get CHAOSQ_URL
sqs://user:[email protected]/queueid
After installing chaosq the application should be configured to fully integrate with the add-on.
If you're using vanilla Ruby, you can interact with Amazon SQS using Amazon's official AWS Ruby SDK.
To install the AWS Ruby SDK, see the official documentation.
Once you've got the AWS Ruby SDK installed, you can easily interact with your new Amazon SQS queue:
- View stats about the queue.
- Add messages to the queue.
- Remove messages from the queue.
- Modify queue settings.
The script below uses the AWS Ruby SDK to connect to your new Amazon SQS queue, add a message to the queue, then consume the message and remove it from the queue.
:::ruby
require 'cgi'
require 'uri'
require 'rubygems'
require 'aws-sdk'
##### GLOBALS
CHAOSQ_URL = URI(ENV['CHAOSQ_URL'])
AWS.config({
:access_key_id => CGI::unescape(CHAOSQ_URL.user),
:secret_access_key => CGI::unescape(CHAOSQ_URL.password),
:sqs_endpoint => CHAOSQ_URL.host,
})
sqs = AWS::SQS.new
##### SQS INTERACTION
queue = sqs.queues.named(CHAOSQ_URL.path.split('/').last)
puts "Fetched queue."
puts "Total messages in queue: #{queue.approximate_number_of_messages}"
queue.send_message('hello, world!')
puts "Sent message to queue: 'hello, world!'"
puts "Total messages in queue: #{queue.approximate_number_of_messages}"
msg = queue.receive_message
puts "Retrieved message: '#{msg.body}'"
msg.delete
puts "Deleted message from queue."
puts "Total messages in queue: #{queue.approximate_number_of_messages}"
For more examples and information, the official AWS Ruby SDK documentation provides lots of example code.
Using SQS with vanilla Python is easy using the boto library.
To get started, you'll need to install boto using pip, the Python package manager:
::: term $ pip install -U boto
Once you've got boto installed, you can perform any standard SQS functions. The script below uses boto to connect to your new Amazon SQS queue, add a message to the queue, then consume the message and remove it from the queue.
::: python
from os import environ
from urllib import unquote
from urlparse import urlparse
from boto.sqs.connection import SQSConnection
from boto.sqs.message import Message
##### GLOBALS
CHAOSQ_URL = urlparse(environ['CHAOSQ_URL'])
conn = SQSConnection(unquote(CHAOSQ_URL.username), unquote(CHAOSQ_URL.password))
##### SQS INTERACTION
queue = conn.get_queue(CHAOSQ_URL.path.split('/')[-1])
print 'Fetched queue.'
print 'Total messages in queue: %d' % queue.count()
message = Message()
message.set_body('hello, world!')
queue.write(message)
print "Sent message to queue: 'hello, world!'"
print 'Total messages in queue: %d' % queue.count()
message = queue.get_messages()[0]
print "Retrieved message: '%s'" % message.get_body()
queue.delete_message(message)
print 'Deleted message from queue.'
For more examples and information, the official boto SQS documentation provides lots of example code.
You can use SQS with Django via the Celery library. Celery allows you to process tasks asynchronously, using Amazon SQS as a message broker.
Before you begin, you'll need to install Celery. Since each Django project's requirements are different, you'll need to follow the official django-celery getting started guide.
Once you've got django-celery configured to your liking, you'll need to configure your broker settings in order to make django-celery use your new Amazon SQS queue.
To do this, open your project's settings.py
file, and add the following:
::: python
from os import environ
from urllib import unquote
from urlparse import urlparse
CHAOSQ_URL = urlparse(environ['CHAOSQ_URL'])
BROKER_URL = '%s://%s:%s@' % (CHAOSQ_URL.scheme, unquote(CHAOSQ_URL.username), unquote(CHAOSQ_URL.password))
Once you've got this setup, you can use Celery like you would normally to process tasks asynchronously!
For more information on django-celery and Amazon SQS, the django-celery SQS documentation contains everything you need to know.
NOTE: In order for django-celery to work properly with Amazon SQS, you'll need to have both the django-celery and boto libraries installed.
chaosq's upgrade process is completely instant and transparent, which means you can freely upgrade or downgrade your chaosq plan at any time with no downtime or application connectivity issues.
Use the heroku addons:upgrade
command to migrate to a new plan.
:::term $ heroku addons:upgrade chaosq:medium
-----> Upgrading chaosq:medium to sharp-mountain-4005... done, v18 ($49/mo)
Your plan has been updated to: chaosq:medium
chaosq can be removed via the CLI.
:::term $ heroku addons:remove chaosq
-----> Removing chaosq from sharp-mountain-4005... done, v20 (free)
All chaosq support and runtime issues should be submitted via one of the Heroku Support channels. Any non-support related issues or product feedback is welcome via email: [email protected]
We should just recommend how to use with Celery, not Django (in my opinion). The new celery 3.0 interface is very simple.