Configuring your Django project to use Celery http://celeryq.org/docs/django-celery/getting-started/first-steps-with-django.html
Run the worker server in the foreground, so we can see what’s going on without consulting the logfile:
$ python manage.py celeryd -l info
For a complete listing of the command line options available, use the help command:
$ python manage.py help celeryd
You can enable better debugging info by running this command:
$ celeryd -Q build,appserver,database,celery -l debug
celeryev - event monitoring celeryctl - sending control messages to cluster
http://ask.github.com/celery/getting-started/broker-installation.html#installing-rabbitmq-on-os-x
Before Celery will work, it needs to have a broker to talk to. The most popular and recommended broker is RabbitMQ.
Start the RabbitMQ server with:
$ rabbitmq-server Activating RabbitMQ plugins ... 0 plugins activated: ... broker running
Or you can run the server in the background with:
$ rabbitmq-server -detached
Stop the server with:
$ rabbitmqctl stop
In the Celery section of the Django settings.py file, you define a user to connect to the broker. By default this is:
BROKER_USER = "guest" BROKER_PASSWORD = "guest"
So to create the user with RabbitMQ, type the following:
$ rabbitmqctl add_user guest guest Creating user "guest" ... ...done.
Set the permissions:
$ rabbitmqctl set_permissions -p / guest ".*" ".*" ".*" Setting permissions for user "guest" in vhost "/" ... ...done.
"/" is the name of the vhost, which you could set with this command (when I tried to run it, it said that this vhost had already been created):
$ rabbitmqctl add_vhost myvhost
First steps using Django with Celery: http://celeryproject.org/docs/django-celery/getting-started/first-steps-with-django.html
You can run Celery in the foreground with "manage.py celeryd -l info":
$ python manage.py celeryd -l info
[2010-12-27 17:35:12,926: WARNING/MainProcess] celery@jazbook.local v2.1.4 is starting.
[2010-12-27 17:35:12,931: WARNING/MainProcess] /Users/nateaune/code/djangozoom/lib/python2.6/site-packages/celery/apps/worker.py:110: UserWarning: Using settings.DEBUG leads to a memory leak, never use this setting in a production environment!
warnings.warn("Using settings.DEBUG leads to a memory leak, "
[2010-12-27 17:35:12,931: WARNING/MainProcess]
Configuration ->
. broker -> amqp://guest@localhost:5672/
. queues ->
. celery -> exchange:celery (direct) binding:celery
. concurrency -> 2
. loader -> djcelery.loaders.DjangoLoader
. logfile -> [stderr]@INFO
. events -> OFF
. beat -> OFF
. tasks ->
. dzworker.tasks.add
[2010-12-27 17:35:13,118: INFO/PoolWorker-1] child process calling self.run()
[2010-12-27 17:35:13,121: INFO/PoolWorker-2] child process calling self.run()
[2010-12-27 17:35:13,124: WARNING/MainProcess] celery@jazbook.local has started.
Running Celery in daemon mode: http://celeryq.org/docs/cookbook/daemonizing.html#example-django-configuration
or using Supervisor with Django: https://github.com/ask/django-celery/blob/master/contrib/supervisord/celeryd.conf
or without Django: https://github.com/ask/celery/tree/master/contrib/supervisord/
Login to the server:
$ cd /usr/local/noderabbit/dztasks/tasks
$ . ../bin/activate
(dztasks)$ celeryctl status
appserver-0001: OK
proxy-0001: OK
web-0001: OK
db-0001: OK
4 nodes online.
(dztasks)$ celeryctl inspect active
<- active
-> appserver-0001: OK
- empty -
-> proxy-0001: OK
- empty -
-> web-0001: OK
- empty -
-> db-0001: OK
- empty -
(dztasks)$ celeryctl inspect scheduled
<- scheduled
-> db-0001: OK
- empty -
-> appserver-0001: OK
- empty -
-> proxy-0001: OK
- empty -
-> web-0001: OK
- empty -
Add these lines to your code:
from celery.contrib import rdb rdb.set_trace()
Then you can connect to the pdb debugger with:
$ telnet localhost 6900
Read more at http://ask.github.com/celery/tutorials/debugging.html
Here was the traceback:
Traceback (most recent call last):
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/core/handlers/base.py", line 111, in get_response
response = callback(request, *callback_args, **callback_kwargs)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/contrib/auth/decorators.py", line 23, in _wrapped_view
return view_func(request, *args, **kwargs)
File "/var/www/nrweb/nrweb/usercontrol/djangozoom/decorators.py", line 39, in wrapper
response = view_func(request, *args, **kwargs)
File "/var/www/nrweb/nrweb/usercontrol/dz2/views.py", line 31, in dashboard
}, RequestContext(request))
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/shortcuts/__init__.py", line 20, in render_to_response
return HttpResponse(loader.render_to_string(*args, **kwargs), **httpresponse_kwargs)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/loader.py", line 188, in render_to_string
return t.render(context_instance)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 123, in render
return self._render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 117, in _render
return self.nodelist.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 744, in render
bits.append(self.render_node(node, context))
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/debug.py", line 73, in render_node
result = node.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/loader_tags.py", line 127, in render
return compiled_parent._render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 117, in _render
return self.nodelist.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 744, in render
bits.append(self.render_node(node, context))
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/debug.py", line 73, in render_node
result = node.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/loader_tags.py", line 127, in render
return compiled_parent._render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 117, in _render
return self.nodelist.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 744, in render
bits.append(self.render_node(node, context))
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/debug.py", line 73, in render_node
result = node.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/loader_tags.py", line 64, in render
result = block.nodelist.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 744, in render
bits.append(self.render_node(node, context))
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/debug.py", line 73, in render_node
result = node.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/loader_tags.py", line 64, in render
result = block.nodelist.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 744, in render
bits.append(self.render_node(node, context))
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/debug.py", line 73, in render_node
result = node.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/loader_tags.py", line 64, in render
result = block.nodelist.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 744, in render
bits.append(self.render_node(node, context))
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/debug.py", line 73, in render_node
result = node.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/defaulttags.py", line 311, in render
return self.nodelist_true.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 744, in render
bits.append(self.render_node(node, context))
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/debug.py", line 73, in render_node
result = node.render(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/defaulttags.py", line 227, in render
nodelist.append(node.render(context))
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/debug.py", line 90, in render
output = self.filter_expression.resolve(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 510, in resolve
obj = self.var.resolve(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 653, in resolve
value = self._resolve_lookup(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/django/template/base.py", line 698, in _resolve_lookup
current = current()
File "/var/www/nrweb/nrweb/usercontrol/dz2/models.py", line 512, in status
return self.get_task_meta()['status']
File "/var/www/nrweb/nrweb/usercontrol/dz2/models.py", line 450, in get_task_meta
taskmeta = default_backend.get_task_meta(task_id)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/celery/backends/base.py", line 187, in get_task_meta
meta = self._get_task_meta_for(task_id)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/celery/backends/database.py", line 63, in _get_task_meta_for
task = session.query(Task).filter(Task.task_id == task_id).first()
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/orm/query.py", line 1620, in first
ret = list(self[0:1])
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/orm/query.py", line 1528, in __getitem__
return list(res)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/orm/query.py", line 1689, in __iter__
return self._execute_and_instances(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/orm/query.py", line 1694, in _execute_and_instances
mapper=self._mapper_zero_or_none())
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/orm/session.py", line 720, in execute
clause, params or {})
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1191, in execute
params)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1271, in _execute_clauseelement
return self.__execute_context(context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1302, in __execute_context
context.parameters[0], context=context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1401, in _cursor_execute
context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/engine/base.py", line 1394, in _cursor_execute
context)
File "/var/www/nrweb/nrweb/usercontrol/lib/python2.6/site-packages/sqlalchemy/engine/default.py", line 299, in do_execute
cursor.execute(statement, parameters)
TemplateSyntaxError: Caught OperationalError while rendering: (OperationalError) terminating connection due to administrator command
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
'SELECT celery_taskmeta.id AS celery_taskmeta_id, celery_taskmeta.task_id AS celery_taskmeta_task_id, celery_taskmeta.status AS celery_taskmeta_status, celery_taskmeta.result AS celery_taskmeta_result, celery_taskmeta.date_done AS celery_taskmeta_date_done, celery_taskmeta.traceback AS celery_taskmeta_traceback \nFROM celery_taskmeta \nWHERE celery_taskmeta.task_id = %(task_id_1)s \n LIMIT 1 OFFSET 0' {'task_id_1': '46a65be4-fd75-456f-ac91-8ffa2f4369cc'}
Buildout recipe for installing RabbitMQ http://pypi.python.org/pypi/rod.recipe.rabbitmq
Introduction to AMQP Messaging with RabbitMQ http://www.slideshare.net/somic/introduction-to-amqp-messaging-with-rabbitmq