Skip to content

Instantly share code, notes, and snippets.

@mdeous
Created March 13, 2011 03:49
Show Gist options
  • Save mdeous/867844 to your computer and use it in GitHub Desktop.
Save mdeous/867844 to your computer and use it in GitHub Desktop.
# container is just a very basic class i use to store objects used in many parts of the code
container.scheduler.add_jobstore(ShelveJobStore(settings.CRONJOBS_DB_FILE), 'crondb')
# this is the part of code used to add jobs
if name == 'reindexer':
target_func = reindex
kwargs = {}
else:
target_func = container.rpc.run_spider
kwargs = {
'name': name,
'is_cronjob': True
}
job = container.scheduler.add_interval_job(
target_func,
minutes=interval,
name=name,
kwargs=kwargs,
)
# here are the 2 functions used for jobs:
def reindex():
if container.reindexer.isrunning:
return
container.reindexer = Reindexer()
container.reindexer.start()
def run_spider(self, **spider_kwargs):
self._jsonrpc_call(self.host, 'crawler/queue', 'append_spider_name', **spider_kwargs)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment