Skip to content

Instantly share code, notes, and snippets.

@pythonhacker

pythonhacker/scrapy_settings.txt

Created February 13, 2019 10:40
Show Gist options
  • Save pythonhacker/619a46a0119d01e09b61f88891273a7c to your computer and use it in GitHub Desktop.
Save pythonhacker/619a46a0119d01e09b61f88891273a7c to your computer and use it in GitHub Desktop.
Download ZIP
Find Scrapy HTTP cache directory
Raw
scrapy_settings.txt
$ scrapy shell -s HTTPCACHE_ENABLED=True
2017-02-28 10:41:41 [scrapy.utils.log] INFO: Scrapy 1.3.2 started (bot: httpbin)
(...)
>>> from scrapy.utils.misc import load_object
>>> storage = load_object(settings['HTTPCACHE_STORAGE'])(settings)
>>> storage.cachedir
'/home/paul/scrapy/httpbin/.scrapy/httpcache'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment