Skip to content

Instantly share code, notes, and snippets.

@atomotic

atomotic/gist:7069028

Created October 20, 2013 12:39
Show Gist options
  • Save atomotic/7069028 to your computer and use it in GitHub Desktop.
Save atomotic/7069028 to your computer and use it in GitHub Desktop.
Download ZIP
warcprox
Raw
gistfile1.txt
~ virtualenv env
~ source env/bin/activate
~ pip install git+https://github.com/nlevitt/warctools@tweaks
~ pip install pyOpenSSL
~ git clone git clone https://github.com/nlevitt/warcprox
~ cd warcprox
~ python warcprox.py --rollover-idle-time=7200
2013-10-20 14:36:07,923 66818 MainThread INFO server_activate(warcprox.py:346) listening on 127.0.0.1:8080
2013-10-20 14:36:07,924 66818 MainThread INFO _read_ca(warcprox.py:75) read CA key+cert from ./warcprox-ca.pem
2013-10-20 14:36:07,928 66818 WarcWriterThread INFO run(warcprox.py:510) WarcWriterThread starting, directory=/private/tmp/warcprox/warcs gzip=False rollover_size=1000000000 rollover_idle_time=7200 prefix=WARCPROX port=8080
~ curl -x http://localhost:8080 http://netpreserve.org
~ ls warcs
WARCPROX-20131020143413490-00000-66630-xtul.lan-8080.warc
~ warcindex warcs/WARCPROX-20131020143413490-00000-66630-xtul.lan-8080.warc
#WARC filename offset warc-type warc-subject-uri warc-record-id content-type content-length
warcs/WARCPROX-20131020143413490-00000-66630-xtul.lan-8080.warc 0 warcinfo None <urn:uuid:eb4f82bd-52d9-439d-9306-1cf874b368f8> application/warc-fields 142
warcs/WARCPROX-20131020143413490-00000-66630-xtul.lan-8080.warc 411 response http://www.netpreserve.org <urn:uuid:b9929da7-28aa-410b-826d-f95ee3b70075> application/http;msgtype=response 46450
warcs/WARCPROX-20131020143413490-00000-66630-xtul.lan-8080.warc 47278 request http://www.netpreserve.org <urn:uuid:5458b4fa-65bd-436c-96fc-34e666845e28> application/http;msgtype=request 180
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment