Skip to content

Instantly share code, notes, and snippets.

@0xKD
Last active June 26, 2016 11:11
Show Gist options
  • Save 0xKD/a7d2711f6e4c96ddc1d7 to your computer and use it in GitHub Desktop.
Save 0xKD/a7d2711f6e4c96ddc1d7 to your computer and use it in GitHub Desktop.
Clone webpage for local browsing
#!/usr/local/bin/python3
from urllib.parse import urlparse
import sys
if __name__ == '__main__':
if len(sys.argv) == 2:
sys.stdout.write(urlparse(sys.argv[1]).netloc)
else:
sys.stdout.write('doesnotexist.lampost')
#!/bin/sh
/usr/local/bin/wget --mirror --no-clobber --page-requisites --html-extension --convert-links --no-parent -U 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4' -e robots=off --span-hosts --domains akamai.net,akamaiedge.net,akamaihd.net,edgesuite.net,edgekey.net,srip.ne,akamaitechnologies.com,akamaitechnologies.fr,llnwd.net,edgecastcdn.net,systemcdn.net,transactcdn.net,v1cdn.net,v2cdn.net,v3cdn.net,v4cdn.net,v5cdn.net,hwcdn.net,simplecdn.net,instacontent.net,footprint.net,fpbns.net,ay1.b.yahoo.com,yahooapis.com,googleusercontent.com,googlehosted.com,gstatic.com,doubleclick.net,insnw.net,inscname.net,internapcdn.net,cloudfront.net,netdna-cdn.com,netdna-ssl.com,netdna.com,kxcdn.com,cotcdn.net,cachefly.net,bo.lt,cloudflare.com,afxcdn.net,lxdns.com,wscdns.com,wscloudcdn.com,ourwebpic.com,att-dsa.net,vo.msecnd.net,azureedge.net,voxcdn.net,bluehatnetwork.com,swiftcdn1.com,cdngc.net,gccdn.net,panthercdn.com,fastly.net,fastlylb.net,nocookie.net,gslb.taobao.com,gslb.tbcache.com,mirror-image.net,yottaa.net,cubecdn.net,cdn77.net,cdn77.org,incapdns.net,bitgravity.com,r.worldcdn.net,r.worldssl.net,tbcdn.cn,taobaocdn.com,ngenix.net,pagerain.net,ccgslb.com,cdn.sfr.net,azioncdn.net,azioncdn.com,azion.net,cdncloud.net.au,rncdn1.com,cdnsun.net,mncdn.com,mncdn.net,mncdn.org,cdn.jsdelivr.net,nyiftw.net,nyiftw.com,resrc.it,zenedge.net,lswcdn.net,lswcdn.eu,revcn.net,revdn.net,caspowa.com,twimg.com,facebook.net,fbcdn.net,cdninstagram.com,rlcdn.com,wp.com,aads1.net,aads-cn.net,aads-cng.net,squixa.net,bisongrid.net,cdn.gocache.net,hiberniacdn.com,cdntel.net,raxcdn.com,unicorncdn.net,optimalcdn.com,`~/parse.py $1` $1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment