Skip to content

Instantly share code, notes, and snippets.

@ifnull
Created December 1, 2014 00:27
Show Gist options
  • Save ifnull/37657fa00ab0f4a74bba to your computer and use it in GitHub Desktop.
Save ifnull/37657fa00ab0f4a74bba to your computer and use it in GitHub Desktop.
Extract all unique URLs from from an access log.

Run brew install gnu-sed on OSX and replace sed with gsed in the command below.

sed -r "s/.*(GET|POST|HEAD|PROPFIND) ([^\?]*?)(\?.*?)? HTTP.*/\2/" \
access_log | grep -v "^/$" | sort | uniq -u
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment