Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save TotallyNotAHaxxer/7ae22233d0184cb41b9822cbd2fe4844 to your computer and use it in GitHub Desktop.
Save TotallyNotAHaxxer/7ae22233d0184cb41b9822cbd2fe4844 to your computer and use it in GitHub Desktop.
Bash One Liner - Iterate through source directories collecting URLs amongst source files
find . -type f -name "*.js" -exec bash -c 'file="{}"; grep -o -E "(https?:\/\/|www\.)[a-zA-Z0-9][a-zA-Z0-9\-]*\.[a-zA-Z0-9][a-zA-Z0-9\-\.]*[a-zA-Z0-9]\b(\/[^\"'\''<>\\s]*)?" "$file" | sed "s/[\"'\'']*$//g" | while read -r url; do echo "{\"file\":\"$file\",\"url\":\"$url\"}"; done' \; | jq -s 'group_by(.url) | map({url: .[0].url, files: map(.file) | unique})' > site_list.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment