Skip to content

Instantly share code, notes, and snippets.

@jasonehines

jasonehines/remove_duplicate_lines.sh

Created June 3, 2019 20:12
Show Gist options
  • Save jasonehines/6fe2c26346f496bb9e0a6562d7e7482a to your computer and use it in GitHub Desktop.
Save jasonehines/6fe2c26346f496bb9e0a6562d7e7482a to your computer and use it in GitHub Desktop.
Download ZIP
Remove duplicate lines preserving order #awk #shell
Raw
remove_duplicate_lines.sh
# https://iridakos.com/how-to/2019/05/16/remove-duplicate-lines-preserving-order-linux.html
awk '!visited[$0]++' your_file > deduplicated_file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment