indygwyn/dedupkeeporder.sh

Last active May 20, 2019 14:10

Star (0) You must be signed in to star a gist
Fork (0) You must be signed in to fork a gist

Select an option

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/indygwyn/39ff19cedc4b3598d2fa67d38486c2bc.js"></script>
Save indygwyn/39ff19cedc4b3598d2fa67d38486c2bc to your computer and use it in GitHub Desktop.

Download ZIP

remove duplicate lines preserve file order #bash #awk

Raw

dedupkeeporder.sh

awk '!visited[$0]++' your_file > deduplicated_file

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment