Skip to content

Instantly share code, notes, and snippets.

@InputBlackBoxOutput

InputBlackBoxOutput/dulicate_entries.py

Created August 17, 2021 06:25
Show Gist options
  • Save InputBlackBoxOutput/b0ff7b6bb25930ab9286f880f37a9d33 to your computer and use it in GitHub Desktop.
Save InputBlackBoxOutput/b0ff7b6bb25930ab9286f880f37a9d33 to your computer and use it in GitHub Desktop.
Download ZIP
Python program to remove duplicate entries in a text or CSV file
Raw
dulicate_entries.py
with open("text.txt", 'r') as infile:
data = infile.read().splitlines()
# print(data)
with open("text.txt", 'w') as outfile:
for each in list(set(data)):
outfile.write(each + '\n')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment