Skip to content

Instantly share code, notes, and snippets.

@SZ-Edward
Created December 7, 2017 11:04
Show Gist options
  • Save SZ-Edward/f6b04c29c151d925e9b83435e82fdc80 to your computer and use it in GitHub Desktop.
Save SZ-Edward/f6b04c29c151d925e9b83435e82fdc80 to your computer and use it in GitHub Desktop.
Remove subsequent duplicates from list with keeping order.
def remove_duplicates_of_list(lst):
if len(lst) < 2:
return lst
d = {}
for i in xrange(len(lst)):
if lst[i] in d.values():
continue
d.update({i: lst[i]})
return [lst[key] for key in sorted(d.keys())]
@hlequien
Copy link

hlequien commented Dec 7, 2017

Hello I have an improvement
lst = list(set(lst)) does the job, is built-in and is way faster.

Try with a 500000 elements list :

a = range(0, 100000)
lst = []
for i in range(0, 5):
    lst.extend(a)
print len(lst)
print len(list(set(lst)))
print len(remove_duplicates_of_list(lst))

:-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment