Skip to content

Instantly share code, notes, and snippets.

@Averroes
Created April 10, 2015 14:11
Show Gist options
  • Select an option

  • Save Averroes/d2a699cb67662ada71af to your computer and use it in GitHub Desktop.

Select an option

Save Averroes/d2a699cb67662ada71af to your computer and use it in GitHub Desktop.
removing duplicates from a sequence while maintaining order
# example2.py
#
# Remove duplicate entries from a sequence while keeping order
def dedupe(items, key=None):
seen = set()
for item in items:
val = item if key is None else key(item)
if val not in seen:
yield item
seen.add(val)
if __name__ == '__main__':
a = [
{'x': 2, 'y': 3},
{'x': 1, 'y': 4},
{'x': 2, 'y': 3},
{'x': 2, 'y': 3},
{'x': 10, 'y': 15}
]
print(a)
print(list(dedupe(a, key=lambda a: (a['x'],a['y']))))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment