Skip to content

Instantly share code, notes, and snippets.

@DArmstrong87
Forked from victorono/remove_duplicates.py
Last active December 1, 2023 19:16
Show Gist options
  • Save DArmstrong87/26da46b1b829db67af5eb31f4297cac4 to your computer and use it in GitHub Desktop.
Save DArmstrong87/26da46b1b829db67af5eb31f4297cac4 to your computer and use it in GitHub Desktop.
Django: Remove duplicate objects where there is more than one field to compare. Keep earliest entry.
from django.db.models import Count, Min
unique_fields = ['field_1', 'field_2']
duplicates = (
MyModel.objects.values(*unique_fields)
.order_by()
.annotate(min_id=Min('id'), count_id=Count('id'))
.filter(count_id__gt=1)
)
for duplicate in duplicates:
(
MyModel.objects
.filter(**{x: duplicate[x] for x in unique_fields})
.exclude(id=duplicate['min_id'])
.delete()
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment