Skip to content

Instantly share code, notes, and snippets.

@fdemmer
Created September 1, 2015 17:46
Show Gist options
  • Save fdemmer/c7568f458d3b3f73c7c6 to your computer and use it in GitHub Desktop.
Save fdemmer/c7568f458d3b3f73c7c6 to your computer and use it in GitHub Desktop.
make huge queries chunk by chunk
def queryset_iterator(query, chunk_size=1000):
count = 0
total = query.count()
while count < total:
for row in query[count:count + chunk_size]:
count += 1
yield row
gc.collect()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment