Skip to content

Instantly share code, notes, and snippets.

@davidwtbuxton
Created January 8, 2018 18:52
Show Gist options
  • Save davidwtbuxton/c4016f93a98a6169bd47979303d8ab68 to your computer and use it in GitHub Desktop.
Save davidwtbuxton/c4016f93a98a6169bd47979303d8ab68 to your computer and use it in GitHub Desktop.
Recipe for chunking an iterable
import itertools
def chunker(iterable, n):
"""Collect data into chunks.
This is different to the grouper recipe in itertools in that the final
chunk may be shorter.
>>> chunks = chunker('abcde', 2)
>>> list(chunks)
[['a', 'b'], ['c', 'd'], ['e']]
>>> chunks = chunker('abcde', 10)
>>> list(chunks)
[['a', 'b', 'c', 'd', 'e']]
"""
iterator = iter(iterable)
chunk = list(itertools.islice(iterator, n))
while chunk:
yield chunk
chunk = list(itertools.islice(iterator, n))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment