Skip to content

Instantly share code, notes, and snippets.

@dsoprea
Created September 18, 2025 08:17
Show Gist options
  • Save dsoprea/3cee7b8f83b957e8c85fbc294bbc187c to your computer and use it in GitHub Desktop.
Save dsoprea/3cee7b8f83b957e8c85fbc294bbc187c to your computer and use it in GitHub Desktop.
Yields sublists of a maximum size given an enumerable
def batched_gen(g, n):
"""Break the given iterator into batches of N items. The last group will
have (0 < x <= N) items.
This replicates `itertools.batched()` and enables it to be available for
Python <3.12 .
"""
# Make sure our input is a decaying generator
g = iter(g)
while True:
batch = itertools.islice(g, n)
batch = list(batch)
if not batch:
break
yield batch
# class Test(object):
# def test_batched_gen(self):
# items = range(10)
# items = list(items)
#
# actual = batched_gen(items, 3)
# actual = list(actual)
#
# expected = [
# [0, 1, 2],
# [3, 4, 5],
# [6, 7, 8],
# [9],
# ]
#
# assert \
# actual == expected, \
# "Batches not correct:\nACTUAL:\n{}\n\nEXPECTED:\n{}".format(
# actual, expected)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment