Created
October 22, 2015 01:30
-
-
Save nicholsn/108a7fb64e0e7567d2f0 to your computer and use it in GitHub Desktop.
processing records in batches
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| def batch(iterable, n=1): | |
| """ | |
| For batch processing of records | |
| :param iterable: | |
| :param n: batch size | |
| :return: generator | |
| """ | |
| l = len(iterable) | |
| for ndx in range(0, l, n): | |
| yield iterable[ndx:min(ndx + n, l)] | |
| # Example: | |
| # for record in batch(range(1, 100), n=10) | |
| # print record |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment