Last active
December 17, 2015 05:09
-
-
Save dcramer/5556161 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| cursor = (shard 0, id 0, group id 0) | |
| > get 10k items from shard 0 | |
| > group id set to max(id of results), say 20000 | |
| > get 10k items from shard 1 < group id (20000) | |
| > adjust id if we hit 10k+ items, and repeat on same shard | |
| > once group id is hit, go to next shard, and repeat same logic used on shard 1 | |
| > once last shard is done, go back to shard 0, get the next 10k results, and reset group id |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment