Last active
June 30, 2016 19:16
-
-
Save easherma/3c17d7ab16242aa6d3232082b9d2d4f5 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
all_results | |
run a call using params, | |
from that call, parse and store only the factual id and the POI name. do the page/offset thing to get 500 max records, in chunks of 50. Store these together | |
make another call with the same params, except filter by POI name NOTIN {our stored list} (the factual api dosen't let you filter this way by factual-id, boo) | |
see if we can get that in chunks of 500 | |
do this until no new results are returned | |
store that in all results | |
this seems like it would have to go in order | |
OR | |
return a count (https://github.com/Factual/factual-nodejs-driver#facets) given provided params | |
count = 1475 | |
params = params | |
run a call, get 50 results, store this in all_results (or whatever) | |
run another call, get 50 results, see if any of these results match the 50 we already have. if not, add them. | |
this comparison could be done via hashing/checksumming or some other fancy computer science thing | |
do this until # of records in all_results == count | |
_prepare these calls all at once/do a bunch at a time to make it go faster_ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment