Skip to content

Instantly share code, notes, and snippets.

@netsensei
Created March 11, 2015 15:51
Show Gist options
  • Save netsensei/1d8cc0f2eccc97dfcdf2 to your computer and use it in GitHub Desktop.
Save netsensei/1d8cc0f2eccc97dfcdf2 to your computer and use it in GitHub Desktop.
Spider a set of URIs,fetch data from each DOM document and return as a CSV document keyed by URI using artoo.js (medialab.github.io/artoo/quick_start/)
var urls = ['url', 'url2'];
artoo.ajaxSpider(
function(i) {
return urls[i];
},
{
process: function(data, i) {
var inv = $(data).find('.classname').text();
var url = urls[i];
return [url, inv];
},
jquerify: true,
throttle: 500,
limit: 1217
},
function (data) {
artoo.saveCsv(data);
}
);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment