Skip to content

Instantly share code, notes, and snippets.

@rippinrobr
Created May 24, 2018 22:25
Show Gist options
  • Save rippinrobr/3a309505e775a085a9cd57cb7bd4fca7 to your computer and use it in GitHub Desktop.
Save rippinrobr/3a309505e775a085a9cd57cb7bd4fca7 to your computer and use it in GitHub Desktop.
Making the calls to Wikipedia to fetch the pages
let links = db::get_links_to_process(&conn, batch_size);
WikiRepo::get_conspiracies(&c, links, title.to_string(), |p2, categories| {
match db::add_conspiracy(&conn, &p2) {
Err(e) => println!("SAVE ERROR: {} {} {}", e ,p2.title, p2.page_id),
Ok(_) => {
let title = &p2.title;
db::add_categories(&conn, categories).unwrap();
db::mark_link_as_processed(&conn, title).expect(&format!("A problem occurred when marking the link '{}' as processed",title));
println!("Added: {} {}", title, p2.page_id)
}
};
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment