Last active
July 21, 2018 21:46
-
-
Save hrbrmstr/28d94c5b979f4c60f6dd to your computer and use it in GitHub Desktop.
Scrape security conference data from Concise Courses
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
library(rvest) | |
library(stringr) | |
library(dplyr) | |
library(readr) | |
extract <- function(doc, x) { | |
doc %>% | |
str_match_all(x) %>% | |
lapply(function(y) { | |
if (length(y)==0) { | |
NA | |
} else { | |
y[1,2] | |
} | |
}) %>% unlist %>% str_trim | |
} | |
pg <- html("http://www.concise-courses.com/security/conferences-of-2015/") | |
# Get all the paragraphs that have the semi-structured conf deets | |
event_text <- pg %>% | |
html_nodes(xpath="//p[contains(., 'Date:')]") %>% | |
html_text | |
date <- event_text %>% | |
extract("Date:([[:alnum:][:punct:] ]+ 2015)\n") | |
title <- event_text %>% | |
extract("Conference Title:([[:alnum:][:punct:] \t]+)\n") | |
where <- event_text %>% | |
extract("Where:([[:alnum:][:punct:] \t]+)Link") | |
# from the original HTML, get all conf links | |
event_link <- pg %>% | |
html_nodes(xpath="//a[@onclick][text()='Link To Event']") %>% | |
html_attr("href") %>% | |
str_trim | |
# build data frame - don't add link as it's not perfectly extracted all the time | |
dat <- data_frame(date, title, where) | |
write_csv(dat, "~/concise.csv") |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
If you are trying to catch your cheating spouse, I recommend that you contact the hacker that helped me monitor my wife’s phone when I was gathering evidence during the divorce. I got virtually every information she has been hiding over the months easily on my own phone: the spy app diverted all her whatsapp, calls, text, facebook, text messages, sent and received through the phone: I also got her phone calls and deleted messages. she could not believe her eyes when she saw the evidence because she had no idea she was hacked.. if you need any type of hack. contact him. Contact: [email protected]