Skip to content

Instantly share code, notes, and snippets.

@hrbrmstr
Created September 22, 2014 21:46
Show Gist options
  • Save hrbrmstr/f2e12b97db6f138cc78d to your computer and use it in GitHub Desktop.
Save hrbrmstr/f2e12b97db6f138cc78d to your computer and use it in GitHub Desktop.
a demotivating httr example - scrape quotes from http://www.despair.com/demotivators.html
library(rvest)
library(httr)
dem <- html_session("http://www.despair.com/demotivators.html",
user_agent("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.78.2 (KHTML, like Gecko) Version/7.0.6 Safari/537.78.2"))
quotes <- data.frame(category=dem %>% html_nodes(xpath="//div/a/h3") %>% html_text(),
text=dem %>% html_nodes(xpath="//div[@class='tilecontents']/p") %>% html_text(),
image_url=dem %>% html_nodes(xpath="//img[@class='tileimg']") %>% html_attr("src"))[-1,]
head(quotes)
## category text image_url
## 2 Caution Life doesn't always want to be grabbed by the horns. http://demotivators.despair.com/viewalldemotivators/caution.jpg
## 3 Wishes (Cow) If wishes were horses then dreamers would ride. But they're much more like cattle, so best grab a shovel. http://demotivators.despair.com/viewalldemotivators/wishes-cow.jpg
## 4 Self Esteem (pawn) It takes genuine talent to see greatness in yourself despite your absence of genuine talent. http://demotivators.despair.com/viewalldemotivators/self-esteem-pawn.jpg
## 5 Preservation Action will be taken to prevent the next disaster as soon as possible after it has occurred. http://demotivators.despair.com/viewalldemotivators/preservation.jpg
## 6 Friendship A friend is just a stranger you haven't alienated yet. http://demotivators.despair.com/viewalldemotivators/friendship.jpg
## 7 Foresight Those who say it cannot be done should not interrupt those busy proving them right. http://demotivators.despair.com/viewalldemotivators/foresight.jpg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment