Skip to content

Instantly share code, notes, and snippets.

@simonwheatley
Forked from stefl/scrape_tweets.rb
Created March 25, 2012 14:50
Show Gist options
  • Select an option

  • Save simonwheatley/2196676 to your computer and use it in GitHub Desktop.

Select an option

Save simonwheatley/2196676 to your computer and use it in GitHub Desktop.
Download all tweets from the twitter search API for a given search term (limited to their maximum of 1500)
#! /usr/bin/env ruby
search_term = ARGV[0]
if search_term
time = Time.now
directory_path = File.dirname(__FILE__) + "/tweets/" + search_term + "_" + time.to_i.to_s
FileUtils.mkdir_p(directory_path)
directory = Dir.new(directory_path)
(1..15).each do |i|
`curl "http://search.twitter.com/search.json?q=#{search_term}&rpp=100&page=#{i}&include_entities=true&result_type=mixed" > #{directory.path}/#{i}.json`
end
puts "Scraped to #{directory_path}"
else
puts "./scrape_tweets.rb <your search term in quotes>"
end
@stefl
Copy link
Copy Markdown

stefl commented Mar 25, 2012

You'll need to change Dir.mkdir_p to FileUtils.mkdir_p

@simonwheatley
Copy link
Copy Markdown
Author

Thanks so much for letting me know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment