Skip to content

Instantly share code, notes, and snippets.

@takumikinjo
Created October 30, 2010 05:33
Show Gist options
  • Save takumikinjo/654996 to your computer and use it in GitHub Desktop.
Save takumikinjo/654996 to your computer and use it in GitHub Desktop.
Gather hash-tagged Twitter tweets and save them as a yaml file.
#!/bin/bash
# needs ruby-1.9.2.
for i in `ls *.yml`; do
j=${i%.yml}
cat <<EOF > $j.xml
<?xml version="1.0" encoding="utf-8" ?>
<?xml-stylesheet href="rdf.xsl" type="text/xsl"?>
<rdf:RDF xmlns="http://purl.org/rss/1.0/"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xml:lang="ja">
<channel rdf:about="http://www">
<title>$j</title>
<link>http://hackers.github.com/</link>
<description></description>
<items>
<rdf:Seq>
EOF
if [ -s $i ]; then
t=`tempfile`
head -$(expr `(grep -n \'^\-\' $i | cut -d\: -f1; expr \`wc -l $i | cut -d\ -f1\` + 1) | head -51 | tail -1` - 1) $i > $t
ruby -ryaml -e "YAML.load_file(\"$t\").collect{|e| puts \"<rdf:li rdf:resource=\\\"http://twitter.com/#{e['from_user']}/status/#{e['id']}\\\" />\"}" >> $j.xml
fi
cat <<EOF >> $j.xml
</rdf:Seq>
</items>
</channel>
EOF
if [ -s $i ]; then
ruby -ryaml -e "YAML.load_file(\"$t\").collect{|e| puts \"<item rdf:about=\\\"http://twitter.com/#{e[\"from_user\"]}/status/#{e[\"id\"]}\\\"><title>【#{e[\"from_user\"]}】#{e[\"text\"]}</title><link>http://twitter.com/#{e[\"from_user\"]}/status/#{e[\"id\"]}</link><description><![CDATA[<a href=\\\"http://twitter.com/#{e[\"from_user\"]}/status/#{e[\"id\"]}\\\"><img src=\\\"#{e[\"profile_image_url\"]}\\\" /></a><br />#{e[\"text\"]}]]></description><dc:date>#{e[\"created_at\"]}</dc:date></item>\"}" >> $j.xml
fi
cat <<EOF >> $j.xml
</rdf:RDF>
EOF
done
#!/bin/bash
# needs ruby-1.9.2.
for j in `ls *.yml`;do mv $j $j.old;wget -q "http://search.twitter.com/search.json?q=%23${j%.yml}&rpp=100" -O- 2>/dev/null|ruby -rjson -p -e 'JSON.parse($_)["results"].each{|e|puts e.to_json};$_=""'|while read i;do echo $i|grep -q "id: `echo $i|ruby -rjson -p -e '$_=JSON.parse($_)["id"]'`" $j.old||(echo $i|ruby -ryaml -rjson -p -e '$_=[JSON.parse($_)].to_yaml');done|grep -v '^---'|sed 's/\\x/%/g'|nkf -w --url-input>>$j;cat $j.old>>$j;done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment