Last active
November 30, 2015 20:03
-
-
Save bradhe/6703bd2426003e4ab47b to your computer and use it in GitHub Desktop.
An example integration for (hypothetical) social media data in a MySQL database with Reflect.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
require 'mysql' | |
# gem install reflect | |
require 'reflect' | |
sql = <<-SQL | |
( | |
SELECT | |
'twitter' AS `Network`, | |
DATE(scrape_time) AS `Scrape Date`, | |
account_id AS `Username`, | |
tweets AS `Posts`, | |
followers AS `Friends` | |
FROM twitter_accounts | |
GROUP BY DATE(scrape_time), account_id | |
) UNION ( | |
SELECT | |
'facebook' AS `Network`, | |
DATE(scrape_time) AS `Scrape Date`, | |
account_id AS `Username`, | |
posts AS `Posts`, | |
friends AS `Friends` | |
FROM facebook_accounts | |
GROUP BY DATE(scrape_time), account_id | |
) UNION ( | |
SELECT | |
'instagram' AS `Network`, | |
DATE(scrape_time) AS `Scrape Date`, | |
account_id AS `Username`, | |
photos AS `Posts`, | |
followers AS `Friends` | |
FROM instagram_accounts | |
GROUP BY DATE(scrape_time), account_id | |
) | |
SQL | |
# NOTE: We're assuming the "social-media" keyspace was already created in | |
# reflect, but if it wasn't created it'll be automatically created for you. | |
$reflect = Reflect::Client.new('<Your API Token>') | |
keyspace = $reflect.keyspace('social-media') | |
db = Mysql::Client.new | |
results = db.query(sql) | |
# Group aggregated tablets by the key components. | |
grouped_tablets = results.group_by { |row| [row["Username"], row["Scrape Date"]] } | |
grouped_tablets.each do |key_components, tablet| | |
# Construct a key from row["Username"] and row["Scrape Date"] | |
key = key_components.join(":") | |
# We have all the data that we want in this tablet here so we can replace | |
# whatever tablet we had for this key with a new tablet. | |
keyspace.replace(key, tablet) | |
end |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment