Skip to content

Instantly share code, notes, and snippets.

@bradhe
Last active November 30, 2015 20:03
Show Gist options
  • Save bradhe/6703bd2426003e4ab47b to your computer and use it in GitHub Desktop.
Save bradhe/6703bd2426003e4ab47b to your computer and use it in GitHub Desktop.
An example integration for (hypothetical) social media data in a MySQL database with Reflect.
require 'mysql'
# gem install reflect
require 'reflect'
sql = <<-SQL
(
SELECT
'twitter' AS `Network`,
DATE(scrape_time) AS `Scrape Date`,
account_id AS `Username`,
tweets AS `Posts`,
followers AS `Friends`
FROM twitter_accounts
GROUP BY DATE(scrape_time), account_id
) UNION (
SELECT
'facebook' AS `Network`,
DATE(scrape_time) AS `Scrape Date`,
account_id AS `Username`,
posts AS `Posts`,
friends AS `Friends`
FROM facebook_accounts
GROUP BY DATE(scrape_time), account_id
) UNION (
SELECT
'instagram' AS `Network`,
DATE(scrape_time) AS `Scrape Date`,
account_id AS `Username`,
photos AS `Posts`,
followers AS `Friends`
FROM instagram_accounts
GROUP BY DATE(scrape_time), account_id
)
SQL
# NOTE: We're assuming the "social-media" keyspace was already created in
# reflect, but if it wasn't created it'll be automatically created for you.
$reflect = Reflect::Client.new('<Your API Token>')
keyspace = $reflect.keyspace('social-media')
db = Mysql::Client.new
results = db.query(sql)
# Group aggregated tablets by the key components.
grouped_tablets = results.group_by { |row| [row["Username"], row["Scrape Date"]] }
grouped_tablets.each do |key_components, tablet|
# Construct a key from row["Username"] and row["Scrape Date"]
key = key_components.join(":")
# We have all the data that we want in this tablet here so we can replace
# whatever tablet we had for this key with a new tablet.
keyspace.replace(key, tablet)
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment