Skip to content

Instantly share code, notes, and snippets.

@dallasmarlow
Created September 17, 2013 23:15
Show Gist options
  • Save dallasmarlow/6602011 to your computer and use it in GitHub Desktop.
Save dallasmarlow/6602011 to your computer and use it in GitHub Desktop.
require 'hdfs/classpath'
fs = Hadoop::DFS::FileSystem.new local: true
dfs = Hadoop::DFS::FileSystem.new host: 'nn-beta-1263dd15.ewr01.tumblr.net'
binlog_path = '/db-binlog'
binlogs = Dir.glob(File.join(binlog_path, '*')).select {|entry| entry.start_with? File.join binlog_path, 'binlog.0'}.sort.tap(&:pop)
binlogs.each {|binlog| fs.copy(binlog, File.join('/blog_shard_merge_binlogs', '7e707b78_master', File.basename(binlog)), dfs)}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment