Skip to content

Instantly share code, notes, and snippets.

@mothra
Last active December 20, 2015 21:09
Show Gist options
  • Save mothra/6195516 to your computer and use it in GitHub Desktop.
Save mothra/6195516 to your computer and use it in GitHub Desktop.
Postgres/Rails "copy from stdin"
#!/usr/bin/env ruby
#https://bitbucket.org/ged/ruby-pg/src/315fa972883163de298acd695f9b7fe65e07e5d3/sample/copyfrom.rb?at=default
require 'pg'
require 'stringio'
$stderr.puts "Opening database connection ..."
conn = PG.connect( :dbname => 'test' )
conn.exec( <<END_SQL )
DROP TABLE IF EXISTS logs;
CREATE TABLE logs (
client_ip inet,
username text,
ts timestamp,
request text,
status smallint,
bytes int
);
END_SQL
copy_data = StringIO.new( <<"END_DATA" )
"127.0.0.1","","30/Aug/2010:08:21:24 -0700","GET /manual/ HTTP/1.1",404,205
"127.0.0.1","","30/Aug/2010:08:21:24 -0700","GET /favicon.ico HTTP/1.1",404,209
END_DATA
### You can test the error case from the database side easily by
### changing one of the numbers at the end of one of the above rows to
### something non-numeric like "-".
$stderr.puts "Running COPY command with data ..."
buf = ''
conn.transaction do
conn.exec( "COPY logs FROM STDIN WITH csv" )
begin
while copy_data.read( 256, buf )
### Uncomment this to test error-handling for exceptions from the reader side:
# raise Errno::ECONNRESET, "socket closed while reading"
$stderr.puts " sending %d bytes of data..." % [ buf.length ]
until conn.put_copy_data( buf )
$stderr.puts " waiting for connection to be writable..."
sleep 0.1
end
end
rescue Errno => err
errmsg = "%s while reading copy data: %s" % [ err.class.name, err.message ]
conn.put_copy_end( errmsg )
else
conn.put_copy_end
while res = conn.get_result
$stderr.puts "Result of COPY is: %s" % [ res.res_status(res.result_status) ]
end
end
end
conn.finish
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment