Skip to content

Instantly share code, notes, and snippets.

@dnch
Created November 23, 2010 01:12
Show Gist options
  • Save dnch/711065 to your computer and use it in GitHub Desktop.
Save dnch/711065 to your computer and use it in GitHub Desktop.
# First thing we need is the raw connection to our database. ActiveRecord makes this quite easy...
pg_connection = ModelName.connection.raw_connection
# Next, tell PG that we're going to be copying a boat load of data into model_names...
pg_connection.exec "COPY model_names (field_a, field_b, field_c) FROM STDIN"
# Process our file line-by-line
IO.foreach("path/to/file.tsv", "r") do |line|
# Convert our tab-delimited string into an array
raw_fields = line.split("\t")
# and re-combine it back into a tab-delmited string, but only using the fields we need, wrapping any fields in quotes that we need
new_row = [raw_fields[0], "'#{raw_fields[5]}'", raw_fields[21]].join("\t")
# send the data to our database
pg_connection.put_copy_data(new_row + "\n")
end
# And finally, tell PG that we're done inserting data.
pg_connection.put_copy_end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment