Skip to content

Instantly share code, notes, and snippets.

@rjurney

rjurney/create_load_statement.py

Created February 13, 2015 01:43
Show Gist options
  • Save rjurney/3b706d32b4259c893442 to your computer and use it in GitHub Desktop.
Save rjurney/3b706d32b4259c893442 to your computer and use it in GitHub Desktop.
Download ZIP
Create Pig LOAD statement from CSV
Raw
create_load_statement.py
import sys, os, re
line = sys.stdin.readline()
columns = line.rstrip("\n").split("|")
load_statement = "full_dump = LOAD 'data/FullDump_clean.csv' AS (\n"
for column in columns:
load_statement += "\t{}:chararray, \n".format(column)
load_statement = load_statement[:-3]
load_statement += "\n);"
print load_statement
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment