Skip to content

Instantly share code, notes, and snippets.

@TimSC
Created September 22, 2016 11:21
Show Gist options
  • Select an option

  • Save TimSC/5d7ff596d655410e3d125388f80af589 to your computer and use it in GitHub Desktop.

Select an option

Save TimSC/5d7ff596d655410e3d125388f80af589 to your computer and use it in GitHub Desktop.
Python manipulation of csv
import gzip, csv, json, sys
csv.field_size_limit(sys.maxsize)
fi = csv.reader(gzip.open("/home/postgres/relations.csv.gz","rb"))
fiout2 = gzip.open("fixedrelations.csv.gz","wb")
fiout = csv.writer(fiout2)
for li in fi:
oldmems = json.loads(li[-1])
assert(len(li)==10)
mems=[]
roles = []
for m in oldmems:
mems.append([m[0], m[1]])
roles.append(m[2])
li[-1] = json.dumps(mems)
li.append(json.dumps(roles))
fiout.writerow(li)
fiout2.close()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment