Skip to content

Instantly share code, notes, and snippets.

@derwiki
Created May 11, 2011 05:01
Show Gist options
  • Select an option

  • Save derwiki/965938 to your computer and use it in GitHub Desktop.

Select an option

Save derwiki/965938 to your computer and use it in GitHub Desktop.
JSON transformer
jeffws:~/x/rawdata/20110511_intl_users$
jeffws:~/x/rawdata/20110511_intl_users$ cat jsontransform.py
#!/usr/bin/python
import sys
import json
if __name__ == "__main__":
if len(sys.argv) < 2:
print "Usage:\n\
Takes a file with a JSON dictionary on each line and extracts keys to a tab seperated CSV output\n\
ex:\n\
$ cat addresses.json id email_address tracking_id\n\
123 [email protected] XYZ"
for line in sys.stdin:
sys.stdout.write('\t'.join(json.loads(line)[key] for key in sys.argv[1:]) + '\n')
jeffws:~/x/rawdata/20110511_intl_users$ head -n 5 20110511_remaining_intl_users_shuffled_133.json | ./jsontransform.py field1 email
BISQBFWSQHPI2LDJXK7ULJU6QI------ [email protected]
TJDA44MYRNJ5EJC2VH2EAWOGUQ------ [email protected]
TLTHKCJTNPMJXSK5JX3VMIPM2A------ [email protected]
HIDNGS64QTHMOLNF7FEZR7735M------ [email protected]
5UZTXLLFXTEI7PEAHDGRT7WOFY------ [email protected]
jeffws:~/x/rawdata/20110511_intl_users$ head -n 5 20110511_remaining_intl_users_shuffled_133.json | ./jsontransform.py field1 email id
Traceback (most recent call last):
File "./jsontransform.py", line 14, in <module>
sys.stdout.write('\t'.join(json.loads(line)[key] for key in sys.argv[1:]) + '\n')
TypeError: sequence item 2: expected string or Unicode, int found
jeffws:~/x/rawdata/20110511_intl_users$ head -n 5 20110511_remaining_intl_users_shuffled_133.json | ./jsontransform.py field1 email id1
Traceback (most recent call last):
File "./jsontransform.py", line 14, in <module>
sys.stdout.write('\t'.join(json.loads(line)[key] for key in sys.argv[1:]) + '\n')
File "./jsontransform.py", line 14, in <genexpr>
sys.stdout.write('\t'.join(json.loads(line)[key] for key in sys.argv[1:]) + '\n')
KeyError: 'id1'
jeffws:~/x/rawdata/20110511_intl_users$
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment