Created
April 14, 2011 18:59
-
-
Save sampsyo/920215 to your computer and use it in GitHub Desktop.
quick & dirty reading/writing of files containing multiple JSON objects
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""A couple of very simple utilities for reading and writing files | |
that contain multiple JSON values. Could be useful in situations where | |
you're generating a bunch of data for later processing and then, later, | |
you want to read it in an element at a time. | |
The json module doesn't really support streaming reads, though, so this | |
is limited by that. If you need real streaming, you probably want to use | |
something like ijson: | |
http://pypi.python.org/pypi/ijson/ | |
""" | |
import json | |
import StringIO | |
_decoder = json.JSONDecoder() | |
_encoder = json.JSONEncoder() | |
_space = '\n' | |
def dump(fp, iterable): | |
"""Write a sequence of values to a file-like object.""" | |
for obj in iterable: | |
for chunk in _encoder.iterencode(obj): | |
fp.write(chunk) | |
fp.write(_space) | |
def dumps(iterable): | |
"""Like `dump` but returns a string.""" | |
fp = StringIO.StringIO() | |
dump(fp, iterable) | |
return fp.getvalue() | |
def loads(s): | |
"""A generator reading a sequence of JSON values from a string.""" | |
while s: | |
s = s.strip() | |
obj, pos = _decoder.raw_decode(s) | |
if not pos: | |
raise ValueError('no JSON object found at %i' % pos) | |
yield obj | |
s = s[pos:] | |
def load(fp): | |
"""Like `loads` but reads from a file-like object.""" | |
return loads(fp.read()) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello @sampsyo
What is the license of this? Is it free to use? The project I want to use it in MIT licensed.