Skip to content

Instantly share code, notes, and snippets.

@justinko
Last active December 21, 2015 13:39
Show Gist options
  • Save justinko/6313779 to your computer and use it in GitHub Desktop.
Save justinko/6313779 to your computer and use it in GitHub Desktop.
Parse streaming JSON
[3] pry(main)> streaming_json = StreamingJSON.new
=> #<StreamingJSON:0x007fe222b9d8e8
@buffer=
#<StreamingJSON::Buffer:0x007fe222b9d898
@complete=[],
@incomplete=[],
@nesting=#<StreamingJSON::Buffer::Nesting:0x007fe222b9d7d0 @level=0>,
@recording=false>>
[4] pry(main)> streaming_json[%({"a":)]
=> []
[5] pry(main)> streaming_json[%("b"},{"c":"d"},{"e")]
=> [{:a=>"b"}, {:c=>"d"}]
[6] pry(main)> streaming_json[%(:"f"},{"g":{"h":["i"]}}])]
=> [{:e=>"f"}, {:g=>{:h=>["i"]}}]
[7] pry(main)>
class StreamingJSON
def initialize
@buffer = Buffer.new
end
def [](chunk)
JSON.parse @buffer[chunk].complete, symbolize_names: true
ensure
@buffer.reset
end
class Buffer
class Nesting
def initialize
@level = 0
end
def increment
@level += 1
end
def decrement
@level -= 1
end
def at_root?
@level.zero?
end
end
LEFT_BRACE = '{'
RIGHT_BRACE = '}'
LEFT_BRACKET = '['
RIGHT_BRACKET = ']'
COMMA = ','
def self.clean(str)
str.join.chomp COMMA
end
def initialize
@incomplete, @complete = [], [] # buffers
@nesting = Nesting.new # handles nested JSON objects
@recording = false
end
def [](chunk)
chunk.each_char do |char|
case char
when LEFT_BRACE
start_recording
@nesting.increment
add_char char
when RIGHT_BRACE
add_char char
@nesting.decrement
if @nesting.at_root?
stop_recording
@complete.concat @incomplete
@incomplete = []
end
when COMMA
if recording?
add_char char
else
@complete << char if @complete.any?
end
when LEFT_BRACKET, RIGHT_BRACKET
add_char char if recording?
else
add_char char
end
end
self
end
def complete
LEFT_BRACKET + Buffer.clean(@complete) + RIGHT_BRACKET
end
def incomplete
Buffer.clean @incomplete
end
def reset
@complete = []
end
private
def add_char(char)
@incomplete << char
end
def start_recording
@recording = true
end
def stop_recording
@recording = false
end
def recording?
!!@recording
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment