Skip to content

Instantly share code, notes, and snippets.

@casperisfine
Last active November 6, 2024 14:56
Show Gist options
  • Save casperisfine/cf4b3a0594fae24b7d0eb93daaf3841a to your computer and use it in GitHub Desktop.
Save casperisfine/cf4b3a0594fae24b7d0eb93daaf3841a to your computer and use it in GitHub Desktop.
== Parsing small nested array (121 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.7.2 45.714k i/100ms
oj 55.058k i/100ms
Oj::Parser 157.848k i/100ms
rapidjson 75.353k i/100ms
Calculating -------------------------------------
json 2.7.2 473.203k (± 1.4%) i/s (2.11 μs/i) - 2.377M in 5.024547s
oj 548.477k (± 1.3%) i/s (1.82 μs/i) - 2.753M in 5.020010s
Oj::Parser 1.623M (± 1.4%) i/s (615.98 ns/i) - 8.208M in 5.057019s
rapidjson 784.788k (± 1.3%) i/s (1.27 μs/i) - 3.994M in 5.089743s
Comparison:
json 2.8.0: 1292068.1 i/s
Oj::Parser: 1623438.8 i/s - 1.26x faster
rapidjson: 784787.9 i/s - 1.65x slower
oj: 548477.3 i/s - 2.36x slower
json 2.7.2: 473202.9 i/s - 2.73x slower
== Parsing small hash (65 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.7.2 170.825k i/100ms
oj 207.606k i/100ms
Oj::Parser 429.119k i/100ms
rapidjson 301.118k i/100ms
Calculating -------------------------------------
json 2.7.2 1.820M (± 2.0%) i/s (549.60 ns/i) - 9.225M in 5.071855s
oj 2.290M (± 1.6%) i/s (436.65 ns/i) - 11.626M in 5.077800s
Oj::Parser 4.677M (± 0.7%) i/s (213.80 ns/i) - 23.602M in 5.046250s
rapidjson 3.237M (± 3.0%) i/s (308.92 ns/i) - 16.260M in 5.028332s
Comparison:
json 2.8.0: 3670085.6 i/s
Oj::Parser: 4677265.1 i/s - 1.27x faster
rapidjson: 3237130.9 i/s - 1.13x slower
oj: 2290147.7 i/s - 1.60x slower
json 2.7.2: 1819505.5 i/s - 2.02x slower
== Parsing test from oj (258 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.7.2 39.144k i/100ms
oj 42.681k i/100ms
Oj::Parser 73.587k i/100ms
rapidjson 52.970k i/100ms
Calculating -------------------------------------
json 2.7.2 391.582k (± 3.3%) i/s (2.55 μs/i) - 1.957M in 5.004386s
oj 444.233k (± 0.7%) i/s (2.25 μs/i) - 2.262M in 5.092362s
Oj::Parser 748.813k (± 1.1%) i/s (1.34 μs/i) - 3.753M in 5.012448s
rapidjson 531.726k (± 1.5%) i/s (1.88 μs/i) - 2.701M in 5.081711s
Comparison:
json 2.8.0: 558504.2 i/s
Oj::Parser: 748813.0 i/s - 1.34x faster
rapidjson: 531725.6 i/s - 1.05x slower
oj: 444232.8 i/s - 1.26x slower
json 2.7.2: 391581.5 i/s - 1.43x slower
== Parsing activitypub.json (58160 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.7.2 638.000 i/100ms
oj 798.000 i/100ms
Oj::Parser 948.000 i/100ms
rapidjson 631.000 i/100ms
Calculating -------------------------------------
json 2.7.2 6.423k (± 1.3%) i/s (155.70 μs/i) - 32.538k in 5.067149s
oj 7.989k (± 1.0%) i/s (125.17 μs/i) - 40.698k in 5.094544s
Oj::Parser 9.472k (± 1.3%) i/s (105.58 μs/i) - 47.400k in 5.005119s
rapidjson 6.354k (± 1.1%) i/s (157.37 μs/i) - 32.181k in 5.064962s
Comparison:
json 2.8.0: 9510.0 i/s
Oj::Parser: 9471.9 i/s - same-ish: difference falls within error
oj: 7989.4 i/s - 1.19x slower
json 2.7.2: 6422.5 i/s - 1.48x slower
rapidjson: 6354.5 i/s - 1.50x slower
== Parsing twitter.json (567916 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.7.2 52.000 i/100ms
oj 64.000 i/100ms
Oj::Parser 76.000 i/100ms
rapidjson 57.000 i/100ms
Calculating -------------------------------------
json 2.7.2 526.860 (± 3.8%) i/s (1.90 ms/i) - 2.652k in 5.042680s
oj 631.234 (± 1.7%) i/s (1.58 ms/i) - 3.200k in 5.070973s
Oj::Parser 764.354 (± 3.5%) i/s (1.31 ms/i) - 3.876k in 5.077736s
rapidjson 579.085 (± 2.8%) i/s (1.73 ms/i) - 2.907k in 5.024620s
Comparison:
json 2.8.0: 884.0 i/s
Oj::Parser: 764.4 i/s - 1.16x slower
oj: 631.2 i/s - 1.40x slower
rapidjson: 579.1 i/s - 1.53x slower
json 2.7.2: 526.9 i/s - 1.68x slower
== Parsing citm_catalog.json (1727030 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.7.2 30.000 i/100ms
oj 35.000 i/100ms
Oj::Parser 45.000 i/100ms
rapidjson 40.000 i/100ms
Calculating -------------------------------------
json 2.7.2 304.584 (± 3.3%) i/s (3.28 ms/i) - 1.530k in 5.029021s
oj 358.572 (± 0.8%) i/s (2.79 ms/i) - 1.820k in 5.076123s
Oj::Parser 450.643 (± 3.1%) i/s (2.22 ms/i) - 2.295k in 5.098150s
rapidjson 395.304 (± 1.5%) i/s (2.53 ms/i) - 2.000k in 5.060537s
Comparison:
json 2.8.0: 449.8 i/s
Oj::Parser: 450.6 i/s - same-ish: difference falls within error
rapidjson: 395.3 i/s - 1.14x slower
oj: 358.6 i/s - 1.25x slower
json 2.7.2: 304.6 i/s - 1.48x slower
== Parsing float parsing (2251051 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.7.2 2.000 i/100ms
oj 3.000 i/100ms
Oj::Parser 4.000 i/100ms
rapidjson 28.000 i/100ms
Calculating -------------------------------------
json 2.7.2 29.292 (± 3.4%) i/s (34.14 ms/i) - 148.000 in 5.055151s
oj 37.769 (± 2.6%) i/s (26.48 ms/i) - 189.000 in 5.005851s
Oj::Parser 42.662 (± 4.7%) i/s (23.44 ms/i) - 216.000 in 5.070201s
rapidjson 269.308 (± 1.1%) i/s (3.71 ms/i) - 1.372k in 5.094996s
Comparison:
json 2.8.0: 33.7 i/s
rapidjson: 269.3 i/s - 7.99x faster
Oj::Parser: 42.7 i/s - 1.27x faster
oj: 37.8 i/s - 1.12x faster
json 2.7.2: 29.3 i/s - 1.15x slower
== Parsing small nested array (121 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.8.0 125.714k i/100ms
oj 53.804k i/100ms
Oj::Parser 155.116k i/100ms
rapidjson 74.779k i/100ms
Calculating -------------------------------------
json 2.8.0 1.301M (± 1.6%) i/s (768.67 ns/i) - 6.537M in 5.026163s
oj 538.505k (± 1.7%) i/s (1.86 μs/i) - 2.744M in 5.097171s
Oj::Parser 1.624M (± 1.1%) i/s (615.87 ns/i) - 8.221M in 5.063838s
rapidjson 769.639k (± 2.1%) i/s (1.30 μs/i) - 3.889M in 5.054685s
Comparison:
json 2.7.2: 462716.2 i/s
Oj::Parser: 1623717.4 i/s - 3.51x faster
json 2.8.0: 1300946.3 i/s - 2.81x faster
rapidjson: 769638.5 i/s - 1.66x faster
oj: 538505.2 i/s - 1.16x faster
== Parsing small hash (65 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.8.0 350.086k i/100ms
oj 212.387k i/100ms
Oj::Parser 439.895k i/100ms
rapidjson 297.023k i/100ms
Calculating -------------------------------------
json 2.8.0 3.709M (± 1.3%) i/s (269.60 ns/i) - 18.555M in 5.003056s
oj 2.199M (± 1.5%) i/s (454.67 ns/i) - 11.044M in 5.022612s
Oj::Parser 4.634M (± 2.5%) i/s (215.78 ns/i) - 23.314M in 5.034142s
rapidjson 3.237M (± 2.8%) i/s (308.96 ns/i) - 16.336M in 5.051646s
Comparison:
json 2.7.2: 1821943.8 i/s
Oj::Parser: 4634447.5 i/s - 2.54x faster
json 2.8.0: 3709253.4 i/s - 2.04x faster
rapidjson: 3236673.7 i/s - 1.78x faster
oj: 2199378.2 i/s - 1.21x faster
== Parsing test from oj (258 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.8.0 55.731k i/100ms
oj 42.508k i/100ms
Oj::Parser 56.066k i/100ms
rapidjson 52.168k i/100ms
Calculating -------------------------------------
json 2.8.0 561.822k (± 3.1%) i/s (1.78 μs/i) - 2.842M in 5.064504s
oj 435.539k (± 0.5%) i/s (2.30 μs/i) - 2.210M in 5.075244s
Oj::Parser 749.045k (± 1.0%) i/s (1.34 μs/i) - 3.756M in 5.015465s
rapidjson 534.906k (± 3.2%) i/s (1.87 μs/i) - 2.713M in 5.077087s
Comparison:
json 2.7.2: 394796.0 i/s
Oj::Parser: 749045.5 i/s - 1.90x faster
json 2.8.0: 561821.7 i/s - 1.42x faster
rapidjson: 534906.0 i/s - 1.35x faster
oj: 435539.2 i/s - 1.10x faster
== Parsing activitypub.json (58160 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.8.0 956.000 i/100ms
oj 805.000 i/100ms
Oj::Parser 964.000 i/100ms
rapidjson 629.000 i/100ms
Calculating -------------------------------------
json 2.8.0 9.404k (± 4.5%) i/s (106.34 μs/i) - 47.800k in 5.095417s
oj 8.069k (± 0.6%) i/s (123.94 μs/i) - 41.055k in 5.088388s
Oj::Parser 9.547k (± 3.0%) i/s (104.74 μs/i) - 48.200k in 5.053709s
rapidjson 6.430k (± 0.7%) i/s (155.53 μs/i) - 32.708k in 5.087335s
Comparison:
json 2.7.2: 6456.3 i/s
Oj::Parser: 9547.3 i/s - 1.48x faster
json 2.8.0: 9403.7 i/s - 1.46x faster
oj: 8068.7 i/s - 1.25x faster
rapidjson: 6429.7 i/s - same-ish: difference falls within error
== Parsing twitter.json (567916 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.8.0 89.000 i/100ms
oj 64.000 i/100ms
Oj::Parser 78.000 i/100ms
rapidjson 55.000 i/100ms
Calculating -------------------------------------
json 2.8.0 881.948 (± 0.5%) i/s (1.13 ms/i) - 4.450k in 5.045747s
oj 646.272 (± 1.1%) i/s (1.55 ms/i) - 3.264k in 5.051033s
Oj::Parser 767.680 (± 3.3%) i/s (1.30 ms/i) - 3.900k in 5.086383s
rapidjson 570.881 (± 0.9%) i/s (1.75 ms/i) - 2.860k in 5.010163s
Comparison:
json 2.7.2: 525.5 i/s
json 2.8.0: 881.9 i/s - 1.68x faster
Oj::Parser: 767.7 i/s - 1.46x faster
oj: 646.3 i/s - 1.23x faster
rapidjson: 570.9 i/s - 1.09x faster
== Parsing citm_catalog.json (1727030 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.8.0 44.000 i/100ms
oj 34.000 i/100ms
Oj::Parser 44.000 i/100ms
rapidjson 40.000 i/100ms
Calculating -------------------------------------
json 2.8.0 450.487 (± 1.1%) i/s (2.22 ms/i) - 2.288k in 5.079615s
oj 348.900 (± 3.2%) i/s (2.87 ms/i) - 1.768k in 5.073216s
Oj::Parser 459.000 (± 1.3%) i/s (2.18 ms/i) - 2.332k in 5.081521s
rapidjson 403.452 (± 1.0%) i/s (2.48 ms/i) - 2.040k in 5.056795s
Comparison:
json 2.7.2: 296.6 i/s
Oj::Parser: 459.0 i/s - 1.55x faster
json 2.8.0: 450.5 i/s - 1.52x faster
rapidjson: 403.5 i/s - 1.36x faster
oj: 348.9 i/s - 1.18x faster
== Parsing float parsing (2251051 bytes)
ruby 3.4.0dev (2024-11-06T07:59:09Z precompute-hash-wh.. 7943f98a8a) +YJIT +PRISM [arm64-darwin24]
Warming up --------------------------------------
json 2.8.0 2.000 i/100ms
oj 3.000 i/100ms
Oj::Parser 4.000 i/100ms
rapidjson 27.000 i/100ms
Calculating -------------------------------------
json 2.8.0 34.165 (± 0.0%) i/s (29.27 ms/i) - 172.000 in 5.035304s
oj 37.290 (± 5.4%) i/s (26.82 ms/i) - 189.000 in 5.079447s
Oj::Parser 42.907 (± 0.0%) i/s (23.31 ms/i) - 216.000 in 5.034699s
rapidjson 263.467 (± 3.4%) i/s (3.80 ms/i) - 1.323k in 5.028218s
Comparison:
json 2.7.2: 28.7 i/s
rapidjson: 263.5 i/s - 9.18x faster
Oj::Parser: 42.9 i/s - 1.50x faster
oj: 37.3 i/s - 1.30x faster
json 2.8.0: 34.2 i/s - 1.19x faster
# gem "json", "2.7.2" #
require "benchmark/ips"
require "json"
require "oj"
require "rapidjson"
if ENV["ONLY"]
RUN = ENV["ONLY"].split(/[,: ]/).map{|x| [x.to_sym, true] }.to_h
RUN.default = false
elsif ENV["EXCEPT"]
RUN = ENV["EXCEPT"].split(/[,: ]/).map{|x| [x.to_sym, false] }.to_h
RUN.default = true
else
RUN = Hash.new(true)
end
def benchmark_parsing(name, json_output)
puts "== Parsing #{name} (#{json_output.size} bytes)"
Benchmark.ips do |x|
x.report("json #{JSON::VERSION}") { JSON.parse(json_output) } if RUN[:json]
x.report("oj") { Oj.load(json_output) } if RUN[:oj]
x.report("Oj::Parser") { Oj::Parser.usual.parse(json_output) } if RUN[:oj]
x.report("rapidjson") { RapidJSON.parse(json_output) } if RUN[:rapidjson]
x.compare!(order: :baseline)
x.save!("/tmp/bench-#{name}.json")
end
puts
end
# NB: Notes are based on ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
# Oj::Parser is significanly faster (~1.3x) on the next 3 micro-benchmarks in large part because its
# cache is persisted across calls. That's not something we can do with the current API, we'd
# need to expose a stateful API as well, but that's no really desirable.
# Other than that we're faster than regular `Oj.load` by a good margin (between 1.3x and 2.4x).
benchmark_parsing "small nested array", JSON.dump([[1,2,3,4,5]]*10)
benchmark_parsing "small hash", JSON.dump({ "username" => "jhawthorn", "id" => 123, "event" => "wrote json serializer" })
benchmark_parsing "test from oj", <<JSON
{"a":"Alpha","b":true,"c":12345,"d":[true,[false,[-123456789,null],3.9676,["Something else.",false],null]],
"e":{"zero":null,"one":1,"two":2,"three":[3],"four":[0,1,2,3,4]},"f":null,
"h":{"a":{"b":{"c":{"d":{"e":{"f":{"g":null}}}}}}},"i":[[[[[[[null]]]]]]]}
JSON
# On these macro-benchmarks, we're on par with `Oj::Parser`, except `twitter.json` where we're `1.14x` faster,
# And between 1.3x and 1.5x faster than `Oj.load`.
benchmark_parsing "activitypub.json", File.read("#{__dir__}/data/activitypub.json")
benchmark_parsing "twitter.json", File.read("#{__dir__}/data/twitter.json")
benchmark_parsing "citm_catalog.json", File.read("#{__dir__}/data/citm_catalog.json")
# rapidjson is 8x faster thanks to its much more performant float parser.
# Unfortunately, there isn't a lot of existing fast float parsers in pure C,
# and including C++ is problematic.
# Aside from that, we're close to the alternatives here.
benchmark_parsing "float parsing", File.read("#{__dir__}/data/canada.json")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment