Skip to content

Commit 5a64fd5

Browse files
committed
Encoding benchmark updates
Remove `rapidjson` as it's 2x slower most benchmarks, and on par on a couple of them, so it's not telling us much here. Configure `Oj` in compat mode so it generate the same JSON on the `many to_json` benchmark. ``` == Encoding small nested array (121 bytes) ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- json (reuse) 220.202k i/100ms json 162.190k i/100ms oj 222.094k i/100ms Calculating ------------------------------------- json (reuse) 2.322M (± 1.3%) i/s (430.72 ns/i) - 11.671M in 5.027655s json 1.707M (± 1.2%) i/s (585.76 ns/i) - 8.596M in 5.035996s oj 2.248M (± 1.4%) i/s (444.94 ns/i) - 11.327M in 5.040712s Comparison: json (reuse): 2321686.9 i/s oj: 2247509.6 i/s - 1.03x slower json: 1707179.3 i/s - 1.36x slower == Encoding small hash (65 bytes) ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- json (reuse) 446.184k i/100ms json 265.594k i/100ms oj 653.226k i/100ms Calculating ------------------------------------- json (reuse) 4.980M (± 1.4%) i/s (200.82 ns/i) - 24.986M in 5.018729s json 2.763M (± 1.8%) i/s (361.94 ns/i) - 13.811M in 5.000434s oj 7.232M (± 1.4%) i/s (138.28 ns/i) - 36.581M in 5.059377s Comparison: json (reuse): 4979642.4 i/s oj: 7231624.4 i/s - 1.45x faster json: 2762890.1 i/s - 1.80x slower == Encoding mixed utf8 (5003001 bytes) ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- json 34.000 i/100ms oj 36.000 i/100ms Calculating ------------------------------------- json 357.772 (± 4.8%) i/s (2.80 ms/i) - 1.802k in 5.047308s oj 327.521 (± 1.5%) i/s (3.05 ms/i) - 1.656k in 5.057241s Comparison: json: 357.8 i/s oj: 327.5 i/s - 1.09x slower == Encoding mostly utf8 (5001001 bytes) ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- json 26.000 i/100ms oj 36.000 i/100ms Calculating ------------------------------------- json 294.357 (±10.5%) i/s (3.40 ms/i) - 1.456k in 5.028862s oj 352.826 (± 8.2%) i/s (2.83 ms/i) - 1.764k in 5.045651s Comparison: json: 294.4 i/s oj: 352.8 i/s - same-ish: difference falls within error == Encoding twitter.json (466906 bytes) ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- json 206.000 i/100ms oj 229.000 i/100ms Calculating ------------------------------------- json 2.064k (± 9.3%) i/s (484.55 μs/i) - 10.300k in 5.056409s oj 2.121k (± 8.4%) i/s (471.47 μs/i) - 10.534k in 5.012315s Comparison: json: 2063.8 i/s oj: 2121.0 i/s - same-ish: difference falls within error == Encoding citm_catalog.json (500298 bytes) ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- json 119.000 i/100ms oj 126.000 i/100ms Calculating ------------------------------------- json 1.317k (± 2.3%) i/s (759.18 μs/i) - 6.664k in 5.061781s oj 1.261k (± 2.9%) i/s (793.11 μs/i) - 6.300k in 5.000714s Comparison: json: 1317.2 i/s oj: 1260.9 i/s - same-ish: difference falls within error == Encoding canada.json (2090234 bytes) ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- json 1.000 i/100ms oj 1.000 i/100ms Calculating ------------------------------------- json 19.590 (± 0.0%) i/s (51.05 ms/i) - 98.000 in 5.004485s oj 19.003 (± 0.0%) i/s (52.62 ms/i) - 95.000 in 5.002276s Comparison: json: 19.6 i/s oj: 19.0 i/s - 1.03x slower == Encoding many #to_json calls (2701 bytes) ruby 3.4.0preview2 (2024-10-07 master 32c733f57b) +YJIT +PRISM [arm64-darwin23] Warming up -------------------------------------- json 2.556k i/100ms oj 2.332k i/100ms Calculating ------------------------------------- json 25.367k (± 1.7%) i/s (39.42 μs/i) - 127.800k in 5.039438s oj 23.743k (± 1.5%) i/s (42.12 μs/i) - 118.932k in 5.010303s Comparison: json: 25367.3 i/s oj: 23743.3 i/s - 1.07x slower ```
1 parent b95d949 commit 5a64fd5

File tree

1 file changed

+11
-6
lines changed

1 file changed

+11
-6
lines changed

benchmark/encoder.rb

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
require "benchmark/ips"
22
require "json"
33
require "oj"
4-
require "rapidjson"
4+
5+
Oj.default_options = Oj.default_options.merge(mode: :compat)
56

67
if ENV["ONLY"]
78
RUN = ENV["ONLY"].split(/[,: ]/).map{|x| [x.to_sym, true] }.to_h
@@ -15,12 +16,10 @@
1516

1617
def implementations(ruby_obj)
1718
state = JSON::State.new(JSON.dump_default_options)
18-
1919
{
2020
json_state: ["json (reuse)", proc { state.generate(ruby_obj) }],
2121
json: ["json", proc { JSON.dump(ruby_obj) }],
2222
oj: ["oj", proc { Oj.dump(ruby_obj) }],
23-
rapidjson: ["rapidjson", proc { RapidJSON.dump(ruby_obj) }],
2423
}
2524
end
2625

@@ -38,6 +37,11 @@ def benchmark_encoding(benchmark_name, ruby_obj, check_expected: true, except: [
3837
result = block.call
3938
if check_expected && expected != result
4039
puts "#{name} does not match expected output. Skipping"
40+
puts "Expected:" + '-' * 40
41+
puts expected
42+
puts "Actual:" + '-' * 40
43+
puts result
44+
puts '-' * 40
4145
next
4246
end
4347
rescue => error
@@ -67,12 +71,13 @@ def benchmark_encoding(benchmark_name, ruby_obj, check_expected: true, except: [
6771

6872
# This benchmark spent the overwhelming majority of its time in `ruby_dtoa`. We rely on Ruby's implementation
6973
# which uses a relatively old version of dtoa.c from David M. Gay.
70-
# Oj is noticeably faster here because it limits the precision of floats, breaking roundtriping. That's not
71-
# something we should emulate.
74+
# Oj in `compat` mode is ~10% slower than `json`, but in its default mode is noticeably faster here because
75+
# it limits the precision of floats, breaking roundtriping. That's not something we should emulate.
76+
#
7277
# Since a few years there are now much faster float to string implementations such as Ryu, Dragonbox, etc,
7378
# but all these are implemented in C++11 or newer, making it hard if not impossible to include them.
7479
# Short of a pure C99 implementation of these newer algorithms, there isn't much that can be done to match
7580
# Oj speed without losing precision.
7681
benchmark_encoding "canada.json", JSON.load_file("#{__dir__}/data/canada.json"), check_expected: false, except: %i(json_state)
7782

78-
benchmark_encoding "many #to_json calls", [{Object.new => Object.new, 12 => 54.3, Integer => Float, Time.now => Date.today}] * 20, except: %i(json_state)
83+
benchmark_encoding "many #to_json calls", [{object: Object.new, int: 12, float: 54.3, class: Float, time: Time.now, date: Date.today}] * 20, except: %i(json_state)

0 commit comments

Comments
 (0)