Skip to content

Instantly share code, notes, and snippets.

@pdyraga

pdyraga/benchmark.md

Last active May 18, 2022 16:16
Download ZIP
Parity benchmarks (24 Sep 2018)
Raw
benchmark.md

About the benchmarks

The Parity client benchmarks are based on the work of Martin Holst Swende done 21 July 2018 for Constantinopole. The new benchmarks have been done for 2.1.1-beta version released 20 September 2018 as well as for master branch (branch as of 25 Oct - 5a1dc3e) with bn dependency pointing to version before and after the optimalisations by Andre Silva

If we compare pre- and post- benchmarks we see there is a significant performance boost justifying reduction of gas prices for ecmul and ecadd operations. Also, pairing is ~2.82 times faster when we compare Martin's benchmark and version 2.1.1-beta what also suggests that pairing gas cost can be reduced.

Martin's benchmark (21 Jul)

Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
ecrecover 3000.0 159077.00 18.8587916544 1590.77 3000.0
Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
modexp_nagydani_1_qube 204.0 9388.00 21.7298679165 93.88 206.835773982
alt_bn128_add_cdetrio12 500.0 16894.00 29.596306381 168.94 372.207452668
alt_bn128_mul_cdetrio11 40000.0 748248.00 53.4582117159 7482.48 16485.3487655
alt_bn128_pairing_ten_point_match_1 900000.0 96298039.00 9.34598470899 962980.39 2121631.8097

2.1.1-beta (released 20 Sep)

Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
ecrecover 3000.0 116516.00 18.8587916544 1165.16 3000.0
Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
modexp_nagydani_1_qube 204.0 8188.00 24.91450904 81.88 210.8208315
alt_bn128_add_cdetrio12 500.0 5551.00 90.07386057 55.51 142.9245769
alt_bn128_mul_cdetrio11 40000.0 229342.00 174.4120135 2293.42 5904.991589
alt_bn128_pairing_ten_point_match_1 900000.0 34076695.00 26.41101198 340766.95 877390.9592

Before bn optimalization

  • master branch as of 25 Oct (5a1dc3e)
  • bn pointing at 786c0d5 (before optimalization from paritytech/bn#9)
Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
ecrecover 3000.0 113570.00 26.41542661 1135.70 3000.0
Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
modexp_nagydani_1_qube 204.0 8196.00 24.89019034 81.96 216.5008365
alt_bn128_add_cdetrio12 500.0 13834.00 36.14283649 138.34 365.4310117
alt_bn128_mul_cdetrio11 40000.0 494407.00 80.90500337 4944.07 13059.97182
alt_bn128_pairing_ten_point_match_1 900000.0 67707776.00 13.29241711 677077.76 1788529.788

After bn optimalization

  • master branch as of 25 Oct (5a1dc3e)
  • bn pointing at 2a71dbd (after optimalization from paritytech/bn#9)
Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
ecrecover 3000.0 123149.00 24.36073375 1231.49 3000.0
Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
modexp_nagydani_1_qube 204.0 9180.00 22.22222222 91.8 223.6315358
alt_bn128_add_cdetrio12 500.0 5179.00 96.54373431 51.79 126.1642401
alt_bn128_mul_cdetrio11 40000.0 214629.00 186.368105 2146.29 5228.519923
alt_bn128_pairing_ten_point_match_1 900000.0 34275714.00 26.2576587 342757.14 834981.5427
Raw
raw-2.1.1-beta.benchmark
$ sysctl -n machdep.cpu.brand_string
Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz
test result: ok. 0 passed; 0 failed; 332 ignored; 0 measured; 0 filtered out
Running `/Users/piotr/git/parity-ethereum/target/release/deps/builtin-1676d3a0791765a3 --bench`
running 57 tests
test alt_bn128_add_cdetrio1 ... bench: 330 ns/iter (+/- 46)
test alt_bn128_add_cdetrio10 ... bench: 630 ns/iter (+/- 71)
test alt_bn128_add_cdetrio11 ... bench: 5,613 ns/iter (+/- 1,000)
test alt_bn128_add_cdetrio12 ... bench: 5,551 ns/iter (+/- 847)
test alt_bn128_add_cdetrio13 ... bench: 5,038 ns/iter (+/- 896)
test alt_bn128_add_cdetrio14 ... bench: 1,311 ns/iter (+/- 129)
test alt_bn128_add_cdetrio2 ... bench: 304 ns/iter (+/- 29)
test alt_bn128_add_cdetrio3 ... bench: 318 ns/iter (+/- 40)
test alt_bn128_add_cdetrio4 ... bench: 310 ns/iter (+/- 26)
test alt_bn128_add_cdetrio5 ... bench: 326 ns/iter (+/- 41)
test alt_bn128_add_cdetrio6 ... bench: 691 ns/iter (+/- 93)
test alt_bn128_add_cdetrio7 ... bench: 650 ns/iter (+/- 89)
test alt_bn128_add_cdetrio8 ... bench: 634 ns/iter (+/- 137)
test alt_bn128_add_cdetrio9 ... bench: 642 ns/iter (+/- 76)
test alt_bn128_add_chfast1 ... bench: 4,743 ns/iter (+/- 303)
test alt_bn128_add_chfast2 ... bench: 4,738 ns/iter (+/- 472)
test alt_bn128_mul_cdetrio1 ... bench: 225,765 ns/iter (+/- 15,520)
test alt_bn128_mul_cdetrio11 ... bench: 229,342 ns/iter (+/- 9,993)
test alt_bn128_mul_cdetrio6 ... bench: 243,032 ns/iter (+/- 34,909)
test alt_bn128_mul_chfast1 ... bench: 50,472 ns/iter (+/- 5,524)
test alt_bn128_mul_chfast2 ... bench: 109,065 ns/iter (+/- 15,625)
test alt_bn128_mul_chfast3 ... bench: 200,649 ns/iter (+/- 15,213)
test alt_bn128_pairing_empty_data ... bench: 29 ns/iter (+/- 5)
test alt_bn128_pairing_jeff1 ... bench: 7,161,114 ns/iter (+/- 313,825)
test alt_bn128_pairing_jeff2 ... bench: 7,076,581 ns/iter (+/- 310,160)
test alt_bn128_pairing_jeff3 ... bench: 7,139,972 ns/iter (+/- 694,744)
test alt_bn128_pairing_jeff4 ... bench: 10,583,223 ns/iter (+/- 1,630,858)
test alt_bn128_pairing_jeff5 ... bench: 10,379,414 ns/iter (+/- 690,564)
test alt_bn128_pairing_jeff6 ... bench: 7,118,354 ns/iter (+/- 379,069)
test alt_bn128_pairing_one_point ... bench: 3,851,247 ns/iter (+/- 991,301)
test alt_bn128_pairing_ten_point_match_1 ... bench: 34,076,695 ns/iter (+/- 3,528,604)
test alt_bn128_pairing_ten_point_match_2 ... bench: 36,357,284 ns/iter (+/- 5,783,316)
test alt_bn128_pairing_ten_point_match_3 ... bench: 7,582,345 ns/iter (+/- 1,299,180)
test alt_bn128_pairing_two_point_match_2 ... bench: 7,126,534 ns/iter (+/- 930,637)
test alt_bn128_pairing_two_point_match_3 ... bench: 7,176,366 ns/iter (+/- 507,984)
test alt_bn128_pairing_two_point_match_4 ... bench: 7,011,596 ns/iter (+/- 492,642)
test ecrecover ... bench: 116,516 ns/iter (+/- 20,167)
test identity ... bench: 79 ns/iter (+/- 14)
test modexp_eip_example1 ... bench: 566,063 ns/iter (+/- 97,608)
test modexp_eip_example2 ... bench: 292 ns/iter (+/- 32)
test modexp_nagydani_1_pow0x10001 ... bench: 42,294 ns/iter (+/- 3,057)
test modexp_nagydani_1_qube ... bench: 8,188 ns/iter (+/- 999)
test modexp_nagydani_1_square ... bench: 5,597 ns/iter (+/- 349)
test modexp_nagydani_2_pow0x10001 ... bench: 87,109 ns/iter (+/- 5,034)
test modexp_nagydani_2_qube ... bench: 15,341 ns/iter (+/- 1,500)
test modexp_nagydani_2_square ... bench: 10,614 ns/iter (+/- 662)
test modexp_nagydani_3_pow0x10001 ... bench: 265,390 ns/iter (+/- 60,985)
test modexp_nagydani_3_qube ... bench: 44,260 ns/iter (+/- 4,956)
test modexp_nagydani_3_square ... bench: 29,629 ns/iter (+/- 1,455)
test modexp_nagydani_4_pow0x10001 ... bench: 823,959 ns/iter (+/- 73,310)
test modexp_nagydani_4_qube ... bench: 145,951 ns/iter (+/- 40,422)
test modexp_nagydani_4_square ... bench: 97,402 ns/iter (+/- 19,660)
test modexp_nagydani_5_pow0x10001 ... bench: 2,956,937 ns/iter (+/- 120,018)
test modexp_nagydani_5_qube ... bench: 497,193 ns/iter (+/- 27,340)
test modexp_nagydani_5_square ... bench: 360,141 ns/iter (+/- 101,174)
test ripemd ... bench: 791 ns/iter (+/- 53)
test sha256 ... bench: 482 ns/iter (+/- 27)
test result: ok. 0 passed; 0 failed; 0 ignored; 57 measured; 0 filtered out
Raw
raw-master-5a1dc3e-bn-2a71dbd.benchmark
$ sysctl -n machdep.cpu.brand_string
Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz
test result: ok. 0 passed; 0 failed; 333 ignored; 0 measured; 0 filtered out
Running `/Users/piotr/git/parity-ethereum/target/release/deps/builtin-a0369717f10190f3 --bench`
running 57 tests
test alt_bn128_add_cdetrio1 ... bench: 305 ns/iter (+/- 9)
test alt_bn128_add_cdetrio10 ... bench: 948 ns/iter (+/- 1,341)
test alt_bn128_add_cdetrio11 ... bench: 5,315 ns/iter (+/- 406)
test alt_bn128_add_cdetrio12 ... bench: 5,179 ns/iter (+/- 594)
test alt_bn128_add_cdetrio13 ... bench: 4,554 ns/iter (+/- 587)
test alt_bn128_add_cdetrio14 ... bench: 1,358 ns/iter (+/- 242)
test alt_bn128_add_cdetrio2 ... bench: 313 ns/iter (+/- 33)
test alt_bn128_add_cdetrio3 ... bench: 313 ns/iter (+/- 69)
test alt_bn128_add_cdetrio4 ... bench: 311 ns/iter (+/- 29)
test alt_bn128_add_cdetrio5 ... bench: 309 ns/iter (+/- 44)
test alt_bn128_add_cdetrio6 ... bench: 641 ns/iter (+/- 124)
test alt_bn128_add_cdetrio7 ... bench: 634 ns/iter (+/- 86)
test alt_bn128_add_cdetrio8 ... bench: 606 ns/iter (+/- 69)
test alt_bn128_add_cdetrio9 ... bench: 593 ns/iter (+/- 58)
test alt_bn128_add_chfast1 ... bench: 4,401 ns/iter (+/- 354)
test alt_bn128_add_chfast2 ... bench: 4,495 ns/iter (+/- 458)
test alt_bn128_mul_cdetrio1 ... bench: 217,850 ns/iter (+/- 15,616)
test alt_bn128_mul_cdetrio11 ... bench: 214,629 ns/iter (+/- 10,053)
test alt_bn128_mul_cdetrio6 ... bench: 217,246 ns/iter (+/- 23,334)
test alt_bn128_mul_chfast1 ... bench: 46,941 ns/iter (+/- 5,343)
test alt_bn128_mul_chfast2 ... bench: 107,822 ns/iter (+/- 13,619)
test alt_bn128_mul_chfast3 ... bench: 198,872 ns/iter (+/- 45,250)
test alt_bn128_pairing_empty_data ... bench: 29 ns/iter (+/- 9)
test alt_bn128_pairing_jeff1 ... bench: 6,810,682 ns/iter (+/- 312,331)
test alt_bn128_pairing_jeff2 ... bench: 6,867,771 ns/iter (+/- 664,455)
test alt_bn128_pairing_jeff3 ... bench: 6,785,325 ns/iter (+/- 186,119)
test alt_bn128_pairing_jeff4 ... bench: 10,181,887 ns/iter (+/- 598,836)
test alt_bn128_pairing_jeff5 ... bench: 10,209,255 ns/iter (+/- 920,372)
test alt_bn128_pairing_jeff6 ... bench: 6,771,495 ns/iter (+/- 255,136)
test alt_bn128_pairing_one_point ... bench: 3,402,253 ns/iter (+/- 163,719)
test alt_bn128_pairing_ten_point_match_1 ... bench: 34,275,714 ns/iter (+/- 3,169,219)
test alt_bn128_pairing_ten_point_match_2 ... bench: 33,779,811 ns/iter (+/- 4,012,588)
test alt_bn128_pairing_ten_point_match_3 ... bench: 6,796,607 ns/iter (+/- 224,947)
test alt_bn128_pairing_two_point_match_2 ... bench: 6,809,049 ns/iter (+/- 409,879)
test alt_bn128_pairing_two_point_match_3 ... bench: 6,948,928 ns/iter (+/- 691,229)
test alt_bn128_pairing_two_point_match_4 ... bench: 6,963,919 ns/iter (+/- 744,114)
test ecrecover ... bench: 123,149 ns/iter (+/- 21,322)
test identity ... bench: 78 ns/iter (+/- 14)
test modexp_eip_example1 ... bench: 585,479 ns/iter (+/- 101,136)
test modexp_eip_example2 ... bench: 293 ns/iter (+/- 67)
test modexp_nagydani_1_pow0x10001 ... bench: 48,475 ns/iter (+/- 11,729)
test modexp_nagydani_1_qube ... bench: 9,180 ns/iter (+/- 2,000)
test modexp_nagydani_1_square ... bench: 5,932 ns/iter (+/- 382)
test modexp_nagydani_2_pow0x10001 ... bench: 90,050 ns/iter (+/- 9,794)
test modexp_nagydani_2_qube ... bench: 15,877 ns/iter (+/- 4,157)
test modexp_nagydani_2_square ... bench: 10,720 ns/iter (+/- 584)
test modexp_nagydani_3_pow0x10001 ... bench: 252,624 ns/iter (+/- 38,900)
test modexp_nagydani_3_qube ... bench: 44,000 ns/iter (+/- 2,469)
test modexp_nagydani_3_square ... bench: 30,180 ns/iter (+/- 2,348)
test modexp_nagydani_4_pow0x10001 ... bench: 820,333 ns/iter (+/- 62,901)
test modexp_nagydani_4_qube ... bench: 143,577 ns/iter (+/- 28,778)
test modexp_nagydani_4_square ... bench: 99,039 ns/iter (+/- 18,831)
test modexp_nagydani_5_pow0x10001 ... bench: 2,942,966 ns/iter (+/- 286,061)
test modexp_nagydani_5_qube ... bench: 499,347 ns/iter (+/- 68,674)
test modexp_nagydani_5_square ... bench: 344,958 ns/iter (+/- 51,048)
test ripemd ... bench: 777 ns/iter (+/- 68)
test sha256 ... bench: 478 ns/iter (+/- 101)
test result: ok. 0 passed; 0 failed; 0 ignored; 57 measured; 0 filtered out
Raw
raw-master-5a1dc3e-bn-786c0d5.benchmark
$ sysctl -n machdep.cpu.brand_string
Intel(R) Core(TM) i7-7920HQ CPU @ 3.10GHz
test result: ok. 0 passed; 0 failed; 333 ignored; 0 measured; 0 filtered out
Running `/Users/piotr/git/parity-ethereum/target/release/deps/builtin-f8cde5d10d9eff41 --bench`
running 57 tests
test alt_bn128_add_cdetrio1 ... bench: 290 ns/iter (+/- 4)
test alt_bn128_add_cdetrio10 ... bench: 977 ns/iter (+/- 153)
test alt_bn128_add_cdetrio11 ... bench: 13,104 ns/iter (+/- 1,189)
test alt_bn128_add_cdetrio12 ... bench: 13,834 ns/iter (+/- 1,614)
test alt_bn128_add_cdetrio13 ... bench: 11,854 ns/iter (+/- 1,938)
test alt_bn128_add_cdetrio14 ... bench: 2,747 ns/iter (+/- 265)
test alt_bn128_add_cdetrio2 ... bench: 279 ns/iter (+/- 23)
test alt_bn128_add_cdetrio3 ... bench: 291 ns/iter (+/- 29)
test alt_bn128_add_cdetrio4 ... bench: 288 ns/iter (+/- 42)
test alt_bn128_add_cdetrio5 ... bench: 275 ns/iter (+/- 46)
test alt_bn128_add_cdetrio6 ... bench: 943 ns/iter (+/- 146)
test alt_bn128_add_cdetrio7 ... bench: 934 ns/iter (+/- 144)
test alt_bn128_add_cdetrio8 ... bench: 972 ns/iter (+/- 109)
test alt_bn128_add_cdetrio9 ... bench: 919 ns/iter (+/- 61)
test alt_bn128_add_chfast1 ... bench: 11,336 ns/iter (+/- 1,018)
test alt_bn128_add_chfast2 ... bench: 11,905 ns/iter (+/- 1,503)
test alt_bn128_mul_cdetrio1 ... bench: 496,267 ns/iter (+/- 97,125)
test alt_bn128_mul_cdetrio11 ... bench: 494,407 ns/iter (+/- 33,830)
test alt_bn128_mul_cdetrio6 ... bench: 499,397 ns/iter (+/- 97,614)
test alt_bn128_mul_chfast1 ... bench: 111,300 ns/iter (+/- 14,779)
test alt_bn128_mul_chfast2 ... bench: 248,520 ns/iter (+/- 14,387)
test alt_bn128_mul_chfast3 ... bench: 437,082 ns/iter (+/- 27,567)
test alt_bn128_pairing_empty_data ... bench: 28 ns/iter (+/- 1)
test alt_bn128_pairing_jeff1 ... bench: 13,969,394 ns/iter (+/- 553,386)
test alt_bn128_pairing_jeff2 ... bench: 13,933,523 ns/iter (+/- 562,737)
test alt_bn128_pairing_jeff3 ... bench: 13,961,327 ns/iter (+/- 509,729)
test alt_bn128_pairing_jeff4 ... bench: 21,983,682 ns/iter (+/- 2,389,361)
test alt_bn128_pairing_jeff5 ... bench: 21,978,623 ns/iter (+/- 3,565,608)
test alt_bn128_pairing_jeff6 ... bench: 13,763,837 ns/iter (+/- 554,839)
test alt_bn128_pairing_one_point ... bench: 6,985,496 ns/iter (+/- 386,617)
test alt_bn128_pairing_ten_point_match_1 ... bench: 67,707,776 ns/iter (+/- 3,632,600)
test alt_bn128_pairing_ten_point_match_2 ... bench: 67,511,217 ns/iter (+/- 3,032,175)
test alt_bn128_pairing_ten_point_match_3 ... bench: 13,710,818 ns/iter (+/- 532,709)
test alt_bn128_pairing_two_point_match_2 ... bench: 13,675,015 ns/iter (+/- 608,874)
test alt_bn128_pairing_two_point_match_3 ... bench: 13,714,703 ns/iter (+/- 641,139)
test alt_bn128_pairing_two_point_match_4 ... bench: 13,769,451 ns/iter (+/- 802,421)
test ecrecover ... bench: 113,570 ns/iter (+/- 16,435)
test identity ... bench: 77 ns/iter (+/- 11)
test modexp_eip_example1 ... bench: 569,781 ns/iter (+/- 70,736)
test modexp_eip_example2 ... bench: 290 ns/iter (+/- 33)
test modexp_nagydani_1_pow0x10001 ... bench: 42,564 ns/iter (+/- 5,066)
test modexp_nagydani_1_qube ... bench: 8,196 ns/iter (+/- 397)
test modexp_nagydani_1_square ... bench: 5,760 ns/iter (+/- 459)
test modexp_nagydani_2_pow0x10001 ... bench: 87,110 ns/iter (+/- 10,860)
test modexp_nagydani_2_qube ... bench: 15,179 ns/iter (+/- 753)
test modexp_nagydani_2_square ... bench: 10,542 ns/iter (+/- 497)
test modexp_nagydani_3_pow0x10001 ... bench: 253,620 ns/iter (+/- 40,738)
test modexp_nagydani_3_qube ... bench: 43,602 ns/iter (+/- 2,690)
test modexp_nagydani_3_square ... bench: 29,735 ns/iter (+/- 2,730)
test modexp_nagydani_4_pow0x10001 ... bench: 855,517 ns/iter (+/- 101,537)
test modexp_nagydani_4_qube ... bench: 140,814 ns/iter (+/- 23,713)
test modexp_nagydani_4_square ... bench: 91,803 ns/iter (+/- 7,332)
test modexp_nagydani_5_pow0x10001 ... bench: 2,919,130 ns/iter (+/- 170,181)
test modexp_nagydani_5_qube ... bench: 499,157 ns/iter (+/- 57,339)
test modexp_nagydani_5_square ... bench: 332,092 ns/iter (+/- 34,205)
test ripemd ... bench: 778 ns/iter (+/- 95)
test sha256 ... bench: 466 ns/iter (+/- 69)
test result: ok. 0 passed; 0 failed; 0 ignored; 57 measured; 0 filtered out
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment