Skip to content

Instantly share code, notes, and snippets.

@timxor

timxor/bench_august.md

Last active August 23, 2017 22:07
Show Gist options
  • Save timxor/87f5fff7e7e1d3966a40b7bc2fc7344a to your computer and use it in GitHub Desktop.
Save timxor/87f5fff7e7e1d3966a40b7bc2fc7344a to your computer and use it in GitHub Desktop.
Download ZIP
tims_geth_benchmarks_august
Raw
bench_august.md

August 23rd *** macbook *** results:

➜  vm git:(master) ✗ go test -bench BenchmarkPrecompiled -benchtime 5s
BenchmarkPrecompiledEcrecover/-Gas=3000-8         	   50000	    144319 ns/op
BenchmarkPrecompiledSha256/128-Gas=108-8          	20000000	       578 ns/op
BenchmarkPrecompiledRipeMD/128-Gas=1080-8         	 5000000	      1662 ns/op
BenchmarkPrecompiledIdentity/128-Gas=27-8         	500000000	        15.3 ns/op
BenchmarkPrecompiledModExp/eip_example1-Gas=2611-8         	  300000	     29852 ns/op
BenchmarkPrecompiledModExp/eip_example2-Gas=2611-8         	 1000000	      7569 ns/op
BenchmarkPrecompiledModExp/nagydani-1-square-Gas=40-8      	 3000000	      2236 ns/op
BenchmarkPrecompiledModExp/nagydani-1-qube-Gas=40-8        	 3000000	      2997 ns/op
BenchmarkPrecompiledModExp/nagydani-1-pow0x10001-Gas=655-8 	  500000	     11987 ns/op
BenchmarkPrecompiledModExp/nagydani-2-square-Gas=133-8     	 2000000	      3717 ns/op
BenchmarkPrecompiledModExp/nagydani-2-qube-Gas=133-8       	 1000000	      5468 ns/op
BenchmarkPrecompiledModExp/nagydani-2-pow0x10001-Gas=2129-8         	  300000	     28191 ns/op
BenchmarkPrecompiledModExp/nagydani-3-square-Gas=378-8              	 1000000	      7345 ns/op
BenchmarkPrecompiledModExp/nagydani-3-qube-Gas=378-8                	  500000	     12177 ns/op
BenchmarkPrecompiledModExp/nagydani-3-pow0x10001-Gas=6062-8         	  100000	     69323 ns/op
BenchmarkPrecompiledModExp/nagydani-4-square-Gas=1116-8             	  500000	     15967 ns/op
BenchmarkPrecompiledModExp/nagydani-4-qube-Gas=1116-8               	  200000	     30117 ns/op
BenchmarkPrecompiledModExp/nagydani-4-pow0x10001-Gas=17858-8        	   50000	    187924 ns/op
BenchmarkPrecompiledModExp/nagydani-5-square-Gas=3573-8             	  200000	     39178 ns/op
BenchmarkPrecompiledModExp/nagydani-5-qube-Gas=3573-8               	  100000	     81914 ns/op
BenchmarkPrecompiledModExp/nagydani-5-pow0x10001-Gas=57180-8        	   10000	    564070 ns/op
BenchmarkPrecompiledBn256Add/chfast1-Gas=500-8                      	  200000	     40977 ns/op
BenchmarkPrecompiledBn256Add/chfast2-Gas=500-8                      	  200000	     43206 ns/op
BenchmarkPrecompiledBn256Add/cdetrio1-Gas=500-8                     	 5000000	      1332 ns/op
BenchmarkPrecompiledBn256Add/cdetrio2-Gas=500-8                     	 5000000	      1365 ns/op
BenchmarkPrecompiledBn256Add/cdetrio3-Gas=500-8                     	 5000000	      1361 ns/op
BenchmarkPrecompiledBn256Add/cdetrio4-Gas=500-8                     	 5000000	      1393 ns/op
BenchmarkPrecompiledBn256Add/cdetrio5-Gas=500-8                     	 5000000	      1337 ns/op
BenchmarkPrecompiledBn256Add/cdetrio6-Gas=500-8                     	 5000000	      1442 ns/op
BenchmarkPrecompiledBn256Add/cdetrio7-Gas=500-8                     	 5000000	      1449 ns/op
BenchmarkPrecompiledBn256Add/cdetrio8-Gas=500-8                     	 5000000	      1474 ns/op
BenchmarkPrecompiledBn256Add/cdetrio9-Gas=500-8                     	 5000000	      1454 ns/op
BenchmarkPrecompiledBn256Add/cdetrio10-Gas=500-8                    	 5000000	      1452 ns/op
BenchmarkPrecompiledBn256Add/cdetrio11-Gas=500-8                    	 1000000	      6221 ns/op
BenchmarkPrecompiledBn256Add/cdetrio12-Gas=500-8                    	 1000000	      6182 ns/op
BenchmarkPrecompiledBn256Add/cdetrio13-Gas=500-8                    	  200000	     40954 ns/op
BenchmarkPrecompiledBn256Add/cdetrio14-Gas=500-8                    	 1000000	      6039 ns/op
BenchmarkPrecompiledBn256ScalarMul/chfast1-Gas=2000-8               	   20000	    420845 ns/op
BenchmarkPrecompiledBn256ScalarMul/chfast2-Gas=2000-8               	    5000	   1634706 ns/op
BenchmarkPrecompiledBn256ScalarMul/chfast3-Gas=2000-8               	    5000	   1612314 ns/op
BenchmarkPrecompiledBn256Pairing/jeff1-Gas=260000-8                 	     500	  17302458 ns/op
BenchmarkPrecompiledBn256Pairing/jeff2-Gas=260000-8                 	     500	  17160573 ns/op
BenchmarkPrecompiledBn256Pairing/jeff3-Gas=260000-8                 	     500	  17041258 ns/op
BenchmarkPrecompiledBn256Pairing/jeff4-Gas=340000-8                 	     300	  21977570 ns/op
BenchmarkPrecompiledBn256Pairing/jeff5-Gas=340000-8                 	     300	  21854866 ns/op
BenchmarkPrecompiledBn256Pairing/jeff6-Gas=260000-8                 	     500	  17257593 ns/op
BenchmarkPrecompiledBn256Pairing/empty_data-Gas=100000-8            	    5000	   1451992 ns/op
BenchmarkPrecompiledBn256Pairing/one_point-Gas=180000-8             	     500	  11918127 ns/op
BenchmarkPrecompiledBn256Pairing/two_point_match_2-Gas=260000-8     	    1000	  10355579 ns/op
BenchmarkPrecompiledBn256Pairing/two_point_match_3-Gas=260000-8     	     500	  16730228 ns/op
BenchmarkPrecompiledBn256Pairing/two_point_match_4-Gas=260000-8     	     500	  16751615 ns/op
BenchmarkPrecompiledBn256Pairing/ten_point_match_1-Gas=900000-8     	     200	  45620049 ns/op
BenchmarkPrecompiledBn256Pairing/ten_point_match_2-Gas=900000-8     	     100	  53389117 ns/op
BenchmarkPrecompiledBn256Pairing/ten_point_match_3-Gas=260000-8     	     500	  16783709 ns/op
PASS
ok  	_/Users/tim.siwula/Dropbox/Projects/go-ethereum/core/vm	478.468s
➜  vm git:(master) ✗ git rev-parse HEAD
3c48a25762dfab9382791c33a2f5832466077ac3

August 22nd *** macbook *** results:

➜  go-ethereum git:(master) git rev-parse HEAD
8d6a5a3581ce6221786eb464bfef7e8c31e7ad95
➜  go-ethereum git:(master) cd core/vm
➜  vm git:(master) go test -bench BenchmarkPrecompiled -benchtime 5s
BenchmarkPrecompiledEcdsa-8      	   50000	    153764 ns/op
BenchmarkPrecompiledSha256-8     	^[^[^C%                                                                                                                                                                                               ➜  vm git:(master) go test -bench BenchmarkPrecompiled -benchtime 5s
BenchmarkPrecompiledEcdsa-8      	   50000	    159277 ns/op
BenchmarkPrecompiledSha256-8     	10000000	       637 ns/op
BenchmarkPrecompiledRipeMD-8     	 3000000	      1982 ns/op
BenchmarkPrecompiledIdentity-8   	500000000	        16.3 ns/op
PASS
ok  	github.com/ethereum/go-ethereum/core/vm	34.428s
➜  vm git:(master) git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working tree clean
➜  vm git:(master) go test -bench BenchmarkPrecompiled -benchtime 5s
BenchmarkPrecompiledEcdsa-8      	   50000	    142948 ns/op
BenchmarkPrecompiledSha256-8     	20000000	       605 ns/op
BenchmarkPrecompiledRipeMD-8     	 5000000	      1806 ns/op
BenchmarkPrecompiledIdentity-8   	500000000	        15.3 ns/op
PASS
ok  	github.com/ethereum/go-ethereum/core/vm	41.468s
➜  vm git:(master)

August 21st:

*** macbook ***

August 19th *** macbook *** results:

=> go test -bench BenchmarkPrecompiled -benchtime 5s
BenchmarkPrecompiledEcdsa/-Gas=3000-8  	   50000	    163069 ns/op
BenchmarkPrecompiledSha256/128-Gas=108-8         	10000000	       645 ns/op
BenchmarkPrecompiledRipeMD/128-Gas=1080-8        	 5000000	      1877 ns/op
BenchmarkPrecompiledIdentity/128-Gas=27-8        	500000000	        17.3 ns/op
BenchmarkPrecompiledModexp/eip_example1-Gas=4197376-8         	  200000	     33746 ns/op
BenchmarkPrecompiledModexp/eip_example2-Gas=4459488-8         	 1000000	      7438 ns/op
BenchmarkPrecompiledBn256Add/chfast1-Gas=0-8                  	  200000	     45821 ns/op
BenchmarkPrecompiledBn256Add/chfast2-Gas=0-8                  	  200000	     48056 ns/op
BenchmarkPrecompiledBn256ScalarMul/chfast1-Gas=0-8            	   20000	    471451 ns/op
BenchmarkPrecompiledBn256ScalarMul/chfast2-Gas=0-8            	    5000	   1765534 ns/op
BenchmarkPrecompiledBn256ScalarMul/chfast3-Gas=0-8            	    5000	   1764253 ns/op
BenchmarkPrecompiledPairing/jeff1-Gas=160000-8                	     500	  18605464 ns/op
BenchmarkPrecompiledPairing/jeff2-Gas=160000-8                	     500	  18702602 ns/op
BenchmarkPrecompiledPairing/jeff3-Gas=160000-8                	     500	  18771734 ns/op
BenchmarkPrecompiledPairing/jeff4-Gas=220000-8                	     300	  23876686 ns/op
BenchmarkPrecompiledPairing/jeff5-Gas=220000-8                	     300	  23989863 ns/op
BenchmarkPrecompiledPairing/jeff6-Gas=160000-8                	     500	  18586363 ns/op
BenchmarkPrecompiledPairing/empty_data-Gas=40000-8            	500000000	        13.6 ns/op
BenchmarkPrecompiledPairing/one_point-Gas=100000-8            	     500	  12694421 ns/op
BenchmarkPrecompiledPairing/two_point_match_2-Gas=160000-8    	    1000	  10988584 ns/op
BenchmarkPrecompiledPairing/two_point_match_3-Gas=160000-8    	     500	  19651165 ns/op
BenchmarkPrecompiledPairing/two_point_match_4-Gas=160000-8    	     500	  19152084 ns/op
BenchmarkPrecompiledPairing/ten_point_match_1-Gas=640000-8    	     100	  52055717 ns/op
BenchmarkPrecompiledPairing/ten_point_match_2-Gas=640000-8    	     100	  58129981 ns/op
BenchmarkPrecompiledPairing/ten_point_match_3-Gas=160000-8    	     500	  17919238 ns/op
PASS
ok  	github.com/ethereum/go-ethereum/core/vm	242.833s
=>

Ran on:

  Model Name:	MacBook Pro
  Model Identifier:	MacBookPro14,3
  Processor Name:	Intel Core i7
  Processor Speed:	3.1 GHz
  Number of Processors:	1
  Total Number of Cores:	4
  L2 Cache (per Core):	256 KB
  L3 Cache:	8 MB
  Memory:	16 GB
  System Version:	macOS 10.12.6 (16G29)
  Kernel Version:	Darwin 16.7.0
@holiman
Copy link

holiman commented Aug 21, 2017

Here are your worst-cases

Name                                         Gascost         Time (ns)      MGas/S    Gasprice for 10MGas/S  Gas/Ideal percent
-----------------------------------------  ---------  ----------------  ----------  -----------------------  -------------------
PrecompiledEcrecover/                           3000  185022              16.2143                   1850.22  162.1 %
PrecompiledSha256/128                            108     728             148.352                       7.28  1483.5 %
PrecompiledRipeMD/128                           1080    2116             510.397                      21.16  5104.0 %
PrecompiledIdentity/128                           27      20            1350                           0.2   13500.0 %
PrecompiledModExp/nagydani-1-qube                 40    3844              10.4058                     38.44  104.1 %
PrecompiledBn256Add/chfast1                      500   65233               7.66483                   652.33  76.6 %
PrecompiledBn256ScalarMul/chfast2               2000       1.89115e+06     1.05756                 18911.5   10.6 %
PrecompiledBn256Pairing/one_point             180000       1.40256e+07    12.8337                 140256     128.3 %

@holiman
Copy link

holiman commented Aug 21, 2017

So your data points to

  • Modexp is pretty right
  • Add should be ~650
  • Scalar mul should ~20K

If they we want them to be in line with ecdsa, they should be multiplied by a factor of ~ 1.6.

@timxor
Copy link
Author

timxor commented Aug 23, 2017

@holiman any idea, why not all of them are running?

➜  go-ethereum git:(master) git rev-parse HEAD
8d6a5a3581ce6221786eb464bfef7e8c31e7ad95
➜  go-ethereum git:(master) cd core/vm
➜  vm git:(master) go test -bench BenchmarkPrecompiled -benchtime 5s
BenchmarkPrecompiledEcdsa-8      	   50000	    153764 ns/op
BenchmarkPrecompiledSha256-8     	^[^[^C%                                                                                                                                                                                               ➜  vm git:(master) go test -bench BenchmarkPrecompiled -benchtime 5s
BenchmarkPrecompiledEcdsa-8      	   50000	    159277 ns/op
BenchmarkPrecompiledSha256-8     	10000000	       637 ns/op
BenchmarkPrecompiledRipeMD-8     	 3000000	      1982 ns/op
BenchmarkPrecompiledIdentity-8   	500000000	        16.3 ns/op
PASS
ok  	github.com/ethereum/go-ethereum/core/vm	34.428s
➜  vm git:(master) git status
On branch master
Your branch is up-to-date with 'origin/master'.

nothing to commit, working tree clean
➜  vm git:(master) go test -bench BenchmarkPrecompiled -benchtime 5s
BenchmarkPrecompiledEcdsa-8      	   50000	    142948 ns/op
BenchmarkPrecompiledSha256-8     	20000000	       605 ns/op
BenchmarkPrecompiledRipeMD-8     	 5000000	      1806 ns/op
BenchmarkPrecompiledIdentity-8   	500000000	        15.3 ns/op
PASS
ok  	github.com/ethereum/go-ethereum/core/vm	41.468s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment